CN112584116B

CN112584116B - Projection correction method, projection correction device, storage medium and electronic equipment

Info

Publication number: CN112584116B
Application number: CN202110206396.5A
Authority: CN
Inventors: 胡震宇; 付啸天; 孙世攀; 张聪
Original assignee: Shenzhen Huole Science and Technology Development Co Ltd
Current assignee: Shenzhen Huole Science and Technology Development Co Ltd
Priority date: 2021-02-24
Filing date: 2021-02-24
Publication date: 2021-06-29
Anticipated expiration: 2041-02-24
Also published as: WO2022179108A1; CN113365040A; WO2022179109A1; CN112584116A; CN113365041B; CN113365041A; CN113365040B

Abstract

The present disclosure relates to a projection correction method, apparatus, storage medium and electronic device, and relates to the field of projection technology, including: the method comprises the steps of projecting a correction image onto an imaging medium to obtain a projection correction image displayed on the imaging medium, acquiring a collected image, determining first position information of each second mark pattern in the collected image, determining a first boundary formed by the projection correction image in the collected image according to the first position information of each second mark pattern and the second position information of each first mark pattern, determining a target projection area on the imaging medium according to the first boundary, and projecting an image to be projected onto the imaging medium according to the target projection area to display information included in the image to be projected in the target projection area. The projection image processing method and device can ensure that the projection image to be projected cannot be distorted when being projected onto an imaging medium, can obtain the projection effect meeting the requirements of a user, does not need manual adjustment of the user, is simple to operate, and is high in projection correction efficiency.

Description

Projection correction method, projection correction device, storage medium and electronic equipment

Technical Field

The present disclosure relates to the field of projection technologies, and in particular, to a projection correction method and apparatus, a storage medium, and an electronic device.

Background

In order to meet the viewing requirements of people for large-size screens, projection devices are widely used in homes, offices, schools and entertainment venues. The projection device may project onto an imaging medium, such as a projection screen, wall surface, or hard screen, to display a corresponding projected image on the imaging medium. However, when the projection device projects the image onto the imaging medium, the projected image may be distorted, which may affect the projection effect of the projection device. For example, for a short-focus or ultra-short-focus projection device, due to the limitations of the design process, the manufacturing process, and the assembly process of the projection optics, a pincushion distortion or barrel distortion may occur to a projection image projected on an imaging medium by the projection device, and the distortion may not disappear as the relative position of the projection device and the imaging medium changes. For another example, when the surface of the imaging medium is uneven (e.g., the projection screen is curved or there are concave and convex portions on the wall surface), the projection image projected on the imaging medium by the projection apparatus may be distorted, which is random and unpredictable. For another example, when a short-focus or ultra-short-focus projection device projects on an imaging medium, if the surface of the imaging medium is uneven, the unevenness will be reflected at the edge of the projected image, so that the edge of the projected image will exhibit a wave-like curve distortion.

Disclosure of Invention

In order to solve the problems in the related art, the present disclosure provides a projection correction method, apparatus, storage medium, and electronic device.

In order to achieve the above object, according to a first aspect of the embodiments of the present disclosure, there is provided a projection correction method applied to a projection apparatus, the method including:

projecting a preset correction image onto an imaging medium to obtain a projection correction image displayed on the imaging medium, and acquiring a collected image; the acquired image is determined according to an image obtained by shooting the projection correction image by an image acquisition device, the correction image comprises a plurality of first mark patterns, and the acquired image comprises second mark patterns corresponding to each first mark pattern;

determining first position information for each of the second marker patterns in the captured image;

determining a first boundary formed by the projection correction image in the acquired image according to the first position information of each second mark pattern and the second position information of each first mark pattern;

determining a target projection area on the imaging medium according to the first boundary;

and projecting the image to be projected onto the imaging medium according to the target projection area so as to show information included in the image to be projected in the target projection area.

According to a second aspect of the embodiments of the present disclosure, there is provided a projection correction apparatus applied to a projection device, the apparatus including:

the device comprises an acquisition module, a correction module and a correction module, wherein the acquisition module is used for projecting a preset correction image onto an imaging medium to obtain a projection correction image displayed on the imaging medium and acquiring a collected image; the acquired image is determined according to an image obtained by shooting the projection correction image by an image acquisition device, the correction image comprises a plurality of first mark patterns, and the acquired image comprises second mark patterns corresponding to each first mark pattern;

a processing module for determining first position information for each of the second marker patterns in the captured image;

the processing module is further configured to determine a first boundary formed by the projection correction image in the acquired image according to the first position information of each of the second mark patterns and the second position information of each of the first mark patterns;

the processing module is further used for determining a target projection area on the imaging medium according to the first boundary;

and the correction module is used for projecting the image to be projected onto the imaging medium according to the target projection area so as to display the information included in the image to be projected in the target projection area.

According to a third aspect of embodiments of the present disclosure, there is provided a computer readable storage medium having stored thereon a computer program which, when executed by a processor, performs the steps of the method of any one of the above first aspects.

According to a fourth aspect of the embodiments of the present disclosure, there is provided an electronic apparatus including:

a memory having a computer program stored thereon;

a processor for executing the computer program in the memory to implement the steps of the method of any one of the first aspect.

According to a fifth aspect of the embodiments of the present disclosure, there is provided a projection correction method applied to a projection apparatus, the method including:

projecting a preset correction image onto an imaging medium to obtain a projection correction image displayed on the imaging medium, and acquiring a collected image; the correction image comprises a plurality of first marker patterns, the acquisition image comprises a second marker pattern corresponding to each first marker pattern, and the acquisition image is determined according to an image obtained by shooting the projection correction image by an image acquisition device;

determining a corresponding correction line of a specified boundary in the first boundaries on the imaging medium, wherein the specified boundary is a boundary in a first direction in the first boundaries;

determining a target number of correction zones on the imaging media based on the first boundary;

determining a correction parameter corresponding to each correction area according to each correction area and each correction line;

and correcting the image to be projected according to the correction parameters, and projecting the corrected image to be projected onto the imaging medium.

According to a sixth aspect of the embodiments of the present disclosure, there is provided a projection correction apparatus applied to a projection device, the apparatus including:

the device comprises an acquisition module, a correction module and a correction module, wherein the acquisition module is used for projecting a preset correction image onto an imaging medium to obtain a projection correction image displayed on the imaging medium and acquiring a collected image; the correction image comprises a plurality of first marker patterns, the acquisition image comprises a second marker pattern corresponding to each first marker pattern, and the acquisition image is determined according to an image obtained by shooting the projection correction image by an image acquisition device;

the processing module is further configured to determine a corresponding correction line of a specified one of the first boundaries on the imaging medium, where the specified one is a boundary in a first direction of the first boundaries;

the processing module is further configured to determine a target number of correction regions on the imaging medium based on the first boundary;

the processing module is further configured to determine a correction parameter corresponding to each correction area according to each correction area and the correction line;

and the correction module is used for correcting the image to be projected according to the correction parameters and projecting the corrected image to be projected onto the imaging medium.

According to a seventh aspect of embodiments of the present disclosure, there is provided a computer-readable storage medium, on which a computer program is stored, which when executed by a processor, implements the steps of the method of any one of the above fifth aspects.

According to an eighth aspect of embodiments of the present disclosure, there is provided an electronic apparatus including:

a memory having a computer program stored thereon;

a processor for executing the computer program in the memory to implement the steps of the method of any one of the fifth aspects.

According to a ninth aspect of the embodiments of the present disclosure, there is provided a projection correction method applied to a projection apparatus, the method including:

determining a corresponding first projection area of the projection correction image in the acquired image according to the first position information of each second mark pattern and the second position information of each first mark pattern;

determining a corresponding second projection area of the first projection area on the imaging medium, the second projection area including a third marker pattern corresponding to each of the first marker patterns;

determining third position information of each third mark pattern and an offset parameter corresponding to the third mark pattern according to the second position information;

under the condition that the offset parameter does not meet a preset condition, determining a target correction matrix corresponding to each first mark pattern according to the first position information and the third position information;

and correcting the image to be projected according to the target correction matrix, and projecting the corrected image to be projected onto the imaging medium.

According to a tenth aspect of the embodiments of the present disclosure, there is provided a projection correction apparatus applied to a projection device, the apparatus including:

the processing module is further configured to determine a corresponding first projection area of the projection correction image in the acquired image according to the first position information of each second marker pattern and the second position information of each first marker pattern;

the processing module is further configured to determine a second projection area corresponding to the first projection area on the imaging medium, where the second projection area includes a third marker pattern corresponding to each of the first marker patterns;

the processing module is further configured to determine third position information of each third mark pattern and an offset parameter corresponding to the third mark pattern according to the second position information;

the processing module is further configured to determine, according to the first position information and the third position information, a target correction matrix corresponding to each first marker pattern when the offset parameter does not satisfy a preset condition;

and the correction module is used for correcting the image to be projected according to the target correction matrix and projecting the corrected image to be projected onto the imaging medium.

According to an eleventh aspect of embodiments of the present disclosure, there is provided a computer-readable storage medium having stored thereon a computer program which, when executed by a processor, implements the steps of the method of any one of the above ninth aspects.

According to a twelfth aspect of an embodiment of the present disclosure, there is provided an electronic apparatus including:

a memory having a computer program stored thereon;

a processor for executing the computer program in the memory to implement the steps of the method of any one of the ninth aspects.

Through the technical scheme, the method comprises the steps of firstly projecting a preset correction image onto an imaging medium to obtain a projection correction image displayed on the imaging medium, and acquiring a collected image, wherein the collected image is determined according to an image obtained by shooting the projection correction image by an image collecting device, the correction image comprises a plurality of first mark patterns, the collected image comprises second mark patterns corresponding to each first mark pattern, then determining first position information of each second mark pattern in the collected image, determining a first boundary formed by the projection correction image in the collected image according to the first position information of each second mark pattern and the second position information of each first mark pattern, then determining a target projection area on the imaging medium according to the first boundary, and then projecting an image to be projected onto the imaging medium according to the target projection area, to present the information comprised by the image to be projected within the projection area of the object. This is disclosed through confirming the projection and correcting the target projection region that does not take place the distortion in the image to treating projection image projection to imaging media according to target projection region on, can ensure will treat that the projection image can not take place the distortion when projecting to imaging media on, can obtain the projection effect that satisfies user's requirement, need not user's manual regulation, easy operation, it is efficient to carry out the correction to the projection.

Additional features and advantages of the disclosure will be set forth in the detailed description which follows.

Drawings

The accompanying drawings, which are included to provide a further understanding of the disclosure and are incorporated in and constitute a part of this specification, illustrate embodiments of the disclosure and together with the description serve to explain the disclosure without limiting the disclosure. In the drawings:

FIG. 1 is a flow chart illustrating a method of projection correction according to an exemplary embodiment;

FIG. 2 is a schematic illustration of a corrected image shown in accordance with an exemplary embodiment;

FIG. 3 is a flow chart of one step 101 shown in the embodiment of FIG. 1;

FIG. 4 is a flow chart of one step 103 shown in the embodiment of FIG. 1;

FIG. 5 is a flow chart illustrating one step 104 of the embodiment shown in FIG. 1;

FIG. 6 is a block diagram illustrating a projection correction device in accordance with an exemplary embodiment;

FIG. 7 is a flow diagram illustrating a method of projection correction according to an exemplary embodiment;

FIG. 8 is a schematic diagram illustrating a first boundary in accordance with an exemplary embodiment;

FIG. 9 is a schematic diagram illustrating a fourth boundary, correction line, and correction zone in accordance with an exemplary embodiment;

FIG. 10 is a flowchart of one step 301 shown in the embodiment of FIG. 7;

FIG. 11 is a flowchart of one step 303 shown in the embodiment of FIG. 7;

FIG. 12 is a flowchart illustrating one step 304 of the embodiment shown in FIG. 7;

FIG. 13 is a flowchart of one step 305 shown in the embodiment of FIG. 7;

FIG. 14 is a block diagram illustrating a projection correction device in accordance with an exemplary embodiment;

FIG. 15 is a flow chart illustrating a method of projection correction according to an exemplary embodiment;

FIG. 16 is a flow chart illustrating one step 501 of the embodiment shown in FIG. 15;

FIG. 17 is a flowchart of one step 503 shown in the embodiment of FIG. 15;

FIG. 18 is a flowchart illustrating one step 504 of the embodiment shown in FIG. 15;

FIG. 19 is a flowchart illustrating one step 505 of the embodiment shown in FIG. 15;

FIG. 20 is a block diagram illustrating a projection correction device in accordance with an exemplary embodiment;

FIG. 21 is a block diagram illustrating an electronic device in accordance with an example embodiment.

Detailed Description

Reference will now be made in detail to the exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, like numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the exemplary embodiments below are not intended to represent all implementations consistent with the present disclosure. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present disclosure, as detailed in the appended claims.

Before describing the projection correction method, apparatus, storage medium, and electronic device provided by the present disclosure, an application scenario related to various embodiments of the present disclosure is first described, where the application scenario may include a projection device, an image capture apparatus, and an imaging medium. The projection device may project the self-played image or video onto an imaging medium to display a corresponding projected image on the imaging medium. The projection device may include, but is not limited to, a projector, a projection television, and the like, for example, the projection device may be a short-focus or ultra-short-focus projector, and may also be a long-focus projector. The image capturing device is used for capturing a projected image displayed on an imaging medium, and may be a camera module (e.g., a camera, an image sensor, etc.) mounted on the projection apparatus itself, or a mobile camera module separated from the projection apparatus by a wired connection or a wireless connection, or a terminal (i.e., third-party hardware) having a capturing function, and the terminal may be, for example, a terminal such as a smart phone, a tablet computer, a smart watch, a smart bracelet, or a PDA (english: Personal Digital Assistant). The imaging medium may be a projection screen, a wall surface, a hard screen, or any other device capable of displaying a projection of a projection device, and the disclosure is not limited thereto.

In one scenario, when a projected image projected by a projection device onto an imaging medium experiences pincushion distortion or barrel distortion, the projected image may be corrected in the manner shown in the following example:

FIG. 1 is a flow chart illustrating a method of projection correction according to an exemplary embodiment. As shown in fig. 1, applied to a projection device, the method may include the steps of:

step 101, projecting a preset correction image onto an imaging medium to obtain a projection correction image displayed on the imaging medium, and acquiring a collected image.

The collected image is determined according to an image obtained by shooting a projection correction image by an image collecting device, the correction image comprises a plurality of first mark patterns, and the collected image comprises a second mark pattern corresponding to each first mark pattern.

For example, when the projection device projects an image onto the imaging medium, if the user finds that the projected image corresponding to the image has pincushion distortion or barrel distortion, a correction instruction may be sent to the projection device to correct the projected image. For example, a corresponding correction button may be provided on the projection device, and the user may press the correction button to send a correction instruction to the projection device. Then the projection equipment can send corresponding prompt information to the user to prompt the user to adjust the position of the image acquisition device to a proper position, so that the image acquisition device can accurately shoot a projection image of the projection equipment on an imaging medium, and the projection image can be corrected more accurately.

Specifically, when the image capturing device is a movable camera module separated from the projection device, the projection device may project a prompt image on the imaging medium after receiving the correction instruction, so as to prompt the user to move the camera module to the specified position. For example, the camera module may capture a prompt image, and send an image obtained by capturing the prompt image to the projection device, and the projection device determines whether the image obtained by capturing the prompt image by the camera module and the prompt image satisfy a preset adjustment condition. If the preset adjustment condition is met, the projection device can determine that the user has adjusted the camera module to a proper position. Otherwise, the projection device may determine that the user does not adjust the camera module to the proper position, and the projection device needs to project a new prompt image again to prompt the user to continue adjusting the camera module. The preset adjustment condition may be, for example, that an image obtained by capturing the hint image by the camera module includes a hint image, and a ratio of the hint image to an image obtained by capturing the hint image by the camera module is greater than or equal to a threshold (e.g., 50%). After the camera module is adjusted to a proper position, the projection device can project the corrected image including the plurality of first mark patterns onto the imaging medium to obtain a projection corrected image displayed on the imaging medium, then shoot the projection corrected image through the camera module, and send an image obtained by shooting the projection corrected image to the projection device. The image obtained by capturing the projection correction image is subjected to preprocessing (preprocessing may be, for example, image format conversion, segmentation processing, image size conversion, and the like) by the projection device to obtain an acquired image. Wherein the captured image includes a second marker pattern corresponding to each of the first marker patterns.

When the image acquisition device is a mobile phone with a shooting function, the projection device can project a prompt image on the imaging medium after receiving the correction instruction so as to prompt a user to use the mobile phone to pair the mobile phone with the projection device. After the pairing is successful, the projection equipment can project the corrected image onto the imaging medium to obtain a projection corrected image, then prompt the user to align the photographing interface of the mobile phone with the projection corrected image, and prompt the user to photograph the projection corrected image by using the mobile phone. And then the projection equipment can acquire an image obtained by shooting the projection correction image by the mobile phone and preprocess the image obtained by shooting the projection correction image to obtain a collected image. Further, in order to reduce the amount of data required to be processed by the projection device, the mobile phone may be further configured to pre-process an image obtained by capturing the projection correction image to obtain a captured image, and send the captured image to the projection device.

Wherein the corrected image may be any one of the images including the plurality of first marker patterns. When the corrected image is a checkerboard image, as shown in a of fig. 2, the first mark pattern may be any one of the checkerboards, and the projection device projects the corrected image onto the imaging medium, actually, the checkerboard is projected onto the imaging medium. At this time, the image obtained by the image capturing device capturing the projection correction image also includes a checkerboard, and then the captured image obtained by preprocessing the image obtained by capturing the projection correction image also includes a checkerboard (i.e., the captured image is also a checkerboard image), and the second mark pattern may be any one of the checkerboards. For another example, as shown in b in fig. 2, when the corrected image is an image including a plurality of circles, the first marker pattern may be any one of the plurality of circles, and at this time, the image obtained by capturing the projection corrected image also includes a plurality of circles, and then the captured image obtained by preprocessing the image obtained by capturing the projection corrected image also includes a plurality of circles (i.e., the captured image is also an image including a plurality of circles), and then the second marker pattern may be any one of the plurality of circles.

First position information for each second marker pattern in the captured image is determined, step 102.

In this step, the projection device may detect each second marker pattern in the captured image and determine first position information of each detected second marker pattern. Taking the corrected image as a checkerboard image and the first mark pattern as any one of the checkerboards as an example, the projection device may detect the checkerboard in the captured image and determine the position information of each of the checkerboards included in the captured image, for example, coordinates of four corner points of each of the checkerboards may be used as the position information of the checkerboard. The projection device may then acquire, as the first position information, position information of each of the tiles included in the checkerboard in the image.

Step 103, determining a first boundary formed by the projection correction image in the acquired image according to the first position information of each second mark pattern and the second position information of each first mark pattern.

For example, the projection device may store a priori information of the corrected image in advance, and the a priori information may include position information of a boundary of the corrected image and second position information of each first marker pattern in the corrected image. For example, when the correction image is a checkerboard image, the coordinates of four corner points of each square (i.e., the first mark pattern) included in the checkerboard in the correction image may be taken as the second position information of the square. The projection device may then determine a positional relationship of each first mark pattern with a boundary of the correction image based on the second positional information of each first mark pattern. Since the captured image is determined based on the image obtained by the image capturing device capturing the projection correction image projected from the correction image. Therefore, the positional relationship of each second marker pattern with the first boundary formed in the captured image by projecting the correction image is the same as the positional relationship of each first marker pattern with the boundary of the correction image. The projection device may determine the first boundary of the projection correction image formed in the captured image based on the positional relationship of each first marker pattern to the boundary of the correction image.

And step 104, determining a target projection area on the imaging medium according to the first boundary.

Further, after determining the first boundary, the projection apparatus may first calculate a homography matrix between an imaging plane of the image acquisition device and a projection plane where the imaging medium is located, for example, when a main optical axis of a projection light engine of the projection apparatus is parallel to a normal vector of the imaging medium, the projection plane and a projection plane of the projection light engine may be regarded as a same plane. Therefore, the homography matrix between the imaging plane and the projection plane can be calculated by using the first position information of each second marker pattern and the second position information of each first marker pattern, and the calculation of the homography matrix between the imaging plane and the projection plane by using the first position information and the second position information can refer to the manner described in the related art, and is not described herein again. The projection device may then map the first boundary on the projection plane via the homography matrix to obtain a corresponding boundary of the first boundary on the imaging medium. The projection device may then determine a target projection area satisfying a predetermined shape condition from the area bounded by the corresponding boundaries on the imaging medium by the first boundary. The preset shape condition may be, for example, a maximum rectangular area with a size of 16:9 in an area surrounded by corresponding boundaries of the first boundary on the imaging medium as the target projection area.

And 105, projecting the image to be projected onto an imaging medium according to the target projection area so as to display information included in the image to be projected in the target projection area.

For example, after determining the target projection area, the projection device may determine third position information of the correction points corresponding to the target projection area, for example, coordinates corresponding to each correction point may be used as the third position information. The correction points include corner points of the target projection region and the middle points of each edge of the target projection region, that is, the correction points include 4 correction corner points and 4 correction middle points. The projection device may then project the image to be projected onto the imaging medium according to the third position information, so as to display information included in the image to be projected in the target projection area, that is, display the projected image corresponding to the image to be projected in the target projection area. Specifically, the projection device may process the image to be projected according to the coordinates of the 8 correction points, so that when the projection device projects the image to be projected onto the imaging medium, the projection image corresponding to the image to be projected is not distorted any more. For example, if barrel distortion occurs in the previous projected image, the image to be projected may be pulled back toward 4 correction midpoints with 4 correction corner points as fixed points. If the projected image has pincushion distortion, the image to be projected can be pulled back to 4 correction corner points by taking 4 correction middle points as fixed points.

It should be noted that, if the projected image corresponding to the image to be projected is still distorted after the correction is completed, steps 101 to 105 may be repeatedly performed until the projected image corresponding to the image to be projected is not distorted any more.

Fig. 3 is a flow chart of one step 101 shown in the embodiment shown in fig. 1. As shown in fig. 3, step 101 may include the steps of:

in step 1011, an image obtained by the image acquisition device capturing the projection correction image is converted into a specified format to obtain a first image.

Step 1012, performing segmentation processing on the first image to remove the region of the first image except the projection corrected image, so as to obtain a second image.

Step 1013, converting the second image into a specified image size to obtain the captured image.

For example, after receiving an image obtained by shooting a projection correction image sent by an image acquisition device, the projection device needs to pre-process the image obtained by shooting the projection correction image to obtain a standard acquired image, so as to correct the projection image in the following process. Specifically, the image obtained by capturing the projection correction image may be first converted into a specified format, such as a BMP (english: Bitmap, chinese: Bitmap file) graphic file format, to obtain the first image. In the image obtained by capturing the projection correction image, other regions than the projection correction image may be included, and if the projection image is corrected by directly using the first image, noise and the amount of data in the subsequent processing may be increased. In order to reduce noise and data amount in the subsequent processing, the projection device may perform segmentation processing (for example, coarse-grained segmentation may be used) on the first image to remove regions of the first image except the projection correction image, so as to obtain the second image. The projection device may then convert the second image to a specified image size (e.g., 720P) and use the converted second image as the captured image. Furthermore, the projection equipment can also perform histogram equalization processing on the acquired image, and adjust the white balance and exposure duration of the acquired image so as to improve the quality of the acquired image.

It should be noted that the second image obtained by performing the segmentation processing on the first image may still include other regions except the projection corrected image. That is, preprocessing an image obtained by capturing a projection correction image is actually a coarse-grained processing method, and the acquired image does not include only the projection correction image.

Fig. 4 is a flow chart illustrating one step 103 of the embodiment shown in fig. 1. As shown in fig. 4, step 103 may include the steps of:

step 1031, determining a position relationship between each first mark pattern and the second boundary of the correction image according to each second position information, wherein the position relationship is used for representing the distance between the first mark pattern and the second boundary.

And 1032, determining a corresponding first area of the projection correction image in the acquired image according to the position relation and each piece of first position information.

For example, the projection device may calculate a distance between each first marker pattern and the second boundary according to the position information of the second boundary of the corrected image and the second position information of each first marker pattern in the prior information of the corrected image to obtain a position relationship between the first marker pattern and the second boundary. Then, the projection device can determine the position relationship between the second mark pattern corresponding to each first mark pattern and the boundary to be determined by using the proportional relationship between the pixel of the corrected image and the pixel of the acquired image according to the position relationship between each first mark pattern and the second boundary. And then the projection equipment can determine the corresponding undetermined boundary of the projection correction image in the acquired image by utilizing the position relation between each second mark pattern and the undetermined boundary. And expanding the undetermined boundary outwards by a specified pixel distance (for example, 20 pixels), and taking a region surrounded by the undetermined boundary after the outward expansion as a first region. The undetermined boundary can be understood as a pre-assumed first boundary, and the first area can be understood as an approximate range of the projection correction image in the acquired image.

Step 1033 determines a second region from the first region based on the color characteristic of the first region.

Step 1034, edge recognition is performed on the second region to obtain a first boundary.

Further, the projection device may extract color features of the first region, and search, according to the color features of the first region, a region with the same color feature difference from the first region by combining color feature differences between the second boundary in the prior information of the corrected image and the surrounding region, as the second region. Then, the projection device may perform edge identification on the second area to identify a boundary corresponding to the second area, and use the boundary as the first boundary. The method for performing edge recognition on the second region to obtain the first boundary may refer to the method described in the related art, and is not described herein again. It should be noted that the process of obtaining the first boundary is actually a fine-grained processing manner to further remove other regions in the acquired image except the projection correction image.

Fig. 5 is a flow chart illustrating one step 104 of the embodiment shown in fig. 1. As shown in fig. 5, step 104 may include the steps of:

step 1041, determining a transformation matrix according to the first position information and the second position information.

And 1042, determining a third boundary corresponding to the first boundary on the imaging medium according to the transformation matrix.

For example, the projection device may first calculate a transformation matrix (e.g., a homography matrix) between the imaging plane and the projection plane based on the first position information of each second marker pattern and the second position information of each first marker pattern after determining the first boundary. The projection device may then map the first boundary on the projection plane using the transformation matrix to obtain a corresponding third boundary of the first boundary on the imaging medium.

And step 1043, determining a target projection area according to the third boundary.

Further, the projection device may use a region surrounded by the third boundary as a region to be corrected, and determine a region satisfying a preset shape condition from the region to be corrected as a target projection region. For example, the projection device may determine a pixel distance of each pixel to a specified one of the second boundaries (e.g., the boundaries below and to the right of each pixel in the second boundary) based on the position information of each pixel in the region to be corrected. Then, the projection device can perform traversal search by using a preset traversal search algorithm according to the pixel distance from each pixel to the specified boundary, and calculate a target projection area satisfying a preset shape condition in the area to be corrected. The preset shape condition may be, for example, a maximum rectangular area with a size of 16:9 in the area to be corrected as the target projection area.

To sum up, the present disclosure first projects a preset calibration image onto an imaging medium to obtain a projected calibration image displayed on the imaging medium, and obtains a captured image, where the captured image is determined according to an image obtained by an image capturing device capturing the projected calibration image, the calibration image includes a plurality of first marker patterns, the captured image includes a second marker pattern corresponding to each first marker pattern, then determines first position information of each second marker pattern in the captured image, and determines a first boundary formed by the projected calibration image in the captured image according to the first position information of each second marker pattern and the second position information of each first marker pattern, and then determines a target projection area on the imaging medium according to the first boundary, and then projects an image to be projected onto the imaging medium according to the target projection area, to present the information comprised by the image to be projected within the projection area of the object. This is disclosed through confirming the projection and correcting the target projection region that does not take place the distortion in the image to treating projection image projection to imaging media according to target projection region on, can ensure will treat that the projection image can not take place the distortion when projecting to imaging media on, can obtain the projection effect that satisfies user's requirement, need not user's manual regulation, easy operation, it is efficient to carry out the correction to the projection.

Fig. 6 is a block diagram illustrating a projection correction apparatus according to an exemplary embodiment. As shown in fig. 6, applied to a projection device, the apparatus 200 includes:

the acquiring module 201 is configured to project a preset correction image onto an imaging medium to obtain a projection correction image displayed on the imaging medium, and acquire a collected image. The captured image is determined from an image obtained by an image capturing device capturing a projection correction image, the correction image includes a plurality of first marker patterns, and the captured image includes a second marker pattern corresponding to each of the first marker patterns.

A processing module 202 for determining first position information for each second marker pattern in the captured image.

The processing module 202 is further configured to determine a first boundary formed by the projection correction image in the captured image according to the first position information of each second marker pattern and the second position information of each first marker pattern.

The processing module 202 is further configured to determine a target projection area on the imaging medium according to the first boundary.

And the correcting module 203 is used for projecting the image to be projected onto the imaging medium according to the target projection area so as to display the information included in the image to be projected in the target projection area.

Optionally, the obtaining module 201 includes:

and the first conversion sub-module is used for converting the image obtained by shooting the projection correction image by the image acquisition device into a specified format so as to obtain a first image.

And the segmentation submodule is used for carrying out segmentation processing on the first image so as to remove the area except the projection correction image in the first image and obtain a second image.

And the second conversion sub-module is used for converting the second image into a specified image size so as to obtain an acquired image.

Optionally, the corrected image is a checkerboard image, the first mark pattern is any one of the checkerboards, and the processing module 202 includes:

and the detection submodule is used for detecting the checkerboards in the acquired image and determining the position information of each square included in the checkerboards in the acquired image.

And the first determining submodule is used for taking the position information of each square included in the checkerboards in the acquired image as first position information.

Optionally, the processing module 202 comprises:

and the second determining submodule is used for determining the position relation between each first mark pattern and the second boundary of the correction image according to each second position information, and the position relation is used for representing the distance between the first mark pattern and the second boundary.

And the second determining submodule is also used for determining a corresponding first area of the projection correction image in the acquired image according to the position relation and each piece of first position information.

And the second determining submodule is also used for determining a second area from the first area according to the color characteristic of the first area.

And the identification submodule is used for carrying out edge identification on the second area so as to obtain the first boundary.

Optionally, the processing module 202 comprises:

and the calculation submodule is used for determining a transformation matrix according to the first position information and the second position information.

And the third determining submodule is used for determining a third boundary corresponding to the first boundary on the imaging medium according to the transformation matrix.

And the third determining submodule is also used for determining a target projection area according to the third boundary.

Optionally, the third determining sub-module is configured to:

and taking the area surrounded by the third boundary as the area to be corrected.

And determining an area satisfying a preset shape condition from the area to be corrected as a target projection area.

Optionally, the correction module 203 comprises:

and the fourth determining submodule is used for determining third position information of the correction point corresponding to the target projection area. The correction points include corner points of the target projection area and the middle point of each edge of the target projection area.

And the correction submodule projects the image to be projected onto the imaging medium according to the third position information.

With regard to the apparatus in the above-described embodiment, the specific manner in which each module performs the operation has been described in detail in the embodiment related to the method, and will not be elaborated here.

In still another scenario, when the edge of the projected image projected on the imaging medium by the projection device exhibits a curve distortion of a wave pattern due to the uneven surface of the imaging medium, the projected image may be corrected in a manner as shown in the following embodiments:

FIG. 7 is a flow chart illustrating a method of projection correction according to an exemplary embodiment. As shown in fig. 7, applied to a projection apparatus, the method may include the steps of:

step 301, projecting a preset correction image onto an imaging medium to obtain a projection correction image displayed on the imaging medium, and acquiring a collected image.

The correction image comprises a plurality of first mark patterns, the collected image comprises a second mark pattern corresponding to each first mark pattern, and the collected image is determined according to an image obtained by shooting the projection correction image by the image collecting device.

For example, when a projection device projects an image onto an imaging medium, if a user finds that an edge of a projected image corresponding to the image exhibits a wave-like curve distortion (for example, an upper edge of the projected image, which should be flat, exhibits a wave-like curve distortion), a correction instruction may be sent to the projection device to correct the projected image. For example, a corresponding correction button may be provided on the projection device, and the user may press the correction button to send a correction instruction to the projection device. Then the projection equipment can send corresponding prompt information to the user to prompt the user to adjust the position of the image acquisition device to a proper position, so that the image acquisition device can accurately shoot a projection image of the projection equipment on an imaging medium, and the projection image can be corrected more accurately.

First position information for each second marker pattern in the captured image is determined, step 302.

Step 303, determining a first boundary formed by the projection correction image in the acquired image according to the first position information of each second mark pattern and the second position information of each first mark pattern.

In step 304, a corresponding correction line of a designated one of the first boundaries on the imaging medium is determined, the designated one being a boundary of the first boundaries in the first direction.

For example, in order to correct the projected image more accurately, the user may set a target edge (i.e., an edge where the curve distortion occurs) that the projection apparatus needs to correct the projected image according to the edge where the curve distortion occurs in the projected image, and send the target edge to the projection apparatus, so that the projection apparatus corrects the target edge, for example, the user may set the target edge by triggering a corresponding button while sending a correction instruction. Further, the target edge may be set by identifying the collected image through a projection device, which is not specifically limited in this disclosure.

Specifically, the projection device may determine a designated boundary in the first boundary according to the target edge, where the designated boundary may be understood as a corresponding boundary of the target edge in the acquired image. For example, when the target edge is an upper edge in the projection image (i.e., an edge above in the projection image), as shown in fig. 8, when the first direction is vertically upward, the specified boundary may be an upper one of the first boundaries (the first boundary includes boundaries 1-4), i.e., boundary 1 in fig. 8. The projection device may then traverse all of the first boundaries (i.e., boundaries 1-4 in fig. 8) through a preset boundary template to determine the specified boundaries from all of the first boundaries. The projection device may then map the specified boundary on a projection plane on which the imaging medium is located to obtain a third boundary corresponding to the specified boundary on the imaging medium, and select a correction line for correcting the target edge according to the third boundary. For example, when the target edge is an upper edge in the projection image, a straight line passing through the lowest point in the third boundary and perpendicular to the first direction may be taken as the correction line, or a straight line passing through a point at a distance above the lowest point in the third boundary and perpendicular to the first direction may be taken as the correction line.

Based on the first boundary, a target number of correction zones on the imaging media is determined, step 305.

Step 306, according to each correction area and correction line, determining the correction parameter corresponding to the correction area.

For example, the projection apparatus may map the first boundary on the projection plane to obtain a fourth boundary (the fourth boundary includes the third boundary) of the first boundary on the imaging medium, divide an area surrounded by the fourth boundary into a target number of sub-areas along a direction perpendicular to the first direction, and use each of the sub-areas as the correction area. The projection device may then use the distance between each correction region and the correction line as the correction parameter for that correction region. Further, the correction parameters corresponding to each correction area may be stored in an array, so that the subsequent projection device can correct the projection image.

When the target edge is an upper edge in the projection image and the number of targets is 12, if a straight line passing through the lowest point (i.e., point a in fig. 9) in the third boundary (i.e., the boundary of the wave pattern in the fourth boundary in fig. 9 which is located at the upper side) and perpendicular to the first direction is taken as the correction line, the fourth boundary, the correction line, and the correction area may be as shown in fig. 9.

And 307, correcting the image to be projected according to the correction parameters, and projecting the corrected image to be projected onto an imaging medium.

In this step, after determining the correction parameters corresponding to each correction region, the projection device may correct the image to be projected according to the correction parameters corresponding to each correction region, and project the corrected image to be projected onto the imaging medium, so that the target edge of the image to be projected corresponding to the image to be projected becomes smooth and is close to the position of the correction line (or directly aligns the target edge with the correction line), thereby preventing the edge of the image to be projected from presenting a curve distortion in a wavy pattern. For example, the projection apparatus may divide the image to be projected into the target number of regions to be corrected in the same manner as the target number of correction regions are divided. And each correction area corresponds to one area to be corrected, and the position of each correction area in the area surrounded by the fourth boundary is the same as the position of the area to be corrected corresponding to the correction area in the image to be projected. And then, the projection equipment can correct the area to be corrected corresponding to each correction area according to the correction parameter corresponding to the correction area so as to counteract curve distortion generated by the area to be corrected corresponding to the correction area when the area to be corrected corresponding to the correction area is projected onto the imaging medium.

It should be noted that, if the edge of the projection image corresponding to the image to be projected still has the curve distortion after the correction is completed, steps 301 to 307 may be repeatedly executed until the edge of the projection image corresponding to the image to be projected no longer has the curve distortion.

Fig. 10 is a flowchart of one step 301 shown in the embodiment of fig. 7. As shown in fig. 10, step 301 may include the steps of:

step 3011, convert the image obtained by the image acquisition device shooting the projection correction image into a specified format to obtain a first image.

Step 3012, perform segmentation on the first image to remove the area of the first image except the projection-corrected image, and obtain a second image.

Step 3013, convert the second image to a specified image size to obtain a captured image.

Fig. 11 is a flowchart of one step 303 shown in the embodiment of fig. 7. As shown in fig. 11, step 303 may include the steps of:

step 3031, according to each second position information, determining the position relation between each first mark pattern and the second boundary of the correction image, wherein the position relation is used for representing the distance between the first mark pattern and the second boundary.

Step 3032, determining a corresponding first region of the projection correction image in the acquired image according to the position relation and each piece of first position information.

For example, the projection device may calculate a distance between each first marker pattern and the second boundary according to the position information of the second boundary of the corrected image and the second position information of each first marker pattern in the prior information of the corrected image to obtain a position relationship between the first marker pattern and the second boundary. Then, the projection device can determine the position relationship between the second mark pattern corresponding to each first mark pattern and the boundary to be determined by using the proportional relationship between the pixel of the corrected image and the pixel of the acquired image according to the position relationship between each first mark pattern and the second boundary. And then, the projection equipment can determine the corresponding undetermined boundary of the projection correction image in the acquired image by using the position relation between each second mark pattern and the undetermined boundary, outwards expand the undetermined boundary by a specified pixel distance (for example, 20 pixels), and use a region surrounded by the outwards expanded undetermined boundary as a first region. The undetermined boundary can be understood as a pre-assumed first boundary, and the first area can be understood as an approximate range of the projection correction image in the acquired image.

Step 3033, determining a second region from the first region according to the color characteristics of the first region.

Step 3034, performing edge recognition on the second region to obtain a first boundary.

Fig. 12 is a flow chart illustrating one step 304 of the embodiment shown in fig. 7. As shown in fig. 12, step 304 may include the steps of:

step 3041, traverse the first boundary using a preset boundary template to determine a specified boundary of the first boundary.

For example, the projection device may determine a specified one of the first boundaries from the target edge and traverse all of the first boundaries through the boundary template to determine the specified one of the first boundaries from all of the first boundaries. The boundary template may be a template image for identifying the first boundary, for example, when the first boundary is stored in a two-dimensional matrix, the template image may be an image with a pixel size of 5 × 5, and when the first boundary is stored in a one-dimensional single list, the template image may be an image with a pixel size of 1 × 5. The manner of determining the specified boundary may be: and traversing all the boundaries in the first boundary, and determining the specified boundary according to the position of each boundary in the traversal result. Still taking fig. 8 as an example for illustration, the projection device may use the corner point at the upper left corner as an origin, and traverse the first boundary in an order from top to bottom and from left to right, and then the boundaries determined by the projection device in order are boundary 2, boundary 4, boundary 3, and boundary 1, and then the projection device may determine the boundary above in the first boundary as boundary 1, that is, boundary 1 is a specified boundary.

Step 3042, determining a transformation matrix according to each first location information and each second location information.

Step 3043, determining a third boundary of the specified boundary on the imaging media based on the transformation matrix.

Further, the projection device may calculate a transformation matrix (e.g., a homography matrix) between the imaging plane of the image acquisition device and the projection plane on which the imaging medium is located, the transformation matrix being used to convert the imaging plane to the projection plane. For example, in the case of transforming the matrix into a homography, when the main optical axis of the projection light engine of the projection apparatus is parallel to the normal vector of the imaging medium, the projection plane and the projection plane of the projection light engine may be treated as the same plane. Accordingly, the homography matrix between the imaging plane and the projection plane may be calculated using the first position information of each second mark pattern and the second position information of each first mark pattern. The calculation of the homography matrix between the imaging plane and the projection plane according to the first position information and the second position information may refer to the manner described in the related art, and is not described herein again. The projection device may then map the specified boundary on the projection plane using the transformation matrix to obtain a third boundary of the specified boundary on the imaging medium.

Step 3044, determine a correction line according to the third boundary.

In this step, in a case where the distance between the first boundary point and the second boundary point is less than or equal to a preset threshold (the preset threshold may be, for example, a distance of 10 pixels), which indicates that the magnitude of the curve distortion occurring at the target edge is small, the projection apparatus may directly use a straight line passing through the second boundary point and perpendicular to the first direction as a correction line. The first boundary point is an extreme point in the third boundary in the first direction, the second boundary point is an extreme point in the second direction, the boundary points are points forming the third boundary, and the second direction is opposite to the first direction. For example, in the case of fig. 9, the first boundary point is the highest point in the third boundary (i.e., point B in fig. 9), and the second boundary point is the lowest point in the third boundary (i.e., point a in fig. 9).

In addition, when the distance between the first boundary point and the second boundary point is greater than the preset threshold, the magnitude of the curve distortion occurring at the target edge is large, and if a straight line which passes through the second boundary point and is perpendicular to the first direction is still used as a correction line, the proportion of the corrected projection image may be out of order. Therefore, the projection apparatus may use a straight line passing through the target point and perpendicular to the first direction as the correction line. The target point is a point which is a specified distance away from the second boundary point in the first direction, for example, the target point may be a point which is 4 pixels away above the second boundary point, that is, the specified distance is 4 pixels away.

Fig. 13 is a flowchart of one step 305 shown in the embodiment of fig. 7. As shown in fig. 13, step 305 may include the steps of:

step 3051, determining a fourth boundary corresponding to the first boundary on the imaging medium according to the transformation matrix.

Step 3052, dividing the region surrounded by the fourth boundary into a target number of sub-regions along a third direction, and taking each sub-region as a correction region, wherein the third direction is perpendicular to the first direction.

For example, the projection device may map the first boundary on the projection plane using a transformation matrix to obtain a fourth boundary, which may be understood as: the projection equipment corrects the corresponding boundary of the image on the projection plane according to the simulated projection of the acquired image. The projection device may then divide the area surrounded by the fourth boundary into a target number of sub-areas along a third direction, and use each sub-area as a correction area, where the third direction is perpendicular to the first direction.

In summary, the present disclosure first projects a calibration image onto an imaging medium to obtain a projected calibration image displayed on the imaging medium, and obtains a collected image, where the calibration image includes a plurality of first mark patterns, the collected image includes second mark patterns corresponding to each of the first mark patterns, then determines first position information of each of the second mark patterns in the collected image, then determines a first boundary formed in the collected image by the projected calibration image according to the first position information of each of the second mark patterns and the second position information of each of the first mark patterns, and determines a calibration line corresponding to a designated boundary in the first boundary on the imaging medium, then determines a number of calibration regions on the imaging medium according to the first boundary, and determines a calibration parameter corresponding to the calibration region according to each of the calibration regions and the calibration line, and finally, correcting the image to be projected according to the correction parameters, and projecting the corrected image to be projected onto an imaging medium. The method and the device can determine the correction lines and the correction areas according to the first boundary formed by the projection correction image in the collected image, and correct the image to be projected according to each correction area and each correction line so as to ensure that the edge of the displayed image does not have curve distortion when the image to be projected is projected onto an imaging medium, the projection correction efficiency is high, and the projection effect of the projection equipment is improved.

Fig. 14 is a block diagram illustrating a projection correction apparatus according to an exemplary embodiment. As shown in fig. 14, applied to a projection device, the apparatus 400 includes:

the obtaining module 401 is configured to project a preset corrected image onto an imaging medium to obtain a projected corrected image displayed on the imaging medium, and obtain a collected image. The correction image comprises a plurality of first mark patterns, the collected image comprises a second mark pattern corresponding to each first mark pattern, and the collected image is determined according to an image obtained by shooting the projection correction image by the image collecting device.

A processing module 402 for determining first position information for each second marker pattern in the captured image.

The processing module 402 is further configured to determine a first boundary formed by the projection correction image in the captured image according to the first position information of each second marker pattern and the second position information of each first marker pattern.

The processing module 402 is further configured to determine a corresponding correction line of a designated one of the first boundaries on the imaging medium, where the designated one is a boundary in the first direction.

The processing module 402 is further configured to determine a target number of correction zones on the imaging medium based on the first boundary.

The processing module 402 is further configured to determine a correction parameter corresponding to each correction area according to each correction area and each correction line.

And a correcting module 403, configured to correct the image to be projected according to the correction parameter, and project the corrected image to be projected onto an imaging medium.

Optionally, the obtaining module 401 includes:

Optionally, the processing module 402 includes:

and the first determining submodule is used for determining the position relation between each first mark pattern and the second boundary of the correction image according to each second position information, and the position relation is used for representing the distance between the first mark pattern and the second boundary.

And the first determining submodule is also used for determining a corresponding first area of the projection correction image in the acquired image according to the position relation and each piece of first position information.

And the first determining submodule is also used for determining the second area from the first area according to the color characteristic of the first area.

Optionally, the processing module 402 includes:

and the second determining submodule is used for traversing the first boundary by using a preset boundary template so as to determine a specified boundary in the first boundary.

And the third determining submodule is used for determining a transformation matrix according to each piece of first position information and each piece of second position information.

A third determining sub-module for determining a third boundary of the specified boundary on the imaging medium based on the transformation matrix.

And the third determining submodule is also used for determining a correction line according to the third boundary.

Optionally, the third determining sub-module is configured to:

and under the condition that the distance between the first boundary point and the second boundary point is less than or equal to a preset threshold value, taking a straight line which passes through the second boundary point and is vertical to the first direction as a correction line. The first boundary point is an extreme point in a first direction among a plurality of boundary points included in the third boundary, and the second boundary point is an extreme point in a second direction among the plurality of boundary points, the second direction being opposite to the first direction.

And under the condition that the distance between the first boundary point and the second boundary point is greater than a preset threshold value, taking a straight line which passes through the target point and is perpendicular to the first direction as a correction line, wherein the target point is a point which is a specified distance away from the second boundary point in the first direction.

Optionally, the processing module 402 includes:

and the fourth determining submodule is used for determining a fourth boundary corresponding to the first boundary on the imaging medium according to the transformation matrix.

And the dividing sub-module is used for dividing the area surrounded by the fourth boundary into a target number of sub-areas along a third direction, each sub-area is used as a correction area, and the third direction is vertical to the first direction.

Optionally, the processing module 402 is configured to:

and taking the distance between each correction area and the correction line as the corresponding correction parameter of the correction area.

In another scenario, when the projection device distorts the inside of the projected image projected on the imaging medium due to the uneven surface of the imaging medium, the projected image may be corrected in the manner shown in the following embodiments:

FIG. 15 is a flow chart illustrating a method of projection correction according to an exemplary embodiment. As shown in fig. 15, applied to a projection apparatus, the method may include the steps of:

step 501, projecting a preset correction image onto an imaging medium to obtain a projection correction image displayed on the imaging medium, and acquiring a collected image.

For example, when a projection device projects an image onto an imaging medium, if a user finds that a projection image corresponding to the image is distorted (for example, distortion, translation, deformation, and the like occur inside the projection image), a correction instruction may be sent to the projection device to correct the projection image. For example, a corresponding correction button may be provided on the projection device, and the user may press the correction button to send a correction instruction to the projection device. Then the projection equipment can send corresponding prompt information to the user to prompt the user to adjust the position of the image acquisition device to a proper position, so that the image acquisition device can accurately shoot a projection image of the projection equipment on an imaging medium, and the projection image can be corrected more accurately.

When the image acquisition device is a mobile phone with a shooting function, the projection device can project a prompt image on the imaging medium after receiving the correction instruction so as to prompt a user to use the mobile phone to pair the mobile phone with the projection device. After the pairing is successful, the projection equipment can project the corrected image onto the imaging medium to obtain a projection corrected image, then prompt the user to align the photographing interface of the mobile phone with the projection corrected image, and prompt the user to photograph the projection corrected image by using the mobile phone. And then the projection equipment can acquire an image obtained by shooting the projection correction image by the image acquisition device and preprocess the image obtained by shooting the projection correction image to obtain an acquired image. Further, in order to reduce the amount of data required to be processed by the projection device, the mobile phone may be further configured to pre-process an image obtained by capturing the projection correction image to obtain a captured image, and send the captured image to the projection device.

The correction image may be any image including a plurality of first marker patterns, and the position of each first marker pattern may be used as a reference for correcting the projection image. When the corrected image is a checkerboard image, as shown in a of fig. 2, the first mark pattern may be any one of the checkerboards, and the projection device projects the corrected image onto the imaging medium, actually, the checkerboard is projected onto the imaging medium. At this time, the image obtained by the image capturing device capturing the projection correction image also includes a checkerboard, and then the captured image obtained by preprocessing the image obtained by capturing the projection correction image also includes a checkerboard (i.e., the captured image is also a checkerboard image), and the second mark pattern may be any one of the checkerboards.

First position information for each second marker pattern in the captured image is determined, step 502.

Step 503, determining a corresponding first projection area of the projection correction image in the acquired image according to the first position information of each second mark pattern and the second position information of each first mark pattern.

For example, the projection device may store a priori information of the corrected image in advance, and the a priori information may include position information of a boundary of the corrected image and second position information of each first marker pattern in the corrected image. For example, when the correction image is a checkerboard image, the coordinates of four corner points of each square (i.e., the first mark pattern) included in the checkerboard in the correction image may be taken as the second position information of the square. The projection device may then determine a positional relationship of each first mark pattern with a boundary of the correction image based on the second positional information of each first mark pattern. Since the captured image is determined based on the image obtained by the image capturing device capturing the projection correction image projected from the correction image. Therefore, the positional relationship of each second marker pattern with the boundary formed in the captured image by projecting the correction image is the same as the positional relationship of each first marker pattern with the boundary of the correction image. The projection device may determine a boundary formed in the captured image by projecting the correction image based on a positional relationship of each of the first mark patterns with the boundary of the correction image, and set an area surrounded by the boundary as the first projection area.

At step 504, a corresponding second projection area of the first projection area on the imaging medium is determined, the second projection area including a third marker pattern corresponding to each of the first marker patterns.

For example, after determining the first projection region, the projection apparatus may first calculate a homography matrix between an imaging plane of the image acquisition device and a projection plane on which the imaging medium is located. For example, when the main optical axis of the projection light engine of the projection device is parallel to the normal vector of the imaging medium, the projection plane and the projection plane of the projection light engine can be regarded as the same plane. Accordingly, the homography matrix between the imaging plane and the projection plane may be calculated using the first position information of each second mark pattern and the second position information of each first mark pattern. The calculation of the homography matrix between the imaging plane and the projection plane according to the first position information and the second position information may refer to the manner described in the related art, and is not described herein again. The projection device may then map the first projection area on the projection plane via the homography matrix to obtain a second projection area. Wherein the second projection area includes a third mark pattern corresponding to each of the first mark patterns. The second projection area may be understood as: the projection device corrects a corresponding area of the image on the projection plane according to the simulated projection of the acquired image, and the third mark pattern can be understood as: the projection device simulates a pattern corresponding to the first mark pattern on the projection plane according to the acquired image, that is, the third mark pattern corresponding to each first mark pattern is: and when the first mark pattern is projected onto a projection plane, the first mark pattern forms a pattern on the projection plane.

Step 505, according to the second position information, determining third position information of each third mark pattern and an offset parameter corresponding to the third mark pattern.

Further, in the case that the projection plane is the same plane as the projection plane of the projection light engine, the second projection area is obtained by mapping the first projection area on the imaging plane onto the projection plane, and therefore the second projection area and the collected image have the same pixel size. For example, when the pixel size of the captured image is 2K, the pixel size of the second projection region is also 2K. However, in an actual case, the pixel size of the projection correction image may be different from the second projection region (the pixel size of the projection correction image is the same as the pixel size of the correction image), and in order to accurately correct the projection image, it is necessary to convert the pixel size of the correction image to the same pixel size as the second projection region and compare the converted correction image with the second projection region to determine the third position information of each third mark pattern in the second projection region and the offset parameter corresponding to each third mark pattern.

For example, the projection device may determine a proportional relationship between the correction image and the captured image according to the pixel size of the correction image and the pixel size of the captured image, and determine an ideal position of a third marker pattern corresponding to each first marker pattern in the second projection region when the projection correction image is not distorted according to the proportional relationship in combination with the second position information of the first marker pattern. And determining third position information of each third mark pattern and an offset parameter corresponding to each third mark pattern according to the ideal position of the third mark pattern corresponding to the first mark pattern. And the offset parameter corresponding to each third mark pattern is used for representing the deviation between the third position information of the third mark pattern and the ideal position.

Step 506, under the condition that the offset parameter does not meet the preset condition, determining a target correction matrix corresponding to each first mark pattern according to the first position information and the third position information.

For example, after determining the offset parameter corresponding to each third mark pattern, the projection apparatus may first determine whether the offset parameter satisfies a preset condition. For example, when the offset parameter is a pixel distance, the preset condition may be: the average value of the pixel distances (i.e., the offset parameters) corresponding to all the third mark patterns is smaller than the average threshold (the average threshold may be 3 pixels, for example), and the largest pixel distance among the pixel distances corresponding to all the third mark patterns is smaller than the distance threshold (the distance threshold may be 5 pixels, for example).

The projection device may determine, for each first mark pattern, an initial correction matrix corresponding to the first mark pattern according to the first position information of the first mark pattern and the third position information of the third mark pattern corresponding to the first mark pattern, when the offset parameter does not satisfy the preset condition. The initial correction matrix may be a homography matrix, for example, in a case where the first mark pattern and the third mark pattern are any squares included in a checkerboard, the second position information of each first mark pattern is coordinates of four corner points of the first mark pattern (i.e., the squares), and the third position information of each third mark pattern is coordinates of four corner points of the third mark pattern (i.e., the squares), the initial correction matrix of the first mark pattern may be calculated from the coordinates of the four corner points of each first mark pattern and the four corner points of the third mark pattern corresponding to the first mark pattern. The projection device may then correct the corrected image based on the initial correction matrix, i.e., adjust each of the first mark patterns based on the initial correction matrix corresponding to the first mark pattern to reduce distortion of the first mark pattern when the first mark pattern is projected onto the imaging medium. Then, the projection device may repeatedly perform the step of projecting the preset correction image onto the imaging medium according to the corrected correction image to obtain the projection correction image displayed on the imaging medium, and perform the step of correcting the correction image according to the initial correction matrix until the offset parameter meets the preset condition. And finally, taking the initial correction matrix as a target correction matrix.

And 507, correcting the image to be projected according to the target correction matrix, and projecting the corrected image to be projected onto an imaging medium.

In this step, the projection device may determine a first target region corresponding to each first marker pattern in the corrected image, and use a region in the image to be projected that is located at the same position as the first target region corresponding to the first marker pattern as a second target region corresponding to the first marker pattern in the image to be projected. And then, the projection equipment can correct the second target area corresponding to the first mark pattern in the image to be projected according to the target correction matrix corresponding to each first mark pattern so as to counteract distortion of the second target area corresponding to the first mark pattern when the second target area corresponding to the first mark pattern is projected onto the imaging medium, so that the projection image corresponding to the image to be projected is not distorted any more when the projection equipment projects the image to be projected onto the imaging medium.

Fig. 16 is a flow chart illustrating one step 501 of the embodiment shown in fig. 15. As shown in fig. 16, step 501 may include the steps of:

in step 5011, an image obtained by capturing a projection correction image by the image capturing device is converted into a specified format to obtain a first image.

Step 5012, performing segmentation processing on the first image to remove the area except the projection correction image in the first image, and obtaining a second image.

Step 5013, convert the second image to a designated image size to obtain a captured image.

Fig. 17 is a flowchart illustrating one step 503 of the embodiment shown in fig. 15. As shown in fig. 17, step 503 may include the steps of:

step 5031, determining the position relationship between each first mark pattern and the first boundary of the corrected image according to each second position information, wherein the position relationship is used for representing the distance between the first mark pattern and the first boundary.

Step 5032, determining a first region of the projection correction image corresponding to the acquired image according to the position relationship and each first position information.

For example, the projection device may calculate a distance between each first mark pattern and the first boundary according to the position information of the first boundary of the corrected image and the second position information of each first mark pattern in the prior information of the corrected image to obtain a position relationship between the first mark pattern and the first boundary. Then, the projection device can determine the position relationship between the second mark pattern corresponding to each first mark pattern and the boundary to be determined by using the proportional relationship between the pixel size of the corrected image and the pixel size of the acquired image according to the position relationship between each first mark pattern and the first boundary. And then the projection equipment can determine the corresponding undetermined boundary of the projection correction image in the acquired image by utilizing the position relation between each second mark pattern and the undetermined boundary. And expanding the undetermined boundary outwards by a specified pixel distance (for example, 20 pixels), and taking a region surrounded by the undetermined boundary after the outward expansion as a first region. The undetermined boundary can be understood as a boundary formed by a presupposed projection correction image in the acquired image, and the first area can be understood as an approximate range of the projection correction image in the acquired image.

Step 5033, determining a second region from the first region according to the color feature of the first region.

Step 5034, performing edge recognition on the second region to obtain a second boundary, and using a region surrounded by the second boundary as the first projection region.

Further, the projection device may extract color features of the first region, and search, according to the color features of the first region, a region that is the same as the color feature difference from the first region as the second region in combination with the color feature difference between the first boundary and the surrounding region in the prior information of the corrected image. Then, the projection device may perform edge recognition on the second area to recognize a boundary corresponding to the second area, and then use the boundary as the second boundary, and use an area surrounded by the second boundary as the first projection area. The method for performing edge recognition on the second region to obtain the second boundary may refer to the method described in the related art, and is not described herein again. It should be noted that the process of obtaining the first projection area is actually a fine-grained processing manner to further remove other areas in the acquired image except the projection correction image.

Fig. 18 is a flowchart illustrating one step 504 of the embodiment shown in fig. 15. As shown in fig. 5, step 504 may include the steps of:

step 5041, a transformation matrix is determined based on each first location information and each second location information.

At step 5042, a second projection area is determined based on the transformation matrix.

For example, after determining the first projection area, the projection apparatus may first calculate a transformation matrix between the imaging plane and the projection plane according to the first position information of each second marker pattern and the second position information of each first marker pattern. Where a transformation matrix is used to convert the imaging plane to a projection plane, the transformation matrix may be, for example, a homography matrix. The projection device may then map the first projection area on the projection plane according to the transformation matrix to obtain a second projection area.

Fig. 19 is a flowchart illustrating a step 505 in the embodiment of fig. 15. As shown in fig. 19, the first mark pattern is any one of first squares included in the first checkerboard, the third mark pattern is any one of second squares included in the second checkerboard, and the second position information includes coordinates of each first corner point of the first squares. Step 505 may include the steps of:

in step 5051, the proportional relationship between the corrected image and the captured image is determined based on the pixel size of the corrected image and the pixel size of the captured image.

For example, in the case where the corrected image is a checkerboard image, the acquired image obtained by the projection device and the second projection area are actually checkerboard images, and the checkerboard in the corrected image may be set as the first checkerboard and the checkerboard in the second projection area may be set as the second checkerboard. At this time, the first mark pattern is any first square included in the first square, the first square includes 4 first corner points, the second position information of the first mark pattern may include coordinates of each first corner point of the first square, and the third mark pattern is any second square included in the second square, the second square includes 4 second corner points. After determining the second position information, the projection device may determine a proportional relationship between the corrected image and the captured image according to a pixel size of the corrected image and a pixel size of the captured image. For example, when the pixel size of the corrected image is 1080P and the pixel size of the captured image is 2K, the proportional relationship is 2.

Step 5052, for each first square, determining coordinates of an ideal corner point of each first corner point of the first square in the second projection area according to the coordinates of each first corner point of the first square by using a proportional relationship.

Step 5053, determining whether the ideal corner point is a second corner point of a target second grid according to the coordinates of the ideal corner point, wherein the target second grid is a second grid corresponding to the first grid in the second projection area.

In step 5054, if the ideal corner point is the second corner point of the target second grid, the coordinates of the ideal corner point are used as the third position information of the target second grid.

In step 5055, if the ideal corner point is not the second corner point of the target second grid, the coordinates of the neighboring corner point closest to the ideal corner point are used as the third position information of the target second grid, and the pixel distance between the neighboring corner point and the ideal corner point is used as the offset parameter of the target second grid.

Specifically, for each first square, the projection device may determine, according to the coordinates of each first corner point of the first square, the coordinates of an ideal corner point of each first corner point of the first square in the second projection area by using a proportional relationship. The ideal corner of each first corner is the corresponding coordinate of the first corner in the second projection area under the ideal condition (i.e. when the projection correction image is not distorted). For example, when the coordinates of the first corner point is (1, 1) and the scale relationship is 2, the coordinates of the ideal corner point may be (2, 2). Then, the projection device may determine, according to the coordinates of the ideal corner of each first corner of the first square grid, whether the ideal corner of the first corner is a second corner of a target second square grid, where the target second square grid is a second square grid corresponding to the first square grid in the second projection area.

If the ideal corner point of the first corner point is the second corner point of the target second square, it is indicated that the first corner point is not distorted when being projected onto the imaging medium, and the ideal corner point of the first corner point is the second corner point of the target second square, then the projection device may use the coordinates of the ideal corner point of the first corner point as the third position information of the target second square. If the ideal corner of the first corner is not the second corner of the target second square, it indicates that the first corner is distorted when being projected onto the imaging medium, and then the projection apparatus may use a neighboring corner closest to the ideal corner as the second corner of the target second square, and use a coordinate of the neighboring corner as the third position information of the target second square. Meanwhile, the projection device may use a pixel distance between the adjacent corner point and the ideal corner point as an offset parameter of the target second square. It should be noted that the third position information of each second square includes coordinates of 4 second corner points, and the coordinates of the 4 second corner points may include only coordinates of an ideal corner point (that is, none of the 4 second corner points is distorted), may also include only coordinates of a neighboring corner point (that is, all of the 4 second corner points are distorted), and may also include both the coordinates of the ideal corner point and the coordinates of the neighboring corner points (that is, some of the 4 second corner points may be distorted).

To sum up, the present disclosure first projects a preset correction image onto an imaging medium to obtain a projection correction image displayed on the imaging medium, and obtains an acquired image, then determines first position information of each second mark pattern in the acquired image, and determines a first projection area corresponding to the projection correction image in the acquired image according to the first position information of each second mark pattern and the second position information of each first mark pattern, and then determines a second projection area corresponding to the first projection area on the imaging medium, and determines third position information of each third mark pattern and an offset parameter corresponding to the third mark pattern according to the second position information and the second projection area, and then determines a target correction matrix corresponding to each first mark pattern according to the first position information and the third position information under the condition that the offset parameter does not satisfy a preset condition, and finally, correcting the image to be projected according to the target correction matrix, and projecting the corrected image to be projected onto an imaging medium. According to the projection image correction method, the projection image to be projected is corrected through the determined target correction matrix corresponding to each first mark pattern, the corrected projection image to be projected is projected onto the imaging medium, distortion cannot occur when the projection image to be projected is projected onto the imaging medium, the projection correction efficiency is high, and the projection effect of the projection equipment is improved.

Fig. 20 is a block diagram illustrating a projection correction apparatus according to an exemplary embodiment. As shown in fig. 20, applied to a projection device, the apparatus 600 includes:

the obtaining module 601 is configured to project a preset correction image onto an imaging medium to obtain a projection correction image displayed on the imaging medium, and obtain a collected image. The correction image comprises a plurality of first mark patterns, the collected image comprises a second mark pattern corresponding to each first mark pattern, and the collected image is determined according to an image obtained by shooting the projection correction image by the image collecting device.

A processing module 602 for determining first position information for each second marker pattern in the captured image.

The processing module 602 is further configured to determine a corresponding first projection area of the projection correction image in the acquired image according to the first position information of each second marker pattern and the second position information of each first marker pattern.

The processing module 602 is further configured to determine a second projection area corresponding to the first projection area on the imaging medium, where the second projection area includes a third mark pattern corresponding to each first mark pattern.

The processing module 602 is further configured to determine third position information of each third mark pattern and an offset parameter corresponding to the third mark pattern according to the second position information.

The processing module 602 is further configured to determine, according to the first position information and the third position information, a target correction matrix corresponding to each first mark pattern when the offset parameter does not satisfy the preset condition.

And the correcting module 603 is configured to correct the image to be projected according to the target correction matrix, and project the corrected image to be projected onto an imaging medium.

Optionally, the obtaining module 601 includes:

Optionally. The processing module 602 includes:

Optionally, the processing module 602 includes:

and the second determining submodule is used for determining the position relation between each first mark pattern and the first boundary of the correction image according to each second position information, and the position relation is used for representing the distance between the first mark pattern and the first boundary.

And the second determining submodule is used for determining a corresponding first area of the projection correction image in the acquired image according to the position relation and each piece of first position information.

And the second determining submodule is used for determining the second area from the first area according to the color characteristic of the first area.

And the identification submodule is used for carrying out edge identification on the second area to obtain a second boundary, and the area surrounded by the second boundary is used as a first projection area.

Optionally, the processing module 602 includes:

and the calculation submodule is used for determining a transformation matrix according to each piece of first position information and each piece of second position information.

And the third determining submodule is used for determining a second projection area according to the transformation matrix.

Optionally, the first mark pattern is any first square included in the first square, the third mark pattern is any second square included in the second square, and the second position information includes coordinates of each first corner point of the first square, and the processing module 602 is configured to:

and determining the proportional relation between the correction image and the collected image according to the pixel size of the correction image and the pixel size of the collected image.

And for each first square, determining the coordinates of the ideal corner points of each first corner point of the first square in the second projection area by utilizing a proportional relation according to the coordinates of each first corner point of the first square.

And determining whether the ideal corner point is a second corner point of a target second square grid according to the coordinates of the ideal corner point, wherein the target second square grid is a second square grid corresponding to the first square grid in the second projection area.

And if the ideal corner point is the second corner point of the target second square grid, taking the coordinates of the ideal corner point as third position information of the target second square grid.

If the ideal corner point is not the second corner point of the target second square, the coordinate of the adjacent corner point closest to the ideal corner point is used as the third position information of the target second square, and the pixel distance between the adjacent corner point and the ideal corner point is used as the offset parameter of the target second square.

Optionally, the processing module 602 is configured to:

and for each first mark pattern, determining an initial correction matrix corresponding to the first mark pattern according to the first position information of the first mark pattern and the third position information of a third mark pattern corresponding to the first mark pattern.

And correcting the corrected image according to the initial correction matrix, repeatedly projecting a preset corrected image onto the imaging medium according to the corrected image to obtain a projected corrected image displayed on the imaging medium, and correcting the corrected image according to the initial correction matrix until the offset parameter meets a preset condition.

And taking the initial correction matrix as a target correction matrix.

Fig. 21 is a block diagram illustrating an electronic device 700 in accordance with an example embodiment. As shown in fig. 21, the electronic device 700 may include: a processor 701 and a memory 702. The electronic device 700 may also include one or more of a multimedia component 703, an input/output (I/O) interface 704, and a communication component 705.

The processor 701 is configured to control the overall operation of the electronic device 700, so as to complete all or part of the steps in the projection correction method. The memory 702 is used to store various types of data to support operation at the electronic device 700, such as instructions for any application or method operating on the electronic device 700 and application-related data, such as contact data, transmitted and received messages, pictures, audio, video, and the like. The Memory 702 may be implemented by any type of volatile or non-volatile Memory device or combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read-Only Memory (EPROM), Programmable Read-Only Memory (PROM), Read-Only Memory (ROM), magnetic Memory, flash Memory, magnetic disk, or optical disk. The multimedia components 703 may include screen and audio components. Wherein the screen may be, for example, a touch screen and the audio component is used for outputting and/or inputting audio signals. For example, the audio component may include a microphone for receiving external audio signals. The received audio signal may further be stored in the memory 702 or transmitted through the communication component 705. The audio assembly also includes at least one speaker for outputting audio signals. The I/O interface 704 provides an interface between the processor 701 and other interface modules, such as a keyboard, mouse, buttons, etc. These buttons may be virtual buttons or physical buttons. The communication component 705 is used for wired or wireless communication between the electronic device 700 and other devices. Wireless Communication, such as Wi-Fi, bluetooth, Near Field Communication (NFC), 2G, 3G, 4G, NB-IOT, eMTC, or other 5G, etc., or a combination of one or more of them, which is not limited herein. The corresponding communication component 705 may thus include: Wi-Fi module, Bluetooth module, NFC module, etc.

In an exemplary embodiment, the electronic Device 700 may be implemented by one or more Application Specific Integrated Circuits (ASICs), Digital Signal Processors (DSPs), Digital Signal Processing Devices (DSPDs), Programmable Logic Devices (PLDs), Field Programmable Gate Arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components for performing the above-described projection correction method.

In another exemplary embodiment, a computer-readable storage medium is also provided, which comprises program instructions, which when executed by a processor, implement the steps of the projection correction method described above. For example, the computer readable storage medium may be the memory 702 described above including program instructions that are executable by the processor 701 of the electronic device 700 to perform the projection correction method described above.

In another exemplary embodiment, a computer program product is also provided, which comprises a computer program executable by a programmable apparatus, the computer program having code portions for performing the projection correction method described above when executed by the programmable apparatus.

The preferred embodiments of the present disclosure are described in detail with reference to the accompanying drawings, however, the present disclosure is not limited to the specific details of the above embodiments, and various simple modifications may be made to the technical solution of the present disclosure within the technical idea of the present disclosure, and these simple modifications all belong to the protection scope of the present disclosure.

It should be noted that, in the foregoing embodiments, various features described in the above embodiments may be combined in any suitable manner, and in order to avoid unnecessary repetition, various combinations that are possible in the present disclosure are not described again.

In addition, any combination of various embodiments of the present disclosure may be made, and the same should be considered as the disclosure of the present disclosure, as long as it does not depart from the spirit of the present disclosure.

Claims

1. A projection correction method is applied to a projection device, and the method comprises the following steps:

correcting a to-be-projected image according to the target correction matrix, and projecting the corrected to-be-projected image onto the imaging medium;

the corrected image and the second projection area are both checkerboard images, the first marker pattern is any first square included in a first checkerboard, the first checkerboard is a checkerboard in the corrected image, the third marker pattern is any second square included in a second checkerboard, the second checkerboard is a checkerboard in the second projection area, and the second position information includes coordinates of each first corner point of the first square; determining third position information of each third mark pattern and an offset parameter corresponding to the third mark pattern according to the second position information, including:

determining a proportional relation between the correction image and the collected image according to the pixel size of the correction image and the pixel size of the collected image;

for each first square grid, determining the coordinates of ideal corner points of each first corner point of the first square grid in the second projection area according to the coordinates of each first corner point of the first square grid and by utilizing the proportional relation;

determining whether the ideal corner point is a second corner point of a target second square grid according to the coordinates of the ideal corner point, wherein the target second square grid is a second square grid corresponding to the first square grid in the second projection area;

if the ideal corner point is a second corner point of the target second square grid, taking the coordinate of the ideal corner point as third position information of the target second square grid;

if the ideal corner point is not the second corner point of the target second square, using the coordinate of the adjacent corner point closest to the ideal corner point as the third position information of the target second square, and using the pixel distance between the adjacent corner point and the ideal corner point as the offset parameter of the target second square.

2. The method of claim 1, wherein the acquiring an acquired image comprises:

converting an image obtained by shooting the projection correction image by the image acquisition device into a specified format to obtain a first image;

performing segmentation processing on the first image to remove the area except the projection correction image in the first image to obtain a second image;

and converting the second image into a specified image size to obtain the acquired image.

3. The method of claim 1, wherein said determining first position information for each of said second marker patterns in said captured image comprises:

detecting the checkerboards in the acquired image, and determining the position information of each square included in the checkerboards in the acquired image;

and taking the position information of each square included in the checkerboards in the acquired image as the first position information.

4. The method of claim 1, wherein determining a corresponding first projection region of the projection corrected image in the acquired image based on the first position information of each of the second marker patterns and the second position information of each of the first marker patterns comprises:

determining the position relation between each first mark pattern and a first boundary of the correction image according to each second position information, wherein the position relation is used for representing the distance between the first mark pattern and the first boundary;

determining a first region corresponding to the projection correction image in the acquired image according to the position relation and each piece of first position information;

determining a second area from the first area according to the color characteristics of the first area;

and performing edge identification on the second area to obtain a second boundary, and taking an area surrounded by the second boundary as the first projection area.

5. The method of claim 1, wherein determining a corresponding second projection area of the first projection area on the imaging media comprises:

determining a transformation matrix according to each first position information and each second position information;

and determining the second projection area according to the transformation matrix.

6. The method of claim 1, wherein determining the target correction matrix for each of the first marker patterns based on the first position information and the third position information comprises:

for each first mark pattern, determining an initial correction matrix corresponding to the first mark pattern according to first position information of the first mark pattern and third position information of a third mark pattern corresponding to the first mark pattern;

correcting the corrected image according to the initial correction matrix, repeatedly projecting a preset corrected image onto an imaging medium according to the corrected image to obtain a projected corrected image displayed on the imaging medium, and correcting the corrected image according to the initial correction matrix until the offset parameter meets the preset condition;

and taking the initial correction matrix as the target correction matrix.

7. A projection correction device, applied to a projection apparatus, the device comprising:

the correction module is used for correcting the image to be projected according to the target correction matrix and projecting the corrected image to be projected onto the imaging medium;

the corrected image and the second projection area are both checkerboard images, the first marker pattern is any first square included in a first checkerboard, the first checkerboard is a checkerboard in the corrected image, the third marker pattern is any second square included in a second checkerboard, the second checkerboard is a checkerboard in the second projection area, and the second position information includes coordinates of each first corner point of the first square; the processing module is used for:

8. A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the steps of the method according to any one of claims 1 to 6.

9. An electronic device, comprising:

a memory having a computer program stored thereon;

a processor for executing the computer program in the memory to carry out the steps of the method of any one of claims 1 to 6.