CN110769218A - Image processing method, projection apparatus, and photographing apparatus - Google Patents
Image processing method, projection apparatus, and photographing apparatus Download PDFInfo
- Publication number
- CN110769218A CN110769218A CN201811177657.XA CN201811177657A CN110769218A CN 110769218 A CN110769218 A CN 110769218A CN 201811177657 A CN201811177657 A CN 201811177657A CN 110769218 A CN110769218 A CN 110769218A
- Authority
- CN
- China
- Prior art keywords
- image
- projection
- original image
- plane
- projected
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N9/00—Details of colour television systems
- H04N9/12—Picture reproducers
- H04N9/31—Projection devices for colour picture display, e.g. using electronic spatial light modulators [ESLM]
- H04N9/3179—Video signal processing therefor
- H04N9/3188—Scale or resolution adjustment
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/80—Camera processing pipelines; Components thereof
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N9/00—Details of colour television systems
- H04N9/12—Picture reproducers
- H04N9/31—Projection devices for colour picture display, e.g. using electronic spatial light modulators [ESLM]
- H04N9/3141—Constructional details thereof
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Projection Apparatus (AREA)
- Controls And Circuits For Display Device (AREA)
Abstract
The invention relates to the technical field of image processing, in particular to an image processing method, projection equipment and shooting equipment. The image processing method comprises the following steps: step 1, projecting an original image on a projection surface according to a preset scaling by a projection module to generate a projection image; step 2, displaying the shooting range and a rectangular prompt box to a user by a display module; step 3, when the receiving module receives a shooting instruction, the shooting module shoots the projection image to generate a shot image; step 4, determining an image adjusting method by a calculating module according to the original image, the shot image, the scaling, the rectangular prompt box and the aspect ratio of the image shape expected to be projected; step 5, adjusting the image to be projected by the computing module according to the image adjusting method determined in the step 4; wherein the photographing module is freely movable relative to the projection module. The invention can realize high-precision trapezoidal correction.
Description
Technical Field
The invention relates to the technical field of image processing, in particular to an image processing method, projection equipment and shooting equipment.
Background
When the projection device is used for projecting images, the projection effect can be ensured only if the position of the projection device is perpendicular to the projection surface as much as possible, otherwise, the projected rectangular images can present trapezoidal pictures on the projection surface. In this case, trapezoidal correction is required.
There are generally two methods for keystone correction: optical keystone correction and digital keystone correction. The optical trapezoid correction is to adjust the trapezoid by adjusting the physical position of the lens, but when the physical position is limited, the method is difficult; digital keystone correction refers to adjustment by software. At present, almost all projector manufacturers adopt a digital trapezoidal correction technology, but the precision of the existing digital trapezoidal correction technology is low.
Therefore, a highly accurate digital keystone correction method is required.
Disclosure of Invention
In view of the above, the present application provides an image processing method, a projection apparatus, and a photographing apparatus capable of accurately performing trapezoidal correction.
In one aspect, the present invention provides an image processing method, performed by a projection device, the method comprising:
step A1, projecting the original image on the projection surface according to the preset scaling to generate a projection image;
a step a2 of receiving a captured image obtained by capturing the projection image on the projection surface by a capturing device that is connected to the projection device in communication and is freely movable relative to the projection device;
step A3, receiving a rectangular prompt box with a preset size and a preset position relative to the shot image;
step A4, determining an image adjusting method according to the length-width ratio of the original image, the shot image, the scaling, the rectangular prompt box and the final image expected to be projected;
step A5, adjusting the image to be projected according to the image adjusting method determined in the step A4.
Preferably, the scaling ratio is smaller than 1, and the aspect ratio of the final image desired to be projected is the aspect ratio of the rectangular prompt box.
Preferably, the step a4 includes:
step A4-1, acquiring a projected image area in the shot image by performing image recognition on the shot image;
step A4-2, calculating an original image area in a shot image plane according to the projected image area and the preset scaling;
step A4-3, calculating a homography matrix T from the original image plane to the shot image plane;
step A4-4, calculating a homography relation matrix C from the projection plane to the shot image plane;
step A4-5, calculating the homography transformation relation P ^ C (-1) × T from the original image plane to the projection plane;
step A4-6, calculating the maximum inscribed rectangle with the length-width ratio in the intersection area of the rectangular prompt box and the original image area;
step A4-7, calculating the change S from the original image to the maximum inscribed rectangle;
step A4-8, calculating a pre-transformation matrix H ═ P ^ (-1) × S;
step A4-9, determining an image adjustment method according to the pre-transformation matrix H.
Preferably, the step a1 further includes: projecting the contour of the original image on the projection surface without scaling.
In another aspect, the present invention provides a projection apparatus, comprising a projection module, a calculation module, and a receiving module, the receiving module being communicatively connected to a photographing apparatus that is freely movable relative to the projection module, the projection apparatus performing the following image processing methods:
step B1, the projection module generates a projection image by projecting the original image on the projection surface according to a preset scaling;
a step B2 of receiving a captured image obtained by capturing the projection image on the projection surface by the capturing device;
step B3, the receiving module receives a rectangular prompt box with preset size and preset position relative to the shot image;
step B4, the calculation module determines an image adjustment method according to the original image, the shot image, the scaling, the rectangular prompt box and the aspect ratio of the final image expected to be projected;
step B5, the calculation module adjusts the image to be projected according to the image adjustment method determined in the step B4.
Preferably, the scaling ratio is smaller than 1, and the aspect ratio of the final image desired to be projected is the aspect ratio of the rectangular prompt box.
Preferably, the step B4 includes:
step B4-1, the calculation module acquires a projection image area in the shot image by performing image recognition on the shot image;
step B4-2, the calculating module calculates an original image area in a shot image plane according to the projected image area and the preset scaling;
step B4-3, the calculation module calculates a homography matrix T from the original image plane to the shot image plane;
step B4-4, the calculation module calculates a homography matrix C from the projection plane to the shot image plane;
step B4-5, the calculation module calculates homography transformation relation P ═ C (-1) × T from the original image plane to the projection plane;
step B4-6, the calculation module calculates the maximum inscribed rectangle with the aspect ratio in the intersection area of the rectangular prompt box and the original image area;
step B4-7, the calculation module calculates the change S of the original image to the maximum inscribed rectangle;
step B4-8, the calculation module calculates a pre-transformation matrix H ═ P ^ (-1) × S;
step B4-9, the calculation module determines an image adjustment method according to the pre-transformation matrix H.
Preferably, the step B1 further includes: projecting the contour of the original image on the projection surface without scaling.
In another aspect, the present invention also provides an image processing method performed by a photographing apparatus, the method including:
step C1, displaying a shooting range and a rectangular prompt box with a preset size and a preset position relative to the shooting range to a user;
a step C2 of acquiring a shot image by shooting a projection image in the shooting range when a shooting instruction is received, wherein the projection image is an image generated by projecting an original image onto a projection surface at a preset zoom ratio by a projection device which is in communication connection with the shooting device and relative to which the shooting device can freely move;
a step C3 of receiving the original image and the scaling from the projection device;
step C4, determining an image adjusting method according to the original image, the shot image, the scaling, the rectangular prompt box and the aspect ratio of the final image expected to be projected;
and step C5, adjusting the image to be projected according to the image adjusting method determined in the step C4.
Preferably, the scaling ratio is smaller than 1, and the aspect ratio of the final image desired to be projected is the aspect ratio of the rectangular prompt box.
Preferably, the step C4 includes:
step C4-1, acquiring a projected image area in the shot image by performing image recognition on the shot image;
step C4-2, calculating an original image area in the shot image plane according to the projected image area and the preset scaling;
step C4-3, calculating a homography matrix T from the original image plane to the shot image plane;
step C4-4, calculating a homography relation matrix C from the projection plane to the shot image plane;
step C4-5, calculating the homography transformation relation P ^ C (-1) × T from the original image plane to the projection plane;
step C4-6, calculating the maximum inscribed rectangle with the length-width ratio in the intersection area of the rectangular prompt box and the original image area;
step C4-7, calculating the change S from the original image to the maximum inscribed rectangle;
step C4-8, calculating a pre-transformation matrix H ═ P ^ (-1) × S;
and step C4-9, determining an image adjusting method according to the pre-transformation matrix H.
In another aspect, the present invention further provides a shooting device, including a display module, a shooting module, a computing module, and a receiving module, where the receiving module is in communication connection with a projection device that is capable of moving freely relative to the shooting device, and the shooting device executes the following image processing methods:
step D1, the display module displays the shooting range and a rectangular prompt box with preset size and position relative to the shooting range to the user;
step D2, when the receiving module receives the shooting instruction, the shooting module obtains the shooting image by shooting the projection image in the shooting range, wherein the projection image is the image generated by the projection device projecting the original image to the projection surface according to the preset scaling;
a step D3, in which the receiving module receives the original image and the scaling from the projection device;
step D4, the calculation module determines an image adjustment method according to the original image, the shot image, the scaling, the rectangular prompt box and the aspect ratio of the final image expected to be projected;
and D5, adjusting the image to be projected according to the image adjusting method determined in the step D4 by the computing module.
Preferably, the scaling ratio is smaller than 1, and the aspect ratio of the final image desired to be projected is the aspect ratio of the rectangular prompt box.
Preferably, the step D4 includes:
step D4-1, the calculation module acquires a projection image area in the shot image by performing image recognition on the shot image;
step D4-2, the calculation module calculates the original image area in the shooting image plane according to the projection image area and the preset scaling;
d4-3, calculating a homography matrix T from the original image plane to the shot image plane by the calculating module;
d4-4, calculating a homography matrix C from the projection plane to the shot image plane by the calculation module;
step D4-5, the calculation module calculates homography transformation relation P ═ C (-1) × T from the original image plane to the projection plane;
step D4-6, the calculation module calculates the maximum inscribed rectangle with the aspect ratio in the intersection area of the rectangular prompt box and the original image area;
step D4-7, the calculation module calculates the change S of the original image to the maximum inscribed rectangle;
step D4-8, the calculation module calculates a pre-transformation matrix H ═ P ^ (-1) × S;
step D4-9, the calculation module determines an image adjustment method according to the pre-transformation matrix H.
Preferably, the photographing apparatus is a mobile phone having a photographing function.
Compared with the prior art, the method and the device can more accurately realize trapezoidal correction.
Drawings
FIG. 1 is a schematic diagram of one embodiment of a projection device of the present invention;
FIG. 2 is a schematic diagram of one embodiment of an image processing method of the present invention;
FIG. 3 is a schematic view of a projection surface and a projected image of one embodiment of the present invention;
FIG. 4 is a schematic diagram of a captured image, a rectangular prompt box, and a projected image area in accordance with one embodiment of the present invention;
FIG. 5 is a schematic diagram of an original image area according to one embodiment of the present invention;
FIG. 6 is a schematic illustration of a pre-adjusted and post-adjusted image according to an embodiment of the present invention;
FIG. 7 is a schematic illustration of pre-adjusted and post-adjusted images according to an embodiment of the present invention.
Detailed Description
In order to make the technical solutions of the present invention better understood by those skilled in the art, the present invention will be further described in detail with reference to the accompanying drawings and specific embodiments.
Referring to fig. 1, in one embodiment, the invention provides a projection device, which comprises a projection module, a shooting module, a display module and a calculation module, wherein the shooting module and the projection module can be connected through wired or wireless local communication, and the shooting module can move freely relative to the projection module. The projection module is used for projecting the image to a projection surface. The shooting module is used for shooting the projected image. Preferably, the shot covers the entire projection surface and the entire projection image, as indicated by the dashed lines in fig. 1.
Referring to fig. 2 to 7, the projection apparatus performs the following image processing methods:
step 1, the projection module projects the original image on a projection surface according to a preset scaling to generate a projection image, as shown in fig. 3. Such as an image with a specific pattern that is easily recognized and located. In fig. 3, a gray area indicates the projection surface, and a quadrangle abcd indicates a projection image generated when the original image is projected onto the projection surface at a scale smaller than 1. It should be noted that the shape and size of the projection surface are not limited in the present invention, and the projection surface may be tangible or intangible, where intangible means that the projection surface has no boundary or the boundary is located at a far position relative to the projection image. The intangible projection surface is, for example, a large wall.
And 2, the display module displays the range which can be shot by the shooting module for a user, and displays a rectangular prompt box with a preset size and a preset position relative to the shooting range for the user, as shown in fig. 4. The rectangular prompt box represents an imaginary curtain range.
And 3, when the receiving module receives a shooting instruction, the shooting module shoots the projection image in the shooting range to obtain a shot image. The captured image is an image obtained by directly capturing a picture within the capturing range. In fig. 4 and 5, a quadrangle IJKL indicates the photographing range or the photographed image, which includes all the projected images.
And 4, the calculation module determines an image adjusting method according to the original image, the shot image, the scaling, the rectangular prompt box and the aspect ratio of the final image expected to be projected.
And 5, adjusting the image to be projected by the computing module according to the image adjusting method determined in the step 4.
The camera module and the projection module of existing projection devices are typically physically located in the same device housing. However, in order not to affect the field of view of the person viewing the projection surface, the projection device is typically placed behind the person viewing the projection surface. In this case, the projection apparatus is far from the projection plane, which results in low trapezoidal correction accuracy.
The invention separates the shooting module from the projection device and enables the shooting module to move freely relative to the projection module. The camera module may be located closer to the projection surface than the projection module when the projection device is not in operation, and may be located closer to the projection surface than the projection module when desired, for example held by a person in front of the projection module who is viewing the projection surface or fixed by a tripod or the like. Because the distance between the shooting module and the projection surface can be shortened, the invention can provide clearer and more accurate images as the basis of the trapezoidal correction, thereby calculating more accurate image adjustment schemes and further realizing more accurate trapezoidal correction.
In addition, the invention provides a display module for displaying the shooting range and the rectangular prompt box to the user, and the rectangular prompt box is used for representing the virtual curtain range, so that the user can intuitively feel the final projected effect. In addition, the user can change the position of the rectangular prompt box relative to the projected image by moving the shooting module, so that the content and the position of the final projected image can be changed. In a preferred embodiment, the shooting module continuously shoots images and sends the shot images to the projection module in real time, and the projection module performs calculation and projection in real time according to the received shot images, so as to realize real-time and continuous correction. By the method, the projection equipment can generate a picture following effect along with the movement of the shooting module by the user. Thus, the user can be given a more interesting experience.
In one embodiment, the step 4 specifically includes:
step 4-1, the calculation module performs image recognition on the shot image to acquire a projected image area in the shot image;
step 4-2, the calculation module calculates an original image area in a shot image plane according to the projected image area and the preset scaling;
4-3, calculating a homography relation matrix T from the original image plane to the shot image plane by the calculating module;
4-4, calculating a homography relation matrix C from the projection plane to the shot image plane by the calculating module;
step 4-5, the calculation module calculates the homography transformation relation P ^ C (-1) × T from the original image plane to the projection plane;
4-6, calculating the maximum inscribed rectangle with the length-width ratio in the intersection region of the rectangular prompt box and the original image region by the calculation module;
4-7, calculating the change S from the original image to the maximum inscribed rectangle by the calculation module;
step 4-8, the calculation module calculates a pre-transformation matrix H ═ P ^ (-1) × S;
and 4-9, determining an image adjusting method according to the pre-transformation matrix H by the computing module.
In one embodiment, the scaling in step 1, i.e. step 4-2, is less than 1, i.e. the original image is reduced and then projected. Referring to fig. 5, a quadrangle abcd in fig. 5 is illustrated as a projected image area obtained by capturing a projected image generated by projecting an original image onto a projection surface at a scaling smaller than 1 by a capturing module. The quadrangle ABCD is an original image area calculated based on the projection image area ABCD and the aforementioned scaling. The original image area is considered as an area in the captured image of the image to be adjusted, which is generated by projecting the original image directly onto the projection surface (onto the projection surface at a scale equal to 1).
The reason for reducing the original image first and then restoring according to the scaling ratio is that if the original image is not reduced, part of the image is projected directly and possibly falls outside the shooting range, and the problems of incomplete data, inaccurate positioning and the like are caused. When a small-scale image is projected, the allowance in the upper direction, the lower direction, the left direction and the right direction of the shooting range is relatively large, the image is less prone to exceeding the projection surface area, and the extracted projection image area is more accurate. Further, the user can easily perform the operation of framing the projected image with the rectangular prompt box.
In a preferred embodiment, step 1 projects the reduced original image and simultaneously projects the outline of the original image onto the projection surface at a scaling ratio equal to 1, or projects the original image onto the projection surface without scaling, so as to help the user to imagine the final projected effect.
In one embodiment, the aspect ratio in step 4, i.e. steps 4-6, can be set by the user. The user may manually enter a desired aspect ratio or select one from several prepared options. Preferably, the aspect ratio defaults to the aspect ratio of the rectangular prompt box.
In one embodiment, when the original image area completely covers the rectangular prompt box, the pre-transformation matrix makes the edge of the corrected image, i.e. the projected final image, completely fit the edge of the projection area represented by the rectangular prompt box, as shown in fig. 6; and when the original image area only partially covers the rectangular prompt box, the corrected image, i.e. the projected final image, is displayed as an image on the projection plane corresponding to the maximum inscribed rectangular area with the desired projected aspect ratio in the intersection area of the original image area and the rectangular prompt box, as shown in fig. 7.
In the embodiment of the invention, the calculation of the pre-transformation matrix H is automatically completed by the calculation module according to the picture taken by the shooting module. For the condition that the user holds the shooting module by hand, the user only needs to shoot a picture containing the projection surface and the projection image through the shooting module, and the user does not need to do any operation, so the operation is very simple for the user. Secondly, the invention has no parameter calibration process, namely, the invention does not need to obtain an internal reference matrix between the shooting module and the projection module, has few characteristic points, and only needs four rectangular intersection points at least, so the invention has lower complexity of calculating the pre-transformation matrix H, thereby having higher trapezoidal correction speed.
It should be noted that, in the above embodiments, the projection module and the shooting module belong to the same projection device. However, the projection module and the photographing module may belong to two devices, as long as data can be exchanged therebetween.
Accordingly, the present invention provides an image processing method, performed by a projection device, the method comprising: step A1, projecting the original image on the projection surface according to the preset scaling to generate a projection image;
a step a2 of receiving a captured image obtained by capturing the projection image on the projection surface by a capturing device that is connected to the projection device in communication and is freely movable relative to the projection device;
step A3, receiving a rectangular prompt box with a preset size and a preset position relative to the shot image;
step A4, determining an image adjusting method according to the length-width ratio of the original image, the shot image, the scaling, the rectangular prompt box and the final image expected to be projected;
step A5, adjusting the image to be projected according to the image adjusting method determined in the step A4.
Preferably, the scaling ratio is smaller than 1, and the aspect ratio of the final image desired to be projected is the aspect ratio of the rectangular prompt box.
Preferably, the step a4 includes:
step A4-1, acquiring a projected image area in the shot image by performing image recognition on the shot image;
step A4-2, calculating an original image area in a shot image plane according to the projected image area and the preset scaling;
step A4-3, calculating a homography matrix T from the original image plane to the shot image plane;
step A4-4, calculating a homography relation matrix C from the projection plane to the shot image plane;
step A4-5, calculating the homography transformation relation P ^ C (-1) × T from the original image plane to the projection plane;
step A4-6, calculating the maximum inscribed rectangle with the length-width ratio in the intersection area of the rectangular prompt box and the original image area;
step A4-7, calculating the change S from the original image to the maximum inscribed rectangle;
step A4-8, calculating a pre-transformation matrix H ═ P ^ (-1) × S;
step A4-9, determining an image adjustment method according to the pre-transformation matrix H.
Preferably, the step a1 further includes: projecting the contour of the original image on the projection surface without scaling.
Correspondingly, the invention further provides the projection equipment. In one embodiment, the projection device comprises a projection module, a calculation module and a receiving module, wherein the receiving module is connected with a shooting device capable of freely moving relative to the projection module in a communication mode, and the projection device executes the following image processing method:
step B1, the projection module generates a projection image by projecting the original image on the projection surface according to a preset scaling;
a step B2 of receiving a captured image obtained by capturing the projection image on the projection surface by the capturing device;
step B3, the receiving module receives a rectangular prompt box with preset size and preset position relative to the shot image;
step B4, the calculation module determines an image adjustment method according to the original image, the shot image, the scaling, the rectangular prompt box and the aspect ratio of the final image expected to be projected;
step B5, the calculation module adjusts the image to be projected according to the image adjustment method determined in the step B4.
In this embodiment, the projection device and the photographing device are separate. One projection apparatus can cooperate with different types of photographing apparatuses manufactured by a plurality of different manufacturers to perform keystone correction. As long as the photographing apparatus can communicate with the projection apparatus. In addition, since the technical change speed of the photographing apparatus may not be synchronized with that of the projection apparatus, the user may separately purchase a new photographing apparatus to replace the old photographing apparatus when the photographing apparatus is updated, or separately purchase a new photographing apparatus to replace the old photographing apparatus when the projection apparatus is updated. Compared with the technical scheme that the shooting module and the projection module are located in the same projection device, the user of the embodiment does not need to replace the whole device when upgrading the device, and therefore cost is saved.
Preferably, the scaling ratio is smaller than 1, and the aspect ratio of the final image desired to be projected is the aspect ratio of the rectangular prompt box.
Preferably, the step B4 includes:
step B4-1, the calculation module acquires a projection image area in the shot image by performing image recognition on the shot image;
step B4-2, the calculating module calculates an original image area in a shot image plane according to the projected image area and the preset scaling;
step B4-3, the calculation module calculates a homography matrix T from the original image plane to the shot image plane;
step B4-4, the calculation module calculates a homography matrix C from the projection plane to the shot image plane;
step B4-5, the calculation module calculates homography transformation relation P ═ C (-1) × T from the original image plane to the projection plane;
step B4-6, the calculation module calculates the maximum inscribed rectangle with the aspect ratio in the intersection area of the rectangular prompt box and the original image area;
step B4-7, the calculation module calculates the change S of the original image to the maximum inscribed rectangle;
step B4-8, the calculation module calculates a pre-transformation matrix H ═ P ^ (-1) × S;
step B4-9, the calculation module determines an image adjustment method according to the pre-transformation matrix H.
Preferably, the step B1 further includes: projecting the contour of the original image on the projection surface without scaling.
Accordingly, the present invention also provides an image processing method, which in one embodiment is performed by a photographing apparatus, the method comprising:
step C1, displaying a shooting range and a rectangular prompt box with a preset size and a preset position relative to the shooting range to a user;
a step C2 of acquiring a shot image by shooting a projection image in the shooting range when a shooting instruction is received, wherein the projection image is an image generated by projecting an original image onto a projection surface at a preset zoom ratio by a projection device which is in communication connection with the shooting device and relative to which the shooting device can freely move;
a step C3 of receiving the original image and the scaling from the projection device;
step C4, determining an image adjusting method according to the original image, the shot image, the scaling, the rectangular prompt box and the aspect ratio of the final image expected to be projected;
and step C5, adjusting the image to be projected according to the image adjusting method determined in the step C4.
Preferably, the scaling ratio is smaller than 1, and the aspect ratio of the final image desired to be projected is the aspect ratio of the rectangular prompt box.
Step C4-1, acquiring a projected image area in the shot image by performing image recognition on the shot image;
step C4-2, calculating an original image area in the shot image plane according to the projected image area and the preset scaling;
step C4-3, calculating a homography matrix T from the original image plane to the shot image plane;
step C4-4, calculating a homography relation matrix C from the projection plane to the shot image plane;
step C4-5, calculating the homography transformation relation P ^ C (-1) × T from the original image plane to the projection plane;
step C4-6, calculating the maximum inscribed rectangle with the length-width ratio in the intersection area of the rectangular prompt box and the original image area;
step C4-7, calculating the change S from the original image to the maximum inscribed rectangle;
step C4-8, calculating a pre-transformation matrix H ═ P ^ (-1) × S;
and step C4-9, determining an image adjusting method according to the pre-transformation matrix H.
Accordingly, the present invention also provides a photographing apparatus, which in one embodiment comprises a display module, a photographing module, a calculation module and a receiving module, wherein the receiving module is in communication connection with a projection apparatus which is freely movable relative to the photographing apparatus, and the photographing apparatus executes the following image processing methods:
step D1, the display module displays the shooting range and a rectangular prompt box with preset size and position relative to the shooting range to the user;
step D2, when the receiving module receives the shooting instruction, the shooting module obtains the shooting image by shooting the projection image in the shooting range, wherein the projection image is the image generated by the projection device projecting the original image to the projection surface according to the preset scaling;
a step D3, in which the receiving module receives the original image and the scaling from the projection device;
step D4, the calculation module determines an image adjustment method according to the original image, the shot image, the scaling, the rectangular prompt box and the aspect ratio of the final image expected to be projected;
and D5, adjusting the image to be projected according to the image adjusting method determined in the step D4 by the computing module.
In this embodiment, the projection device and the photographing device are separate. One camera may cooperate with different models of projection devices from a plurality of different manufacturers to perform keystone correction. As long as the photographing apparatus can communicate with the projection apparatus. In addition, since the technical change speed of the photographing apparatus may not be synchronized with that of the projection apparatus, the user may separately purchase a new photographing apparatus to replace the old photographing apparatus when the photographing apparatus is updated, or separately purchase a new photographing apparatus to replace the old photographing apparatus when the projection apparatus is updated. Compared with the technical scheme that the shooting module and the projection module are located in the same projection device, the user of the embodiment does not need to replace the whole device when upgrading the device, and therefore cost is saved.
Preferably, the scaling ratio is smaller than 1, and the aspect ratio of the final image desired to be projected is the aspect ratio of the rectangular prompt box.
Preferably, the step D4 includes:
step D4-1, the calculation module acquires a projection image area in the shot image by performing image recognition on the shot image;
step D4-2, the calculation module calculates the original image area in the shooting image plane according to the projection image area and the preset scaling;
d4-3, calculating a homography matrix T from the original image plane to the shot image plane by the calculating module;
d4-4, calculating a homography matrix C from the projection plane to the shot image plane by the calculation module;
step D4-5, the calculation module calculates homography transformation relation P ═ C (-1) × T from the original image plane to the projection plane;
step D4-6, the calculation module calculates the maximum inscribed rectangle with the aspect ratio in the intersection area of the rectangular prompt box and the original image area;
step D4-7, the calculation module calculates the change S of the original image to the maximum inscribed rectangle;
step D4-8, the calculation module calculates a pre-transformation matrix H ═ P ^ (-1) × S;
step D4-9, the calculation module determines an image adjustment method according to the pre-transformation matrix H.
In a preferred embodiment, the camera device is a mobile phone with a camera function. Mobile phones have become almost everyone's own personal belongings today. The owner of the mobile phone can therefore use his familiar mobile phone to take a picture to assist the projection device in reminder calibration. This eliminates the time and effort required for the user to learn and become accustomed to using a new device. In addition, compared with the technical scheme that the shooting module and the projection module are located in the same projection device, the embodiment has the advantage of low cost. This is because the mobile phone has become a necessary personal belonging, which is equivalent to people carrying a shooting device, and if a shooting module is provided for the projection device, this is equivalent to a waste of resources, and the cost is also increased.
Preferably, the shooting device is provided with a gyroscope which can prompt the current declination angle and help the shooting device to shoot the image perpendicular to the projection plane.
It should be noted that, in the present specification, the embodiments are all described in a progressive manner, each embodiment focuses on differences from other embodiments, and the same and similar parts among the embodiments may be referred to each other. The order of the steps in each embodiment is not limited as long as the desired functions can be achieved.
The above is only a preferred embodiment of the present invention, and it should be noted that the above preferred embodiment should not be considered as limiting the present invention, and the protection scope of the present invention should be subject to the scope defined by the claims. It will be apparent to those skilled in the art that various modifications and adaptations can be made without departing from the spirit and scope of the invention, and these modifications and adaptations should be considered within the scope of the invention.
Claims (16)
1. An image processing method performed by a projection device, the method comprising:
step A1, projecting the original image on the projection surface according to the preset scaling to generate a projection image;
a step a2 of receiving a captured image obtained by capturing the projection image on the projection surface by a capturing device that is connected to the projection device in communication and is freely movable relative to the projection device;
step A3, receiving a rectangular prompt box with a preset size and a preset position relative to the shot image;
step A4, determining an image adjusting method according to the original image, the shot image, the scaling, the rectangular prompt box and the aspect ratio of the final image expected to be projected;
step A5, adjusting the image to be projected according to the image adjusting method determined in the step A4.
2. The method of claim 1, wherein the scaling is less than 1, and wherein the aspect ratio of the final image desired to be projected is the aspect ratio of the rectangular prompt box.
3. The method according to claim 1, wherein said step a4 comprises:
step A4-1, acquiring a projected image area in the shot image by performing image recognition on the shot image;
step A4-2, calculating an original image area in a shot image plane according to the projected image area and the preset scaling;
step A4-3, calculating a homography matrix T from the original image plane to the shot image plane;
step A4-4, calculating a homography relation matrix C from the projection plane to the shot image plane;
step A4-5, calculating the homography transformation relation P ^ C (-1) × T from the original image plane to the projection plane;
step A4-6, calculating the maximum inscribed rectangle with the length-width ratio in the intersection area of the rectangular prompt box and the original image area;
step A4-7, calculating the change S from the original image to the maximum inscribed rectangle;
step A4-8, calculating a pre-transformation matrix H ═ P ^ (-1) × S;
step A4-9, determining an image adjustment method according to the pre-transformation matrix H.
4. The method of claim 1, wherein step a1 further comprises: projecting the contour of the original image on the projection surface without scaling.
5. A projection device comprising a projection module, a calculation module, and a receiving module, the receiving module being communicatively connected to a camera device that is freely movable relative to the projection module, the projection device performing the following image processing methods:
step B1, the projection module generates a projection image by projecting the original image on the projection surface according to a preset scaling;
a step B2 of receiving a captured image obtained by capturing the projection image on the projection surface by the capturing device;
step B3, the receiving module receives a rectangular prompt box with preset size and preset position relative to the shot image;
step B4, the calculation module determines an image adjustment method according to the original image, the shot image, the scaling, the rectangular prompt box and the aspect ratio of the final image expected to be projected;
step B5, the calculation module adjusts the image to be projected according to the image adjustment method determined in the step B4.
6. The device of claim 5, wherein the scaling is less than 1, and wherein the aspect ratio of the desired projected final image is the aspect ratio of the rectangular prompt box.
7. The apparatus of claim 5, wherein said step B4 comprises:
step B4-1, the calculation module acquires a projection image area in the shot image by performing image recognition on the shot image;
step B4-2, the calculating module calculates an original image area in a shot image plane according to the projected image area and the preset scaling;
step B4-3, the calculation module calculates a homography matrix T from the original image plane to the shot image plane;
step B4-4, the calculation module calculates a homography matrix C from the projection plane to the shot image plane;
step B4-5, the calculation module calculates homography transformation relation P ═ C (-1) × T from the original image plane to the projection plane;
step B4-6, the calculation module calculates the maximum inscribed rectangle with the aspect ratio in the intersection area of the rectangular prompt box and the original image area;
step B4-7, the calculation module calculates the change S of the original image to the maximum inscribed rectangle;
step B4-8, the calculation module calculates a pre-transformation matrix H ═ P ^ (-1) × S;
step B4-9, the calculation module determines an image adjustment method according to the pre-transformation matrix H.
8. The apparatus of claim 7, wherein said step B1 further comprises: projecting the contour of the original image on the projection surface without scaling.
9. An image processing method performed by a photographing apparatus, the method comprising:
step C1, displaying a shooting range and a rectangular prompt box with a preset size and a preset position relative to the shooting range to a user;
a step C2 of acquiring a shot image by shooting a projection image in the shooting range when a shooting instruction is received, wherein the projection image is an image generated by projecting an original image onto a projection surface at a preset zoom ratio by a projection device which is in communication connection with the shooting device and relative to which the shooting device can freely move;
a step C3 of receiving the original image and the scaling from the projection device;
step C4, determining an image adjusting method according to the original image, the shot image, the scaling, the rectangular prompt box and the aspect ratio of the final image expected to be projected;
and step C5, adjusting the image to be projected according to the image adjusting method determined in the step C4.
10. The method of claim 9, wherein the scaling is less than 1, and wherein the aspect ratio of the final image desired to be projected is the aspect ratio of the rectangular prompt box.
11. The method according to claim 9, wherein the step C4 comprises:
step C4-1, acquiring a projected image area in the shot image by performing image recognition on the shot image;
step C4-2, calculating an original image area in the shot image plane according to the projected image area and the preset scaling;
step C4-3, calculating a homography matrix T from the original image plane to the shot image plane;
step C4-4, calculating a homography relation matrix C from the projection plane to the shot image plane;
step C4-5, calculating the homography transformation relation P ^ C (-1) × T from the original image plane to the projection plane;
step C4-6, calculating the maximum inscribed rectangle with the length-width ratio in the intersection area of the rectangular prompt box and the original image area;
step C4-7, calculating the change S from the original image to the maximum inscribed rectangle;
step C4-8, calculating a pre-transformation matrix H ═ P ^ (-1) × S;
and step C4-9, determining an image adjusting method according to the pre-transformation matrix H.
12. A photographing apparatus comprising a display module, a photographing module, a calculation module, and a reception module, the reception module being in communication connection with a projection apparatus with which the photographing apparatus is freely movable relative thereto, the photographing apparatus performing the following image processing method:
step D1, the display module displays the shooting range and a rectangular prompt box with preset size and position relative to the shooting range to the user;
step D2, when the receiving module receives the shooting instruction, the shooting module obtains the shooting image by shooting the projection image in the shooting range, wherein the projection image is the image generated by the projection device projecting the original image to the projection surface according to the preset scaling;
a step D3, in which the receiving module receives the original image and the scaling from the projection device;
step D4, the calculation module determines an image adjustment method according to the original image, the shot image, the scaling, the rectangular prompt box and the aspect ratio of the final image expected to be projected;
and D5, adjusting the image to be projected according to the image adjusting method determined in the step D4 by the computing module.
13. The device of claim 12, wherein the scaling is less than 1, and wherein the aspect ratio of the desired projected final image is the aspect ratio of the rectangular prompt box.
14. The apparatus according to claim 12, wherein said step D4 comprises:
step D4-1, the calculation module acquires a projection image area in the shot image by performing image recognition on the shot image;
step D4-2, the calculation module calculates the original image area in the shooting image plane according to the projection image area and the preset scaling;
d4-3, calculating a homography matrix T from the original image plane to the shot image plane by the calculating module;
d4-4, calculating a homography matrix C from the projection plane to the shot image plane by the calculation module;
step D4-5, the calculation module calculates homography transformation relation P ═ C (-1) × T from the original image plane to the projection plane;
step D4-6, the calculation module calculates the maximum inscribed rectangle with the aspect ratio in the intersection area of the rectangular prompt box and the original image area;
step D4-7, the calculation module calculates the change S of the original image to the maximum inscribed rectangle;
step D4-8, the calculation module calculates a pre-transformation matrix H ═ P ^ (-1) × S;
step D4-9, the calculation module determines an image adjustment method according to the pre-transformation matrix H.
15. The photographing apparatus according to claim 12, wherein the photographing apparatus is a mobile phone having a photographing function.
16. The photographing apparatus according to claim 15, wherein the photographing apparatus has a gyroscope.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811177657.XA CN110769218B (en) | 2018-10-10 | 2018-10-10 | Image processing method, projection apparatus, and photographing apparatus |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811177657.XA CN110769218B (en) | 2018-10-10 | 2018-10-10 | Image processing method, projection apparatus, and photographing apparatus |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110769218A true CN110769218A (en) | 2020-02-07 |
| CN110769218B CN110769218B (en) | 2021-10-22 |
Family
ID=69328949
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811177657.XA Active CN110769218B (en) | 2018-10-10 | 2018-10-10 | Image processing method, projection apparatus, and photographing apparatus |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110769218B (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111405258A (en) * | 2020-04-30 | 2020-07-10 | 平安科技（深圳）有限公司 | Projection method, device, equipment and computer readable storage medium |
| CN115174879A (en) * | 2022-07-18 | 2022-10-11 | 峰米(重庆)创新科技有限公司 | Projection picture correction method, projection picture correction device, computer equipment and storage medium |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105554486A (en) * | 2015-12-22 | 2016-05-04 | Tcl集团股份有限公司 | Projection calibration method and device |
| CN106162124A (en) * | 2016-08-02 | 2016-11-23 | 上海唱风信息科技有限公司 | The calibration steps of scialyscope output image |
| US20170374331A1 (en) * | 2016-06-27 | 2017-12-28 | Intel Corporation | Auto keystone correction and auto focus adjustment |
-
2018
- 2018-10-10 CN CN201811177657.XA patent/CN110769218B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105554486A (en) * | 2015-12-22 | 2016-05-04 | Tcl集团股份有限公司 | Projection calibration method and device |
| US20170374331A1 (en) * | 2016-06-27 | 2017-12-28 | Intel Corporation | Auto keystone correction and auto focus adjustment |
| CN106162124A (en) * | 2016-08-02 | 2016-11-23 | 上海唱风信息科技有限公司 | The calibration steps of scialyscope output image |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111405258A (en) * | 2020-04-30 | 2020-07-10 | 平安科技（深圳）有限公司 | Projection method, device, equipment and computer readable storage medium |
| CN111405258B (en) * | 2020-04-30 | 2023-06-20 | 平安科技（深圳）有限公司 | Projection method, apparatus, device and computer readable storage medium |
| CN115174879A (en) * | 2022-07-18 | 2022-10-11 | 峰米(重庆)创新科技有限公司 | Projection picture correction method, projection picture correction device, computer equipment and storage medium |
| CN115174879B (en) * | 2022-07-18 | 2024-03-15 | 峰米(重庆)创新科技有限公司 | Projection screen correction method, apparatus, computer device and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110769218B (en) | 2021-10-22 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| JP6263623B2 (en) | Image generation method and dual lens apparatus | |
| CN107238996B (en) | Projection system and correction method of projection picture | |
| KR100834638B1 (en) | Method and apparatus for adjusting mage color for beam projector | |
| JP6645687B2 (en) | Display device and control method | |
| CN102404537B (en) | Projector and method of controlling projector | |
| CN109934931B (en) | Method and device for collecting image and establishing target object recognition model | |
| CN109840939B (en) | Three-dimensional reconstruction method, three-dimensional reconstruction device, electronic equipment and storage medium | |
| CN110769217A (en) | Image processing method, projection apparatus, and photographing apparatus | |
| KR20190012465A (en) | Electronic device for acquiring image using plurality of cameras and method for processing image using the same | |
| CN109377446B (en) | Face image processing method and device, electronic equipment and storage medium | |
| CN110769218B (en) | Image processing method, projection apparatus, and photographing apparatus | |
| US10987198B2 (en) | Image simulation method for orthodontics and image simulation device thereof | |
| US10909748B2 (en) | Image projection method and image projection device for three-dimensional object for projection | |
| CN114170324A (en) | Calibration method and device, electronic equipment and storage medium | |
| US20150022697A1 (en) | Projector Auto-Focus Correction with the Aid of a Camera | |
| JP2016085380A (en) | Controller, control method, and program | |
| JP2004228824A (en) | Stack projection device and its adjusting method | |
| JP2006513509A (en) | Composite information projection | |
| CN109644248B (en) | Projection type image display device and method for adjusting projection image | |
| CN114125411A (en) | Projection equipment correction method and device, storage medium and projection equipment | |
| CN106973275A (en) | The control method and device of projector equipment | |
| CN112365406B (en) | Image processing method, device and readable storage medium | |
| CN114339179B (en) | Projection correction method, apparatus, storage medium and projection device | |
| CN112529781B (en) | Image processing method, device and readable storage medium | |
| JP2009302814A (en) | Imaging control device and method of imaging apparatus, and projection system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |