Detailed Description
In order to make those skilled in the art better understand the technical solution of the present invention, the technical solution in the embodiment of the present invention will be clearly and completely described below with reference to the drawings in the embodiment of the present invention, and it is obvious that the described embodiment is only a part of the embodiment of the present invention, and not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Currently, in the field of image processing, in some scenes, an image captured by a camera is required to be completely consistent with a target, that is, an imaging surface of the camera is required to be consistent with the target in the same coordinate plane. In practice, however, the camera will have a slight angle to the target, which the human eye cannot recognize. Such a deviation, when introduced into the image data acquired by the camera, will hinder the processing such as image recognition and feature matching in the later stage. That is, the tilt between the camera and the target cannot be effectively adjusted at present.
The embodiment of the application provides a scheme for assisting in adjusting the inclination angle of the camera, aiming at the problems, and is used for solving the problems. In the technical scheme provided by the embodiment of the application, in the process of one adjustment, the camera acquires continuous multi-frame target images from the target, acquires a first window image and a second window image which are symmetrical about a central line in the target image, determines the similarity values of the first window image and the second window image, and prompts the maximum value of the similarity values of the multi-frame target images, so that the camera obtains the inclination angle adjustment with the maximum value as the reference. The technical scheme provided by the embodiment of the application can indicate the inclination degree of the camera relative to the target in the same coordinate plane by determining the similarity of the images of two symmetrical window regions in the target image, and prompt the similarity to enable the camera to obtain the inclination angle adjustment taking the similarity as a reference, so that the problem that the inclination exists between the camera and the target cannot be effectively adjusted in the prior art can be solved.
In the embodiment of the present application, the camera tilt angle refers to the rotation angle of the camera in two directions of a two-dimensional plane.
In the embodiment of the application, the target has a figure with a symmetrical center line, and the center line of the target is arranged in the same direction as the center line of the camera imaging surface. For example, when the target and the imaging plane of the camera are arranged on the XY plane with the center line of the imaging plane of the camera in the Y direction, the center line of the pattern on the target is arranged in the Y direction, that is, the pattern on the target is a Y-axis symmetric pattern. For another example, when the target and the imaging plane of the camera are arranged on the XZ plane, the center line of the imaging plane of the camera is in the X direction, and the center line of the pattern on the target may be arranged in the X direction, the pattern of the target is a pattern that is X-axis symmetric. Further, the same row of pixels in the pattern of the target perpendicular to the centerline has the same resolution, and different rows of pixels may have different resolutions.
The embodiments of the present application will be described by taking an example of auxiliary adjustment of the tilt angle of the camera on a two-dimensional plane.
Fig. 1 illustrates a processing flow of a method for assisting in adjusting a tilt angle of a camera according to an embodiment of the present application, including:
step 101, in a primary adjustment process, an adjustment device acquires continuous multi-frame target images acquired by a camera on a target; wherein the target comprises a figure with symmetrical central lines;
102, acquiring a first window image and a second window image which are symmetrical to each other in a central line in each frame of target image, and determining the similarity value of the first window image and the second window image;
and 103, prompting the maximum value in the similarity values of the multi-frame target images to enable the camera to obtain the inclination angle adjustment taking the maximum value as a reference, wherein the inclination angle is the angle between the imaging surface of the camera and the target in the same coordinate plane.
When the angle of the camera is adjusted, multiple adjustments may be required, and in each adjustment process, the above steps 101 to 103 are performed.
In some embodiments, in step 101, the adjusting device may be a separate device disposed separately from the camera, and the adjusting device may read a frame of target image currently acquired by the camera or receive a current frame of target image from the camera; the adjusting device may also be combined with the camera, may be arranged inside or outside the camera, and reads the current frame target image acquired by the camera, or acquires the current frame target image synchronized by the camera. Further, the adjustment device is also connected to the drive device, and the adjustment device controls the drive device to adjust the tilt angle of the pan/tilt head and to adjust the tilt angle of the camera connected to the pan/tilt head according to the result of performing the method shown in fig. 1.
In some embodiments, as described above, the target comprises a pattern that is centrosymmetric. In some embodiments, for example, the pattern of the target is arranged in an XY plane and is a Y-axis symmetric pattern, and the pattern of the target has the same resolution of pixels in the same row in the X-axis direction, and may have different resolutions between different rows. The pattern of the target is usually a rectangular pattern, but may be a pattern of other shapes and axial symmetry. An example of a target graphic is shown in fig. 2. In the embodiment of the present application, a rectangle in which the graph of the target is symmetrical with respect to the vertical center line is taken as an example for description. When the pattern of the target is an axisymmetric pattern having another shape, the following processing method is also applicable.
In some embodiments, the process of acquiring the first window image and the second window image with central line symmetry in the target image in step 102 includes:
step 301, a first window image of a designated window area is acquired in a target image.
In some embodiments, step 301 comprises a process flow as shown in FIG. 4.
The process flow shown in fig. 4 includes:
and 3011, acquiring at least two coordinate points on the target image.
Step 3011 is described by taking two coordinate points as an example in some embodiments, but the present embodiment is not limited to the following two obtaining manners, and in a specific application scenario, other obtaining manners may be set according to a scenario requirement.
The first acquisition mode is that the coordinate point input by the user can be acquired through a human-computer interaction interface. For example, when a user uses a mouse or selects two coordinate points in a displayed target image on a touch screen, the two coordinate points selected by the user are obtained through a human-computer interaction interface; or when a user drags a slant line on the displayed target image by using a mouse or on a touch screen, coordinate points of a starting point and an end point of the slant line are obtained through the human-computer interface.
And the second acquisition mode is to acquire two preset image coordinate points. In this way, two fixed image coordinate points set in advance can be acquired; or a group of image coordinate points may be selected from a plurality of preset pairs of image coordinate points, where the group of image coordinate points includes two image coordinate points, and the selection may be random, sequential, or according to a predetermined priority.
In other embodiments of the present application, other greater numbers of image coordinate points may also be obtained through the two manners. Further, in some other embodiments, the at least two image coordinate points may be obtained in other manners, which are not listed herein.
And step 3012, determining the designated window area according to the coordinate values of the acquired at least two coordinate points.
In the above example, when there are two coordinate points, the two coordinate points may be used as the vertices of opposite angles to determine a resulting rectangular area. For example, a rectangular region is determined from the coordinate point a (x1, y1) and the coordinate point b (x2, y2), and the coordinates of four vertices of the rectangular region are (x1, y1), (x2, y1), (x1, y2), (x2, y 2).
In the above description, the present embodiment has been described taking, as an example, a specified window region in which a rectangle is obtained from two coordinate points. In other embodiments, the designated window area may also include other shapes, such as an axisymmetric pattern of circles or the like. In other embodiments, the designated window region may also be determined or generated based on a greater number of image coordinate points. One of ordinary skill in the art can determine other shapes of the designated window area based on other numbers of points according to the process described above.
Step 3013, a first window image of the determined designated window area is obtained in the target image. That is, the image of the designated window area is acquired in the target image.
And 302, acquiring a second window image which is symmetrical to the first window image in terms of a central line according to the central line of the imaging surface of the camera.
And according to the coordinates of the appointed window area and the central line of the camera imaging surface, a second window image which is symmetrical to the first window image in the target image can be obtained.
In some embodiments, the determining the similarity value of the first window image and the second window image in step 102 may include taking the pixel value of the first window image and the pixel value of the second window image as an input of the similarity function, and obtaining the similarity value output by the similarity function. The similarity function may be any function for representing similarity.
For example, the following formula is a similarity function, and the embodiment of the present application applies the formula one, where a is the first window image, B is the second window image,
is the average of the pixel values in a,
is the average of the pixel values in B. The a pixel matrix and the B pixel matrix are each m × n in size. A. the
ij、B
ijRespectively, the pixel point values with coordinates (i, j) in the first and second window images.
The application can also use other similarity functions to determine and obtain the similarity value of the first window image and the second window image. The present application is not specifically limited herein.
Whether the camera imaging surface and the target incline on the same coordinate plane can be measured through the similarity value between the first window image and the second window image. When the target image acquired by the camera is consistent with the target pattern, there is no inclined angle between the camera imaging plane and the target in the same coordinate plane, as shown in example (a) of fig. 5, otherwise there is an inclined angle α between the camera imaging plane and the target, as shown in example (b) or example (c) of fig. 5. Since the graph of the target is a graph whose center line is symmetrical, in the case where the camera imaging plane coincides with the target without inclination, the first window image and the second window image in the target image acquired by the camera will be coincident or symmetrical images, and the similarity value between the two will be larger, as shown in the example of (a) in fig. 6; in the case where the camera imaging plane is not consistent with the target and there is a tilt, the first window image and the second window image will not be consistent or asymmetric images, and the smaller the similarity value between the two is, as shown in (b) or (c) example in fig. 6, where window 1 represents the first window image and window 2 represents the second window image. So that the presence or absence of tilt of the camera relative to the target can be measured by the similarity value.
After the similarity value between the first window image and the second window image is obtained, the similarity value can be prompted, so that the camera can obtain the inclination angle adjustment according to the similarity value.
In the step 103, the operation of prompting to determine the obtained similarity value may include the following three ways. In a specific application scenario, other acquisition modes may also be set, and the present application is not limited specifically here.
And in the first prompting mode, after the similarity values of the multiple frames of target images are determined, prompting the maximum value in the similarity values of the multiple frames of target images and/or the similarity value of each frame of target image.
Wherein only the maximum value in the similarity values of the multi-frame target images is prompted, the best adjustment condition in the current adjustment stage can be shown. The similarity value of each frame of target image is prompted, so that the historical situation of the current adjusting stage can be displayed, and a historical and global reference is provided for adjusting the camera. The maximum value of the similarity values of the multiple frames of target images and the similarity value of each frame of target image are prompted, and the two situations can be integrated for integrated display.
And a second prompting mode, comparing the maximum value of the similarity values of the multi-frame target images with a preset first reference value, prompting the maximum value of the multi-frame target images and the similarity value of each frame of target images under the condition that the maximum value is larger than the first reference value, and otherwise, prompting the similarity value of each frame of target images.
In the present embodiment, the first reference value may be set to a value closer to 1 (in the present embodiment, 1 is a maximum value of the similarity value), and in the process of adjusting the camera, once a similarity value greater than the first reference value is obtained, it may be considered that the camera has been adjusted to a better state, and then the camera may be kept in the better state or adjusted to the better state. The maximum similarity value and the similarity value of each frame of target image can be presented at this time, so as to provide a reference for determining whether further adjustment is needed. When a similarity value larger than the first reference value is not obtained, the camera can be considered to need further adjustment, and only the similarity value of each frame of target image can be prompted to provide reference for further adjustment of the camera.
And a third prompting mode, comparing the maximum value of the similarity values of the multi-frame target images with a preset first reference value and a preset second reference value, prompting the maximum value of the similarity values of the multi-frame target images when the maximum value is smaller than the second reference value, prompting the preset second prompting information, the maximum value of the similarity values of the multi-frame target images and the similarity value of each frame of target image when the maximum value is larger than or equal to the second reference value and smaller than the first reference value, and prompting the preset first prompting information, the maximum value of the similarity values of the multi-frame target images and the similarity value of each frame of target image when the maximum value is larger than or equal to the first reference value.
The first reference value and the second reference value are preset values, and the first reference value is larger than the second reference value. When the maximum value in the similarity values of the multiple frames of target images is smaller than the second reference value, the inclination angle of the camera relative to the target is considered to be larger, the camera needs to be adjusted by a larger margin, and at the moment, only the maximum value in the similarity values of the multiple frames of target images can be prompted, so that the optimal adjustment condition of the current adjustment process is shown.
When the maximum value of the similarity values of the multiple frames of target images is greater than or equal to the second reference value and smaller than the first reference value, it can be considered that the camera still has a certain inclination angle relative to the target, and the camera needs to be further adjusted. At this time, the maximum value of the similarity values of the multiple frames of target images and the similarity value of each frame of target image can be prompted, the optimal adjustment condition of the current adjustment stage and the historical global adjustment condition can be displayed, and further, second prompt information can be prompted, so that reference can be provided for mastering the optimal adjustment opportunity in the next adjustment. The specific content of the second prompt message may be set according to the needs of a specific application scenario, for example, may be set to "approach to the optimal state, and still need to be adjusted", which is not specifically limited herein.
When the maximum value among the similarity values of the multiple frame target images is greater than the first reference value, the case is the same as the second prompting mode. Further, when the maximum value of the similarity values of the multiple frame target images is greater than the first reference value, the first prompt message can be further prompted. The prompted information provides reference for determining whether further adjustments to the camera are needed. And will not be described in detail herein. The specific content of the first prompt information may be set according to the needs of specific application scenarios, for example, "achieve the best state", and the present application is not limited in this respect.
In the embodiment of the application, the prompt operation may include display and/or acousto-optic prompt. That is, the information to be prompted can be displayed through the display device, or the corresponding prompt can be performed through the devices such as the sound playing device and the optical prompting device.
In the embodiment of the application, the similarity value is prompted, and reference can be provided for manual or automatic adjustment of the camera.
According to the technical scheme provided by the embodiment of the application, the degree of inclination of the target image relative to the target in the same coordinate plane is indicated by determining the similarity of the images of two symmetrical window regions in the target image, and the similarity is prompted so that the camera can obtain the inclination angle adjustment taking the similarity as the reference, so that the problem that the inclination exists between the camera and the target cannot be effectively adjusted in the prior art can be solved.
Based on the same inventive concept, the embodiment of the application also provides a device for assisting in adjusting the inclination angle of the camera.
Fig. 7 shows a structure of an apparatus for assisting adjustment of a tilt angle of a camera according to an embodiment of the present application, which includes a processor 71 and at least one memory 72, where the at least one memory 72 stores at least one machine executable instruction, and the processor 71 executes the at least one machine executable instruction to perform:
in the primary camera adjustment process, acquiring continuous multi-frame target images acquired by a camera on a target; wherein the target comprises a figure with symmetrical central lines;
for each frame of target image, acquiring a first window image and a second window image which are symmetrical in terms of a central line in the target image, and determining the similarity value of the first window image and the second window image;
and prompting the determined similarity value so that the camera obtains the inclination angle adjustment taking the similarity value as a reference, wherein the inclination angle is the angle between the imaging surface of the camera and the target in the same coordinate plane.
In some embodiments, the processor 71 executing at least one machine executable instruction performs acquiring a first window image and a second window image symmetric with respect to a center line in the target image, including: acquiring a first window image of a specified window area in a target image; and acquiring a second window image which is symmetrical to the first window image in terms of a central line according to the central line of the imaging surface of the camera.
In some embodiments, the processor 71 executing at least one machine executable instruction performs acquiring a first window image of the specified region in the target image, comprising: acquiring at least two coordinate points input through a human-computer interaction interface, or acquiring at least two preset image coordinate points; determining a designated window area according to the coordinate values of the at least two coordinate points; a first window image of the determined specified window region is acquired in the target image.
In some embodiments, the processor 71 executes at least one machine executable instruction to perform acquiring two predetermined image coordinate points, including: reading at least two preset fixed image coordinate points; or selecting a group of image coordinate points from a plurality of preset groups of image coordinate points, wherein the group of image coordinate points comprises at least two image coordinate points.
In some embodiments, the processor 71 executing at least one machine executable instruction performs determining a similarity value for the first window image and the second window image, comprising: and taking the pixel values of the first window image and the second window image as the input of the similarity function to obtain the similarity value output by the similarity function.
In some embodiments, execution of at least one machine executable instruction by processor 71 to perform hinting at the determined similarity values comprises: prompting the maximum value in the similarity values of the multiple frames of target images and/or the similarity value of each frame of target image; or comparing the maximum value of the similarity values of the multiple frames of target images with a preset first reference value, and prompting the maximum value of the multiple frames of target images and the similarity value of each frame of target images under the condition that the maximum value is greater than the first reference value, otherwise, prompting the similarity value of each frame of target images; or comparing the maximum value of the similarity values of the multiple frame target images with a preset first reference value and a preset second reference value, prompting the maximum value of the similarity values of the multiple frame target images when the maximum value is smaller than the second reference value, prompting the preset second prompting information, the maximum value of the similarity values of the multiple frame target images and the similarity value of each frame target image when the maximum value is larger than or equal to the second reference value and smaller than the first reference value, and prompting the preset first prompting information, the maximum value of the similarity values of the multiple frame target images and the similarity value of each frame target image when the maximum value is larger than or equal to the first reference value.
In some embodiments, execution of the at least one machine-executable instruction by processor 71 to perform the prompting includes display and/or acousto-optic prompting.
According to the technical scheme provided by the embodiment of the application, the degree of inclination of the target image relative to the target in the same coordinate plane is indicated by determining the similarity of the images of two symmetrical window regions in the target image, and the similarity is prompted so that the camera can obtain the inclination angle adjustment taking the similarity as the reference, so that the problem that the inclination exists between the camera and the target cannot be effectively adjusted in the prior art can be solved.
Based on the same inventive concept, the embodiment of the present application further provides a non-transitory machine-readable storage medium, in which at least one machine executable instruction is stored, and the machine executes the at least one machine executable instruction to perform:
in the primary camera adjustment process, acquiring continuous multi-frame target images acquired by a camera on a target; wherein the target comprises a figure with symmetrical central lines;
for each frame of target image, acquiring a first window image and a second window image which are symmetrical in terms of a central line in the target image, and determining the similarity value of the first window image and the second window image;
and prompting the determined similarity value so that the camera obtains the inclination angle adjustment taking the similarity value as a reference, wherein the inclination angle is the angle between the imaging surface of the camera and the target in the same coordinate plane.
In some embodiments, machine-executed at least one machine-executable instruction performs acquiring a first window image and a second window image that are centrosymmetric in a target image, comprising: acquiring a first window image of a specified window area in a target image; and acquiring a second window image which is symmetrical to the first window image in terms of a central line according to the central line of the imaging surface of the camera.
In some embodiments, machine-executed at least one machine-executable instruction performs acquiring a first window image of a specified region in a target image, comprising: acquiring at least two coordinate points input through a human-computer interaction interface, or acquiring at least two preset image coordinate points; determining a designated window area according to the coordinate values of the at least two coordinate points; a first window image of the determined specified window region is acquired in the target image.
In some embodiments, the machine-executable instructions perform obtaining the predetermined two image coordinate points by machine execution of at least one machine-executable instruction, comprising: reading at least two preset fixed image coordinate points; or selecting a group of image coordinate points from a plurality of preset groups of image coordinate points, wherein the group of image coordinate points comprises at least two image coordinate points.
In some embodiments, machine-executed at least one machine-executable instruction performs determining a similarity value for a first window image and a second window image, comprising: and taking the pixel values of the first window image and the second window image as the input of the similarity function to obtain the similarity value output by the similarity function.
In some embodiments, machine-executing the at least one machine-executable instruction to perform hinting the determined similarity values comprises: prompting the maximum value in the similarity values of the multiple frames of target images and/or the similarity value of each frame of target image; or comparing the maximum value of the similarity values of the multiple frames of target images with a preset first reference value, and prompting the maximum value of the multiple frames of target images and the similarity value of each frame of target images under the condition that the maximum value is greater than the first reference value, otherwise, prompting the similarity value of each frame of target images; or comparing the maximum value of the similarity values of the multiple frame target images with a preset first reference value and a preset second reference value, prompting the maximum value of the similarity values of the multiple frame target images when the maximum value is smaller than the second reference value, prompting the preset second prompting information, the maximum value of the similarity values of the multiple frame target images and the similarity value of each frame target image when the maximum value is larger than or equal to the second reference value and smaller than the first reference value, and prompting the preset first prompting information, the maximum value of the similarity values of the multiple frame target images and the similarity value of each frame target image when the maximum value is larger than or equal to the first reference value.
In some embodiments, the machine-executable instruction to perform the prompting includes displaying and/or acousto-optic prompting.
According to the technical scheme provided by the embodiment of the application, the degree of inclination of the target image relative to the target in the same coordinate plane is indicated by determining the similarity of the images of two symmetrical window regions in the target image, and the similarity is prompted so that the camera can obtain the inclination angle adjustment taking the similarity as the reference, so that the problem that the inclination exists between the camera and the target cannot be effectively adjusted in the prior art can be solved.
Based on the same inventive concept, the embodiment of the present application further provides a camera, which includes a device for assisting in adjusting the tilt angle of the camera, as shown in fig. 7. The device may be arranged inside or outside the camera. When the device is disposed outside the camera, the device may be connected to the camera in a wired or wireless manner.
The camera provided by the embodiment of the application can automatically acquire the image of the target and prompt the similarity of the images of two symmetrical window regions in the acquired target image. When the device shown in fig. 7 is provided within a camera, the device may prompt through a display device or a prompting device already existing in the camera. When the device shown in fig. 7 is disposed outside the camera, the device may prompt through a display device or a prompting device already provided in the camera, or prompt through a display device or a prompting device further provided in the device.
It will be apparent to those skilled in the art that various changes and modifications may be made in the present invention without departing from the spirit and scope of the invention. Thus, if such modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include such modifications and variations.