CN110651299A - Image water ripple detection method and device, unmanned aerial vehicle and storage device - Google Patents

Image water ripple detection method and device, unmanned aerial vehicle and storage device Download PDF

Info

Publication number
CN110651299A
CN110651299A CN201880029303.2A CN201880029303A CN110651299A CN 110651299 A CN110651299 A CN 110651299A CN 201880029303 A CN201880029303 A CN 201880029303A CN 110651299 A CN110651299 A CN 110651299A
Authority
CN
China
Prior art keywords
image
calibration
determining
objects
group
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN201880029303.2A
Other languages
Chinese (zh)
Inventor
林家荣
唐克坦
张培科
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
SZ DJI Technology Co Ltd
Shenzhen Dajiang Innovations Technology Co Ltd
Original Assignee
Shenzhen Dajiang Innovations Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shenzhen Dajiang Innovations Technology Co Ltd filed Critical Shenzhen Dajiang Innovations Technology Co Ltd
Publication of CN110651299A publication Critical patent/CN110651299A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B64—AIRCRAFT; AVIATION; COSMONAUTICS
    • B64U—UNMANNED AERIAL VEHICLES [UAV]; EQUIPMENT THEREFOR
    • B64U20/00—Constructional aspects of UAVs
    • B64U20/80—Arrangement of on-board electronics, e.g. avionics systems or wiring
    • B64U20/87—Mounting of imaging devices, e.g. mounting of gimbals
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Microelectronics & Electronic Packaging (AREA)
  • Mechanical Engineering (AREA)
  • Remote Sensing (AREA)
  • Aviation & Aerospace Engineering (AREA)
  • Image Analysis (AREA)
  • Studio Devices (AREA)
  • Image Processing (AREA)

Abstract

A method of moire detection of an image, comprising: acquiring an image (301) obtained by shooting an image calibration device (20), wherein the image calibration device (20) comprises a plurality of calibration objects (302); detecting an image object of a designated object in an image; matching the detected image object of the calibration object with the calibration object in the image device; and determining whether the water ripple exists in the image according to the position of the image object in the image and the position of the calibration object matched with the image object in the image calibration device. Through such mode, can realize intellectual detection system ripple, improve detection efficiency. Also discloses a water ripple detection device, an unmanned aerial vehicle and a storage device.

Description

Image water ripple detection method and device, unmanned aerial vehicle and storage device [ technical field ] A method for producing a semiconductor device
The application relates to the technical field of image processing, in particular to a water ripple detection method and device of an image, an unmanned aerial vehicle and a storage device.
[ background of the invention ]
The unmanned aerial vehicle is provided with a shooting device and is widely applied to executing shooting tasks. Unmanned aerial vehicle is at the in-process of flight, and the fuselage can produce the vibration, and the vibration of fuselage can directly or indirectly transmit to shooting device. Thus, when the shooting device vibrates, the shooting device shoots and acquires an image, and relative displacement is generated among lines. At this time, the photographed image is distorted, and moire deformation, i.e., a jelly effect, is observed in the image with naked eyes, which affects the photographing quality of the image. At present, the detection of the water ripple of an image is generally judged manually, the detection efficiency is low, and the professional requirement on an operator is high.
[ summary of the invention ]
The application mainly solves the technical problem of providing a water ripple detection method and device of an image, an unmanned aerial vehicle and a storage device, which can realize intelligent detection of water ripple and improve detection efficiency.
In order to solve the above technical problem, a first aspect of the present application provides a method for detecting water ripple of an image, including: acquiring an image shot by an image calibration device, wherein the image calibration device comprises a plurality of calibration objects; detecting an image object of a labeled object in the image; matching the detected image object of the calibration object with a calibration object in the image calibration device; and determining whether water ripples exist in the image according to the position of the image object in the image and the position of a calibration object matched with the image object in the image calibration device.
In order to solve the above technical problem, a second aspect of the present application provides a ripple detection apparatus, including a processor and a memory, where the memory is used for storing program instructions; the processor executing the program instructions to: acquiring an image shot by an image calibration device, wherein the image calibration device comprises a plurality of calibration objects; detecting an image object of a labeled object in the image; matching the detected image object of the calibration object with a calibration object in the image calibration device; and determining whether water ripples exist in the image according to the position of the image object in the image and the position of a calibration object matched with the image object in the image calibration device.
In order to solve the above technical problem, a third aspect of the present application provides a water ripple detection system, which includes a shooting device and the water ripple detection device, wherein the shooting device is used for shooting the image calibration device.
In order to solve the technical problem, the present application fourth aspect provides an unmanned aerial vehicle, including above-mentioned ripple detecting system.
In order to solve the above technical problem, a fifth aspect of the present application provides a storage device storing program instructions that, when executed on a processor, perform the method of the first aspect.
According to the scheme, the image object of the calibration object is obtained by detecting the image shot by the image calibration device, and whether the water ripple exists in the image is determined according to the position of the image object in the image and the position of the corresponding matched calibration object in the image calibration device, so that the intelligent detection of the water ripple of the image is realized, the manual detection is not needed, the detection efficiency is improved, and the false detection rate is reduced.
[ description of the drawings ]
FIG. 1 is a schematic flow chart diagram illustrating an embodiment of a method for detecting moire in an image according to the present application;
FIG. 2a is a schematic diagram of an image calibration apparatus used in an application scenario of the present application;
FIG. 2b is a schematic diagram of an image calibration apparatus used in another application scenario of the present application;
FIG. 3 is a schematic diagram of an image and an image object therein taken in an application scenario of the present application;
FIG. 4 is a schematic diagram illustrating a matching relationship between a calibration object and an image object in an application scenario of the present application;
FIG. 5 is a schematic flowchart of step S14 in another embodiment of the method for detecting water ripples of an image according to the present application;
FIG. 6 is a schematic diagram of a selected set of image objects in an application scenario of the present application;
FIG. 7 is a schematic flowchart of step S14 in yet another embodiment of the present invention;
FIG. 8 is a schematic diagram illustrating a positional relationship between a projection position of an image in an application scene and a position of a calibration object matching the projection position;
FIG. 9 is a schematic structural diagram of an embodiment of the present invention;
FIG. 10 is a schematic diagram of an embodiment of the present invention;
fig. 11 is a schematic structural diagram of an embodiment of the drone of the present application;
FIG. 12 is a schematic structural diagram of an embodiment of a memory device according to the present application.
[ detailed description ] embodiments
The technical solutions in the embodiments of the present invention will be described clearly below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
It will be understood that when an element is referred to as being "secured to" another element, it can be directly on the other element or intervening elements may also be present. When a component is referred to as being "connected" to another component, it can be directly connected to the other component or intervening components may also be present.
Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. The terminology used in the description of the invention herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. As used herein, the term "and/or" includes any and all combinations of one or more of the associated listed items.
Referring to fig. 1, fig. 1 is a schematic flow chart of an embodiment of a method for detecting water ripples of an image according to the present application, where the method may be applied to an unmanned aerial vehicle, and is specifically used to determine whether water ripples exist in an image captured by a capturing device configured on the unmanned aerial vehicle. The method specifically comprises the following steps:
s11: and acquiring an image obtained by shooting the image calibration device.
Specifically, the executing subject of the method of this embodiment may be a water wave detection apparatus, and further, the executing subject may be a processor of the water wave detection apparatus, where the processor may be a general-purpose or special-purpose processor, where the processor may be one or more processors, and is not limited specifically herein. The water ripple detection device can be configured on the unmanned aerial vehicle, and in the process of water ripple detection of images, the shooting device configured on the unmanned aerial vehicle can shoot the image calibration device and output the shot images, wherein the water ripple detection device can acquire the images.
The image calibration device may be any calibration device having an image calibration function, wherein the calibration device includes a plurality of calibration objects, and accordingly, the captured image includes an image object of the calibration object, which is an image area representing the calibration object in the image.
Further, the image calibration device may be a calibration board, and the calibration object is a calibration point on the calibration board. For example, as shown in fig. 2a, the image calibration apparatus is a checkerboard calibration board 20, and the calibration objects are corner points 201 on the checkerboard calibration board; or the image calibration device is a random point calibration board, as shown in fig. 2b, the calibration object is a random point on the calibration board 21, the random point may be a circle or other shape, the sizes of the calibration objects on the calibration board 20 may be the same, and in some cases, the calibration objects on the calibration board 20 include at least two sizes of calibration objects, i.e., at least two different sizes of calibration objects, which are schematically illustrated here as two different sizes of calibration objects 211 and 212 for convenience of description: the image calibration device comprises a carrier device and calibration objects of at least two size types arranged on the carrier device. Further, the image calibration device may further include a textured image disposed on the carrier device, the textured image serving as a background image of the calibration object. The carrier device may be a substrate. Optionally, the color of the outer ring and the color of the inner part of the outer ring of at least one of the at least two size types of calibration objects are different, for example, the outer ring is black, and the inner part of the outer ring is white; or the outer ring is white and the inner part of the outer ring is black. Optionally, the color of the central part of at least one of the at least two size types of calibration objects is different from the color of the central part of the other of the at least two size types of calibration objects. Compared with a chessboard pattern calibration board, the matching between the angular points and the image objects in the image can be realized by taking a picture integrally, and the random point calibration board can realize the matching between the random points and the image objects in the image by partially taking the picture of the calibration board because the random point distribution around each random point can be identified uniquely.
S12: an image object of a target object in the image is detected.
Specifically, after acquiring an image obtained by shooting the image calibration device, the moire detection device identifies an image object of the calibration object from the image, wherein the image object is an image area of the shot calibration object in the image. Since the calibration object on the image calibration device is an obvious object, the ripple detection device can identify the image object of the calibration object from the image according to the characteristic of the calibration object. For example, for a checkerboard calibration board, the water ripple detection apparatus may extract the corner points in the image by using a corner point extraction algorithm, and for a random point calibration board, the water ripple detection apparatus may extract the random points in the image by using a dot extraction (blob detector) algorithm. The dot extraction algorithm has higher precision than the checkerboard corner algorithm, so that the detection precision of the image object can be improved by shooting in S11 by adopting an image calibration device with a circular calibration object.
For convenience of explanation, taking the image calibration device as a checkerboard calibration board as an example, as shown in fig. 3, the moire detection device acquires an image 301 captured on the checkerboard calibration board, and then may extract a plurality of corner points 302 from the image 301.
S13: and matching the detected image object of the calibration object with the calibration object in the image calibration device.
Specifically, after detecting an image object of the detected image target, a matching relationship between the detected image object and the calibration object on the image calibration device needs to be established, that is, it is determined which calibration object on the image calibration device corresponds to the detected image object. For example, as shown in fig. 4, for an image captured by a checkerboard calibration board 401, a plurality of corner points 402 may be extracted from the image, each of the corner points 402 may be matched with a corner point in the checkerboard calibration board 401, and after the matching is completed, each of the corner points 402 establishes a one-to-one correspondence relationship with the checkerboard calibration board 401.
In some embodiments, the matching the detected image object of the calibration object with the calibration object in the image calibration device includes: and determining the position characteristic parameters of the detected image object according to the position of the image object in the image, and matching the detected image object of the calibration object with the calibration object in the image device according to the determined position characteristic parameters and the pre-stored position characteristic parameters of the calibration object. In addition, the moire detection device may prestore a position characteristic parameter of a calibration object in the image calibration device, where the position characteristic parameter may represent a position relationship of a certain image object or the calibration object with respect to one or more other image objects and the calibration object, respectively, and specifically, the position characteristic parameter may be a characteristic vector, and the detected image object of the calibration object is matched with the calibration object in the image device according to the determined position characteristic parameter and the prestored position characteristic parameter of the calibration object. Alternatively, when the position characteristic parameter of the image object is the same as or similar to a pre-stored position characteristic parameter of one of the calibration objects, it may be determined that the image object and the calibration object are matched.
S14: and determining whether the water ripple exists in the image according to the position of the image object in the image and the position of the calibration object matched with the image object in the image calibration device.
In particular, the moire detection device may determine a position of the image object in the image, i.e. where the image object is located in the image, wherein the position of the image object in the image may be coordinates of the image object in an image coordinate system. In addition, the moire detection device may determine a position of a calibration object matching the image object in the image calibration device, and for convenience of description, the calibration object matching the image object may be simply referred to as a target calibration object. The position of the calibration object in the image calibration device may be pre-stored in the ripple detection device, and after the target calibration object matching the image object is determined, the position of the target calibration object on the image calibration device may be obtained from the pre-stored position of the calibration object in the image calibration device. For example, when the image calibration device is a checkerboard calibration board, the position of the image object in the image may be the coordinates of the image area of the corner point in the image coordinate system, and the position of the target calibration object on the image calibration device may be the position of the corner point on the checkerboard calibration board.
After the position of the image object in the image and the position of the target calibration object on the image calibration device are obtained, whether the water ripple exists in the image can be determined according to the position of the image object in the image and the position of the target calibration object on the image calibration device.
In the embodiment, the image object of the calibration object is obtained by detecting the image shot by the image calibration device, and whether the water ripple exists in the image is determined according to the position of the image object in the image and the position of the corresponding matched calibration object in the image calibration device, so that the intelligent detection of the water ripple of the image is realized, the manual detection is not needed, the detection efficiency is further improved, the situation of false detection or missed detection can be reduced by the intelligent detection mode, and the detection accuracy is improved.
Referring to fig. 5, fig. 5 is a flowchart illustrating a step S14 in another embodiment of the method for detecting water ripple of an image according to the present application. In this embodiment, the geometric parameters of the image object and the geometric parameters of the calibration object matched with the image object are used to determine whether the calibration object on the image calibration device and the image object corresponding to the calibration object satisfy projective invariance, and further determine whether the image has water ripples. Specifically, compared with the embodiment shown in fig. 1, S14 of the method of the present embodiment further includes the following sub-steps:
s541: the geometric parameters of the image object are determined according to the position of the image object in the image.
Specifically, the image calibration device is shot, that is, a calibration object on the image device is projectively transformed onto a plane where an image sensor of the shooting device is located. After the position of the image object in the image is determined, the geometric parameters of the image object can be determined according to the position, wherein the geometric parameters comprise a projective invariant parameter, and the projective invariant parameter is a characteristic of projective transformation and refers to a parameter that the image is invariant after any projective transformation.
In some embodiments, the geometric parameter may include an intersection ratio parameter, and further, the projection invariant parameter includes an intersection ratio parameter. Step S13 includes the following substeps: selecting at least one group of image objects from the detected image objects, wherein the group of image objects comprises 5 image objects, and determining a calibration object in at least one group of image calibration devices matched with the at least one group of image objects. The determining the cross-ratio parameter of the image object according to the position of the image object in the image comprises: and determining the cross ratio parameter of each group of image objects according to the position of each group of image objects in the at least one group in the image. Specifically, as shown in fig. 6, 5 image objects A, B, C, D and E may be selected from the detected number of image objects to form a set of image objects, with at least one different image object between different sets of image objects. The number of groups of the selected image objects can be adjusted according to actual requirements, and generally, in an application scene requiring high detection accuracy, the number of the selected groups is large, and the selected image objects are distributed in different areas of the image.
Further, the determining the cross-ratio parameter of each group of image objects according to the position of each group of image objects in the image in the at least one group may comprise: determining the areas of 4 crossed triangles corresponding to each group of image objects according to the position of each group of image objects in the image; and determining the intersection ratio parameter of each group of image objects according to the area of the 4 intersection triangles corresponding to each group of image objects. With continued reference to FIG. 6, a set of image objects comprises 5 image objects A, B, C, D, E, 4 intersecting triangles are formed from the 5 image objects, and the area of the 4 intersecting triangles is used to obtain an intersection ratio parameter S for the set of image objectsΔABCSΔADE/SΔABDSΔACE。
Optionally, to ensure the accuracy of the cross-ratio parameter determined in this step, it may be determined whether the area of the cross triangle of each set of calibration objects meets the area requirement, and when it is determined that the area requirement is met, the area of the cross triangle is used to determine the cross-ratio parameter. Alternatively, it may be determined whether the cross ratio parameter obtained according to the area of the triangle meets the cross ratio requirement, and the cross ratio parameter is determined to be selected when the cross ratio requirement is met.
Specifically, after the areas of the 4 intersecting triangles corresponding to each group of image objects are determined, it is determined whether each of the areas of the 4 intersecting triangles corresponding to each group is greater than or equal to a preset area threshold value; the determining the intersection ratio parameter of each group of image objects according to the area of the 4 intersecting triangles corresponding to each group of images comprises: and when the area of each group of corresponding 4 crossed triangles is larger than or equal to a preset area threshold value, determining the intersection ratio parameter of each group of image objects according to the area of each group of images corresponding to 4 crossed triangles. And when one or more areas of each group of the corresponding 4 crossed triangles are smaller than a preset area threshold value, reselecting a group of image objects, and determining the cross ratio parameters of the group of image objects by using the step.
In addition, after determining the cross-ratio parameter of the set of image objects, it can be determined whether the cross-ratio parameter is greater than a set cross-ratio threshold, if so, the cross-ratio parameter is determined to be selected for a subsequent step, otherwise, a set of image objects is reselected, and the cross-ratio parameter of the set of image objects is determined by using the step.
S542: the geometric parameters of the calibration object matched with the image object are determined according to the position of the calibration object matched with the image object in the image calibration device.
Specifically, after the position of the target calibration object in the image calibration device is determined, the geometric parameters of the target calibration object may be determined according to the position, and as described above, the geometric parameters include the projection invariant parameters.
In some embodiments, the geometric parameter may include an intersection parameter. Specifically, the determining the geometric parameter of the calibration object matched with the image object according to the position of the calibration object matched with the image object in the image calibration device includes: and determining the cross ratio parameter of each set of calibration objects according to the positions of the calibration objects in the at least one set of image calibration devices in the image calibration devices.
Specifically, similarly to the above manner of determining the cross ratio parameter of each group of image objects, the determining the cross ratio parameter of each group of calibration objects according to the position of the calibration object in the at least one group of image calibration devices at the image calibration device includes: and determining the area of 4 crossed triangles corresponding to each group of calibration objects according to the position of the calibration objects in the at least one group of image calibration devices in the image calibration devices, and determining the intersection ratio parameter of each group of calibration objects according to the area of 4 crossed triangles corresponding to each group of calibration objects. The specific method is similar to the determination method of the cross-ratio parameter of each group of image objects in the foregoing section, and is not described herein again.
S543: and determining whether water ripples exist in the image according to the geometric parameters of the image object and the geometric parameters of the calibration object matched with the image object.
Specifically, according to the projective principle, when the image has no water ripple, the geometric parameters of the image object and the geometric parameters of the calibration object matching the image object should be the same, so that whether the water ripple exists in the image can be determined according to the geometric parameters of the image object and the geometric parameters of the calibration object matching the image object. Furthermore, a geometric parameter difference value can be determined according to the geometric parameters of the image object and the geometric parameters of the calibration object matched with the image object, and then whether water ripples exist in the image or not is determined according to the difference value.
As mentioned above, the geometric parameters include cross-ratio parameters, and the step S543 includes: and determining whether water ripples exist in the image according to the difference between the cross ratio parameter of each group of image objects in the at least one group and the cross ratio parameter of the corresponding group of calibration objects. Specifically, based on the principle of invariance of projective transformation, because the calibration object in the image calibration device and the image corresponding to the calibration object are in projective transformation relationship, when the image has no water ripple, the cross ratio parameter of the image object in the image is equal to or has a smaller error than the cross ratio parameter of the corresponding calibration object on the image calibration device, and when the image has water ripple, the cross ratio parameter of the image object in the image area having water ripple in the image has a larger difference with the cross ratio parameter of the corresponding calibration object on the image calibration device due to distortion of the image, so that whether the image has water ripple can be determined by comparing the cross ratio parameters of the image object and the calibration object.
Further, when the number of sets of image objects selected in step S13 is at least two, the determining whether water ripples exist in the image according to the difference between the cross ratio parameter of each set of image objects in the at least one set and the cross ratio parameter of the corresponding set of calibration objects may include: and determining an average cross ratio difference value according to the difference value, determining that the image has water ripples when the average cross ratio difference value is greater than or equal to a preset threshold value, and determining that the image does not have water ripples when the average cross ratio difference value is less than the preset threshold value. For example, the number of groups of image objects is 10, the cross ratio parameters of 10 groups of image objects and the cross ratio parameters of 10 corresponding groups of calibration objects are respectively calculated in the above steps S541 and S542, the difference between the cross ratio parameter of each group of image objects and the cross ratio parameter of the corresponding calibration object is calculated to obtain 10 differences, the 10 differences are arithmetically averaged to obtain an average cross ratio difference, and the average cross ratio difference is compared with a preset threshold to determine whether the image has the water ripple.
Referring to fig. 7, fig. 7 is a flowchart illustrating a step S14 in a further embodiment of the method for detecting water ripple of an image according to the present application. The embodiment is based on the geometric distance error, that is, the projective position of the image object on the plane where the calibration object is located in the image calibration device is utilized, and whether the projective position of the image object and the position of the corresponding calibration object in the image calibration device meet the position consistency is determined, so that whether water ripples exist in the image is determined. Specifically, compared with the embodiment shown in fig. 1, S14 of the method of the present embodiment further includes the following sub-steps:
s741: and projectively transforming the position of the image object in the image to a plane where the calibration object is located in the image calibration device to obtain the projective position of the image object.
Specifically, as described above, when there is no water ripple in the image, there is a projective transformation relationship between the image object in the image and the calibration object in the image calibration apparatus, that is, there is a projective transformation relationship between the position of the image object in the image and the position of the calibration object in the image calibration apparatus corresponding to the image. The water ripple detection device can project the position of the image object in the image to the plane where the calibration object in the image calibration device is located, and further, the position of the image object in the image is projected to the plane where the calibration object matched with the image object is located in the image calibration device according to the position of the image object in the image and the position of the calibration object matched with the image object in the image calibration device to obtain the projection position of the image object, so that the projection position of the image object on the image calibration device can be obtained.
In some embodiments, a projective parameter between an image object in a first predetermined area of the image and a calibration object matched with the image object may be determined, and the projective parameter may be used to determine a projective position of the image object. Specifically, the projective parameters are determined according to the position of the image object of the first preset area in the image and the position of the calibration object matched with the image object of the first preset area in the image calibration device, wherein the projective parameters may be transformation parameters for projectively transforming the position of the image object of the first area in the image into the position of the calibration object matched with the image object in the image calibration device, and the projective parameters may be calculated on the premise that the position of the image object of the first area in the image and the position of the calibration object matched with the image object of the first preset area in the image calibration device are known, wherein the projective parameters may be a homography matrix. And then, the position of the image object in the image can be projectively transformed to the plane of the calibration object in the image calibration device according to the projective parameters to obtain the projective position of the image object.
In some embodiments, the projective transformation of the position of the image object in the image to the plane of the calibration object in the image calibration apparatus to obtain the projective position of the image object according to the position of the image object in the image and the position of the calibration object matching with the image object in the image calibration apparatus includes: and projectively transforming the position of the image object in the second preset area in the image into the plane of the calibration object in the image calibration device according to the position of the image object in the image and the position of the calibration object matched with the image object in the image calibration device to obtain the projective position of the image object. Specifically, the water ripple detection device selects an image of a second preset region from the image, determines a projection position of an image object of the second preset region, and then determines whether water ripple exists in the image according to the projection position of the image object in the second preset region.
Further, the step of calibrating the image calibration device according to the position of the image object in the image and the position of the calibration object matched with the image object comprises: determining a projective parameter according to the position of the image object in the first preset area in the image and the position of the calibration object matched with the image object in the first preset area in the image calibration device, and projectively transforming the position of the image object in the second preset area in the image into a plane where the calibration object is located in the image calibration device according to the projective parameter to obtain the projective position of the image object.
Alternatively, the first preset region and the second preset region may be provided as regions that do not completely overlap. For example, the first preset region is a region near one or more of the four corners of the image. In practical situations, the probability of water ripples existing at four corners of an image is low, and the water ripples generally exist in the center of the image or an area close to the center of the image, so that the position of an image object in a first preset area in the image and the position of a calibration object matched with the image object in the first preset area in the image calibration device are selected to calculate the photographic parameters, so that the calculation accuracy of the photographic parameters can be effectively improved. Optionally, the center of the second preset area is the center of the image, that is, the second preset area is located in the center area of the image.
S742: and determining whether water ripples exist in the image according to the projection position of the image object.
Specifically, when the image has no water ripples, the projection position of the image object and the position of the calibration object matching the image object in the image calibration device should be the same or have only a small error. When the image has water ripples, the projective position of the image object and the position of a calibration object matched with the image object in the image calibration device should have a large error, and whether the water ripples exist in the image is determined according to the projective position of the image object and the position of the calibration object matched with the image object in the image calibration device.
Optionally, when the image object at the shooting position is in the second preset area, determining whether water ripple exists in the image according to the shooting position of the image object and the position of the calibration object matched with the image object in the second preset area of the image in the image calibration device.
Alternatively, whether or not the image has water ripples may be determined by comparing the obtained projection position with the position of the corresponding calibration object. For example, a difference between the projection position of the image object and the position of a calibration object matching the image object in the image calibration device is determined; and determining whether water ripples exist in the image according to the difference value.
Optionally, the determining whether the water ripple exists in the image according to the difference may specifically include: and determining the distance between the projection position of the image object and the position of a calibration object matched with the image object in the image calibration device according to the difference, and determining whether water ripples exist in the image or not based on the distance. For example, when the distance is higher than a preset distance threshold value by a preset distance threshold value, it is determined that the image has water ripples, otherwise, it is determined that the image does not have water ripples. When a plurality of image objects at the projective positions are obtained, a plurality of distances can be correspondingly obtained, at the moment, the distances are sequenced, and a median value is selected to be compared with a preset distance threshold value; or, the distances are respectively compared with a preset distance threshold, if only one distance is higher than the preset distance threshold, the image is determined to have the water ripple, and if all the distances are lower than the preset distance threshold, the image is determined to have no water ripple.
Optionally, to improve the accuracy of water ripple detection, a distance vector cluster between the position of each calibration object in the image calibration device and the corresponding projection position may be determined according to the projection position of the image object in each frame of image in multiple frames of images captured by the image calibration device, and whether water ripple exists in the image may be determined according to an area surrounded by the distance vector cluster.
When the number of the image objects for acquiring the projection position in each frame of image is multiple, the number of the corresponding calibration objects is also multiple, so that a distance vector cluster of the multiple calibration objects can be obtained. At this time, the average value or the median value of the plurality of distance vector clusters may be selected to be compared with the preset area threshold. For example, the determining whether the image has the water ripple according to the area surrounded by the distance vector cluster may specifically include: determining the average value of the area surrounded by the distance vector clusters; when the average value of the areas is larger than a preset area threshold value, determining that water ripples exist in the image, otherwise, determining that the water ripples do not exist in the image; or determining a median value in an area surrounded by a plurality of distance vector clusters; and when the median is larger than a preset area threshold value, determining that the image has water ripple, otherwise, determining that the image does not have water ripple.
For ease of understanding, the following examples are given.
In an application scene, a video shot by an image collector of the unmanned aerial vehicle is acquired, and water ripple detection is carried out on the video. Firstly, extracting a plurality of frames of images on a video, supposing that three frames of images shot by an image calibration device are obtained, after matching between an image object on each frame of image and a calibration object on the image calibration device is completed, selecting 5 image objects which are closest to each of four corners of the image on each frame of image as image objects of a first preset area, and according to the position X 'of the image object of the first preset area on the image in each frame of image'1And the position X of the calibration object corresponding to the image object of the first preset area1Calculating the projection parameter H corresponding to each frame image, specifically, according to equation X1＝HX’1Namely, the projective parameters H can be calculated by fitting. Then, the position X 'of the image object in the second preset area in the image in each frame of image can be determined according to the corresponding shooting parameter H of each frame'2Projective transformation to the plane of the target object in the image calibration device to obtain projective position X'2＝HX’2. The second preset area can be within a quarter of the area of the center of the image.
As shown in fig. 8, for convenience of description, taking an example of a calibration object matching an image object in the second preset region, if the position of the calibration object in the image calibration device is position a, the projection positions B, C and D of the image object matching the calibration object a in the three frames of images are obtained in the manner as described above for the one calibration object, so as to obtain vector clusters AB, AC and AD, and further, the vector can be determinedArea S surrounded by measuring clustersΔBCDWherein the area enclosed by the vector clusters can be realized by a convex hull algorithm, and the current area S isΔBCDWhen the size is larger, the water ripple exists in the image, that is, whether the water ripple exists in the image can be determined according to the area surrounded by the distance vector clusters. Further, by adopting the method, the area surrounded by the plurality of distance vector clusters can be obtained, the areas surrounded by the plurality of distance vector clusters can be sequenced, the area surrounded by the distance vector cluster at the intermediate value is selected to be compared with the preset area threshold, and if the area is larger than the preset area threshold, the video is determined to have water ripples.
Referring to fig. 9, fig. 9 is a schematic structural diagram of an embodiment of a water ripple detection device according to the present application. In this embodiment, the ripple detection device 900 includes a processor 91 and a memory 92 connected to each other.
Memory 901 may include both read-only memory and random access memory, and provides instructions and data to processor 902. A portion of memory 901 may also include non-volatile random access memory.
The Processor 902 may be a Central Processing Unit (CPU), or other general-purpose Processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), an off-the-shelf Programmable Gate Array (FPGA) or other Programmable logic device, discrete Gate or transistor logic device, discrete hardware component, or the like. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
The memory 901 is used to store program instructions.
A processor 902, invoking the program instructions, that when executed, is configured to:
acquiring an image shot by an image calibration device, wherein the image calibration device comprises a plurality of calibration objects;
detecting an image object of a labeled object in the image;
matching the detected image object of the calibration object with a calibration object in the image calibration device;
and determining whether water ripples exist in the image according to the position of the image object in the image and the position of a calibration object matched with the image object in the image calibration device.
In some embodiments, the processor 902, when determining whether water ripple exists in the image according to the position of the image object in the image and the position of the calibration object matching the image object in the image calibration device, is specifically configured to: determining the geometric parameters of the image object according to the position of the image object in the image; determining the geometric parameters of the calibration object matched with the image object according to the position of the calibration object matched with the image object in the image calibration device; and determining whether water ripples exist in the image according to the geometric parameters of the image object and the geometric parameters of the calibration object matched with the image object.
Further, when determining whether the image has water ripples according to the geometric parameters of the image object and the geometric parameters of the calibration object matched with the image object, the processor 902 may be specifically configured to: determining a geometric parameter difference value according to the geometric parameters of the image object and the geometric parameters of the calibration object matched with the image object; and determining whether water ripples exist in the image according to the difference value.
Wherein the geometric parameter may comprise an intersection parameter.
In some embodiments, when matching the detected image object of the calibration object with the calibration object in the image calibration apparatus, the processor 902 is specifically configured to: selecting at least one group of image objects from the detected image objects, wherein the group of image objects comprises 5 image objects; determining a calibration object in at least one set of image calibration devices that matches the at least one set of image objects;
the processor 902, when determining the geometric parameter of the image object according to the position of the image object in the image, is specifically configured to: determining an intersection ratio parameter of each group of image objects according to the position of each group of image objects in the at least one group in the image;
the processor 902, when determining the geometric parameter of the calibration object matching the image object according to the position of the calibration object matching the image object in the image calibration apparatus, is specifically configured to: determining the cross ratio parameter of each group of calibration objects according to the positions of the calibration objects in the at least one group of image calibration devices in the image calibration devices;
when determining the geometric parameter difference according to the geometric parameter of the image object and the geometric parameter of the calibration object matched with the image object, the processor 902 is specifically configured to: determining a difference value between the cross ratio parameter of each group of image objects in the at least one group and the cross ratio parameter of the corresponding group of calibration objects;
when determining whether the image has water ripples according to the difference, the processor 902 is specifically configured to: and determining whether water ripples exist in the image according to the difference between the cross ratio parameter of each group of image objects in the at least one group and the cross ratio parameter of the corresponding group of calibration objects.
Further, the processor 902, when determining whether there is water ripple in the image according to the difference between the cross ratio parameter of each image object in the at least one set and the cross ratio parameter of the corresponding calibration object, may be specifically configured to: determining an average cross ratio difference value according to the difference value; and when the average cross ratio difference value is larger than or equal to a preset threshold value, determining that water ripples exist in the image.
In some embodiments, the processor 902, when determining the cross-ratio parameter for each group of image objects according to the position of each group of image objects in the image in the at least one group, is specifically configured to: determining the areas of 4 crossed triangles corresponding to each group of image objects according to the position of each group of image objects in the image; determining the intersection ratio parameter of each group of image objects according to the area of 4 intersection triangles corresponding to each group of image objects;
the processor 902, when determining the cross-ratio parameter of each set of calibration objects according to the positions of the calibration objects in the at least one set of image calibration devices in the image calibration devices, is specifically configured to: determining the area of 4 crossed triangles corresponding to each group of calibration objects according to the positions of the calibration objects in the at least one group of image calibration devices in the image calibration devices; and determining the cross ratio parameter of each group of calibration objects according to the area of the 4 crossed triangles corresponding to each group of calibration objects.
Further, the processor 902 is also operable to: determining whether each of the areas of each set of corresponding 4 triangles is greater than or equal to a preset area threshold;
when determining the intersection ratio parameter of each group of image objects according to the area of the 4 intersecting triangles corresponding to each group of images, the processor 902 may be specifically configured to: and when the area of each group of corresponding 4 triangles is larger than or equal to a preset area threshold value, determining an intersection ratio parameter of each group of image objects according to the area of each group of images corresponding to 4 intersecting triangles.
In some embodiments, the processor 902, when determining whether water ripple exists in the image according to the position of the image object in the image and the position of the calibration object matching the image object in the image calibration device, is specifically configured to: according to the position of the image object in the image and the position of a calibration object matched with the image object in the image calibration device, projectively transforming the position of the image object in the image into a plane where the calibration object is located in the image calibration device to obtain the projective position of the image object; and determining whether water ripples exist in the image according to the projection position of the image object.
Further, the processor 902, when projectively transforming the position of the image object in the image to the plane of the calibration object in the image calibration apparatus according to the position of the image object in the image and the position of the calibration object matching with the image object in the image calibration apparatus to obtain the projective position of the image object, is specifically configured to: determining a projective parameter according to the position of the image object in the first preset area in the image and the position of the calibration object matched with the image object in the first preset area in the image calibration device; and projectively transforming the position of the image object in the image into a plane where the calibration object is located in the image calibration device according to the projective parameters to obtain the projective position of the image object.
The first preset area may be an area near one or more of four corners of the image.
In some embodiments, the processor 902 is specifically configured to, when projectively transforming the position of the image object in the image to the plane of the calibration object in the image calibration apparatus to obtain the projective position of the image object according to the position of the image object in the image and the position of the calibration object matching with the image object in the image calibration apparatus: projectively transforming the position of the image object in the second preset area in the image into a plane where the calibration object in the image calibration device is located according to the position of the image object in the image and the position of the calibration object matched with the image object in the image calibration device to obtain the projective position of the image object;
when determining whether the water ripple exists in the image according to the projection position of the image object, the processor 902 is specifically configured to: and determining whether water ripples exist in the image according to the projection position of the image object in the second preset area.
Wherein, the center of the second preset area may be the center of the image.
In some embodiments, the processor 902, when determining whether a water ripple is present in the image according to the projection position of the image object, is specifically configured to: determining a difference value between the projection position of the image object and the position of a calibration object matched with the image object in the image calibration device; and determining whether water ripples exist in the image according to the difference value.
Further, the processor 902, when determining whether there is a water ripple in the image according to the difference value, may specifically be configured to: determining the distance between the projection position of the image object and the position of a calibration object matched with the image object in the image calibration device according to the difference value; determining whether water ripples exist in the image based on the distance.
In some embodiments, the processor 902, when determining whether a water ripple is present in the image according to the projection position of the image object, is specifically configured to: determining a distance vector cluster between the position of each calibration object in the image calibration device and the corresponding projection position according to the projection position of the image object in each frame of image in a plurality of frames of images shot by the image calibration device; and determining whether water ripples exist in the image according to the area surrounded by the distance vector clusters.
In some embodiments, the processor 902, when determining whether there is a water ripple in the image according to the area enclosed by the distance vector cluster, is specifically configured to: determining the average value of the area surrounded by the distance vector clusters; and when the average value of the areas is larger than a preset area threshold value, determining that water ripples exist in the image.
In some embodiments, the processor 902, when determining whether there is a water ripple in the image according to the area enclosed by the distance vector cluster, is specifically configured to: determining a median value in an area surrounded by the plurality of distance vector clusters; and when the median value is larger than a preset area threshold value, determining that water ripples exist in the image.
In some embodiments, the projection parameters are homography matrices.
In some embodiments, when matching the detected image object of the calibration object with the calibration object in the image apparatus, the processor 902 is specifically configured to: determining characteristic parameters of the detected image object; and matching the detected image object of the calibration object with the calibration object in the image device according to the determined characteristic parameter and the characteristic parameter of the pre-stored calibration object in the image calibration device.
The apparatus of this embodiment may be configured to implement the technical solution of the method embodiment of the present application, and the implementation principle and the technical effect are similar, which are not described herein again.
Referring to fig. 10, fig. 10 is a schematic structural diagram of an embodiment of a water ripple detection system according to the present application. The detection system 1000 includes a photographing device 1001 and a ripple detection device 1002 connected to each other. The imaging device 1001 is used for imaging the image calibration device to obtain an image. The water ripple detection device 1002 is the water ripple detection device described in the above embodiments, and is not described herein again.
Please refer to fig. 11, fig. 11 is a schematic structural diagram of an embodiment of the unmanned aerial vehicle according to the present application. In this embodiment, the unmanned aerial vehicle includes a ripple detection system, wherein the ripple detection system may specifically include a ripple detection device 1101 and a camera 1002 as described in the above system embodiments.
Further, the unmanned aerial vehicle may further include a carrying device 1103, wherein the carrying device 1103 is used for carrying the photographing device 1002. In some embodiments, the drone is a rotorcraft, and the camera 1002 may be the main camera of the drone. The carrying device 1103 may be a two-axis or three-axis pan/tilt head.
Optionally, the unmanned aerial vehicle is further provided with functional circuits such as a visual sensor and an inertia measuring device according to actual needs.
Referring to fig. 12, fig. 12 is a schematic structural diagram of a memory device according to an embodiment of the present application. In this embodiment, the storage device 1200 stores a program instruction 1201, and when the program instruction 1201 runs on a processor, the technical solution of the above method embodiment of the present application is executed.
The storage device 1200 may be a medium that can store computer instructions, such as a usb disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk, or an optical disk, or may also be a server that stores the program instructions, and the server may send the stored program instructions to other devices for operation, or may self-operate the stored program instructions.
According to the scheme, the image object of the calibration object is obtained by detecting the image shot by the image calibration device, and whether the water ripple exists in the image is determined according to the position of the image object in the image and the position of the corresponding matched calibration object in the image calibration device, so that the intelligent detection of the water ripple of the image is realized, the manual detection is not needed, the detection efficiency is improved, and the false detection rate is reduced.
In the several embodiments provided in the present application, it should be understood that the disclosed method and apparatus may be implemented in other ways. For example, the above-described apparatus embodiments are merely illustrative, and for example, a division of a module or a unit is merely a logical division, and an actual implementation may have another division, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may be in an electrical, mechanical or other form.
Units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the embodiment.
In addition, functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.
The integrated unit, if implemented in the form of a software functional unit and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present application may be substantially implemented or contributed to by the prior art, or all or part of the technical solution may be embodied in a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, a network device, or the like) or a processor (processor) to execute all or part of the steps of the method according to the embodiments of the present application. And the aforementioned storage medium includes: a U disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program instructions.
The above embodiments are merely examples and are not intended to limit the scope of the present disclosure, and all modifications, equivalents, and flow charts using the contents of the specification and drawings of the present disclosure or those directly or indirectly applied to other related technical fields are intended to be included in the scope of the present disclosure.

Claims (43)

  1. A method for detecting water ripple of an image, comprising:
    acquiring an image shot by an image calibration device, wherein the image calibration device comprises a plurality of calibration objects;
    detecting an image object of a labeled object in the image;
    matching the detected image object of the calibration object with a calibration object in the image calibration device;
    and determining whether water ripples exist in the image according to the position of the image object in the image and the position of a calibration object matched with the image object in the image calibration device.
  2. The method of claim 1,
    the determining whether the water ripple exists in the image according to the position of the image object in the image and the position of the calibration object matched with the image object in the image calibration device comprises:
    determining the geometric parameters of the image object according to the position of the image object in the image;
    determining the geometric parameters of the calibration object matched with the image object according to the position of the calibration object matched with the image object in the image calibration device;
    and determining whether water ripples exist in the image according to the geometric parameters of the image object and the geometric parameters of the calibration object matched with the image object.
  3. The method of claim 2,
    the determining whether the water ripple exists in the image according to the geometric parameters of the image object and the geometric parameters of the calibration object matched with the image object comprises:
    determining a geometric parameter difference value according to the geometric parameters of the image object and the geometric parameters of the calibration object matched with the image object;
    and determining whether water ripples exist in the image according to the difference value.
  4. The method of claim 3,
    the geometric parameters include cross-ratio parameters.
  5. The method of claim 4,
    the matching of the detected image object of the calibration object with the calibration object in the image calibration device comprises:
    selecting at least one group of image objects from the detected image objects, wherein the group of image objects comprises 5 image objects;
    determining a calibration object in at least one set of image calibration devices that matches the at least one set of image objects;
    said determining a geometric parameter of an image object depending on the position of said image object in the image comprises:
    determining an intersection ratio parameter of each group of image objects according to the position of each group of image objects in the at least one group in the image;
    the determining the geometric parameters of the calibration object matched with the image object according to the position of the calibration object matched with the image object in the image calibration device comprises the following steps:
    determining the cross ratio parameter of each group of calibration objects according to the positions of the calibration objects in the at least one group of image calibration devices in the image calibration devices;
    the determining a geometric parameter difference value according to the geometric parameter of the image object and the geometric parameter of the calibration object matched with the image object includes:
    determining a difference value between the cross ratio parameter of each group of image objects in the at least one group and the cross ratio parameter of the corresponding group of calibration objects;
    the determining whether water ripples exist in the image according to the difference value comprises:
    and determining whether water ripples exist in the image according to the difference between the cross ratio parameter of each group of image objects in the at least one group and the cross ratio parameter of the corresponding group of calibration objects.
  6. The method of claim 5, wherein determining whether water ripples exist in the image according to the difference between the cross-ratio parameter of each set of image objects in the at least one set and the cross-ratio parameter of the corresponding set of calibration objects comprises:
    determining an average cross ratio difference value according to the difference value;
    and when the average cross ratio difference value is larger than or equal to a preset threshold value, determining that water ripples exist in the image.
  7. The method according to claim 5 or 6,
    the determining the cross-ratio parameter of each group of image objects according to the position of each group of image objects in the image in the at least one group comprises:
    determining the areas of 4 crossed triangles corresponding to each group of image objects according to the position of each group of image objects in the image;
    determining the intersection ratio parameter of each group of image objects according to the area of 4 intersection triangles corresponding to each group of image objects;
    the determining the cross-ratio parameter of each set of calibration objects according to the positions of the calibration objects in the at least one set of image calibration devices at the image calibration devices comprises:
    determining the area of 4 crossed triangles corresponding to each group of calibration objects according to the positions of the calibration objects in the at least one group of image calibration devices in the image calibration devices;
    and determining the cross ratio parameter of each group of calibration objects according to the area of the 4 crossed triangles corresponding to each group of calibration objects.
  8. The method of claim 7, further comprising:
    determining whether each of the areas of each set of corresponding 4 triangles is greater than or equal to a preset area threshold;
    the determining the intersection ratio parameter of each group of image objects according to the area of the 4 intersecting triangles corresponding to each group of images comprises:
    and when the area of each group of corresponding 4 triangles is larger than or equal to a preset area threshold value, determining an intersection ratio parameter of each group of image objects according to the area of each group of images corresponding to 4 intersecting triangles.
  9. The method of claim 1,
    the determining whether the water ripple exists in the image according to the position of the image object in the image and the position of the calibration object matched with the image object in the image calibration device comprises:
    according to the position of the image object in the image and the position of a calibration object matched with the image object in the image calibration device, projectively transforming the position of the image object in the image into a plane where the calibration object is located in the image calibration device to obtain the projective position of the image object;
    and determining whether water ripples exist in the image according to the projection position of the image object.
  10. The method of claim 9,
    the projective transformation of the position of the image object in the image to the plane of the calibration object in the image calibration device according to the position of the image object in the image and the position of the calibration object matched with the image object in the image calibration device to obtain the projective position of the image object comprises:
    determining a projective parameter according to the position of the image object in the first preset area in the image and the position of the calibration object matched with the image object in the first preset area in the image calibration device;
    and projectively transforming the position of the image object in the image into a plane where the calibration object is located in the image calibration device according to the projective parameters to obtain the projective position of the image object.
  11. The method of claim 10,
    the first preset region is a region near one or more of the four corners of the image.
  12. The method according to any one of claims 9 to 11,
    the projective transformation of the position of the image object in the image to the plane of the calibration object in the image calibration device according to the position of the image object in the image and the position of the calibration object matched with the image object in the image calibration device to obtain the projective position of the image object comprises:
    projectively transforming the position of the image object in the second preset area in the image into a plane where the calibration object in the image calibration device is located according to the position of the image object in the image and the position of the calibration object matched with the image object in the image calibration device to obtain the projective position of the image object;
    the determining whether water ripples exist in the image according to the projection position of the image object comprises:
    and determining whether water ripples exist in the image according to the projection position of the image object in the second preset area.
  13. The method of claim 12,
    the center of the second preset area is the center of the image.
  14. The method according to any one of claims 9 to 13,
    the determining whether water ripples exist in the image according to the projection position of the image object comprises:
    determining a difference value between the projection position of the image object and the position of a calibration object matched with the image object in the image calibration device;
    and determining whether water ripples exist in the image according to the difference value.
  15. The method of claim 14, wherein determining whether water ripples exist in the image according to the difference comprises:
    determining the distance between the projection position of the image object and the position of a calibration object matched with the image object in the image calibration device according to the difference value;
    determining whether water ripples exist in the image based on the distance.
  16. The method according to any one of claims 9 to 13,
    the determining whether water ripples exist in the image according to the projection position of the image object comprises:
    determining a distance vector cluster between the position of each calibration object in the image calibration device and the corresponding projection position according to the projection position of the image object in each frame of image in a plurality of frames of images shot by the image calibration device;
    and determining whether water ripples exist in the image according to the area surrounded by the distance vector clusters.
  17. The method of claim 16,
    the determining whether the water ripple exists in the image according to the area surrounded by the distance vector cluster comprises:
    determining the average value of the area surrounded by the distance vector clusters;
    and when the average value of the areas is larger than a preset area threshold value, determining that water ripples exist in the image.
  18. The method of claim 16,
    the determining whether the water ripple exists in the image according to the area surrounded by the distance vector cluster comprises:
    determining a median value in an area surrounded by the plurality of distance vector clusters;
    and when the median value is larger than a preset area threshold value, determining that water ripples exist in the image.
  19. The method of claim 10, wherein the projection parameters are homography matrices.
  20. The method of claim 1,
    the matching of the detected image object of the calibration object with the calibration object in the image device includes:
    determining a position characteristic parameter of the detected image object according to the position of the image object in the image;
    and matching the detected image object of the calibration object with the calibration object in the image device according to the determined position characteristic parameter and the pre-stored position characteristic parameter of the calibration object.
  21. A water ripple detection device is characterized by comprising a processor and a memory, wherein,
    the memory to store program instructions;
    the processor executing the program instructions to:
    acquiring an image shot by an image calibration device, wherein the image calibration device comprises a plurality of calibration objects;
    detecting an image object of a labeled object in the image;
    matching the detected image object of the calibration object with a calibration object in the image calibration device;
    and determining whether water ripples exist in the image according to the position of the image object in the image and the position of a calibration object matched with the image object in the image calibration device.
  22. The apparatus of claim 21,
    when determining whether water ripples exist in the image according to the position of the image object in the image and the position of the calibration object matched with the image object in the image calibration device, the processor is specifically configured to:
    determining the geometric parameters of the image object according to the position of the image object in the image;
    determining the geometric parameters of the calibration object matched with the image object according to the position of the calibration object matched with the image object in the image calibration device;
    and determining whether water ripples exist in the image according to the geometric parameters of the image object and the geometric parameters of the calibration object matched with the image object.
  23. The apparatus of claim 22,
    when determining whether the image has water ripples according to the geometric parameters of the image object and the geometric parameters of the calibration object matched with the image object, the processor is specifically configured to:
    determining a geometric parameter difference value according to the geometric parameters of the image object and the geometric parameters of the calibration object matched with the image object;
    and determining whether water ripples exist in the image according to the difference value.
  24. The apparatus of claim 23,
    the geometric parameters include cross-ratio parameters.
  25. The apparatus of claim 24,
    when the processor matches the detected image object of the calibration object with the calibration object in the image calibration device, the processor is specifically configured to:
    selecting at least one group of image objects from the detected image objects, wherein the group of image objects comprises 5 image objects;
    determining a calibration object in at least one set of image calibration devices that matches the at least one set of image objects;
    when determining the geometric parameter of the image object according to the position of the image object in the image, the processor is specifically configured to:
    determining an intersection ratio parameter of each group of image objects according to the position of each group of image objects in the at least one group in the image;
    when determining the geometric parameter of the calibration object matching the image object according to the position of the calibration object matching the image object in the image calibration device, the processor is specifically configured to:
    determining the cross ratio parameter of each group of calibration objects according to the positions of the calibration objects in the at least one group of image calibration devices in the image calibration devices;
    when determining the geometric parameter difference according to the geometric parameter of the image object and the geometric parameter of the calibration object matched with the image object, the processor is specifically configured to:
    determining a difference value between the cross ratio parameter of each group of image objects in the at least one group and the cross ratio parameter of the corresponding group of calibration objects;
    when determining whether the image has water ripples according to the difference, the processor is specifically configured to:
    and determining whether water ripples exist in the image according to the difference between the cross ratio parameter of each group of image objects in the at least one group and the cross ratio parameter of the corresponding group of calibration objects.
  26. The apparatus of claim 25, wherein the processor, when determining whether water ripples exist in the image based on a difference between the cross-ratio parameter of each set of image objects in the at least one set and the cross-ratio parameter of the corresponding set of calibration objects, is specifically configured to:
    determining an average cross ratio difference value according to the difference value;
    and when the average cross ratio difference value is larger than or equal to a preset threshold value, determining that water ripples exist in the image.
  27. The apparatus of claim 25 or 26,
    the processor, when determining the cross-ratio parameter of each group of image objects according to the position of each group of image objects in the image, is specifically configured to:
    determining the areas of 4 crossed triangles corresponding to each group of image objects according to the position of each group of image objects in the image;
    determining the intersection ratio parameter of each group of image objects according to the area of 4 intersection triangles corresponding to each group of image objects;
    when determining the cross-ratio parameter of each set of calibration objects according to the positions of the calibration objects in the at least one set of image calibration devices in the image calibration device, the processor is specifically configured to:
    determining the area of 4 crossed triangles corresponding to each group of calibration objects according to the positions of the calibration objects in the at least one group of image calibration devices in the image calibration devices;
    and determining the cross ratio parameter of each group of calibration objects according to the area of the 4 crossed triangles corresponding to each group of calibration objects.
  28. The apparatus of claim 27, wherein the processor is further configured to:
    determining whether each of the areas of each set of corresponding 4 triangles is greater than or equal to a preset area threshold;
    when determining the intersection ratio parameter of each group of image objects according to the area of the 4 intersecting triangles corresponding to each group of images, the processor is specifically configured to:
    and when the area of each group of corresponding 4 triangles is larger than or equal to a preset area threshold value, determining an intersection ratio parameter of each group of image objects according to the area of each group of images corresponding to 4 intersecting triangles.
  29. The apparatus of claim 21,
    when determining whether water ripples exist in the image according to the position of the image object in the image and the position of the calibration object matched with the image object in the image calibration device, the processor is specifically configured to:
    according to the position of the image object in the image and the position of a calibration object matched with the image object in the image calibration device, projectively transforming the position of the image object in the image into a plane where the calibration object is located in the image calibration device to obtain the projective position of the image object;
    and determining whether water ripples exist in the image according to the projection position of the image object.
  30. The apparatus of claim 29,
    when the processor performs projective transformation on the position of the image object in the image to the plane of the calibration object in the image calibration device according to the position of the image object in the image and the position of the calibration object matched with the image object in the image calibration device to obtain the projective position of the image object, the processor is specifically configured to:
    determining a projective parameter according to the position of the image object in the first preset area in the image and the position of the calibration object matched with the image object in the first preset area in the image calibration device;
    and projectively transforming the position of the image object in the image into a plane where the calibration object is located in the image calibration device according to the projective parameters to obtain the projective position of the image object.
  31. The apparatus of claim 30,
    the first preset region is a region near one or more of the four corners of the image.
  32. The apparatus of any one of claims 29-31,
    when the processor performs projective transformation on the position of the image object in the image to the plane of the calibration object in the image calibration device according to the position of the image object in the image and the position of the calibration object matched with the image object in the image calibration device to obtain the projective position of the image object, the processor is specifically configured to:
    projectively transforming the position of the image object in the second preset area in the image into a plane where the calibration object in the image calibration device is located according to the position of the image object in the image and the position of the calibration object matched with the image object in the image calibration device to obtain the projective position of the image object;
    when determining whether water ripples exist in the image according to the projection position of the image object, the processor is specifically configured to:
    and determining whether water ripples exist in the image according to the projection position of the image object in the second preset area.
  33. The apparatus of claim 32,
    the center of the second preset area is the center of the image.
  34. The apparatus of any one of claims 29-33,
    when determining whether water ripples exist in the image according to the projection position of the image object, the processor is specifically configured to:
    determining a difference value between the projection position of the image object and the position of a calibration object matched with the image object in the image calibration device;
    and determining whether water ripples exist in the image according to the difference value.
  35. The apparatus of claim 34, wherein the processor, in determining whether water ripples exist in the image based on the difference, is specifically configured to:
    determining the distance between the projection position of the image object and the position of a calibration object matched with the image object in the image calibration device according to the difference value;
    determining whether water ripples exist in the image based on the distance.
  36. The apparatus of any one of claims 29-33,
    when determining whether water ripples exist in the image according to the projection position of the image object, the processor is specifically configured to:
    determining a distance vector cluster between the position of each calibration object in the image calibration device and the corresponding projection position according to the projection position of the image object in each frame of image in a plurality of frames of images shot by the image calibration device;
    and determining whether water ripples exist in the image according to the area surrounded by the distance vector clusters.
  37. The apparatus of claim 36,
    when determining whether the image has water ripples according to the area surrounded by the distance vector clusters, the processor is specifically configured to:
    determining the average value of the area surrounded by the distance vector clusters;
    and when the average value of the areas is larger than a preset area threshold value, determining that water ripples exist in the image.
  38. The apparatus of claim 36,
    when determining whether the image has water ripples according to the area surrounded by the distance vector clusters, the processor is specifically configured to:
    determining a median value in an area surrounded by the plurality of distance vector clusters;
    and when the median value is larger than a preset area threshold value, determining that water ripples exist in the image.
  39. The apparatus of claim 30, wherein the projection parameters are homography matrices.
  40. The apparatus of claim 21,
    when the processor matches the detected image object of the calibration object with the calibration object in the image device, the processor is specifically configured to:
    determining a position characteristic parameter of the detected image object according to the position of the image object in the image;
    and matching the detected image object of the calibration object with the calibration object in the image device according to the determined position characteristic parameter and the pre-stored position characteristic parameter of the calibration object.
  41. A ripple detection system comprising a camera and a ripple detection device of any one of claims 21 to 40, wherein,
    the shooting device is used for shooting the image calibration device.
  42. A drone, characterized in that it comprises a water ripple detection system of claim 41.
  43. A storage device storing program instructions for performing the method of any one of claims 1-20 when the program instructions are run on a processor.
CN201880029303.2A 2018-02-28 2018-02-28 Image water ripple detection method and device, unmanned aerial vehicle and storage device Pending CN110651299A (en)

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2018/077658 WO2019165611A1 (en) 2018-02-28 2018-02-28 Method and device for detecting water ripple of image, and unmanned aerial vehicle and storage device

Publications (1)

Publication Number Publication Date
CN110651299A true CN110651299A (en) 2020-01-03

Family

ID=67804783

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201880029303.2A Pending CN110651299A (en) 2018-02-28 2018-02-28 Image water ripple detection method and device, unmanned aerial vehicle and storage device

Country Status (2)

Country Link
CN (1) CN110651299A (en)
WO (1) WO2019165611A1 (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113240595A (en) * 2021-05-06 2021-08-10 腾讯科技（深圳）有限公司 Image detection method, image detection device, storage medium and computer equipment

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101271573A (en) * 2008-05-05 2008-09-24 南京师范大学 Image deformation calibration method irrespective with shooting apparatus
CN101636745A (en) * 2006-12-29 2010-01-27 格斯图尔泰克股份有限公司 Manipulation of virtual objects using enhanced interactive system
US20120033098A1 (en) * 2010-08-03 2012-02-09 Canon Kabushiki Kaisha Image pickup apparatus and image processing method of a picked-up image
US20120106837A1 (en) * 2010-10-29 2012-05-03 Canon Kabushiki Kaisha Foreground background separation in a scene with unstable textures
CN102868858A (en) * 2011-07-06 2013-01-09 株式会社摩如富 Image processing device, image processing method
CN105163046A (en) * 2015-08-17 2015-12-16 成都鹰眼视觉科技有限公司 Video stabilization method based on grid point non-parametric motion model

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101636745A (en) * 2006-12-29 2010-01-27 格斯图尔泰克股份有限公司 Manipulation of virtual objects using enhanced interactive system
CN101271573A (en) * 2008-05-05 2008-09-24 南京师范大学 Image deformation calibration method irrespective with shooting apparatus
US20120033098A1 (en) * 2010-08-03 2012-02-09 Canon Kabushiki Kaisha Image pickup apparatus and image processing method of a picked-up image
US20120106837A1 (en) * 2010-10-29 2012-05-03 Canon Kabushiki Kaisha Foreground background separation in a scene with unstable textures
CN102868858A (en) * 2011-07-06 2013-01-09 株式会社摩如富 Image processing device, image processing method
CN105163046A (en) * 2015-08-17 2015-12-16 成都鹰眼视觉科技有限公司 Video stabilization method based on grid point non-parametric motion model

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113240595A (en) * 2021-05-06 2021-08-10 腾讯科技（深圳）有限公司 Image detection method, image detection device, storage medium and computer equipment
CN113240595B (en) * 2021-05-06 2023-09-08 腾讯科技（深圳）有限公司 Image detection method, device, storage medium and computer equipment

Also Published As

Publication number Publication date
WO2019165611A1 (en) 2019-09-06

Similar Documents

Publication Publication Date Title
US11503275B2 (en) Camera calibration system, target, and process
US10594941B2 (en) Method and device of image processing and camera
KR101657039B1 (en) Image processing apparatus, image processing method, and imaging system
US10872436B2 (en) Spatial positioning method, spatial positioning device, spatial positioning system and computer readable storage medium
CN109474780B (en) Method and device for image processing
WO2020014909A1 (en) Photographing method and device and unmanned aerial vehicle
WO2021098448A1 (en) Sensor calibration method and device, storage medium, calibration system, and program product
WO2020237565A1 (en) Target tracking method and device, movable platform and storage medium
JP2012088114A (en) Optical information processing device, optical information processing method, optical information processing system and optical information processing program
CN108734738B (en) Camera calibration method and device
CN112399033B (en) Camera assembly and monitoring camera
CN113538587A (en) Camera coordinate transformation method, terminal and storage medium
US10742852B2 (en) Image processing apparatus, object shape estimation method, and storage medium
JP6897082B2 (en) Computer program for face orientation estimation, face orientation estimation device and face orientation estimation method
WO2018102990A1 (en) System and method for rectifying a wide-angle image
JP2020197989A5 (en) Image processing systems, image processing methods, and programs
CN107680035B (en) Parameter calibration method and device, server and readable storage medium
WO2019165612A1 (en) Method for positioning a movable platform, and related device and system
CN110651299A (en) Image water ripple detection method and device, unmanned aerial vehicle and storage device
JPH09231370A (en) Picture information input device
CN115375741A (en) Registration method and related device for panoramic image and laser point cloud
WO2021138856A1 (en) Camera control method, device, and computer readable storage medium
CN111383262B (en) Occlusion detection method, occlusion detection system, electronic terminal and storage medium
CN110796596A (en) Image splicing method, imaging device and panoramic imaging system
CN117115242B (en) Identification method of mark point, computer storage medium and terminal equipment

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
WD01 Invention patent application deemed withdrawn after publication

Application publication date: 20200103

WD01 Invention patent application deemed withdrawn after publication