CN112203018A - Camera anti-shake self-adaptive adjustment method and system based on artificial intelligence - Google Patents
Camera anti-shake self-adaptive adjustment method and system based on artificial intelligence Download PDFInfo
- Publication number
- CN112203018A CN112203018A CN202011069335.0A CN202011069335A CN112203018A CN 112203018 A CN112203018 A CN 112203018A CN 202011069335 A CN202011069335 A CN 202011069335A CN 112203018 A CN112203018 A CN 112203018A
- Authority
- CN
- China
- Prior art keywords
- image
- camera
- module
- polar coordinate
- key points
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Withdrawn
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/60—Control of cameras or camera modules
- H04N23/68—Control of cameras or camera modules for stable pick-up of the scene, e.g. compensating for camera body vibrations
- H04N23/682—Vibration or motion blur correction
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/181—Segmentation; Edge detection involving edge growing; involving edge linking
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30168—Image quality inspection
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Molecular Biology (AREA)
- Artificial Intelligence (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Health & Medical Sciences (AREA)
- Evolutionary Computation (AREA)
- General Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Mathematical Physics (AREA)
- Signal Processing (AREA)
- Multimedia (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Studio Devices (AREA)
Abstract
The invention relates to the field of artificial intelligence, in particular to a camera anti-jitter self-adaptive adjusting method and system based on artificial intelligence. Acquiring a current image, performing polar coordinate transformation on the current image, marking two types of key points on the polar coordinate image, processing the key points through a key point detection network, obtaining key point coordinate information through coordinate transformation, calculating the average distance of the two types of key points, and outputting the distance difference of adjacent frame pictures as the camera shaking degree; carrying out edge extraction on the image by using an edge extraction network, and sending the image into a first fully-connected neural network for classification after gradient processing to obtain the quality grade of the image; acquiring illumination intensity, and constructing a camera parameter model according to the illumination intensity, the jitter degree and the quality grade; and adjusting the camera parameters in real time according to the model. In the special environment of the photovoltaic power station, the influence of shaking on camera imaging can be rapidly and effectively eliminated by adjusting the camera parameters in real time.
Description
Technical Field
The invention relates to the field of artificial intelligence, in particular to a camera anti-jitter self-adaptive adjusting method and system based on artificial intelligence.
Background
With the development of artificial intelligence, the photovoltaic power generation industry also wants to develop in an intelligent direction. When the image on the surface of the photovoltaic cell panel is collected, the camera shakes to cause the obtained image to be uneven in splicing and blurred in edge.
In the prior art, a method for eliminating jitter generally adopts the steps of shooting a plurality of pictures, and splicing and synthesizing one picture according to different parameters of different pictures.
In practice, the inventors found that the above prior art has the following disadvantages:
in the camera shaking process, a large number of pictures with shaking effects can be shot, and the influence of camera shaking on the shot pictures can not be effectively reduced only through processing operation on the pictures. Moreover, the image shearing processing time is long, the shooting speed in the photovoltaic power station environment is slow, the image shearing processing method is not suitable for the working environment of the photovoltaic power station, and the influence of shake on the shot image cannot be eliminated from the camera.
Disclosure of Invention
In order to solve the above technical problems, an object of the present invention is to provide a camera anti-shake adaptive adjustment method and system based on artificial intelligence, wherein the adopted technical scheme is as follows:
the invention provides an artificial intelligence-based camera anti-jitter self-adaptive adjusting method, which comprises the following steps:
acquiring a current image, and performing polar coordinate transformation on the current image to obtain a current polar coordinate image;
processing the current polar coordinate image by utilizing a pre-trained key point detection network to obtain heat map information containing two types of key points; obtaining coordinate information of the two key points according to the heat map information; obtaining the distance average value h of adjacent key points of different categories according to the coordinate information of the two types of key points, and further obtaining the difference delta h of the distance average values h of adjacent frame images reflecting the camera shaking degree;
performing edge extraction on the current image by using an edge extraction network, inputting the current image after gradient processing into a first fully-connected neural network, and classifying according to a preset image quality grade to obtain a quality grade G of the image;
constructing a camera shutter speed model v ═ a × ln (G + Δ h) + k; where a and k are fitting parameters of a first type and v represents the camera shutter speed;
obtaining illumination intensity I and constructing a phaseMechanical sensitivity model
Wherein b and c are a second class of fitting parameters, and s represents the camera sensitivity;
adjusting camera parameters using the camera shutter speed model and the camera sensitivity model.
Further, the polar coordinate transformation of the current image comprises the following steps:
setting the coordinate of the center point of the current image as (x)0,y0) For any point (x, y) on the plane, with (x)0,y0) For the center point, a transformation is performed using log-polar coordinates.
Further, the training process of the key point detection network comprises the following steps:
acquiring a training image, and converting the training image into a training polar coordinate image through a polar coordinate;
drawing a circle with the radius of R and a circle with the radius of R in the training polar coordinate image by taking the image central point as an origin, wherein R is greater than R and R is not greater than the radius of a circumscribed circle of the picture;
taking the intersection point of the circle with the radius of R and the spiral line of the current polar coordinate image as a first type key point, and taking the intersection point of the circle with the radius of R and the spiral line of the current polar coordinate image as a second type key point;
generating Gaussian hot spots at the positions of the two types of key points by utilizing a Gaussian convolution kernel, and using the Gaussian hot spots as the marking information of the key points;
and training the key point detection network by using the labeling information and the polar coordinate image as a training set.
Further, the obtaining of the coordinate information of the two types of key points specifically includes the following steps:
acquiring image coordinates of two types of key points on the heat map information;
obtaining a connecting line between the heat map information center point and a second type key point;
extending the connecting line to obtain an intersection point of the connecting line on a circle with the radius of R, and obtaining coordinate information of the intersection point; the intersection point coordinate information represents coordinate information of the second type of key points on a circle with a radius of R.
Further, the first-class fitting parameter obtaining process includes:
and inputting a plurality of sample data with known quality grades and jitter degrees into a second fully-connected neural network, and fitting to obtain the first-class fitting parameters a and k.
Further, the second class of fitting parameter obtaining process includes:
and inputting sample data of a plurality of known cameras with light sensitivity, illumination intensity, quality grade and shaking degree into a third fully-connected neural network, and fitting to obtain the second type of fitting parameters b and c.
The invention also provides an artificial intelligence-based camera anti-shake self-adaptive adjusting system, which comprises an image acquisition module, a shake detection module, an image quality analysis module, an illumination intensity acquisition module and an online learning module;
the image acquisition module is used for acquiring images, wherein the images comprise the first training image, the second training image and the current image;
the shake detection module is used for carrying out polar coordinate transformation on the current image, automatically adding feature points, and outputting feature point position information by using a key point detection network to obtain the shake degree of the camera;
the image quality analysis module is used for extracting the edge characteristics of the current image, judging the quality of the image through a first full-connection neural network after gradient processing, and outputting an image quality grade G;
the illumination intensity acquisition module is used for acquiring the illumination intensity of the current environment;
the online adjusting module stores the camera shutter speed model and the camera sensitivity model and is used for adapting to a changing environment and adjusting camera parameters in real time.
Further, the jitter detection module comprises a polar coordinate transformation module, an image annotation module, a key point detection network and a jitter degree calculation module;
the polar coordinate transformation module is used for carrying out polar coordinate transformation on the current image to obtain a current polar coordinate image;
the image marking module is used for marking two key points on the current polar coordinate image and obtaining marking information after processing;
the key point detection network uses an encoding-decoding structure, a current image is input into an encoder to carry out feature extraction to obtain a feature map, the feature map is sampled by a decoder, and finally a key point heat map is output as two channels, wherein each channel contains one type of key point information.
And the jitter degree calculation module processes the key point heat map to obtain the image coordinates of key points, calculates the distances of different key points through coordinate transformation to obtain the average value of the distances, and outputs the average value difference of adjacent frame pictures as jitter degree delta h.
Further, the image quality analysis module comprises an edge extraction module, a gradient processing module and a first fully-connected neural network;
the edge extraction module is used for extracting edge features of an input picture;
the gradient processing module performs gradient processing on the edge characteristics and outputs a gradient map;
the first fully-connected neural network is used for processing the quality grade G of the gradient map output image.
The invention has the following beneficial effects:
1. and the plurality of neural networks are utilized to process each parameter of the image, so that the calculation amount is reduced, the storage space is saved, the processing effect is faster, simpler and more convenient, and the output effect is better.
2. And a camera parameter adjusting model is constructed, so that the camera adjusts the camera parameters on line according to the real-time condition, and the influence of jitter on camera imaging is effectively eliminated by adjusting the camera.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions and advantages of the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and other drawings can be obtained by those skilled in the art without creative efforts.
Fig. 1 is a flowchart of an artificial intelligence-based adaptive camera anti-shake adjustment method according to an embodiment of the present invention;
FIG. 2 is a diagram illustrating a specific stitching method for image acquisition according to an embodiment of the present invention;
FIG. 3 is a diagram illustrating two types of key points labeled according to an embodiment of the present invention;
FIG. 4 is a schematic diagram of distance calculation of two types of key points according to an embodiment of the present invention
Fig. 5 is a block diagram of an artificial intelligence-based adaptive camera anti-jitter adjustment system according to an embodiment of the present invention.
Detailed Description
To further illustrate the technical means and effects of the present invention adopted to achieve the predetermined object, the following detailed description of the embodiments, structures, features and effects of the camera anti-shake adaptive adjustment method and system based on artificial intelligence according to the present invention will be provided with reference to the accompanying drawings and preferred embodiments. In the following description, different "one embodiment" or "another embodiment" refers to not necessarily the same embodiment. Furthermore, the particular features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.
Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.
The following describes a specific scheme of an artificial intelligence-based camera anti-shake adaptive adjustment method and system in detail with reference to the accompanying drawings.
Referring to fig. 1, a flowchart of an artificial intelligence-based camera anti-shake adaptive adjustment method according to an embodiment of the present invention is shown, and for convenience of description, only the relevant portions of the embodiment of the present application are shown.
The method comprises the following steps:
step S1: and acquiring a current image, and performing polar coordinate transformation on the current image.
The method comprises the steps of collecting images on the surface of the photovoltaic cell panel by using a track camera, enabling the camera to run at a constant speed, combining the idea of line scanning, and cutting and splicing the shot pictures to obtain a panoramic image of the photovoltaic cell panel. As shown in fig. 2, the specific splicing method includes: setting the speed of the track camera as V; the sampling interval of the camera is T, and the actual size of each pixel is represented by dx; let the coefficient N denote the number of pixels sampled in a time interval, which can be expressed as N ═ V × T)/dx.
When the track camera works, the visual angle is a depression angle, and a far picture is distorted due to distance, so that only the N columns of pixels closest to the camera are reserved at the time t1, as shown in fig. 2, the track camera collects images according to the set running direction, only 2N columns of pixels are reserved in image information at each later time, and the information at the current time and the information at the previous time, namely the time t2, are spliced at the time t3 to obtain the surface image of the photovoltaic cell panel finally collected at the time t 3.
As more obvious jitter characteristics can be obtained under a polar coordinate system, the detection is more accurate. The acquired image is thus transformed into a corresponding polar image by a polar transformation. And realizing pixel point assignment under a polar coordinate system by an interpolation method. For any point (x, y) on the image plane, let (x)0,y0) For the central point coordinate, a logarithmic polar coordinate is adopted, and the corresponding polar coordinate transformation formula is as follows:
ρ＝log(γ)
where γ represents a distance between an arbitrary point and the center point, ρ represents a polar diameter in a logarithmic polar coordinate, and θ represents a polar angle in a polar coordinate.
Step S2: processing the polar coordinate image by using a pre-trained key point detection network to obtain heat map information containing two types of key points, obtaining the coordinate information of the two types of key points according to the heat map information, calculating the average distance of the two types of key points, and outputting the distance difference of adjacent frame pictures as the camera shake degree.
The specific training process of the pre-trained key point detection network is as follows:
and (3) acquiring and labeling a training image, and transforming the training image through the polar coordinates in the step (1) to obtain a training polar coordinate image. As shown in fig. 3, two circles with radii of R and R are drawn on the training polar coordinate image by using the image center point, and satisfy 0<r<R≤RmaxWherein R ismaxThe radius of the circumscribed circle of the polar coordinate image is trained. And taking the intersection point of the circle with the radius of R and the spiral line of the training polar coordinate image as a first class key point, and taking the intersection point of the circle with the radius of R and the spiral line of the training polar coordinate image as a second class key point. And generating Gaussian hot spots at the positions of the two types of key points by utilizing a Gaussian convolution kernel, wherein the Gaussian hot spots are used as the labeling information of the key points, and the generated labeled data comprises two channels, and each channel is one type of key point.
And training by taking the processed training polar coordinate image and the corresponding marking information as a training set, and updating model parameters by adopting a mean square error loss function as a loss function. The key point detection network adopts an encoding-decoding (Encoder-Decoder) structure, and an Encoder performs feature extraction on an input polar coordinate image to obtain a feature map; the decoder samples the feature map, and finally outputs a key point heat map with the size equal to that of the original image, wherein the output is two channels, and each channel comprises Gaussian hot spot information of one type of key point.
Preferably, the key point detection network may adopt an hourglass network structure to facilitate training and convergence of the network. The encoder can be implemented by using code blocks of several common lightweight networks, such as a High-Resolution Network (HR Network), a shuffle Network (shuffle Network), a compression-and-Excitation Network (SENet), and so on.
And processing the obtained key point heat map by a softargmax function to obtain the image coordinates of the two types of key points.
The Softargmax function can be combined with the softmax function to achieve argmax, and the process is made to be conductive while the index normalization is performed.
And drawing a connecting line between the central point of the key point heat map and the second type of key points, and extending the connecting line to a circle with the radius of R to obtain intersection point coordinate information, wherein the intersection point coordinate information represents the coordinate information of the second type of key points on the circle with the radius of R. The processing of the circle with the radius R has coordinate information of two types of key points.
And transforming the processed key point heat map into a rectangular coordinate system through coordinate system transformation, and transforming a circle with the radius of R into a picture as shown in figure 4. Calculating the distances between adjacent key points of different categories, and calculating the average value to obtain a value h related to the straight line features in the original image:
where i represents the subscript of the distance l and n is the total number of distance segments. And obtaining h values of different frames according to the formula, and recording the difference of h of adjacent frame pictures as delta h, wherein the delta h is used for reflecting the jitter degree of the camera.
Step S3: and carrying out edge extraction on the image by using an edge extraction network, and sending the image into a first fully-connected neural network for classification after gradient processing to obtain the quality grade of the image.
Because the shaking of the camera can cause the blurring of the image edge, the edge extraction network is utilized to extract the edge of the image to obtain the edge feature. The characteristics are more obvious through gradient processing. And sending the processed gradient map into a first fully-connected neural network for classification to obtain the quality grade G of the image.
The quality level is a preset level, and in the embodiment of the present invention, the level is designed to be 5, and a higher level indicates better image quality. The data set of the first fully-connected neural network training is a preset label for judging the image quality, and the network is trained through gradient processing.
Step S4: and obtaining the illumination intensity, and constructing a camera parameter model according to the illumination intensity, the jitter degree and the quality grade.
The illumination intensity is obtained more commonly, and can be obtained by a corresponding sensor or a local meteorological database. In the embodiment of the invention, the illumination intensity is set as data obtained from a meteorological database.
The faster the shutter speed of the camera is, the smaller the influence of shaking on the image is, when the shutter reaches a certain value, the shaking effect can be eliminated, the image quality is improved, the shutter speed is set as v, and a camera shutter speed model is constructed:
v＝a*ln(G+Δh)+k
in the above formula, a and k represent the first type of fitting parameters of the model.
The light sensitivity of the camera is increased, the blurring of the camera caused by shaking can be reduced, and the picture quality is improved. Setting the sensitivity of the camera as s, and because the larger the illumination intensity I is, the enough illumination can be obtained in a short time, the sensitivity s of the camera is reduced; the jitter degree deltah of the camera is increased, and the light sensitivity s of the camera is adjusted to be large. Constructing a camera sensitivity model:
in the above formula, b and c represent the second class of fitting parameters of the model.
In the two models, a, b, c and k are fitting parameters to be solved. Each of the two models has only two fitting parameters, so that good effect can be obtained by simply using the training output of the fully-connected neural network. The training processes of the two models are independent of each other.
The sample data for training the camera shutter speed model v is sample data with high image quality with a plurality of known quality levels and jitter degrees, the input is { G, Δ h }, and the second fully-connected neural network has two outputs respectively representing a and k.
The sample data for training the model camera sensitivity model s is sample data with high image quality and known camera sensitivity, illumination intensity, quality grade and jitter degree, the input is { s, I, G, delta h }, and the third fully-connected neural network has two outputs which respectively represent b and c.
For the present embodiment, the above-mentioned high image quality represents samples corresponding to image levels of 4 and 5.
Step S5: and adjusting the camera parameters in real time according to the model.
And (4) obtaining the environment-adaptive nonlinear model through the steps, and adjusting camera parameters according to the two models to eliminate the influence of jitter on camera imaging.
The specific adjustment method is as follows:
1) setting image quality grade G of camera shooting0For the image blurring threshold, when the image quality grade G is G0Triggering an adaptive adjustment mechanism.
2) And acquiring the current jitter degree delta h and the current illumination intensity I, and adjusting the camera shutter speed and the camera sensitivity according to the camera shutter speed model and the camera sensitivity model.
3) And adjusting camera parameters to improve the image quality grade G through real-time change of the parameters.
In summary, in the embodiment, the camera parameters are adjusted in real time by acquiring a series of parameters, and the influence of camera shake on image quality can be rapidly solved through the adjustment of the camera. And various parameters are obtained through the network, so that the calculation amount is effectively reduced, and the storage space is saved.
Referring to fig. 2, a block diagram of an artificial intelligence based adaptive camera anti-shake adjustment system according to an embodiment of the present invention is shown. The system comprises: the system comprises an image acquisition module 101, a shake detection module 102, an image quality analysis module 103, an illumination intensity acquisition module 104 and an online adjustment module 105.
The image acquisition module 101 acquires a surface image of the photovoltaic cell panel by using the track camera, and cuts and splices the shot picture by using a line scanning idea to obtain a panoramic image of the surface of the photovoltaic cell panel.
The jitter detection module 102 specifically includes: a polar coordinate transformation module 1101, an image annotation module 1102, a key point detection network 1103, and a shake degree calculation module 1104. The polar coordinate conversion module 1101 performs polar coordinate conversion on the image data input by the image acquisition module 101, and outputs a corresponding polar coordinate image. The image annotation module 1102 annotates the polar coordinate image and obtains annotation key point information, and sends both the polar coordinate image and the annotation information to the key point detection network 1103. The keypoint detection network 1103 is processed to output heat map information containing keypoints. The heat map information is subjected to function processing by the jitter degree calculation module 1104 to obtain image coordinates of key points, distances between adjacent key points of different categories are obtained according to the coordinate information through coordinate transformation, a distance average value h is output, and delta h is used as a difference between h of adjacent frame pictures to reflect the jitter degree of the camera.
The image quality analysis module 103 specifically includes: an edge extraction module 2101, a gradient processing module 2102, and a first fully-connected neural network 2103. The edge extraction module 2101 is configured to extract edge features of an image using an edge extraction network, and the gradient processing module 2102 performs gradient processing on the edge features to make the features more obvious and output a gradient map. The first fully-connected neural network 2103 classifies the gradient map according to a preset image grade, and outputs an image quality grade G.
The illumination intensity obtaining module 104 is configured to obtain the illumination intensity I of the current environment from the weather database.
The online adjusting module 105 includes the set model, adjusts parameters of the camera in real time by combining the jitter degree h, the image quality grade G and the illumination intensity I, and reduces the influence of jitter on the imaging of the camera.
It should be noted that: the precedence order of the above embodiments of the present invention is only for description, and does not represent the merits of the embodiments. And specific embodiments thereof have been described above. Other embodiments are within the scope of the following claims. In some cases, the actions or steps recited in the claims may be performed in a different order than in the embodiments and still achieve desirable results. In addition, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. In some embodiments, multitasking and parallel processing may also be possible or may be advantageous.
The embodiments in the present specification are described in a progressive manner, and the same and similar parts among the embodiments are referred to each other, and each embodiment focuses on the differences from the other embodiments.
The above description is only for the purpose of illustrating the preferred embodiments of the present invention and is not to be construed as limiting the invention, and any modifications, equivalents, improvements and the like that fall within the spirit and principle of the present invention are intended to be included therein.
Claims (9)
1. An artificial intelligence based camera anti-shake adaptive adjustment method is characterized by comprising the following steps:
acquiring a current image, and performing polar coordinate transformation on the current image to obtain a current polar coordinate image;
processing the current polar coordinate image by utilizing a pre-trained key point detection network to obtain heat map information containing two types of key points; obtaining coordinate information of the two key points according to the heat map information; obtaining the distance average value h of adjacent key points of different categories according to the coordinate information of the two types of key points, and further obtaining the difference delta h of the distance average values h of adjacent frame images reflecting the camera shaking degree;
performing edge extraction on the current image by using an edge extraction network, inputting the current image after gradient processing into a first fully-connected neural network, and classifying according to a preset image quality grade to obtain a quality grade G of the image;
constructing a camera shutter speed model v ═ a × ln (G + Δ h) + k; where a and k are fitting parameters of a first type and v represents the camera shutter speed;
obtaining illumination intensity I and constructing a camera sensitivity model
Wherein b and c are a second class of fitting parameters, and s represents the camera sensitivity;
adjusting camera parameters using the camera shutter speed model and the camera sensitivity model.
2. The adaptive camera anti-shake adjustment method based on artificial intelligence of claim 1, wherein: the polar coordinate transformation of the current image comprises the following steps:
setting the coordinate of the center point of the current image as (x)0,y0) For any point (x, y) on the plane, with (x)0,y0) For the center point, a transformation is performed using log-polar coordinates.
3. The adaptive camera anti-shake adjusting method based on artificial intelligence of claim 1 or 2, wherein: the training process of the key point detection network comprises the following steps:
acquiring a training image, and converting the training image into a training polar coordinate image through a polar coordinate;
drawing a circle with the radius of R and a circle with the radius of R in the training polar coordinate image by taking the image central point as an origin, wherein R is greater than R and R is not greater than the radius of a circumscribed circle of the picture;
taking the intersection point of the circle with the radius of R and the spiral line of the current polar coordinate image as a first type key point, and taking the intersection point of the circle with the radius of R and the spiral line of the current polar coordinate image as a second type key point;
generating Gaussian hot spots at the positions of the two types of key points by utilizing a Gaussian convolution kernel, and using the Gaussian hot spots as the marking information of the key points;
and training the key point detection network by using the labeling information and the polar coordinate image as a training set.
4. The adaptive camera anti-shake adjustment method based on artificial intelligence of claim 1, wherein: the obtaining of the coordinate information of the two types of key points specifically comprises the following steps:
acquiring image coordinates of two types of key points on the heat map information;
obtaining a connecting line between the heat map information center point and a second type key point;
extending the connecting line to obtain an intersection point of the connecting line on a circle with the radius of R, and obtaining coordinate information of the intersection point; the intersection point coordinate information represents coordinate information of the second type of key points on a circle with a radius of R.
5. The adaptive camera anti-shake adjustment method based on artificial intelligence of claim 1, wherein: the first-class fitting parameter obtaining process comprises the following steps:
and inputting a plurality of sample data with known quality grades and jitter degrees into a second fully-connected neural network, and fitting to obtain the first-class fitting parameters a and k.
6. The adaptive camera anti-shake adjustment method based on artificial intelligence of claim 1, wherein: the second class of fitting parameter obtaining process comprises:
and inputting sample data of a plurality of known cameras with light sensitivity, illumination intensity, quality grade and shaking degree into a third fully-connected neural network, and fitting to obtain the second type of fitting parameters b and c.
7. The utility model provides a camera anti-shake self-adaptation governing system based on artificial intelligence which characterized in that: the system comprises an image acquisition module, a shake detection module, an image quality analysis module, an illumination intensity acquisition module and an online learning module;
the image acquisition module is used for acquiring images, wherein the images comprise the first training image, the second training image and the current image;
the shake detection module is used for carrying out polar coordinate transformation on the current image, automatically adding feature points, and outputting feature point position information by using a key point detection network to obtain the shake degree of the camera;
the image quality analysis module is used for extracting the edge characteristics of the current image, judging the quality of the image through a first full-connection neural network after gradient processing, and outputting an image quality grade G;
the illumination intensity acquisition module is used for acquiring the illumination intensity of the current environment;
the online adjusting module stores the camera shutter speed model and the camera sensitivity model and is used for adapting to a changing environment and adjusting camera parameters in real time.
8. The artificial intelligence based camera anti-shake adaptive adjustment system of claim 7, wherein: the jitter detection module comprises a polar coordinate transformation module, an image annotation module, a key point detection network and a jitter degree calculation module;
the polar coordinate transformation module is used for carrying out polar coordinate transformation on the current image to obtain a current polar coordinate image;
the image marking module is used for marking two key points on the current polar coordinate image and obtaining marking information after processing;
the key point detection network uses an encoding-decoding structure, a current image is input into an encoder to carry out feature extraction to obtain a feature map, the feature map is sampled by a decoder, and finally a key point heat map is output as two channels, wherein each channel contains one type of key point information.
And the jitter degree calculation module processes the key point heat map to obtain the image coordinates of key points, calculates the distances of different key points through coordinate transformation to obtain the average value of the distances, and outputs the average value difference of adjacent frame pictures as jitter degree delta h.
9. The artificial intelligence based camera anti-shake adaptive adjustment system of claim 7, wherein: the image quality analysis module comprises an edge extraction module, a gradient processing module and a first fully-connected neural network;
the edge extraction module is used for extracting edge features of an input picture;
the gradient processing module performs gradient processing on the edge characteristics and outputs a gradient map;
the first fully-connected neural network is used for processing the quality grade G of the gradient map output image.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011069335.0A CN112203018A (en) | 2020-09-30 | 2020-09-30 | Camera anti-shake self-adaptive adjustment method and system based on artificial intelligence |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011069335.0A CN112203018A (en) | 2020-09-30 | 2020-09-30 | Camera anti-shake self-adaptive adjustment method and system based on artificial intelligence |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112203018A true CN112203018A (en) | 2021-01-08 |
Family
ID=74012992
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011069335.0A Withdrawn CN112203018A (en) | 2020-09-30 | 2020-09-30 | Camera anti-shake self-adaptive adjustment method and system based on artificial intelligence |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112203018A (en) |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112954314A (en) * | 2021-02-08 | 2021-06-11 | 南昌欧菲光电技术有限公司 | Graph card, device and method for testing optical jitter of camera module |
| CN113406659A (en) * | 2021-05-28 | 2021-09-17 | 浙江大学 | Mobile robot position re-identification method based on laser radar information |
| CN114627215A (en) * | 2022-05-16 | 2022-06-14 | 山东捷瑞数字科技股份有限公司 | Method and device for camera shake animation production based on three-dimensional software |
| CN115035282A (en) * | 2022-08-12 | 2022-09-09 | 南通华豪巨电子科技有限公司 | High-quality photovoltaic cell panel orthographic image acquisition method and system based on track camera |
| CN117221727A (en) * | 2023-10-09 | 2023-12-12 | 深圳市启惠智能科技有限公司 | Image stabilization control method for moving camera and related equipment |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| RU2486590C1 (en) * | 2012-04-06 | 2013-06-27 | Федеральное государственное бюджетное образовательное учреждение высшего профессионального образования "Юго-Западный государственный университет" (ЮЗ ГУ) | Method and device of invariant identification of fingerprints by key points |
| CN110175544A (en) * | 2019-05-14 | 2019-08-27 | 广州虎牙信息科技有限公司 | Construction method, device, electronic equipment and the storage medium of object module |
| CN110598554A (en) * | 2019-08-09 | 2019-12-20 | 中国地质大学（武汉） | Multi-person posture estimation method based on counterstudy |
| US20200042776A1 (en) * | 2018-08-03 | 2020-02-06 | Baidu Online Network Technology (Beijing) Co., Ltd. | Method and apparatus for recognizing body movement |
| CN111353325A (en) * | 2018-12-20 | 2020-06-30 | 北京京东尚科信息技术有限公司 | Key point detection model training method and device |
| CN111402330A (en) * | 2020-04-03 | 2020-07-10 | 山东省科学院激光研究所 | Laser line key point extraction method based on plane target |
| CN111563446A (en) * | 2020-04-30 | 2020-08-21 | 郑州轻工业大学 | Human-machine interaction safety early warning and control method based on digital twin |
-
2020
- 2020-09-30 CN CN202011069335.0A patent/CN112203018A/en not_active Withdrawn
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| RU2486590C1 (en) * | 2012-04-06 | 2013-06-27 | Федеральное государственное бюджетное образовательное учреждение высшего профессионального образования "Юго-Западный государственный университет" (ЮЗ ГУ) | Method and device of invariant identification of fingerprints by key points |
| US20200042776A1 (en) * | 2018-08-03 | 2020-02-06 | Baidu Online Network Technology (Beijing) Co., Ltd. | Method and apparatus for recognizing body movement |
| CN111353325A (en) * | 2018-12-20 | 2020-06-30 | 北京京东尚科信息技术有限公司 | Key point detection model training method and device |
| CN110175544A (en) * | 2019-05-14 | 2019-08-27 | 广州虎牙信息科技有限公司 | Construction method, device, electronic equipment and the storage medium of object module |
| CN110598554A (en) * | 2019-08-09 | 2019-12-20 | 中国地质大学（武汉） | Multi-person posture estimation method based on counterstudy |
| CN111402330A (en) * | 2020-04-03 | 2020-07-10 | 山东省科学院激光研究所 | Laser line key point extraction method based on plane target |
| CN111563446A (en) * | 2020-04-30 | 2020-08-21 | 郑州轻工业大学 | Human-machine interaction safety early warning and control method based on digital twin |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112954314A (en) * | 2021-02-08 | 2021-06-11 | 南昌欧菲光电技术有限公司 | Graph card, device and method for testing optical jitter of camera module |
| CN113406659A (en) * | 2021-05-28 | 2021-09-17 | 浙江大学 | Mobile robot position re-identification method based on laser radar information |
| CN114627215A (en) * | 2022-05-16 | 2022-06-14 | 山东捷瑞数字科技股份有限公司 | Method and device for camera shake animation production based on three-dimensional software |
| CN115035282A (en) * | 2022-08-12 | 2022-09-09 | 南通华豪巨电子科技有限公司 | High-quality photovoltaic cell panel orthographic image acquisition method and system based on track camera |
| CN115035282B (en) * | 2022-08-12 | 2022-11-22 | 南通华豪巨电子科技有限公司 | High-quality photovoltaic cell panel orthographic image acquisition method and system based on track camera |
| CN117221727A (en) * | 2023-10-09 | 2023-12-12 | 深圳市启惠智能科技有限公司 | Image stabilization control method for moving camera and related equipment |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112203018A (en) | Camera anti-shake self-adaptive adjustment method and system based on artificial intelligence | |
| CN111325203B (en) | American license plate recognition method and system based on image correction | |
| CN112507793B (en) | Ultra-short term photovoltaic power prediction method | |
| CN111461134B (en) | Low-resolution license plate recognition method based on generation countermeasure network | |
| CN108875821A (en) | The training method and device of disaggregated model, mobile terminal, readable storage medium storing program for executing | |
| CN112150493B (en) | Semantic guidance-based screen area detection method in natural scene | |
| CN110379020B (en) | Laser point cloud coloring method and device based on generation countermeasure network | |
| CN111950453A (en) | Optional-shape text recognition method based on selective attention mechanism | |
| CN108154102A (en) | A kind of traffic sign recognition method | |
| CN104217404A (en) | Video image sharpness processing method in fog and haze day and device thereof | |
| CN112287941B (en) | License plate recognition method based on automatic character region perception | |
| CN113139896A (en) | Target detection system and method based on super-resolution reconstruction | |
| CN111507353B (en) | Chinese field detection method and system based on character recognition | |
| CN116665092A (en) | Method and system for identifying sewage suspended matters based on IA-YOLOV7 | |
| CN110222217B (en) | Shoe print image retrieval method based on segmented weighting | |
| CN113989608A (en) | Student experiment classroom behavior identification method based on top vision | |
| CN115410240A (en) | Intelligent face pockmark and color spot analysis method and device and storage medium | |
| CN116994068A (en) | Target detection method and device based on knowledge distillation | |
| CN112116569A (en) | Photovoltaic power station power generation power prediction method based on shadow recognition | |
| CN110414517B (en) | Rapid high-precision identity card text recognition algorithm used for being matched with photographing scene | |
| CN114913523B (en) | Yolox-based multifunctional real-time intelligent plant stomata recognition system | |
| CN114283431B (en) | Text detection method based on differentiable binarization | |
| CN115641592A (en) | Contrast optimization-based electro-optical character recognition method and system | |
| CN115527128A (en) | Semantic segmentation based aerial image rapid positioning method | |
| CN115588130A (en) | Cross-domain YOLO detection method based on domain self-adaptation |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WW01 | Invention patent application withdrawn after publication |
Application publication date: 20210108 |
|
| WW01 | Invention patent application withdrawn after publication |