CN114979470A - Camera rotation angle analysis method, device, equipment and storage medium - Google Patents
Camera rotation angle analysis method, device, equipment and storage medium Download PDFInfo
- Publication number
- CN114979470A CN114979470A CN202210515853.3A CN202210515853A CN114979470A CN 114979470 A CN114979470 A CN 114979470A CN 202210515853 A CN202210515853 A CN 202210515853A CN 114979470 A CN114979470 A CN 114979470A
- Authority
- CN
- China
- Prior art keywords
- field
- image
- rotation angle
- camera
- feature
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000004458 analytical method Methods 0.000 title claims description 37
- 239000013598 vector Substances 0.000 claims abstract description 176
- 239000011159 matrix material Substances 0.000 claims abstract description 52
- 238000000034 method Methods 0.000 claims abstract description 51
- 238000012549 training Methods 0.000 claims description 80
- 230000006870 function Effects 0.000 claims description 66
- 238000000605 extraction Methods 0.000 claims description 33
- 238000006243 chemical reaction Methods 0.000 claims description 29
- 230000009466 transformation Effects 0.000 claims description 23
- 238000010586 diagram Methods 0.000 claims description 15
- 238000013507 mapping Methods 0.000 abstract description 6
- 238000012545 processing Methods 0.000 abstract description 3
- 230000008569 process Effects 0.000 description 12
- 238000004891 communication Methods 0.000 description 5
- 238000010606 normalization Methods 0.000 description 5
- 238000013461 design Methods 0.000 description 4
- 238000004364 calculation method Methods 0.000 description 3
- 230000004913 activation Effects 0.000 description 2
- 230000008901 benefit Effects 0.000 description 2
- 239000002131 composite material Substances 0.000 description 2
- 239000000284 extract Substances 0.000 description 2
- 230000007246 mechanism Effects 0.000 description 2
- 238000001069 Raman spectroscopy Methods 0.000 description 1
- 230000002860 competitive effect Effects 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000003708 edge detection Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000002349 favourable effect Effects 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 230000003595 spectral effect Effects 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/60—Control of cameras or camera modules
- H04N23/695—Control of camera direction for changing a field of view, e.g. pan, tilt or based on tracking of objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/048—Activation functions
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
- G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/74—Image or video pattern matching; Proximity measures in feature spaces
- G06V10/75—Organisation of the matching processes, e.g. simultaneous or sequential comparisons of image or video features; Coarse-fine approaches, e.g. multi-scale approaches; using context analysis; Selection of dictionaries
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/74—Image or video pattern matching; Proximity measures in feature spaces
- G06V10/761—Proximity, similarity or dissimilarity measures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/10—Terrestrial scenes
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10016—Video; Image sequence
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Evolutionary Computation (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Multimedia (AREA)
- Computing Systems (AREA)
- Health & Medical Sciences (AREA)
- Artificial Intelligence (AREA)
- General Health & Medical Sciences (AREA)
- Software Systems (AREA)
- Mathematical Physics (AREA)
- General Engineering & Computer Science (AREA)
- Biomedical Technology (AREA)
- Computational Linguistics (AREA)
- Molecular Biology (AREA)
- Biophysics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Medical Informatics (AREA)
- Signal Processing (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a method, a device, equipment and a storage medium for analyzing a rotation angle of a camera, and belongs to the technical field of image processing. The method comprises the steps of obtaining a field image to be detected, generating a field side line graph based on the field image to be detected, extracting a field side line characteristic vector corresponding to the field side line graph, performing characteristic matching on the field side line characteristic vector in a characteristic database to obtain a corresponding camera parameter matrix, and decoding to obtain a camera rotation angle. According to the method and the device, the field sideline vector in the field image is extracted, and the mapping database of the field sideline characteristic vector and the camera parameter matrix is established, so that the camera rotation angle can be obtained in real time in a full-automatic manner without depending on the experience of professionals.
Description
Technical Field
The invention relates to the technical field of image processing, in particular to a method, a device, equipment and a storage medium for analyzing a rotation angle of a camera.
Background
With the development of modern competitive sports, people have higher and higher attention to sports events, and a large number of sports events are broadcasted to audiences in a live broadcast mode. In a sports event, a plurality of video capture scenes are usually distributed around the playing field, and a director makes a plurality of paths of video signals captured in real time on site, and clips and plays corresponding videos according to the rules of the director.
The current event director usually needs professionals to estimate the rotation angle of a camera according to own experience, and real-time tracking of targets in the event is realized. However, the manual estimation method has high requirements on professionals, and the operation often brings large errors. Such errors may further mislead the judgment of the automated director system, thereby affecting the quality of the event director. In addition, this professional-dependent camera rotation tracking method is not scalable.
Disclosure of Invention
The invention mainly aims to provide a camera rotation angle analysis method, a camera rotation angle analysis device, equipment and a storage medium, and aims to solve the problem of how to estimate the camera lens rotation angle.
In order to achieve the above object, the present invention provides a camera rotation angle analysis method, including the steps of:
acquiring an image of a field to be detected;
generating a field boundary diagram based on the field image to be detected;
obtaining a field sideline feature vector corresponding to the field sideline graph;
performing feature matching on the field sideline feature vector and an image feature vector in a feature database, and acquiring a camera parameter matrix corresponding to the image feature vector;
and decoding the camera parameter matrix obtained by matching to obtain the corresponding camera rotation angle.
Preferably, the step of establishing the feature database comprises:
acquiring a field sideline image dataset synthesized by a camera under different rotation angles;
generating and storing camera parameter matrix parameters corresponding to the sideline site images;
correspondingly generating the image feature vector based on the synthesized field sideline image dataset;
and establishing a corresponding relation between the image characteristic vector and the camera parameter matrix to obtain a characteristic database.
Preferably, the step of generating a field and ground boundary line map based on the field image to be detected comprises;
inputting the field image to be detected into a mask generator and a field edge generator to generate a corresponding initial field mask image and an initial field edge image;
and after splicing the initial field mask map and the initial field boundary map, inputting a boundary enhancement network to obtain an enhanced field boundary map.
Preferably, the training of the mask generator and the field edge generator comprises:
inputting training data image frames and random noise into an initial mask generator to obtain a generated site mask image;
acquiring a real site mask image corresponding to the training data image frame, inputting the real site mask image, the generated site mask image and the training data image frame into a first target loss function, iteratively calculating the first target loss function until the first target loss function is converged, and training to obtain a mask generator;
inputting training data image frames and random noise into an initial field edge generator to obtain a generated field edge line graph;
and acquiring a real field side line graph corresponding to the training data image frame, inputting the real field side line graph, the generated field side line graph and the training data image frame into a first target loss function, iteratively calculating the first target loss function until the first target loss function is converged, and training to obtain the field side generator.
Preferably, the training step of the edge enhancement network includes:
splicing the generated field boundary graph and the generated field mask graph, inputting the spliced field boundary graph and the generated field mask graph into an initial boundary enhancement generator, and outputting a boundary enhancement graph;
inputting the edge enhancement graph, the generated field boundary graph and the generated field mask graph into an enhancement loss function, iteratively calculating the enhancement loss function until the enhancement loss function is converged, and training to obtain the edge enhancement network.
Preferably, the step of obtaining a field edge feature vector corresponding to the field edge map includes:
inputting the field contour map into an image feature extraction network, and extracting to obtain an initial feature vector;
and inputting the initial characteristic vector into an image characteristic conversion network to generate a field sideline characteristic vector.
Preferably, the image feature extraction and training step of the image feature transformation network includes:
acquiring a field sideline image pair, inputting the field sideline image pair into a feature extractor, and generating a first feature vector and a second feature vector, wherein the field sideline image pair comprises a first image and a second image;
inputting the first feature vector and the second feature vector into a feature converter to obtain a first space conversion vector and a second space conversion vector after space conversion;
inputting the edge image pair, the label of the edge image pair, the first space transformation vector and the second space transformation vector into a second target loss function, and iteratively training the second target loss function until the second target loss function meets a convergence condition to obtain the image feature extraction network and the image feature transformation network.
Preferably, the step of performing feature matching on the field sideline feature vector and an image feature vector in a feature database, and acquiring a camera parameter matrix corresponding to the image feature vector includes:
and matching the field sideline characteristic vector with the image characteristic vector in the characteristic database through a local sensitive Hash algorithm to obtain a corresponding camera parameter matrix.
Further, to achieve the above object, the present invention provides a camera rotation angle analyzing apparatus, comprising:
the acquisition module is used for acquiring an image of a field to be detected;
the image generation module is used for generating a field boundary diagram based on the field image to be detected;
the characteristic extraction module is used for acquiring a field sideline characteristic vector corresponding to the field sideline graph;
the matching module is used for performing feature matching on the field sideline feature vector and an image feature vector in a feature database and acquiring a camera parameter matrix corresponding to the image feature vector;
and the decoding module is used for decoding the camera parameter matrix obtained by matching to obtain the corresponding camera rotation angle.
Preferably, the obtaining module is further configured to:
acquiring a field sideline image dataset synthesized by a camera under different rotation angles;
generating and storing a camera parameter matrix corresponding to the sideline site image;
correspondingly generating the image feature vector based on the synthesized field sideline image dataset;
and establishing a corresponding relation between the image feature vector and the camera parameter matrix to obtain a feature database.
Preferably, the image generation module is further configured to:
inputting the field image to be detected into a mask generator and a field edge generator to generate a corresponding initial field mask image and an initial field edge image;
and after splicing the initial field mask map and the initial field boundary map, inputting a boundary enhancement network to obtain an enhanced field boundary map.
Preferably, the camera rotation angle analysis device further includes:
the model training module is used for inputting training data image frames and random noise into the initial mask generator to obtain a generated site mask image;
acquiring a real site mask image corresponding to the training data image frame, inputting the real site mask image, the generated site mask image and the training data image frame into a first target loss function, iteratively calculating the first target loss function until the first target loss function is converged, and training to obtain a mask generator;
inputting training data image frames and random noise into an initial field edge generator to obtain a generated field edge line graph;
and acquiring a real field side line graph corresponding to the training data image frame, inputting the real field side line graph, the generated field side line graph and the training data image frame into a first target loss function, iteratively calculating the first target loss function until the first target loss function is converged, and training to obtain the field side generator.
Preferably, the model training module is further configured to:
splicing the generated field boundary map and the generated field mask map, inputting the spliced field boundary map and the generated field mask map into an initial boundary enhancement generator, and outputting a boundary enhancement map;
inputting the edge enhancement graph, the generated field boundary graph and the generated field mask graph into an enhancement loss function, iteratively calculating the enhancement loss function until the enhancement loss function is converged, and training to obtain the edge enhancement network.
Preferably, the feature extraction module is further configured to:
inputting the field contour map into an image feature extraction network, and extracting to obtain an initial feature vector;
and inputting the initial characteristic vector into an image characteristic conversion network to generate a field sideline characteristic vector.
Preferably, the model training module is further configured to:
acquiring a field sideline image pair, inputting the field sideline image pair into a feature extractor, and generating a first feature vector and a second feature vector, wherein the field sideline image pair comprises a first image and a second image;
inputting the first feature vector and the second feature vector into a feature converter to obtain a first space conversion vector and a second space conversion vector after space conversion;
inputting the edge image pair, the label of the edge image pair, the first space transformation vector and the second space transformation vector into a second target loss function, and iteratively training the second target loss function until the second target loss function meets a convergence condition to obtain the image feature extraction network and the image feature transformation network.
Preferably, the matching module is further configured to:
and matching the field sideline characteristic vector with the image characteristic vector in the characteristic database through a local sensitive Hash algorithm to obtain a corresponding camera parameter matrix.
Further, to achieve the above object, the present invention also provides a camera rotation angle analyzing apparatus, comprising: a memory, a processor and a camera rotation angle analysis program stored on the memory and executable on the processor, the camera rotation angle analysis program being configured to implement the steps of the camera rotation angle analysis method as described above.
Further, to achieve the above object, the present invention also provides a storage medium having a camera rotation angle analysis program stored thereon, which when executed by a processor, implements the steps of the camera rotation angle analysis method as described above.
The camera rotation angle analysis method provided by the invention comprises the steps of acquiring a field image to be detected, generating a field side line graph based on the field image to be detected, extracting a field side line characteristic vector corresponding to the field side line graph, carrying out characteristic matching on the field side line characteristic vector in a characteristic database to obtain a corresponding camera parameter matrix, and decoding to obtain a camera rotation angle. In the embodiment, the field sideline vector is used as a keyword of the database index to organize the camera parameter matrix library, the incidence relation between the field sideline characteristic vector and the camera parameter matrix is established and stored as the characteristic database, and the field sideline vector in the field image is extracted, so that the camera rotation angle can be obtained in real time in a fully automatic manner without depending on the experience of a professional.
Drawings
Fig. 1 is a schematic structural diagram of a camera rotation angle analysis device in a hardware operating environment according to an embodiment of the present invention;
FIG. 2 is a schematic flowchart of a first embodiment of a method for analyzing a rotation angle of a camera according to the present invention;
FIG. 3 is a flowchart of a technical solution for automatically tracking a rotation angle of a camera according to an embodiment of a method for analyzing a rotation angle of a camera of the present invention;
fig. 4 is a detailed flowchart of step S20 in an embodiment of the camera rotation angle analysis method of the present invention;
fig. 5 is a network structure diagram generated by a field boundary according to an embodiment of the method for analyzing a rotation angle of a camera of the present invention;
fig. 6 is a schematic flowchart of a detailed process of step S30 in an embodiment of the method for analyzing a rotation angle of a camera according to the present invention;
FIG. 7 is a network structure diagram of the field sideline image feature extraction and feature transformation according to an embodiment of the method for analyzing the rotation angle of the camera of the present invention;
fig. 8 is a functional block diagram of an embodiment of the camera rotation angle analysis apparatus according to the present invention.
The implementation, functional features and advantages of the objects of the present invention will be further explained with reference to the accompanying drawings.
Detailed Description
It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
Referring to fig. 1, fig. 1 is a schematic structural diagram of a camera rotation angle analysis device in a hardware operating environment according to an embodiment of the present invention.
As shown in fig. 1, the camera rotation angle analyzing apparatus may include: a processor 1001, such as a Central Processing Unit (CPU), a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. Wherein a communication bus 1002 is used to enable connective communication between these components. The user interface 1003 may include a Display screen (Display), an input unit such as a Keyboard (Keyboard), and the optional user interface 1003 may also include a standard wired interface, a wireless interface. The network interface 1004 may optionally include a standard wired interface, a WIreless interface (e.g., a WIreless-FIdelity (WI-FI) interface). The Memory 1005 may be a Random Access Memory (RAM) Memory, or may be a Non-Volatile Memory (NVM), such as a disk Memory. The memory 1005 may alternatively be a storage device separate from the processor 1001 described previously.
Those skilled in the art will appreciate that the configuration shown in fig. 1 does not constitute a limitation of the camera rotation angle analysis apparatus, and may include more or fewer components than those shown, or some components in combination, or a different arrangement of components.
As shown in fig. 1, a memory 1005, which is a storage medium, may include therein an operating system, a data storage module, a network communication module, a user interface module, and a camera rotation angle analysis program.
In the camera rotation angle analysis device shown in fig. 1, the network interface 1004 is mainly used for data communication with other devices; the user interface 1003 is mainly used for data interaction with a user; the processor 1001 and the memory 1005 in the camera rotation angle analysis apparatus of the present invention may be provided in the camera rotation angle analysis apparatus which calls the camera rotation angle analysis program stored in the memory 1005 through the processor 1001 and executes the camera rotation angle analysis method provided by the embodiment of the present invention.
An embodiment of the present invention provides a method for analyzing a rotation angle of a camera, and referring to fig. 2, fig. 2 is a schematic flow chart of a first embodiment of the method for analyzing a rotation angle of a camera according to the present invention.
In this embodiment, the method for analyzing the rotation angle of the camera includes:
step S10, acquiring an image of a site to be detected;
step S20, generating a field boundary diagram based on the field image to be detected;
step S30, obtaining a field sideline characteristic vector corresponding to the field sideline graph;
step S40, performing feature matching on the field sideline feature vector and an image feature vector in a feature database, and acquiring a camera parameter matrix corresponding to the image feature vector;
and step S50, decoding the camera parameter matrix obtained by matching to obtain the corresponding camera rotation angle.
The camera rotation angle analysis method is used in an event director system. In order to solve the problem that the quality of a broadcasting-directing picture greatly depends on the professional skill of an operator and large-scale intelligent broadcasting directing is difficult to realize because the conventional camera rotation angle tracking mainly depends on the professional skill of a professional, an automatic camera rotation tracking technology needs to be developed.
The respective steps will be described in detail below:
step S10, acquiring an image of a site to be detected;
in an embodiment, an image of a rotation angle of a camera to be detected is acquired, and specifically, an image of a field to be detected is acquired by performing frame picture extraction on a video shot by the camera. It can be understood that, according to the scheme, the field image is analyzed, so that the information related to the rotation angle of the camera is acquired, and therefore, the field image to be detected needs to be acquired first. The scheme is mainly used in the event broadcasting guide system, and usually, frame pictures are extracted from videos shot by a camera in real time for real-time judgment.
Step S20, generating a field boundary diagram based on the field image to be detected;
in one embodiment, a field map is generated from the field image to be detected. It can be understood that, in order to determine the mapping relationship between the field image and the rotation angle of the camera, an identification feature with invariance needs to be found from the field image, and the identification feature is the identification of the field edge. Generally, the field lines of various events are all fixed and standard, so that the field lines can play a role in positioning in the video. Specifically, a field sideline image generated from the field image can be used for training the field sideline to generate a network through machine learning, the network can effectively divide a current picture acquired by the camera into a court and a background through learning sideline characteristics, and the foreground court is further converted into a field sideline image at a current angle.
Step S30, obtaining a field sideline characteristic vector corresponding to the field sideline graph;
in one embodiment, according to the generated field boundary map, a corresponding boundary feature vector is obtained. It will be appreciated that a large database of image-to-camera pose (angle) correspondences needs to be built. However, the images are all high latitude data, which can present speed and accuracy problems. The problem is how to accurately map the high-dimensional field edge map to a specific space where the feature vectors of the field edge have strong resolvability. If the sideline image is directly corresponding to the rotation angle of the camera lens, the correspondence is slow and difficult, so the sideline image is converted into a low-dimensional feature vector with higher resolution from a high-dimensional image, namely, the sideline feature extraction vector corresponding to the field sideline image is extracted. And extracting the field edge feature vector, and directly converting the field edge image into a feature vector by using a pre-trained feature extractor on ImageNet.
Step S40, performing feature matching on the field sideline feature vector and an image feature vector in a feature database, and acquiring a camera parameter matrix corresponding to the image feature vector;
in an embodiment, after the low-dimensional feature of the field sideline is obtained, that is, after the field sideline feature vector is obtained, the field sideline feature vector is matched with feature data in a feature database to obtain a corresponding feature vector, so that a camera parameter matrix corresponding to the feature vector is obtained. It can be understood that, in the training stage, images shot by different camera rotation angles are collected, then edge image generation and feature extraction are performed, and then the extracted image feature vector and the parameters during shooting are stored in an associated manner, so that when it is detected that the field edge feature vector of the field image to be detected is similar to or identical to the existing image feature vector, the shooting parameters of the image can be considered to be identical to the shooting parameters of the image in the training data at that time, and thus a camera parameter matrix stored in an associated manner is obtained. It should be noted that, the camera parameter matrix is usually a combination of internal and external parameters used in imaging of the camera, and the internal parameter includes a parameter capable of indicating the rotation angle of the camera, so that the rotation angle of the camera can be obtained by analyzing the camera parameter matrix.
And step S50, decoding the camera parameter matrix obtained by matching to obtain the corresponding camera rotation angle.
In an embodiment, the camera parameter matrix obtained by matching is decoded, and the decoding process is to obtain the left and right rotation angles and the up and down rotation angles of the camera from the camera parameter matrix again by using a rodgers rotation formula, and provide services for other downstream tasks in the director system after obtaining the angle result. The rodlike rotation formula is a formula for calculating a new vector obtained by rotating a vector around a rotation axis by a given angle in a three-dimensional space, and is relatively suitable for an application scenario of camera lens rotation in the embodiment.
Further, in an embodiment, the step of establishing the feature database includes:
step S11, acquiring a field sideline image dataset synthesized by the camera under different rotation angles;
step S12, generating and storing camera parameter matrix parameters corresponding to the sideline site images;
step S13, correspondingly generating the image feature vector based on the synthesized field side line image dataset;
and step S14, establishing a corresponding relation between the image feature vector and the camera parameter matrix to obtain a feature database.
In one embodiment, the system needs to determine the current camera's Pose (Pose), i.e., the angle of rotation of the camera relative to the zero homing state, from the input images. For this reason, we need to establish an accurate mapping relationship between the image and the rotation angle of the camera, specifically:
we first need to establish a mapping database of field sideline images (feature vectors) and camera poses (poses), and this process uses a synthetic field sideline image dataset by controlling the camera rotation angle. That is, each composite edge map obtained offline in the data set simultaneously corresponds to a camera parameter matrix (which is used to generate the composite edge image). The method is supposed to be used for the football match director, because the camera is a fixed first machine position on the central line of the football court, the left-right rotation angle can obtain the information of the current left-right half court, namely the football is currently in the left half court or the right half court, therefore, the data of the whole court can be obtained through the rotation of the camera. And then inputting the acquired image data into a field sideline generating network to obtain a synthesized field sideline graph, and extracting the network by using the characteristic vector to obtain a field sideline image data set, namely a characteristic data set. And taking the feature vector of the field sideline as a keyword of the database index to organize a camera parameter matrix library. The new mechanism of one-to-one correspondence of the sideline feature vector and the camera parameter matrix is the basis of the automation of camera rotation angle estimation.
Referring to fig. 3, fig. 3 is a flowchart of a technical scheme for automatically tracking a camera rotation angle according to an embodiment of a camera rotation angle analysis method of the present invention, where the flowchart of the technical scheme provides a step of obtaining a rotation direction of a camera lens of a sports field in real time, so as to help an intelligent director to determine a field position where the camera lens focuses on. Specifically, the flow chart illustrates two work routes:
(1) and (5) generating a route by the edge image feature database, and performing step 1 → 3 → 4 → 5. This is the step of building a library of edge maps. The system trains the feature extraction network and the feature conversion network of the boundary graph by using the synthesized field boundary graph, thereby establishing effective indexes for a large number of boundary graphs in a database and facilitating the retrieval of the boundary graphs. Each of the edge maps in the database is accompanied by a corresponding matrix of camera parameters.
(2) The camera rotation angle generates a route, step 2 → 3 → 4 → 6 → 7 → the result. This is a step of automatically calculating the rotation angle of the camera in real time through the current frame of the camera. Firstly, the system automatically generates a field side line graph for an input scene, and then converts the graph into a feature vector, wherein the feature vector and the feature vector of the field side line in the database are in the same feature space, so that the best matching field side line can be found in the database through feature matching, and the rotation angle of the camera can be estimated according to the best matching field side line.
In the embodiment, a field image to be detected is acquired, a field side line graph is generated according to the field image to be detected, corresponding field side line feature vectors are generated for the field side line graph, feature matching is performed on the field side line feature vectors and image feature vectors in a feature database to acquire a camera parameter matrix corresponding to the image feature vectors, and camera rotation angles corresponding to the field image to be detected are obtained by decoding camera parameters. In the embodiment, the one-to-one correlation between the field sideline characteristic vector and the camera parameter matrix is established, so that the system can automatically acquire the corresponding camera rotation angle through the matching of the field sideline image characteristics, and the whole estimation process is full-automatic in real time without manual intervention.
Further, based on the first embodiment of the camera rotation angle analysis method of the present invention, a second embodiment of the camera rotation angle analysis method of the present invention is proposed.
Referring to fig. 4, fig. 4 is a detailed flowchart of step S20 in an embodiment of the method for analyzing a rotation angle of a camera according to the present invention, and in a second embodiment, the step of generating a field contour map based on the image of the field to be detected includes;
step S21, inputting the site image to be detected into a mask generator and a field edge generator to generate a corresponding initial site mask image and an initial site edge image;
and step S22, splicing the initial site mask map and the initial site edge map, and inputting an edge enhancement network to obtain an enhanced site edge map.
After the field image to be detected is obtained, the field image to be detected is input into the mask generator and the field edge generator to generate the corresponding initial field mask image and the initial field edge image, and the enhanced field edge image is obtained through the edge enhancement network.
The respective steps will be described in detail below:
step S21, inputting the site image to be detected into a mask generator and a field edge generator to generate a corresponding initial site mask image and an initial site edge image;
in one embodiment, the field image to be detected generates a field contour map in two stages. The first stage is to generate a field Mask (Mask) map and an initial field boundary map. This phase is mainly composed of two homogeneous generating networks, namely a mask generator and a field edge generator. The input of the two generators is the current image frame, and the output is the field Mask and the field edge Map respectively. The architecture of the network is a few lines of basic units BU "convolutional layers- > batch normalization layer- > nonlinear activation functions", all convolutional layers use 3x3 filter kernel companion stride 2. BU/N64-BU/N128-BU/N256-BU/N512-BU/N512-BU/N512, wherein BU/Nk indicates that the basic unit BU has k characteristic filtering cores. The decoding subnetwork structure after encoding is as follows: BU/N512-BU/N1024-BU/N1024-BU/N512-BU/N256-BU/N128. In order for the decoder to obtain more original information, each layer of the encoder is copied to the corresponding layer of the decoder, i.e. the information association of the encoder and the decoder is realized.
And step S22, splicing the initial site mask map and the initial site edge map, and inputting an edge enhancement network to obtain an enhanced site edge map.
In an embodiment, the initial bottom line graph generated by the mask generator and the field edge generator is further enhanced. Generally, the pictures of the playing field often relate to the complex background contents of the field, including auditoriums and billboards, and meanwhile, the sidelines of the field are also shielded by players, referees and the like on the field, so that the detection quality of the sidelines is influenced. In response to these unique problems, we have designed a second stage enhanced generation network. The input of the stage is the splicing of the field Mask graph generated in the first stage and the field initial edge graph. The purpose of our input of the venue mask together is that the network can automatically learn the venue foreground and auditorium background. The site mask information is added to reduce the interference of site background. As shown in fig. 5 (right), fig. 5 is a network structure diagram generated by a field edge according to an embodiment of the camera rotation angle analysis method of the present invention, and the edge enhanced network first extracts features and reduces dimensions through a convolution layer of 3 layers 3x3 (i.e. K3x 3C 32Conv → K3x 3C 64 Conv → K3x 3C 128 Conv in fig. 5), so as to reduce the calculation amount of the following dense sub-node network. The feature map is further refined through two residual Dense sub-networks (i.e., two sense blocks in fig. 5). The depth and the complexity of the network are increased by the residual error dense sub-networks, so that the capacity of the network is improved, the sideline information can be better extracted, and a more accurate field sideline graph is generated. After dense subnetting, the decoding part, the main design feature convolution (K3x 3C 128C)onv) and Upsampling (Upsampling Block), thereby generating a final field-edge graph. Suppose that the mask and edge maps generated in the first stage are I mask And
the second stage edge enhancement generation calculation can be expressed as the following formula:
wherein f is θ Namely, the network is generated by edge enhancement, and theta is the network weight needing to be learned. Also, to boost the performance of the generator, we purposely increase the L1 penalty, i.e.
Further, in an embodiment, the training of the mask generator and the field edge generator includes:
step S23, inputting training data image frames and random noise into an initial mask generator to obtain a generated site mask image;
step S24, acquiring a real site mask image corresponding to the training data image frame, inputting the real site mask image, the generated site mask image and the training data image frame into a first target loss function, iteratively calculating the first target loss function until the first target loss function is converged, and training to obtain a mask generator;
step S25, inputting training data image frames and random noise into an initial field edge generator to obtain a generated field edge graph;
step S26, obtaining a real field edge graph corresponding to the training data image frame, inputting the real field edge graph, the generated field edge graph and the training data image frame into a first target loss function, iteratively calculating the first target loss function until the first target loss function is converged, and training to obtain a field edge generator.
In an embodiment, the field edge generator and the mask generator are isomorphic, that is, the training mode is the same, the training data image frame and the random noise are used as input, the real field mask image is used as auxiliary data, a first target loss function is adopted for training, and the field edge generator and the mask generator are obtained when the first target function meets the requirement.
Specifically, if we assume that the input image (training data image frame) is x and the generated field mask (mask) or field-side map (map) is y, then our generator can be represented as G (x, z), where z is random noise, and the output of the generator is the generated image, i.e., given the input image x as the condition and the random noise z, G can generate an image that is as similar as possible to the real field mask or map y; while the discriminator can be expressed as D (x, y) and D (x, G (x, z)) to discriminate whether the image is real or generated. Given all training data x and y, the first objective loss function of network training can be expressed as:
Respectively inputting training data image frame and random noise into initial mask generator/initial field edge generator, respectively calculating first objective function, and when the first objective function is reached
And after convergence is achieved, training is finished, and a field edge generator and a mask generator are respectively obtained.
The method comprises the steps that a field image to be detected is input into a mask generator and a field edge generator, and an initial field mask image and an initial field edge image are generated in a first stage; and in the second stage, after the initial site mask map and the initial site sideline map are spliced, a sideline enhanced network is input, noise outside the site is eliminated by integrating site mask information, and meanwhile, the sideline enhanced network is designed, so that a clearer and more complete site sideline map can be generated. By the aid of the two-stage field sideline generation algorithm, the field sideline is effectively detected, and simultaneously, boundary diagrams generated by auditoria outside the field, advertising boards and players on the field are successfully eliminated, which cannot be achieved by a traditional Edge detection method.
Further, a third embodiment of the camera rotation angle analyzing method of the present invention is proposed based on the previous embodiment of the camera rotation angle analyzing method of the present invention.
Referring to fig. 6, fig. 6 is a schematic view of a detailed flow of step S30 in an embodiment of the method for analyzing a rotation angle of a camera according to the present invention, in which the step of obtaining a feature vector of a field boundary corresponding to the field boundary map includes:
step S31, inputting the field boundary map into an image feature extraction network, and extracting to obtain an initial feature vector;
and step S32, inputting the initial feature vector into an image feature conversion network to generate a field side line feature vector.
In this embodiment, feature extraction and feature conversion are performed on the field edge graph, so as to obtain a field edge feature vector for feature database retrieval. Specifically, the image feature extraction network may be obtained by training a published classical feature extraction network, and feature transformation is performed to make similar field edge maps closer in space and farther away, and specifically, some common methods are also used for feature transformation, such as: standard, Min-Max Scaling/Normalization, etc.
Further, in an embodiment, the image feature extraction and training of the image feature transformation network includes:
step S33, acquiring a field side line image pair, inputting the field side line image pair into a feature extractor, and generating a first feature vector and a second feature vector, wherein the field side line image pair comprises a first image and a second image;
step S34, inputting the first eigenvector and the second eigenvector into a feature converter to obtain a first space conversion vector and a second space conversion vector after space conversion;
step S35, inputting the edge image pair, the label of the edge image pair, the first spatial transformation vector, and the second spatial transformation vector into a second target loss function, and iteratively training the second target loss function until the second target loss function satisfies a convergence condition, to obtain the image feature extraction network and the image feature transformation network.
Compared with the common feature extraction method based on ImageNet pre-training, the image feature extraction and image feature conversion network provided by the embodiment can generate feature vectors with higher resolution, and is favorable for accurate matching of field edges.
Referring to fig. 7, fig. 7 is a network structure diagram of the field sideline image feature extraction and feature transformation according to an embodiment of the method for analyzing the rotation angle of the camera of the present invention, which is specifically described as follows:
and a field sideline image feature generator. Directly searching and matching generated field edge images (ultra-high dimensional data) is an impossible task, and effective and concise features need to be extracted from the images for efficient storage and retrieval. A commonly used method is to directly convert the field edge image into a feature vector using a pre-trained feature extractor on ImageNet. However, the features extracted in this way cannot well represent edge images, and therefore, the distances between different image features cannot be effectively measured, and therefore, the method is not suitable for image retrieval. For this reason, we specially design a field sideline image feature extraction and conversion network. As shown in FIG. 7, the input for network training is a pair of edge images, i.e., a first image and a second image (x) 1, x 2 ) Feature encoder f θ (extractor) then extracts features (f) for the images, respectively θ (x 1 ),f θ (x 2 ) I.e. the first feature vector and the second feature vector. Our design has versatility, the feature encoder can be of any network structure such as AlexNet and deep residual network. We use in our application an 18-layer residual network.
And a field sideline image feature converter. The signature converter consists of 3 fully-coupled layers (i.e., FC in fig. 7) DxD (D1024), each convolution being followed by a batch normalization operation (i.e., BatchNorm in fig. 7) and a nonlinear activation (i.e., RELU in fig. 7), except for the last layer. The input in the network training is the output of the feature code, namely a first feature vector and a second feature vector (f) θ (x 1 ),f θ (x 2 )). The converters are respectively aligned with the field side line x 1 And x 2 The feature vector of (a) is spatially transformed to obtain a first spatially transformed vector and a second spatially transformed vector, i.e. f π (f θ (x 1 ) And f) and π (f θ (x 2 ) The goal of this transformation is to make similar field edge graphs closer in space, and farther otherwise. Therefore, the process of feature generation and transformation network training is to optimize the following second objective loss function:
y*D(x 1 ,x 2 )+(1-y)*{max(0,m-D(x 1 ,x 2 ))}
wherein D (x) 1 ,x 2 )＝sqrt((f π (f θ (x 1 ))-f π (f θ (x 2 ))) 2 ) Mapping two field boundaries in a new feature space
The Euclidean distance of the inner face, y is 1/0, which is the edge line image pair (x 1, x 2 ) The notation of (1) indicates that the two field edges are similar. m is a threshold to ensure that dissimilar edge distances are not less than the threshold.
The network structure of the feature transformer, although appearing somewhat similar to a generic multi-layer perceptron, has a totally different effect due to the batch normalization operation. In essence, such normalization makes the features comparable between feature vectors. In summary, the feature transformer allows the system to transform features in multiple non-linear spaces by adding a full-link layer, on one hand, to increase the learning capacity/capability of the network, and on the other hand, to project the image into a more highly condensed space (or semantically more advanced space). Such a design has two major benefits: firstly, the Euclidean distance of the feature space can accurately represent the similarity of the field sidelines, namely, the feature vector has higher resolution, and secondly, the feature vector with relatively low dimension can effectively improve the retrieval efficiency of the image library.
In this embodiment, the field side line graph is input to an image feature extraction network, an initial feature vector is obtained by extraction, and the initial feature vector is input to an image feature conversion network to generate a field side line feature vector. Based on the self-supervision learning mechanism and the network provided by the embodiment, the accurate feature extraction network and the feature conversion network of the field sideline image can be trained. The training of the process of projecting the edge images to the low-dimensional feature space with higher resolution does not need data identification, and is an automatic supervision learning process. Therefore, the field sideline feature vector with higher resolution can be obtained, and the retrieval efficiency of the image library can be effectively improved by the feature vector with relatively low dimension.
Further, a fourth embodiment of the camera rotation angle analyzing method of the present invention is proposed based on the previous embodiment of the camera rotation angle analyzing method of the present invention.
In this embodiment, the step of performing feature matching on the field sideline feature vector and an image feature vector in a feature database, and acquiring a camera parameter matrix corresponding to the image feature vector includes:
and step S41, matching the field sideline characteristic vector with the image characteristic vector in the characteristic database through a local sensitive hash algorithm to obtain a corresponding camera parameter matrix.
In one embodiment, the field edge feature vectors are matched. The general search method is to find the matched field edge feature vector from the database by using the nearest neighbor traversal method. However, the nearest neighbor algorithm generally has low matching accuracy, is time-consuming, and is difficult to meet the real-time requirement. To this end, this patent proposes two innovations to overcome this problem. Firstly, the field sideline image is subjected to feature conversion, so that the feature vector is more compact, and high-performance quick matching is easier to realize. Secondly, when the image feature vectors are matched with the image feature vectors of the database, a faster and more accurate locality sensitive hashing algorithm is used. In a Local Sensitive Hashing (LSH) algorithm, LSH can implement dimension reduction + local search matching, where the LSH steps are exemplified as follows: the method comprises the steps of firstly compressing high-dimensional sparse data through Min Hashing so as to improve the calculation efficiency, compressing a large matrix into a small matrix through Hash mapping through Min Hashing, and simultaneously keeping the similarity among columns so as to reduce the complexity. On the basis of the signature vectors obtained by Min Hashing, each vector is divided into a plurality of sections, which are called bands (namely each band comprises a plurality of lines), hash bucket division is respectively carried out on the signature vectors of all users on each band to calculate the similarity, users divided into the same bucket on any band are mutually similar users, and thus the similar user group of each user can be found only by calculating the similarity of all candidate users.
In the embodiment, the field sideline feature vectors are matched and retrieved in the feature database through Hash, so that the corresponding camera parameter matrix is obtained, the speed of obtaining the rotation angle of the camera can be increased, and the function of image feature conversion can be better exerted.
The invention also provides a camera rotation angle analysis device. As shown in fig. 8, fig. 8 is a functional block diagram of an embodiment of the camera rotation angle analysis device according to the present invention.
The camera rotation angle analysis device of the present invention includes:
the acquisition module 10 is used for acquiring an image of a field to be detected;
the image generation module 20 is configured to generate a field-side line map based on the field image to be detected;
the feature extraction module 30 is configured to obtain a field edge feature vector corresponding to the field edge map;
the matching module 40 is configured to perform feature matching on the field sideline feature vector and an image feature vector in a feature database, and acquire a camera parameter matrix corresponding to the image feature vector;
and the decoding module 50 is configured to decode the camera parameter matrix obtained by matching to obtain a corresponding camera rotation angle.
Preferably, the obtaining module is further configured to:
acquiring a field sideline image dataset synthesized by a camera under different rotation angles;
generating and storing camera parameter matrix parameters corresponding to the sideline site images;
correspondingly generating the image feature vector based on the synthesized field sideline image dataset;
and establishing a corresponding relation between the image feature vector and the camera parameter matrix to obtain a feature database.
Preferably, the image generation module is further configured to:
inputting the field image to be detected into a mask generator and a field edge generator to generate a corresponding initial field mask image and an initial field edge image;
and after the initial site mask map and the initial site sideline map are spliced, inputting a sideline enhancement network to obtain an enhanced site sideline map.
Preferably, the camera rotation angle analysis device further includes:
the model training module is used for inputting training data image frames and random noise into the initial mask generator to obtain a generated site mask image;
acquiring a real site mask image corresponding to the training data image frame, inputting the real site mask image, the generated site mask image and the training data image frame into a first target loss function, iteratively calculating the first target loss function until the first target loss function is converged, and training to obtain a mask generator;
inputting training data image frames and random noise into an initial field edge generator to obtain a generated field edge line graph;
and acquiring a real field side line graph corresponding to the training data image frame, inputting the real field side line graph, the generated field side line graph and the training data image frame into a first target loss function, iteratively calculating the first target loss function until the first target loss function is converged, and training to obtain the field side generator.
Preferably, the model training module is further configured to:
splicing the generated field boundary map and the generated field mask map, inputting the spliced field boundary map and the generated field mask map into an initial boundary enhancement generator, and outputting a boundary enhancement map;
inputting the edge enhancement graph, the generated field boundary graph and the generated field mask graph into an enhancement loss function, iteratively calculating the enhancement loss function until the enhancement loss function is converged, and training to obtain the edge enhancement network.
Preferably, the feature extraction module is further configured to:
inputting the field contour map into an image feature extraction network, and extracting to obtain an initial feature vector;
and inputting the initial characteristic vector into an image characteristic conversion network to generate a field sideline characteristic vector.
Preferably, the model training module is further configured to:
acquiring a field sideline image pair, inputting the field sideline image pair into a feature extractor, and generating a first feature vector and a second feature vector, wherein the field sideline image pair comprises a first image and a second image;
inputting the first feature vector and the second feature vector into a feature converter to obtain a first space conversion vector and a second space conversion vector after space conversion;
inputting the edge image pair, the label of the edge image pair, the first space transformation vector and the second space transformation vector into a second target loss function, and iteratively training the second target loss function until the second target loss function meets a convergence condition to obtain the image feature extraction network and the image feature transformation network.
Preferably, the matching module is further configured to:
and matching the field sideline characteristic vector with the image characteristic vector in the characteristic database through a local sensitive Hash algorithm to obtain a corresponding camera parameter matrix.
The invention also provides a storage medium.
The storage medium of the present invention stores a camera rotation angle analysis program, which when executed by a processor implements the steps of the camera rotation angle analysis method as described above.
The method implemented when the camera rotation angle analysis program running on the processor is executed may refer to each embodiment of the camera rotation angle analysis method of the present invention, and details thereof are not repeated herein.
It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or system. Without further limitation, an element defined by the phrase "comprising a raman spectral data process" does not exclude the presence of another like element in a process, method, article, or system that comprises the element.
The above-mentioned serial numbers of the embodiments of the present invention are merely for description and do not represent the merits of the embodiments.
Through the above description of the embodiments, those skilled in the art will clearly understand that the method of the above embodiments can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware, but in many cases, the former is a better implementation manner. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium (e.g., ROM/RAM, magnetic disk, optical disk) as described above and includes instructions for enabling a terminal device (e.g., a mobile phone, a computer, a server, or a network device) to execute the method according to the embodiments of the present invention.
The above description is only a preferred embodiment of the present invention, and not intended to limit the scope of the present invention, and all modifications of equivalent structures and equivalent processes, which are made by using the contents of the present specification and the accompanying drawings, or directly or indirectly applied to other related technical fields, are included in the scope of the present invention.
Claims (11)
1. A camera rotation angle analysis method, characterized by comprising the steps of:
acquiring a field image to be detected;
generating a field boundary diagram based on the field image to be detected;
obtaining a field sideline feature vector corresponding to the field sideline graph;
performing feature matching on the field sideline feature vector and an image feature vector in a feature database, and acquiring a camera parameter matrix corresponding to the image feature vector;
and decoding the camera parameter matrix obtained by matching to obtain the corresponding camera rotation angle.
2. The camera rotation angle analysis method according to claim 1, wherein the characteristic database creating step includes:
acquiring a field sideline image dataset synthesized by a camera under different rotation angles;
generating and storing a camera parameter matrix corresponding to the field sideline image;
correspondingly generating the image feature vector based on the synthesized field sideline image dataset;
and establishing a corresponding relation between the image feature vector and the camera parameter matrix to obtain a feature database.
3. The camera rotation angle analysis method according to claim 1, wherein the step of generating a field-ground-boundary map based on the field image to be detected includes;
inputting the field image to be detected into a mask generator and a field edge generator to generate a corresponding initial field mask image and an initial field edge image;
and after splicing the initial field mask map and the initial field boundary map, inputting a boundary enhancement network to obtain an enhanced field boundary map.
4. The camera rotation angle analysis method according to claim 3, wherein the training of the mask generator and the field edge generator comprises:
inputting training data image frames and random noise into an initial mask generator to obtain a generated site mask image;
acquiring a real site mask image corresponding to the training data image frame, inputting the real site mask image, the generated site mask image and the training data image frame into a first target loss function, iteratively calculating the first target loss function until the first target loss function is converged, and training to obtain a mask generator;
inputting training data image frames and random noise into an initial field edge generator to obtain a generated field edge line graph;
and acquiring a real field side line graph corresponding to the training data image frame, inputting the real field side line graph, the generated field side line graph and the training data image frame into a first target loss function, iteratively calculating the first target loss function until the first target loss function is converged, and training to obtain the field side generator.
5. The camera rotation angle analysis method according to claim 4, wherein the training step of the edge enhancement network includes:
splicing the generated field boundary graph and the generated field mask graph, inputting the spliced field boundary graph and the generated field mask graph into an initial boundary enhancement generator, and outputting a boundary enhancement graph;
inputting the edge enhancement graph, the generated field boundary graph and the generated field mask graph into an enhancement loss function, iteratively calculating the enhancement loss function until the enhancement loss function is converged, and training to obtain the edge enhancement network.
6. The method for analyzing rotation angle of a camera according to claim 1, wherein the step of obtaining a feature vector of a field edge corresponding to the field edge map comprises:
inputting the field contour map into an image feature extraction network, and extracting to obtain an initial feature vector;
and inputting the initial characteristic vector into an image characteristic conversion network to generate a field sideline characteristic vector.
7. The camera rotation angle analysis method according to claim 6, wherein the image feature extraction and training of the image feature conversion network comprises:
acquiring a field sideline image pair, inputting the field sideline image pair into a feature extractor, and generating a first feature vector and a second feature vector, wherein the field sideline image pair comprises a first image and a second image;
inputting the first feature vector and the second feature vector into a feature converter to obtain a first space conversion vector and a second space conversion vector after space conversion;
inputting the edge image pair, the label of the edge image pair, the first space transformation vector and the second space transformation vector into a second target loss function, and iteratively training the second target loss function until the second target loss function meets a convergence condition to obtain the image feature extraction network and the image feature transformation network.
8. The method for analyzing the rotation angle of a video camera according to claim 1, wherein the step of performing feature matching on the field edge feature vector and an image feature vector in a feature database and obtaining a camera parameter matrix corresponding to the image feature vector comprises:
and matching the field sideline characteristic vector with the image characteristic vector in the characteristic database through a local sensitive Hash algorithm to obtain a corresponding camera parameter matrix.
9. A camera rotation angle analyzing apparatus, characterized in that the apparatus comprises:
the acquisition module is used for acquiring an image of a field to be detected;
the image generation module is used for generating a field and ground boundary diagram based on the field image to be detected;
the characteristic extraction module is used for acquiring a field sideline characteristic vector corresponding to the field sideline graph;
the matching module is used for performing feature matching on the field sideline feature vector and an image feature vector in a feature database and acquiring a camera parameter matrix corresponding to the image feature vector;
and the decoding module is used for decoding the camera parameter matrix obtained by matching to obtain the corresponding camera rotation angle.
10. A camera rotation angle analysis apparatus, characterized in that the apparatus comprises: a memory, a processor and a camera rotation angle analysis program stored on the memory and executable on the processor, the camera rotation angle analysis program being configured to implement the steps of the camera rotation angle analysis method according to any one of claims 1 to 8.
11. A storage medium, characterized in that the storage medium has stored thereon a camera rotation angle analysis program, which when executed by a processor implements the steps of the camera rotation angle analysis method according to any one of claims 1 to 8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210515853.3A CN114979470A (en) | 2022-05-12 | 2022-05-12 | Camera rotation angle analysis method, device, equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210515853.3A CN114979470A (en) | 2022-05-12 | 2022-05-12 | Camera rotation angle analysis method, device, equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114979470A true CN114979470A (en) | 2022-08-30 |
Family
ID=82980473
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210515853.3A Pending CN114979470A (en) | 2022-05-12 | 2022-05-12 | Camera rotation angle analysis method, device, equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114979470A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116416671A (en) * | 2023-06-12 | 2023-07-11 | 太平金融科技服务(上海)有限公司深圳分公司 | Face image correcting method and device, electronic equipment and storage medium |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105069809A (en) * | 2015-08-31 | 2015-11-18 | 中国科学院自动化研究所 | Camera positioning method and system based on planar mixed marker |
| US10186038B1 (en) * | 2017-07-18 | 2019-01-22 | Siemens Healthcare Gmbh | Segmentation and representation network for pose discrimination |
| CN110458060A (en) * | 2019-07-30 | 2019-11-15 | 暨南大学 | A kind of vehicle image optimization method and system based on confrontation study |
| CN112949508A (en) * | 2021-03-08 | 2021-06-11 | 咪咕文化科技有限公司 | Model training method, pedestrian detection method, electronic device and readable storage medium |
| US20210192271A1 (en) * | 2019-12-23 | 2021-06-24 | Beijing Institute Of Technology | Method and Apparatus for Pose Planar Constraining on the Basis of Planar Feature Extraction |
| KR20210133083A (en) * | 2020-04-28 | 2021-11-05 | 삼성전자주식회사 | Device and method for obtaining depth of space using camera |
| WO2022027987A1 (en) * | 2020-08-04 | 2022-02-10 | 杰创智能科技股份有限公司 | Image recognition model training method, and image recognition method |
-
2022
- 2022-05-12 CN CN202210515853.3A patent/CN114979470A/en active Pending
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105069809A (en) * | 2015-08-31 | 2015-11-18 | 中国科学院自动化研究所 | Camera positioning method and system based on planar mixed marker |
| US10186038B1 (en) * | 2017-07-18 | 2019-01-22 | Siemens Healthcare Gmbh | Segmentation and representation network for pose discrimination |
| CN110458060A (en) * | 2019-07-30 | 2019-11-15 | 暨南大学 | A kind of vehicle image optimization method and system based on confrontation study |
| US20210192271A1 (en) * | 2019-12-23 | 2021-06-24 | Beijing Institute Of Technology | Method and Apparatus for Pose Planar Constraining on the Basis of Planar Feature Extraction |
| KR20210133083A (en) * | 2020-04-28 | 2021-11-05 | 삼성전자주식회사 | Device and method for obtaining depth of space using camera |
| WO2022027987A1 (en) * | 2020-08-04 | 2022-02-10 | 杰创智能科技股份有限公司 | Image recognition model training method, and image recognition method |
| CN112949508A (en) * | 2021-03-08 | 2021-06-11 | 咪咕文化科技有限公司 | Model training method, pedestrian detection method, electronic device and readable storage medium |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116416671A (en) * | 2023-06-12 | 2023-07-11 | 太平金融科技服务(上海)有限公司深圳分公司 | Face image correcting method and device, electronic equipment and storage medium |
| CN116416671B (en) * | 2023-06-12 | 2023-10-03 | 太平金融科技服务(上海)有限公司深圳分公司 | Face image correcting method and device, electronic equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Li et al. | Dual-resolution correspondence networks | |
| Liu et al. | Densernet: Weakly supervised visual localization using multi-scale feature aggregation | |
| Liao et al. | DR-GAN: Automatic radial distortion rectification using conditional GAN in real-time | |
| CN101504761B (en) | Image splicing method and apparatus | |
| KR101528081B1 (en) | Object recognition using incremental feature extraction | |
| CN112270710B (en) | Pose determining method, pose determining device, storage medium and electronic equipment | |
| Lin et al. | Hnip: Compact deep invariant representations for video matching, localization, and retrieval | |
| Stathopoulou et al. | Semantic photogrammetry–boosting image-based 3d reconstruction with semantic labeling | |
| CN115171165A (en) | Pedestrian re-identification method and device with global features and step-type local features fused | |
| CN111696136B (en) | Target tracking method based on coding and decoding structure | |
| CN114821390B (en) | Method and system for tracking twin network target based on attention and relation detection | |
| JP2004045565A (en) | Method and device for detecting similar time series, program, and recording medium | |
| Luo et al. | Image steganalysis with convolutional vision transformer | |
| CN114979470A (en) | Camera rotation angle analysis method, device, equipment and storage medium | |
| CN115393404A (en) | Double-light image registration method, device and equipment and storage medium | |
| CN114241007A (en) | Multi-target tracking method based on cross-task mutual learning, terminal equipment and medium | |
| Zhou et al. | Explore spatio-temporal aggregation for insubstantial object detection: benchmark dataset and baseline | |
| Liu et al. | Two-stream refinement network for RGB-D saliency detection | |
| CN117094895B (en) | Image panorama stitching method and system | |
| Kong et al. | Hole-robust wireframe detection | |
| CN114694185B (en) | Cross-modal target re-identification method, device, equipment and medium | |
| CN116311384A (en) | Cross-modal pedestrian re-recognition method and device based on intermediate mode and characterization learning | |
| Taya et al. | Detecting tampered regions in JPEG images via CNN | |
| CN115619637A (en) | Automatic splicing and fusing method of panoramic image capable of being transplanted to ARM end | |
| CN113610906B (en) | Multi-parallax image sequence registration method based on fusion image guidance |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |