CN116797961A

CN116797961A - Picture acquisition method and device for moving sphere, computer equipment and storage medium

Info

Publication number: CN116797961A
Application number: CN202310335637.5A
Authority: CN
Inventors: 张伟俊; 侯俊; 马龙祥; 卢睿华
Original assignee: Insta360 Innovation Technology Co Ltd
Current assignee: Insta360 Innovation Technology Co Ltd
Priority date: 2023-03-24
Filing date: 2023-03-24
Publication date: 2023-09-22

Abstract

The application relates to a picture acquisition method, a picture acquisition device, a picture acquisition computer device, a picture acquisition storage medium and a picture acquisition computer program product for a moving sphere. The method comprises the following steps: identifying a moving sphere according to the sphere region track in the multi-frame picture; determining a target sphere from each sports sphere according to the confidence that each sports sphere belongs to a preset sphere; and acquiring pictures of the target sphere according to the position information of the target sphere. According to the method, the moving sphere is identified through the sphere region track formed by the sphere regions of the multiple frames, so that the movement trend of the sphere can be included in the sphere detection range, the static sphere is eliminated, and the identification accuracy of the sphere region in the picture is improved; determining target spheres from the sports spheres according to the confidence that the sports spheres belong to preset spheres, and more accurately identifying the target spheres such as basketball; and acquiring pictures of the target sphere according to the position information of the target sphere, so as to realize automatic shooting of the target sphere.

Description

Picture acquisition method and device for moving sphere, computer equipment and storage medium

Technical Field

The present application relates to the field of image processing technology, and in particular, to a method, an apparatus, a computer device, a storage medium, and a computer program product for capturing images of a moving sphere.

Background

With the wide application of image processing in civil and commercial fields, the technology of image acquisition according to the position of a target has more and more application scenes.

When the image acquisition is carried out on the ball body of the sports video, people heads in the audience at which the competition field is disordered, static balls on the field side, balls in the poster on the wall and the like can trigger the detector and further trigger the tracker, and the target ball body is difficult to accurately determine.

Disclosure of Invention

Based on the foregoing, it is necessary to provide a method, an apparatus, a computer device, a computer readable storage medium and a computer program product for acquiring a picture of a moving sphere, which can increase accuracy of target sphere identification and avoid occurrence of false alarm.

In a first aspect, the present application provides a method for capturing images of a moving sphere. The method comprises the following steps:

identifying a moving sphere according to the sphere region track in the multi-frame picture;

determining a target sphere from each moving sphere according to the confidence coefficient that each moving sphere belongs to a preset sphere;

And acquiring pictures of the target sphere according to the position information of the target sphere.

In one embodiment, the identifying the moving sphere according to the sphere region track in the multi-frame picture includes:

in a multi-frame picture, sequentially carrying out sphere region identification based on the classification model of the preset sphere to obtain a plurality of sphere regions;

selecting related sphere areas with the intervals meeting the interval conditions from the sphere areas; the related sphere area is an area formed by the same sphere in different frames of pictures;

combining the associated sphere areas of different frames of pictures according to time sequence to obtain each sphere area chain;

and determining the moving sphere in the picture of each frame according to each sphere region chain.

In one embodiment, the frame obtained by frame acquisition of the target sphere is a current frame, and the multi-frame is a forward frame of the current frame.

In one embodiment, the determining the moving sphere in the picture of each frame according to each sphere region chain includes:

determining the sphere accumulated displacement of each sphere regional chain among the pictures of each frame;

Removing spheres with the accumulated displacement of the spheres smaller than a movement sphere displacement threshold value from spheres corresponding to the sphere regional chains to obtain filtered spheres;

and determining the moving spheres in each picture according to each filtered sphere.

In one embodiment, the determining the cumulative displacement of the sphere of each sphere regional chain between the pictures of each frame includes:

in the pictures of each frame, determining each sphere corresponding to each sphere region chain;

and determining the sphere accumulated displacement of each sphere regional chain according to the relative position change of each sphere and the basket between the frames.

determining standard pictures in the pictures of each frame;

calibrating the pictures of each frame according to the matching points of the pictures of each frame and the standard picture to obtain aligned pictures;

and in the aligned picture, according to the sphere track corresponding to each sphere regional chain, determining the sphere accumulated displacement of each sphere regional chain.

In one embodiment, the determining the target sphere from the moving spheres according to the confidence that each moving sphere belongs to the preset sphere includes:

in the pictures of each frame, determining a sphere region chain where each moving sphere is located in sequence;

determining the confidence that spheres corresponding to the sphere region chains belong to preset spheres;

determining target sphere region chains in each sphere region chain according to the confidence coefficient;

and determining the target sphere according to the sphere corresponding to the target sphere region chain.

In one embodiment, the determining the confidence that the sphere corresponding to each sphere region chain belongs to the preset sphere includes:

in each sphere region chain, determining associated spheres in at least two frames of pictures forward to a target timestamp;

and determining the confidence coefficient of each sphere region chain according to the confidence coefficient of the associated sphere in the at least two frames of pictures relative to the preset sphere.

In one embodiment, the determining the target sphere region chain in each sphere region chain according to the confidence comprises:

determining a reference chain in each sphere region chain;

And if the fact that each sphere region chain has a replacement chain meeting the reference chain adjustment confidence coefficient condition is determined, adjusting the reference chain according to the replacement chain until each sphere region chain does not have the replacement chain meeting the reference chain adjustment confidence coefficient condition, and determining the current reference chain as a target sphere region chain.

In one embodiment, the reference chain is a target sphere region chain of adjacent time stamps to a target time stamp.

In one embodiment, the determining that each sphere region chain has an alternate chain that satisfies a reference chain adjustment confidence condition includes:

and taking the sphere region chain with the confidence degree larger than that of the reference chain as a replacement chain in each sphere region chain.

In one embodiment, a ratio between the confidence of the replacement chain and the confidence of the reference chain exceeds a region chain adjustment threshold that is a preset confidence multiple of the reference chain.

In one embodiment, said adjusting said reference chain according to said replacement chain comprises:

and if one replacement chain exists in the picture of the target timestamp, adjusting the replacement chain to be the reference chain.

if a plurality of the replacement chains exist in the picture with the target timestamp, determining the associated sphere area with the largest area in each replacement chain in the picture with the target timestamp; and determining the replacement chain to which the associated sphere region with the largest area belongs as the reference chain.

In one embodiment, the acquiring the picture of the target sphere according to the position information of the target sphere includes:

determining a picture offset according to the position information of the target sphere;

controlling the rotation of the cradle head according to the picture offset, and collecting the moving picture of the target sphere in the rotation process; wherein the target sphere is in the motion picture.

In one embodiment, the acquiring the picture of the target sphere according to the position information of the target sphere further includes:

determining a target moving picture of the target sphere entering the basket according to the path of the target sphere in the moving picture of each frame;

and combining the target moving picture with the associated moving picture of the target moving picture to obtain a target moving fragment.

In one embodiment, the determining a target moving picture of the target sphere into the basket according to a path of the target sphere in each moving picture includes:

extracting an entrance area screen and an exit area screen of the target sphere from each of the moving pictures according to a path of the target sphere in each of the moving pictures;

determining a moving picture between the entrance area picture and the exit area picture as a target moving picture;

the entrance area picture is a moving picture of the target sphere in the basket entrance area; the outlet area picture refers to a moving picture in which the target sphere is located in the basket outlet area.

In one embodiment, the determining the target moving picture of the target sphere entering the basket according to the track of the target sphere in each moving picture includes:

determining a reference area path of the target sphere relative to the basket according to the relative position of the target sphere and the basket in each frame of the moving picture;

determining candidate goal pictures in the moving pictures of each frame according to the reference area path;

And in the candidate goal pictures, determining a target moving picture of the target sphere entering the basket according to the shielding relation between the target sphere and the basket.

In one embodiment, the method further comprises:

when the target sphere belongs to basketball, extracting gesture key points of a target object from each moving picture according to the human body gesture corresponding to basketball scene information;

if the action of the target object is recognized to belong to shooting action according to the gesture key points, combining a moving picture where the shooting action is positioned to obtain shooting fragments;

determining a set of related pictures of the shot segments from the moving pictures in a time-stamped order;

performing frame rate adjustment on the shooting segments to obtain slow-motion shooting segments with target frame rates;

and synthesizing a target video based on the related picture set and the slow motion shooting segment.

In a second aspect, the application also provides a picture acquisition device of the sports sphere. The device comprises:

the sphere identification module is used for identifying a moving sphere according to the sphere region track in the multi-frame picture;

the sphere determining module is used for determining a target sphere from each moving sphere according to the confidence that each moving sphere belongs to a preset sphere;

And the picture acquisition module is used for carrying out picture acquisition on the target sphere according to the position information of the target sphere.

In a third aspect, the present application further provides a handheld pan-tilt, including a motor and a processor, where the motor is configured to control rotation of the pan-tilt, and the processor implements the step of capturing a picture of a moving sphere in any of the foregoing embodiments when executing the computer program.

In a fourth aspect, the present application also provides a computer device. The computer device comprises a memory storing a computer program and a processor which when executing the computer program performs the steps of the moving sphere picture acquisition in any of the embodiments described above.

In a fifth aspect, the present application also provides a computer-readable storage medium. The computer readable storage medium having stored thereon a computer program which when executed by a processor performs the steps of picture acquisition of a moving sphere in any of the embodiments described above.

In a sixth aspect, the application also provides a computer program product. The computer program product comprises a computer program which, when executed by a processor, carries out the steps of the picture acquisition of a moving sphere in any of the embodiments described above.

According to the picture acquisition method, the picture acquisition device, the computer equipment, the storage medium and the computer program product for the moving spheres, the moving spheres are identified through the sphere region tracks formed by the sphere regions of the multiple frames, so that the movement trend of the spheres can be included in the sphere detection range, static spheres are eliminated, and the identification accuracy of the sphere regions in the picture is improved; determining target spheres from the sports spheres according to the confidence that the sports spheres belong to preset spheres, and more accurately identifying the target spheres such as basketball; in this case, the image of the target sphere is acquired according to the position information of the target sphere, so that automatic shooting of the target sphere is realized.

Drawings

FIG. 1 is an application environment diagram of a picture acquisition method of a moving sphere in one embodiment;

FIG. 2 is a flow chart of a method for capturing images of a moving sphere according to one embodiment;

FIG. 3 is an environmental view of a basketball scene of a method of scene acquisition of a sport ball in one embodiment;

FIG. 4 is an effect diagram of a picture acquisition method of a moving sphere in one embodiment;

FIG. 5 is an effect diagram of a picture acquisition method of a moving sphere according to another embodiment;

FIG. 6 is a shot view of a method of view acquisition of a moving sphere in one embodiment;

FIG. 7 is a diagram of a target motion picture in one embodiment;

FIG. 8 is a schematic diagram of a reference area and a reference area path in one embodiment;

FIG. 9 is a block diagram of a motion sphere frame acquisition device in one embodiment;

fig. 10 is an internal structural view of a computer device in one embodiment.

Detailed Description

The present application will be described in further detail with reference to the drawings and examples, in order to make the objects, technical solutions and advantages of the present application more apparent. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the application.

The image acquisition method of the moving sphere provided by the embodiment of the application can be applied to an application environment shown in figure 1. The terminal 102 may be, but not limited to, various cameras, video cameras, panoramic cameras, motion cameras, personal computers, notebook computers, smartphones, tablet computers, and portable wearable devices, which may be smart watches, smart bracelets, headsets, etc. The terminal 102 may be fixed to the holder body by welding or the like, and may also be detachably connected or rotatably connected to the holder body.

In one embodiment, as shown in fig. 2, a method for capturing images of a moving sphere is provided, and the method is applied to the terminal 102 in fig. 1 for illustration, and includes the following steps:

step 202, identifying a moving sphere according to the sphere region track in the multi-frame picture.

The sphere regions in the multi-frame screen may each contain spheres that can be classified and identified, or may be estimated from the trajectories of the identified spheres at a certain stage. For example: for the frames a, B and C arranged in time sequence, wherein each of the frames a and C has a identifiable sphere region, and the frame B does not have a identifiable sphere region, a sphere region track is formed by the identifiable sphere regions in the frames a and C, and the sphere region in the frame B is determined by the sphere region track.

The sphere region track is the region where each sphere is respectively located in a plurality of frames, and the regions which are sequentially changed in the frames and belong to one sphere are subjected to inter-frame association to obtain the sphere region track of the sphere. The spheres are detected by respective detection methods, and the spheres can be detected based on manually designed features (such as a template matching method, a key point matching method, a key feature method and the like), or can be detected by using a Convolutional Neural Network (CNN) technology. The method is preferably used for acquiring a large amount of basketball data, training a detector based on a convolutional neural network and detecting through the trained detector. Among them, convolutional neural networks, including but not limited to YOLO (You Only Look Once, you look only once), SSD (Single Shot MultiBox, single shot multi-box), R-CNN (Region-based Convolutional Neural Networks, region-based convolutional neural network) or Mask R-CNN (Mask Region-based Convolutional Neural Networks, masked Region-based convolutional neural network), etc., as long as the convolutional neural network can recognize the characteristics of a sphere.

It can be understood that, because the proportion of the moving sphere in the picture is small, and the moving sphere may be in a semi-shielding state or in a motion blurring state in the air, the characteristic is not very typical, and the detector easily detects spherical marks (spherical LOGO), heads, faces and the like on the poster as the moving sphere by mistake, so as to generate false alarms. Therefore, the sphere areas of the multiple frames are inter-frame related to form a sphere area track so that the moving sphere can be identified from spheres stably detected by the continuous multiple frames. A moving sphere is a sphere that moves in a multi-frame picture, i.e., a sphere that is not stationary.

In an alternative embodiment, identifying a moving sphere from a sphere region trajectory in a multi-frame picture includes: sequentially identifying the interested areas with preset ball characteristics in a multi-frame picture to obtain a plurality of ball areas; determining an associated sphere region set of regions formed by the same sphere in different frames of pictures in a plurality of sphere regions; and determining the moving sphere in each frame of picture according to each sphere region set. Therefore, the feature recognition is respectively carried out through the multi-frame pictures, and the sphere area in each frame of video picture is obtained; and then carrying out inter-frame association on the sphere areas in each frame of video picture, determining an associated sphere area set of each sphere, representing sphere area tracks by the associated sphere area set, and compensating the possibility of false recognition of the neural network model by the sphere area set.

In an alternative embodiment, the sphere areas in the sphere area set are arranged according to the time sequence to form a sphere area sequence, the weights of the different sphere areas are determined according to the distance between the sphere areas and the current picture timestamp, and the moving sphere is determined according to the weights; even part of the sphere area is filtered out, so that the filtered sphere area determines the moving sphere.

In an alternative embodiment, identifying a moving sphere from a sphere region trajectory in a multi-frame picture includes: in a multi-frame picture, sequentially carrying out sphere region identification based on a classification model of a preset sphere to obtain a plurality of sphere regions; selecting related sphere areas with the intervals meeting the interval conditions from the sphere areas; the related sphere area is an area formed by the same sphere in different frames; combining the associated sphere areas of different frames of pictures according to time sequence to obtain each sphere area chain; and determining the moving sphere in each frame of picture according to each sphere region chain.

A sphere region chain is a sequence of sphere regions, and may also be referred to as a sphere tracking chain. Each sphere region chain characterizes the region positions of one sphere which the sphere region chain belongs to, the region positions pass through sequentially at all times, and the track of the sphere is formed by the region positions which pass through sequentially at all times. Illustratively, the moving sphere X is at a position within the region a at a first time, the moving sphere X is at a position within the region B at a second time, and the first time is before the second time, the sphere region chain of the sphere X includes: the moving sphere X is in the region a at a first time and the moving sphere X is in the region B at a second time.

In one embodiment, in a multi-frame picture, sphere region identification is sequentially performed based on a classification model of a preset sphere to obtain a plurality of sphere regions, including: and in each frame of picture, respectively carrying out sphere region identification based on a neural network model aiming at the preset sphere to obtain each sphere region belonging to the preset sphere. Wherein each frame of picture may correspond to one or more sphere regions therein, and a portion of the picture does not have sphere regions. Alternatively, the multi-frame pictures are consecutive to be processed separately for each frame picture.

In one embodiment, selecting an associated sphere region whose sphere region pitch satisfies a pitch condition among a plurality of sphere regions, comprises: measuring the distance between detection results according to the intersection ratio between the sphere areas; and determining the sphere region to which the intersection ratio exceeding the preset value belongs as the associated sphere region meeting the spacing condition.

In one embodiment, determining a motion sphere in each frame of picture according to each sphere region chain comprises: determining a moving sphere area chain with a path conforming to the preset ball movement condition from each sphere area chain; and determining the sphere to which the moving sphere region chain belongs as the moving sphere in each frame of picture.

The sphere region identification is carried out through a classification model of the preset sphere, so that a part of spheres which do not have the preset sphere characteristics can be removed, and a sphere region of a multi-frame picture is obtained; in each frame of picture, determining an associated sphere region according to the sphere region distance, and realizing the inter-frame association of the sphere regions; and then, combining the associated sphere areas of different frame images according to the time sequence to obtain each sphere area chain, and forming ordered inter-frame association, so that each sphere area chain reflects the covered area of the sphere, thereby reflecting the movement track of the sphere more accurately, and further determining the movement sphere in each frame image.

The picture obtained by picture acquisition of the target sphere is a current picture, and the multi-frame picture is a forward picture of the current picture. The current picture is a picture whose current timestamp is to be acquired, and the forward picture of the current picture is a picture whose timestamp is acquired before the current timestamp, that is, the timestamp of the multi-frame picture is before the timestamp of the picture acquisition of the target sphere, in this case, the acquisition parameters of the current picture are determined through the forward picture, and the acquisition parameters of the current picture are gradually adjusted according to the position of the target sphere in the moving picture, so that real-time tracking of the target sphere is formed.

In an exemplary embodiment, one or more methods of a simple online real-time Tracking (SORT) algorithm, a simple online real-time Tracking (Deep SORT) algorithm based on depth features, and the like may be used to perform object Tracking of the sphere on the forward screen of the current screen, so as to obtain a multi-object Tracking result of the current screen, i.e., each moving sphere of the forward screen. For example, assuming that the current video frame is the t-th video frame and the current video frame includes n basketball, the plurality of target tracking results corresponding to the current video frame may be expressed as:

T＝{T ₁ ,T ₂ ,…,T _n and } wherein,wherein t is ₀ Frame number indicating that the nth basketball first appears in the video frame,/for example>Representing the basketball in the detection frame of the t-th video frame.

In one embodiment, determining a motion sphere in each frame of picture from each sphere region chain includes: determining the sphere accumulated displacement of each sphere regional chain among frames; removing spheres with the accumulated displacement of the spheres smaller than the displacement threshold of the moving spheres from the spheres corresponding to the regional chains of the spheres to obtain filtered spheres; and determining the moving sphere in each picture according to each filtered sphere.

The cumulative displacement of the sphere is the displacement generated by the sphere belonging to the sphere area chain in the video picture segment formed by each frame picture. The cumulative displacement of the sphere is used for representing the motion degree of the sphere between frames. For the sphere to which the sphere regional chain belongs, when the cumulative displacement of the sphere is smaller than the displacement threshold of the moving sphere, the sphere belongs to the sphere to be removed; when the cumulative displacement of the sphere is greater than or equal to the moving sphere displacement threshold, the sphere belongs to the filtered sphere. The filtered spheres are potential moving spheres of each frame picture after the spheres are removed from the spheres and the accumulated displacement of the spheres is smaller than the displacement threshold value of the moving spheres.

In an alternative embodiment, determining a moving sphere in each frame from each filtered sphere includes: the filtered sphere is directly used as a sport sphere, and the confidence that the sport sphere belongs to the preset sphere directly determines a target sphere from the filtered sphere.

In an alternative embodiment, determining a moving sphere in each frame from each filtered sphere includes: screening the filtered spheres according to the movement rule of preset spheres to obtain screened spheres conforming to the movement rule; the screened sphere is determined to be a sport sphere. Thereby, the accuracy of the identification of the moving sphere can be increased. When the preset ball is a basketball, the filtered ball is screened again according to the parabolic angle or related parameters, so that the filtered ball conforming to the basketball sport rules is obtained, and the sport ball of the basketball is determined more accurately.

According to the sphere accumulated displacement of each sphere regional chain, the spheres with the sphere accumulated displacement smaller than the movement sphere displacement threshold are removed, so that the spheres with too small displacement such as heads and posters are removed more accurately, the probability of false alarms is reduced, and the calculated amount is relatively small.

In an alternative embodiment, determining the sphere cumulative displacement of each sphere region chain between frames comprises: in each frame of picture, determining each sphere corresponding to each sphere region chain; and determining the sphere accumulated displacement of each sphere regional chain according to the relative position change of each sphere and the basket between each frame of pictures.

The relative position change of the sphere and the basket is the position change generated by the regional chain of each sphere on the premise of taking the basket as a reference object. The relative position change of the spheres and the basket can accurately reflect the accumulated displacement of the spheres in the range close to the basket.

In one embodiment, in each frame of picture, each sphere corresponding to each sphere region chain is determined, including: in each frame of forward pictures of the current picture, respectively determining spheres of each sphere region chain corresponding to each frame of forward pictures; wherein each sphere region chain may correspond to one sphere in a frame of picture, and a frame of picture may have respective spheres of the multi-sphere region chain.

In one embodiment, determining the sphere cumulative displacement of each sphere area chain according to the relative position change of each sphere and the basket between frames comprises: in each frame of picture, determining the basket position and the sphere position in each frame of picture respectively; in the same frame of picture, calculating the position information of the positions of the spheres relative to the positions of the baskets; and determining the sphere accumulated displacement of each sphere regional chain according to the position information among different frames.

The accumulated displacement is determined through the relative position change of each sphere and the basket, so that the trend of the target sphere entering the basket can be reflected more accurately, the time period before and after the target sphere enters the basket is detected accurately, the moving sphere is identified more accurately, and the key moment when the sphere enters and leaves the basket is recorded conveniently.

Specifically, the accumulated displacement is determined through the relative position change of each ball and the basket, in the process of rotation of the lens, even if the terminal follows the rotation of the cradle head to cause the displacement of the static ball at the field edge on the picture, the ball with small relative position change of the basket can be filtered by using the relative coordinates of the basketball relative to the basket, and the basket is ensured to be in the picture when the cradle head carries the mirror, so that the mirror can be stopped in time when the basket reaches the edge of the picture, and the basket is kept in the picture acquired by the lens. And each moving picture of the ball entering the basket is a key picture of a basketball court related video, and the key pictures are collected according to the basketball position mirror, so that the basket and the target ball are both in the key pictures, and the goal judgment and the picture automatic indirect process are facilitated.

In an alternative embodiment, determining the sphere cumulative displacement of each sphere region chain between frames comprises: determining standard pictures in each frame of pictures; calibrating each frame of picture according to the matching points of each frame of picture and the standard picture to obtain an aligned picture; in the aligned picture, according to the sphere track corresponding to each sphere regional chain, the sphere accumulated displacement of each sphere regional chain is respectively determined.

The standard picture is used for converting the associated sphere area of each frame picture, so that sphere track in each frame picture is clearly reflected through one frame picture. Optionally, determining a standard picture in each frame of pictures includes: one frame of picture is selected from each group of pictures as a standard picture. For example, the first frame screen at the start of shooting may be selected as the standard screen, and the current screen may be selected as the standard screen.

In one embodiment, calibrating each frame according to the matching point of each frame and the standard frame to obtain an aligned frame includes: determining key points of a standard picture and key points in each frame of picture; carrying out random consistency pairing and registration processing on key points in each frame of picture and key points of a standard picture to obtain homography transformation relations corresponding to the matched points after registration; calibrating the positions of the key points in each frame of picture according to a homography transformation relation, and superposing the positions of the key points calibrated by each frame of picture into the current picture to obtain an aligned picture; the aligned picture comprises aligned associated sphere regions corresponding to each sphere region chain. Therefore, even if the terminal rotates along with the cradle head to cause the static sphere to displace on the picture, the static sphere can be aligned to the same position, and the static sphere is filtered from the picture in a key point matching mode.

The random consistency pairing and registering processing (Random Sample Consensus) method removes the influence of noise points on the model in a probability mode; the homography transformation relationship is a one-to-one correspondence relationship of coordinates, can be characterized by homography matrix, vector or other forms, and can also adopt perspective transformation, rigid transformation, similarity transformation, affine transformation and other types of image transformation.

Because the matching points of each frame picture and the current picture are calibrated, the coordinates of each frame picture are calibrated in the current picture, the lens information of each frame picture collecting process is converted into the lens information of the current picture collecting process, and even if the terminal rotates along with the cradle head to cause the displacement of the field-side static sphere on the picture, the static sphere can be filtered from the picture by a key point matching and coordinate mapping method. Even if the terminal rotates along with the cradle head to cause the displacement of the static sphere at the field side on the picture, the static sphere can be filtered from the picture by a method of key point matching and coordinate mapping.

Step 204, determining a target sphere from the sports spheres according to the confidence that each sports sphere belongs to the preset sphere.

The preset balls are preset round balls, and the sizes of the preset round balls can be set for a certain specification or for various specifications respectively; preset balls include, but are not limited to, basketball, football, volleyball, and table tennis. Optionally, the preset balls are set according to the sport scene; when the sports scene is a basketball court, the preset ball is basketball; when the sports scene is a soccer field, the preset ball is a soccer ball, and the relationship between the sports scene and the preset ball is similarly deduced.

The confidence that the sport ball belongs to the preset ball is calculated based on whether the characteristics of the sport ball are consistent with the characteristics of the preset ball. Optionally, the confidence of the moving sphere in one frame of picture can be determined based on the image features, the semantic features, the associated features or other features, and then the confidence of the moving sphere in each frame of picture is comprehensively calculated. The confidence coefficient of a certain moving sphere in each frame of pictures is subjected to averaging treatment according to the number of pictures passed by the moving sphere, so that the confidence coefficient of the moving sphere belonging to the preset sphere is obtained; or screening the confidence coefficient of a certain moving sphere in each frame of picture according to the time sequence, and determining the confidence coefficient of the picture obtained by screening as the confidence coefficient of the moving sphere belonging to the preset sphere.

In an alternative embodiment, determining the target sphere from each sport sphere based on the confidence that each sport sphere belongs to the preset ball comprises: in each frame of picture, respectively determining an associated sphere region image set of each moving sphere; aiming at each sport ball, calculating the confidence coefficient of each associated ball region image belonging to the preset ball class respectively to obtain the confidence coefficient of each associated ball region image of each sport ball; based on the respective quantity of the associated sphere region images of each moving sphere, carrying out averaging treatment on the confidence coefficient of the associated sphere region images to obtain the confidence coefficient of each moving sphere belonging to a preset sphere; determining the motion sphere with the highest confidence from the motion spheres; and determining the motion sphere with the highest confidence as the target sphere. Thus, the target ball of the preset ball is quickly identified based on the confidence of the moving ball.

In one possible embodiment, determining the target sphere from each sport sphere based on the confidence that each sport sphere belongs to the preset ball comprises: and selecting a proper target sphere region chain from the sphere region chain set T as a tracking target of the current frame according to a preset rule. The preset rule considers the confidence coefficient of basketball and the continuity and stability of the sphere regional chain; wherein the sphere region chain is a tracking chain; confidence is the confidence between 0 and 1 using the basketball detector output, with higher probability of basketball being represented as higher probability of other shape being liked as lower probability; the continuity and stability of the sphere area chain means that the basketball on the same tracking chain tends to be continuously tracked.

In an alternative embodiment, determining the target sphere from each sport sphere based on the confidence that each sport sphere belongs to the preset ball comprises: in each frame of picture, determining a sphere region chain in which each moving sphere is positioned in sequence; determining the confidence that the sphere corresponding to each sphere region chain belongs to a preset sphere; determining a target sphere region chain in each sphere region chain according to the confidence coefficient; and determining the target sphere according to the sphere corresponding to the target sphere region chain.

Alternatively, the sphere region chains may be generated according to any embodiment scheme of step 202, and may also be more accurately identified by a moving sphere-specific neural network model. Alternatively, the sphere-specific neural network model may be a neural network model for target detection based on the law of motion of the sphere.

Optionally, the confidence that the sphere corresponding to each sphere region chain belongs to the preset sphere may be calculated according to the motion track of each sphere, or may be calculated according to the shape of each sphere. Illustratively, in a chain of sphere regions, confidence is determined based on the reference regions through which the sphere corresponding to the chain of sphere regions passes, and based on the order in which the sphere passes through the reference regions.

Optionally, determining a target sphere region chain of each sphere region chain according to the confidence comprises: determining a sphere region chain with the highest confidence as a target sphere region chain in each sphere region chain; or comparing the confidence coefficient of each sphere region chain with that of the reference chain to obtain a confidence coefficient comparison result, and determining the reference chain as a target sphere region chain under the current time stamp if the reference chain is judged not to be adjusted according to the confidence coefficient comparison result.

Optionally, determining the target sphere according to the sphere corresponding to the target sphere region chain includes: and taking the sphere corresponding to the target sphere region chain as a target sphere, or further screening the target sphere from the target sphere region chain according to a certain parameter value of a picture.

The confidence coefficient calculation is carried out on the moving sphere through the sphere region chain of the moving sphere, so that the continuity and stability of the target sphere tracking process can be better ensured, the target sphere can be more accurately identified from the moving sphere, and the sending probability of the false alarm condition is reduced.

In one embodiment, determining the confidence that the sphere corresponding to each sphere region chain belongs to a preset sphere comprises: in each sphere region chain, determining the associated spheres in at least two frames of pictures forward to the target timestamp; and determining the confidence coefficient of each sphere region chain according to the confidence coefficient of the associated sphere in at least two frames of pictures relative to the preset sphere.

The target timestamp may be a timestamp of the current picture, or may be another designated timestamp.

In one embodiment, determining the associated sphere in at least two frames of pictures forward of the target timestamp in each sphere region chain comprises: selecting pictures recorded by at least two time stamps in front of the target time stamp to obtain at least two frames of pictures in front of the target time stamp; in the at least two frames of pictures, determining the associated sphere area of each sphere area chain in each frame of picture according to the corresponding relation of the time stamps; and respectively containing the associated spheres through the respectively determined associated sphere regions.

Optionally, the confidence coefficient of each sphere region chain may be a confidence coefficient average value of the associated sphere in at least two frames of pictures relative to a preset sphere, and the confidence coefficient average value may be weighted according to a duration between each frame of pictures and a target timestamp.

Therefore, the related spheres in at least two frames of pictures forward to the target timestamp are selected by taking the timestamp as a medium, and confidence calculation is carried out on the related spheres in the at least two frames of pictures to obtain the confidence of the related sphere area chain, so that the target spheres in the basketball court are more accurately identified.

In one embodiment, determining a target sphere region chain from each sphere region chain based on the confidence comprises: determining a reference chain in each sphere region chain; if it is determined that each sphere region chain has a replacement chain meeting the reference chain adjustment confidence coefficient condition, adjusting the reference chain according to the replacement chain until each sphere region chain does not have the replacement chain meeting the reference chain adjustment confidence coefficient condition, and determining the current reference chain as the target sphere region chain.

The reference chain and the replacement chain are candidate chains in each sphere region chain, the candidate chains being one or more sphere region chains that can be selected as the target sphere region chain in different cases. The reference chain is one candidate chain in each sphere region chain, the replacement chain includes other candidate chains in each sphere region chain, and the replacement chain is a chain that satisfies a reference chain adjustment confidence condition.

The reference chain adjustment confidence coefficient condition is calculated based on the current reference chain confidence coefficient and is used for screening out the replacement chain of the reference chain from other sphere region chains outside the reference chain. The reference chain adjustment confidence condition is used to evaluate the respective confidence comparisons of the reference chain to other sphere regions outside the reference chain to determine whether a replacement chain is present.

In one embodiment, determining that each sphere region chain has an alternate chain that satisfies a reference chain adjustment confidence condition comprises: comparing the sphere region chains outside the reference chain with the reference chain according to the confidence degree of each sphere region chain; and according to the comparison result, determining the sphere region chain with the confidence degree of the reference chain meeting the confidence degree adjustment condition of the reference chain as a replacement chain of the reference chain.

In one embodiment, adjusting the reference chain according to the replacement chain until each sphere region chain does not have a replacement chain that satisfies the reference chain adjustment confidence condition, comprising: adjusting the reference chain through a replacement chain set of the initial reference chain to obtain a current reference chain; if the current reference chain is determined, each sphere region chain has a replacement chain meeting the reference chain adjustment confidence coefficient condition, a replacement chain set of the current reference chain is obtained, the current reference chain is adjusted through the replacement chain set of the current reference chain, the reference chain adjustment confidence coefficient condition is updated according to the current reference chain, and until each sphere region chain does not meet the updated current reference chain adjustment confidence coefficient condition, namely, no replacement chain of the current reference chain exists.

Correspondingly, when each sphere region chain does not have a replacement chain meeting the reference chain adjustment confidence condition, the current reference chain is determined to be the target sphere region chain.

The method comprises the steps of determining a reference chain, judging whether a replacement chain exists or not through a reference chain adjustment confidence coefficient condition, and adjusting in real time so that the tracking process of a control target sphere can be corrected in time under the condition that the reference chain is wrong.

Wherein the reference chain is a target sphere region chain of adjacent time stamps to the target time stamp.

The adjacent time stamp of the target time stamp can be the time stamp of the previous frame of the target time stamp, or can be the time stamp which is spaced with the time length of the target time stamp by a preset length and is before the target time stamp. Optionally, the picture to which the target timestamp belongs is a current picture, and the target timestamp can be used as an identifier of the current picture, and the current picture can be a moving picture of a target sphere or other types of pictures; the picture to which the adjacent time stamp belongs is an adjacent forward picture of the current picture, and the adjacent time stamp can be used as an identification of the adjacent forward picture, and the adjacent forward picture can be a moving picture of the target sphere or at least one frame picture of other types. Thus, the reference chain can be efficiently determined by updating the sphere region chain by the target sphere regions of adjacent time stamps.

In an alternative embodiment, determining that each sphere region chain has an alternate chain that satisfies a reference chain adjustment confidence condition includes: among the sphere region chains, a sphere region chain with a confidence greater than that of the reference chain is taken as a replacement chain.

Optionally, in each sphere region chain, taking the sphere region chain with the confidence degree larger than that of the reference chain as a replacement chain, including: and taking the sphere region chain with the confidence degree larger than that of the reference chain as a replacement chain of the reference chain to be replaced in each sphere region chain corresponding to the current picture. The reference chain to be replaced is one of the current reference chains, and a replacement chain meeting the confidence degree adjustment condition of the reference chain exists in the sphere region chain to which the current reference chain belongs.

The sphere region chain with the confidence coefficient larger than that of the reference chain is used as a replacement chain, so that the sphere region chain can be easily adjusted, errors of the sphere region chain can be quickly corrected, and the sphere region chain where the target sphere is located can be accurately identified.

The ratio between the confidence coefficient of the replacement chain and the confidence coefficient of the reference chain exceeds a regional chain adjustment threshold, and the regional chain adjustment threshold is a preset confidence coefficient multiple of the reference chain.

The ratio between the confidence coefficient of the replacement chain and the confidence coefficient of the reference chain is to judge whether the replacement chain is adjusted by taking the confidence coefficient of the reference chain as a reference. Therefore, the dynamic region chain adjustment threshold is set through the preset confidence coefficient multiple of the reference chain, and the reference chain adjustment confidence coefficient condition of the reference chain adjustment threshold is updated in real time through the dynamic region chain adjustment threshold. The switching of the reference chain is relatively strict, the same sphere region chain can be tracked more continuously, the continuity is high, and the target sphere is prevented from being frequently switched.

In one embodiment, adjusting the reference chain according to the replacement chain includes: if there is a replacement chain in the picture of the target timestamp, the replacement chain is adjusted to be the reference chain. Therefore, when only one replacement chain exists in the reference chain, the calculation of the area is not involved, the processing of related data is not involved, the replacement chain is directly adjusted to the reference chain, and the data processing speed is relatively high.

In one embodiment, adjusting the reference chain according to the replacement chain includes: if a plurality of replacement chains exist in the picture with the target time stamp, determining the associated sphere area with the largest area in each replacement chain in the picture with the target time stamp; and determining the replacement chain with the largest area of the associated sphere area as a reference chain. Therefore, the replacement chain to which the associated sphere region with the largest area belongs is determined from the replacement chains to be the reference chain through the associated sphere region with the largest area, and the replacement chain is selected more quickly.

In one exemplary embodiment, the target sphere region chain of the picture of the previous frame of the current picture is defined as the reference chain T by a time stamp _a If for the current picture there is a replacement chain for its reference chain for the sphere region chain (assumed to be T _b ) If the confidence coefficient of the continuous K frames (K is at least two frames) is larger than that of the reference chain, switching the target sphere region chain, and replacing the chain T _b Adjust to target sphere region chain, otherwise keep current tracking chain T _a . If a plurality of replacement chains meeting the requirements exist, selecting the area of the associated sphere area with the largest area in the current picture.

In another exemplary embodiment, the target sphere region chain of the picture of the previous frame of the current picture is defined as the reference chain T by a time stamp _a If for the current picture there is a replacement chain for its reference chain for the sphere region chain (assumed to be T _b ) Continuous K frames (K is at least two frames), and the confidence ratio of the reference chain to the reference chain is greater than 1 time of the confidence of the reference chain, switching the target sphere region chain, and replacing the chain T _b Adjust to target sphere region chain, otherwise keep current tracking chain T _a . If a plurality of replacement chains meeting the requirements exist, selecting the area of the associated sphere area with the largest area in the current picture.

And 206, acquiring a picture of the target sphere according to the position information of the target sphere.

The position information of the target sphere is the position information of the target sphere in a certain frame or a plurality of frames of pictures and is used for determining the acquisition parameters corresponding to the target sphere. Optionally, the position information of the target sphere may be the coordinate position of the target sphere in the screen, or may be a region set of coordinates where the target sphere is located, and the target sphere is determined by using the region set.

In an alternative embodiment, the image acquisition of the target sphere according to the position information of the target sphere includes: determining acquisition parameters of a picture according to the relative positions of the target sphere and the basket; and acquiring the moving picture of the target sphere according to the acquisition parameters.

In another alternative embodiment, the image acquisition of the target sphere according to the position information of the target sphere includes: acquiring position information of a target sphere; updating the acquisition parameters of the picture according to the position information of the target sphere; and acquiring the moving picture of the target sphere based on the updated acquisition parameters.

In one embodiment, the image acquisition of the target sphere according to the position information of the target sphere comprises: determining a picture offset according to the position information of the target sphere; controlling the rotation of the cradle head according to the picture offset, and collecting the moving picture of the target sphere in the rotation process; wherein the target sphere is in a moving picture.

The screen offset is an offset of composition information of the forward screen from the target composition information, which is determined according to whether or not the target sphere of the previous screen is located at the target position in the target composition information. If the position difference between the target sphere and the target position of the previous picture is smaller than the position threshold value, the picture offset of the previous picture can be considered to be smaller, the terminal can not be adjusted, and smaller adjustment can be performed; if the position difference between the target sphere and the target position of the previous picture is greater than the position threshold, the picture offset of the previous picture is considered to be large, and the acquisition mode of the first image is adjusted. The image acquisition offset at least comprises adjustment of acquisition directions such as pitch direction (pitch), heading direction (yaw) and the like, and can also comprise adjustment of focal length.

In an exemplary embodiment, controlling rotation of a pan/tilt head according to a screen offset, and capturing a moving screen of a target sphere during rotation, includes: when the picture offset comprises an offset angle, controlling the picture acquisition equipment to rotate according to the offset angle, and controlling the rotated picture acquisition equipment to acquire a picture to obtain a current picture; and/or when the picture offset comprises the zoom coefficient of the picture acquisition equipment, controlling the picture acquisition equipment to adjust according to the zoom coefficient, and controlling the picture acquisition equipment after zooming to acquire the picture to obtain the current picture.

In the picture acquisition method of the moving sphere, the moving sphere is identified through the sphere region track formed by the sphere regions of the multiple frames, so that the movement trend of the sphere can be included in the sphere detection range, the static sphere is eliminated, and the identification accuracy of the sphere region in the picture is improved; determining target spheres from the sports spheres according to the confidence that the sports spheres belong to preset spheres, and more accurately identifying the target spheres such as basketball; in this case, the image of the target sphere is acquired according to the position information of the target sphere, and automatic shooting of the target sphere is realized.

Taking the basketball application scene as an example, the scheme can inhibit the false alarm ball interference and realize basketball tracking. According to the scheme, the interference spheres such as people heads in disordered audience seats, static basketball on the field, false alarm spheres such as spheres in wall posters and the like can be removed in the competition field, basketball moving on the field is locked, and the basketball is matched with the tripod head equipment to be used for automatic tracking shooting of basketball scenes. Even if basketball encounters a human body or other object to block so as to cause tracking loss, or the basketball moves too fast so as to cause tracking loss, the basketball can be shot according to continuity and stability, and global re-detection is not easy to trigger frequently. The cradle head mirror is not controlled by the global re-detection position signal, so that tracking jamming is reduced.

In one embodiment, the image of the target sphere is acquired according to the position information of the target sphere, and the method further comprises: determining a target moving picture of the target sphere entering the basket according to the path of the target sphere in each frame moving picture; and combining the target moving picture with the associated moving picture of the target moving picture to obtain the target moving fragment.

The target moving picture includes a moving picture when the target sphere enters the basket, and the associated moving picture of the target motion picture is a picture according to an association relationship of time stamps or picture features. Alternatively, the multi-frame target moving picture forms a key segment of the goal process, and the associated moving picture forms a relevant segment before and after the goal. For example, each frame of pictures to be spliced in the first 4 seconds and the last 1 second of the timestamp of the target moving picture is determined, and the target moving picture and each frame of pictures to be spliced are spliced according to the time sequence, so as to obtain the target moving segment, namely the goal segment highlight.

In an alternative embodiment, determining a target moving picture of the target sphere into the basket according to a path of the target sphere in each frame moving picture includes: sequentially determining each coordinate position of a target sphere in each frame of moving picture; combining the coordinate positions of the target sphere according to the time sequence to obtain a path of the target sphere; according to the track of the target sphere, a path section of the target sphere entering the basket exists, and if the path section exists, each moving picture to which the path section belongs is determined as a target moving picture.

In an alternative embodiment, determining a target moving picture of the target sphere into the basket according to a path of the target sphere in each frame moving picture includes: determining each trigger area of the target sphere in each frame of moving picture; combining the trigger areas according to time sequence to obtain a path of the target sphere; according to the path of the target sphere, the target sphere enters a path section of the basket, and if the path section exists, each moving picture to which the path section belongs is determined as a target moving picture.

The target moving picture is determined by the track of the target ball in the moving picture, the characteristic extraction can be carried out on the moving process of the target ball at different moments, so that the target moving picture can be more accurately identified, and then the video clip of the ball feeding process is formed by combining the target moving picture with the associated moving picture.

In one embodiment, determining a target motion picture of a target sphere into a basket according to a trajectory of the target sphere in each motion picture includes: extracting an entrance area picture and an exit area picture of the target sphere from each moving picture according to the path of the target sphere in each moving picture; determining a moving picture between the entrance area picture and the exit area picture as a target moving picture; the entrance area picture is a moving picture of the target sphere in the basket entrance area; the outlet area picture refers to a moving picture in which the target sphere is located in the outlet area of the basket.

The inlet area and the outlet area are both areas divided according to the basket, the picture of the inlet area is the picture that the target sphere is positioned in the inlet area, and the picture of the outlet area is the picture that the target sphere is positioned in the outlet area; the entrance area picture is used for representing the movement trend of basketball entering the basket, and the exit area picture is used for representing the movement trend of basketball leaving from the basket.

In an alternative embodiment, extracting an entry area screen and an exit area screen of a target sphere from each moving picture according to a path of the target sphere in each moving picture, includes: in each moving picture, determining an inlet area above the basket and an outlet area below the basket according to an inlet area range and an outlet area range corresponding to the basket, and determining a trigger area surrounding the inlet area according to a trigger area range; sequentially determining pictures of the positions of the target spheres entering the entrance area from the trigger area between all moving pictures to obtain an entrance area picture; and, among the moving pictures, the picture of the position of the target sphere from the entrance area to the exit area is sequentially determined, and the exit area picture is obtained. The trigger area is used for judging the period of detecting the inlet area and the outlet area of the target sphere; when the target sphere enters the trigger zone, it starts to detect if the target sphere reaches the entrance zone.

Under the condition that the target moving picture is used for determining the moment that basketball enters the basket, compared with the process of directly identifying the target moving picture, the number of pictures of the entrance area picture and the exit area picture is more, the duration is longer, the entrance area picture and the exit area picture have a certain sequence, the accuracy is higher, and whether the target sphere enters the basket can be judged under the condition that the terminal carries out the mirror.

In another embodiment, determining a target moving picture of the target sphere into the basket according to a trajectory of the target sphere in each moving picture includes: determining a reference area path of the target sphere relative to the basket according to the relative positions of the target sphere and the basket in each frame of moving picture; determining candidate goal pictures in each frame of moving pictures according to the reference area path; in the candidate goal picture, a target moving picture of the target sphere entering the basket is determined according to a shielding relation between the target sphere and the basket.

The reference area path is a reference area through which the target sphere sequentially passes in each frame of moving picture, and is used for determining the state of the target sphere. Each reference zone includes a trigger zone, an inlet zone, and an outlet zone of the basket, and may also include an event end zone surrounding the outlet zone. For example, if a certain reference area path of the target sphere is that of passing through the trigger area, the exit area and the end area in sequence, a candidate goal screen does not exist in each frame of moving picture, and a candidate backboard screen may appear.

The candidate goal picture is a moving picture in which a goal event may occur after the state judgment is performed according to the reference area path, however, the reference area path represents a positional relationship in a two-dimensional space such as a picture, and a problem of spatial misalignment may be encountered, so that whether the target sphere enters the basket is judged through a shielding relationship between the target sphere and the basket.

Because the relative position of the target sphere and the basket cannot be changed due to the lens, a reference area path is generated according to the relative position, and the goal state is judged according to the reference area path, so that a candidate goal picture is determined; on the basis, the problem of space misjudgment in the mirror transporting process is solved through the characteristic of the shielding relation between the target sphere and the basket, so that whether the target sphere enters the basket or not is judged more accurately, and a target moving picture is acquired accurately.

In one embodiment, the method further comprises: when the target sphere belongs to basketball, extracting gesture key points of the target object from each moving picture according to the human body gesture corresponding to basketball scene information; if the action of the target object is identified according to the gesture key points to belong to the shooting action, combining the moving picture of the shooting action to obtain a shooting segment; determining a set of related pictures of the shooting segment from the moving pictures in a time stamp order; performing frame rate adjustment on the shooting segments to obtain slow-motion shooting segments with target frame rates; the target video is synthesized based on the set of related frames and the slow motion shooting segment.

The basketball scene information characterizes that the target object is located in basketball scenes such as basketball courts or basketball halls, and the basketball scenes have preset actions such as shooting, basketball buckling and the like, so that the accuracy of identifying key events is improved. The basketball scene information can be basketball sport scenes, basketball teaching scenes and other scenes, and can also be refined scenes obtained by further dividing the coarse-granularity scenes, and the accuracy of key event identification of the refined scenes can be further improved. The video model for identifying basketball scene information specifically refers to a deep learning model using multi-frame video as input, such as an extended three-dimensional time-space domain network (x 3 d), a slow network (slow), and the like.

The gesture key points may be points in a gesture extraction template, which includes any type of gesture key points such as shape edge points, muscle change points, bone key points, and the like, which are not limited to the parts of the human body. User actions can be planned through various types of gesture key points to improve recognition accuracy. In a basketball scene, human body key points of a target object are estimated based on the human body gesture of deep learning, the human body gesture is accurately estimated through relatively low data volume, the process of shooting actions is clear, and a moving picture where the shooting actions are located is combined to obtain shooting segments.

The shooting segments and the related picture sets thereof are all obtained by combining moving pictures. The shooting segment is a video segment formed by determining shooting pictures according to shooting events and then based on the shooting pictures; relevant segment identifiers can be determined through the shooting segments, and mapping is performed based on the relevant segment identifiers to obtain relevant segments of the shooting segments. When the playing speed of a shooting segment changes, the relevant picture set of the shooting segment can be adaptively adjusted, so that the smoothness of a target video or the fluctuation of other video evaluation data is small, and the video quality is ensured.

The identification of the related picture set and the shooting segment identification are in an association relation; the relevant segment identification of the shot segment includes a timestamp. The relevant segment identification may be a time period before and after the shooting segment to determine moving pictures in the time period before and after the shooting segment as relevant pictures in a time stamp order, and then combine the relevant pictures into a relevant picture set. The time stamp order is an order of each moving picture on the time axis for determining moving pictures of time periods before and after the shooting segment.

The corresponding relation exists between the time period of the shooting event and the shooting segment identifier, the shooting segment identifier can be mapped based on the time period of the key event, and the time period of the key event can also be directly used as the shooting segment identifier.

Therefore, the shooting pictures are identified, the shooting pictures are combined into shooting segments, different times of speeds are used for shooting segments and related picture sets, shooting events are prominent, the rhythm sense is improved, the rhythm is high and low, the visual shock is strong, and the wonderful degree is far better than that of the existing method.

In a specific embodiment, the application effect of the present embodiment is illustrated by a plurality of sets of images, such as the basketball scene in fig. 3, where the basket area 302 is disposed on one side of the basketball court, and the sphere area 304 where the target sphere identified according to the sphere area track and the confidence level is located can be identified although it is blocked by the human body. In the application scenario of fig. 4, although there are multiple heads, the sphere area 304 where the target sphere is located can be identified by the scheme; in the application scenario of fig. 5, although there is separation between the human body and the target sphere, the sphere region 304 where the target sphere is located can be identified by the present scheme. In the moving picture of the shooting action as shown in fig. 6, there are a head, a sphere area 304 where a target sphere is, and a basket area 302 at the same time, wherein the sphere area 304 and the basket area 302 are respectively drawn by rectangular frames of different categories; and the target moving picture is shown in fig. 7. In another specific embodiment, each reference area is shown in fig. 8 (a), and a reference area path for determining a target moving picture is shown in fig. 8 (a).

It should be understood that, although the steps in the flowcharts related to the embodiments described above are sequentially shown as indicated by arrows, these steps are not necessarily sequentially performed in the order indicated by the arrows. The steps are not strictly limited to the order of execution unless explicitly recited herein, and the steps may be executed in other orders. Moreover, at least some of the steps in the flowcharts described in the above embodiments may include a plurality of steps or a plurality of stages, which are not necessarily performed at the same time, but may be performed at different times, and the order of the steps or stages is not necessarily performed sequentially, but may be performed alternately or alternately with at least some of the other steps or stages.

Based on the same inventive concept, the embodiment of the application also provides a picture acquisition device of the moving sphere for realizing the picture acquisition method of the moving sphere. The implementation of the solution provided by the device is similar to the implementation described in the above method, so the specific limitation in the embodiments of the image capturing device for one or more moving spheres provided below may be referred to the limitation of the image capturing method for a moving sphere hereinabove, and will not be repeated herein.

In one embodiment, as shown in fig. 9, there is provided a picture collecting apparatus for a moving sphere, comprising:

the sphere identification module 902 is configured to identify a moving sphere according to a sphere region track in a multi-frame picture;

the sphere determining module 904 is configured to determine a target sphere from the moving spheres according to a confidence coefficient that each moving sphere belongs to a preset sphere;

and the image acquisition module 906 is used for acquiring images of the target sphere according to the position information of the target sphere.

In one embodiment, the sphere identification module 902 is configured to:

determining standard pictures in the pictures of each frame;

In one embodiment, the sphere determination module 904 is configured to:

determining a reference chain in each sphere region chain;

In one embodiment, the sphere determination module 904 is configured to:

In one embodiment, the screen capturing module 906 is configured to:

The above-mentioned various modules in the picture acquisition device for moving sphere can be implemented in whole or in part by software, hardware and their combination. The above modules may be embedded in hardware or may be independent of a processor in the computer device, or may be stored in software in a memory in the computer device, so that the processor may call and execute operations corresponding to the above modules.

In one embodiment, a computer device is provided, which may be a terminal, and an internal structure diagram thereof may be as shown in fig. 10. The computer device includes a processor, a memory, an input/output interface, a communication interface, a display unit, and an input means. The processor, the memory and the input/output interface are connected through a system bus, and the communication interface, the display unit and the input device are connected to the system bus through the input/output interface. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The input/output interface of the computer device is used to exchange information between the processor and the external device. The communication interface of the computer device is used for carrying out wired or wireless communication with an external terminal, and the wireless mode can be realized through WIFI, a mobile cellular network, NFC (near field communication) or other technologies. The computer program, when executed by a processor, implements a method of motion sphere frame acquisition. The display unit of the computer equipment is used for forming a visual picture, and can be a display screen, a projection device or a virtual reality imaging device, wherein the display screen can be a liquid crystal display screen or an electronic ink display screen, the input device of the computer equipment can be a touch layer covered on the display screen, can also be a key, a track ball or a touch pad arranged on a shell of the computer equipment, and can also be an external keyboard, a touch pad or a mouse and the like.

It will be appreciated by those skilled in the art that the structure shown in FIG. 10 is merely a block diagram of some of the structures associated with the present inventive arrangements and is not limiting of the computer device to which the present inventive arrangements may be applied, and that a particular computer device may include more or fewer components than shown, or may combine some of the components, or have a different arrangement of components.

In one embodiment, the application further provides a handheld cradle head, which comprises a motor and a processor, wherein the motor is used for controlling the cradle head to rotate, and the processor realizes the steps in the method embodiments when executing the computer program.

In an embodiment, there is also provided a computer device comprising a memory and a processor, the memory having stored therein a computer program, the processor implementing the steps of the method embodiments described above when the computer program is executed.

In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored which, when executed by a processor, carries out the steps of the method embodiments described above.

In an embodiment, a computer program product is provided, comprising a computer program which, when executed by a processor, implements the steps of the method embodiments described above.

It should be noted that, the user information (including but not limited to user equipment information, user personal information, etc.) and the data (including but not limited to data for analysis, stored data, presented data, etc.) related to the present application are information and data authorized by the user or sufficiently authorized by each party, and the collection, use and processing of the related data need to comply with the related laws and regulations and standards of the related country and region.

Those skilled in the art will appreciate that implementing all or part of the above described methods may be accomplished by way of a computer program stored on a non-transitory computer readable storage medium, which when executed, may comprise the steps of the embodiments of the methods described above. Any reference to memory, database, or other medium used in embodiments provided herein may include at least one of non-volatile and volatile memory. The nonvolatile Memory may include Read-Only Memory (ROM), magnetic tape, floppy disk, flash Memory, optical Memory, high density embedded nonvolatile Memory, resistive random access Memory (ReRAM), magnetic random access Memory (Magnetoresistive Random Access Memory, MRAM), ferroelectric Memory (Ferroelectric Random Access Memory, FRAM), phase change Memory (Phase Change Memory, PCM), graphene Memory, and the like. Volatile memory can include random access memory (Random Access Memory, RAM) or external cache memory, and the like. By way of illustration, and not limitation, RAM can be in the form of a variety of forms, such as static random access memory (Static Random Access Memory, SRAM) or dynamic random access memory (Dynamic Random Access Memory, DRAM), and the like. The databases referred to in the embodiments provided herein may include at least one of a relational database and a non-relational database. The non-relational database may include, but is not limited to, a blockchain-based distributed database, and the like. The processor referred to in the embodiments provided in the present application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic unit, a data processing logic unit based on quantum computing, or the like, but is not limited thereto.

The technical features of the above embodiments may be arbitrarily combined, and all possible combinations of the technical features in the above embodiments are not described for brevity of description, however, as long as there is no contradiction between the combinations of the technical features, they should be considered as the scope of the description.

The foregoing examples illustrate only a few embodiments of the application and are described in detail herein without thereby limiting the scope of the application. It should be noted that it will be apparent to those skilled in the art that several variations and modifications can be made without departing from the spirit of the application, which are all within the scope of the application. Accordingly, the scope of the application should be assessed as that of the appended claims.

Claims

1. A picture acquisition method of a moving sphere, the method comprising:

2. The method of claim 1, wherein the identifying a moving sphere from a sphere region trajectory in a multi-frame picture comprises:

3. The method of claim 2, wherein the frame of the target sphere from which the frame is acquired is a current frame, and wherein the multi-frame is a forward frame of the current frame.

4. The method of claim 2, wherein said determining moving spheres in said picture for each frame from each of said chains of sphere regions comprises:

5. The method of claim 4, wherein said determining the cumulative displacement of spheres of each chain of sphere regions between frames of said frames, respectively, comprises:

6. The method of claim 4, wherein said determining the cumulative displacement of spheres of each chain of sphere regions between frames of said frames, respectively, comprises:

determining standard pictures in the pictures of each frame;

7. The method of claim 1, wherein determining a target sphere from each of the sport spheres based on a confidence that each of the sport spheres belongs to a preset sphere comprises:

8. The method of claim 7, wherein determining the confidence that the sphere corresponding to each of the sphere region chains belongs to a preset sphere comprises:

9. The method of claim 7, wherein said determining a target sphere region chain from each of said sphere region chains based on said confidence comprises:

Determining a reference chain in each sphere region chain;

10. The method of claim 9, wherein the reference chain is a target sphere region chain of adjacent time stamps to a target time stamp.

11. The method of claim 9, wherein said determining that each of said sphere region chains has an alternate chain that satisfies a reference chain adjustment confidence condition comprises:

12. The method of claim 11, wherein a ratio between the confidence of the replacement chain and the confidence of the reference chain exceeds a region chain adjustment threshold that is a preset confidence multiple of the reference chain.

13. The method of claim 11, wherein said adjusting the reference chain according to the replacement chain comprises:

14. The method of claim 9, wherein said adjusting the reference chain according to the replacement chain comprises:

15. The method according to claim 1, wherein the performing image acquisition on the target sphere according to the position information of the target sphere comprises:

16. The method of claim 1, wherein the acquiring the picture of the target sphere according to the position information of the target sphere further comprises:

Determining a target moving picture of the target sphere entering the basket according to the path of the target sphere in each frame of moving picture;

17. The method of claim 16, wherein determining a target motion picture for the target sphere into the basket based on a path of the target sphere in each of the motion pictures comprises:

18. The method of claim 16, wherein determining a target motion picture for the target sphere into the basket based on the trajectory of the target sphere in each of the motion pictures comprises:

19. The method according to any one of claims 1-18, further comprising:

20. A picture acquisition device for a moving sphere, the device comprising:

21. A handheld cradle head comprising a motor for controlling rotation of the cradle head and a processor for carrying out the steps of the method of any one of claims 1 to 19.

22. A computer device comprising a memory and a processor, the memory storing a computer program, characterized in that the processor implements the steps of the method of any one of claims 1 to 19 when the computer program is executed.

23. A computer readable storage medium, on which a computer program is stored, characterized in that the computer program, when being executed by a processor, implements the steps of the method of any of claims 1 to 19.