CN110059584B - Event naming method combining boundary distribution and correction - Google Patents

Abstract

The invention provides an event naming method combining boundary distribution and correction, which forms an event naming network by constructing a starting point distribution network, an end point distribution network and a boundary circulation correction network; training and updating the event nomination network by constructing an event nomination network loss function; performing nomination prediction on the video event by using the event nomination network after training and updating; the starting point distribution network and the ending point distribution network are used for predicting event nomination; the boundary circulation correction network is used for generating predicted event nomination bias information and correcting event nomination boundaries. According to the event naming method combining boundary distribution and correction, provided by the invention, the event naming fitting the real event distribution is generated by combining the event starting and ending point distribution rule in the real video, and the boundary of the event naming is corrected by utilizing a cyclic correction network, so that the event naming which is more in line with the real event and enables the boundary of the event to be more accurate is obtained.

Description

An event nomination method combining boundary distribution and correction

Technical Field

The present invention relates to the field of computer vision technology, and more specifically, to an event nomination method combining boundary distribution and correction.

Background Art

With the rapid development of the Internet and portable devices, video shooting has become more convenient and easy. A large number of videos have been uploaded to the Internet. The content and length of these videos vary greatly. Most video-based computer vision algorithms perform some trimming on long videos and then analyze and process the trimmed short videos, such as action recognition. However, trimming long videos consumes a lot of manpower and time costs. In order to meet the needs of real life, it becomes necessary to process and analyze the untrimmed long videos.

Dense video description [1] is a new task based on untrimmed long videos. Its goal is to use natural language to describe multiple events in the video. Dense video description can be divided into two parts: one is to locate the events in the video and find the start and end time of all events in the video, that is, to extract event nominations in the video; the other is to describe the located events using natural language. Its potential applications are very wide, such as early childhood education, daily tutoring for the blind, movie subtitles, video retrieval and classification, etc.

Robust dense video description relies on high-quality event nominations, which not only requires that the generated event nominations cover the time span of all possible events in terms of time scale, but also requires that the boundaries of the generated event nominations be the same as the real events in the video. Compared with image-based tasks such as object detection, high-quality event nominations require not only the extraction of object information in the video, but also the extraction and analysis of the temporal motion information of the objects in the video to obtain relevant dynamic information. In real life, video generation conditions are not restricted, multiple events in the video overlap in time, and the shooting angle and shooting distance of the video vary to a certain extent, which brings great challenges to event nomination.

Event nomination in videos is similar to object detection in images. Currently, many research works on event nomination are inspired by object detection. However, in event nomination, we need to pay attention not only to the apparent features of the video, but also to the dynamic features in the time series, and the time span of events often varies greatly. Shou et al. [2] applied sliding windows of different scales to video feature sequences, and used [3] to extract features in the window and further predict event nominations. However, this method can only obtain event nominations of a limited number of predetermined lengths and cannot flexibly adapt to the actual length of the event. In addition, repeated scanning of the same frame by sliding windows of different scales will also lead to redundant calculations. Gao et al. [4] further regressed the event nomination boundary based on the sliding window to obtain a more flexible and accurate nomination boundary. Chao et al. [5] extended the Faster R-CNN [6] framework into a two-stream network structure for video, and applied dilated convolution to the time series to expand the receptive field, thereby obtaining event nominations with a larger time span. Thanks to the memory of video feature extraction by recurrent neural networks, [7] input the video features in the sliding window into the recurrent neural network to predict the start and end time and confidence of event nominations of different scales. However, sliding windows usually lead to a large number of repeated operations. [8] By predicting multiple event nominations of different scales at each node of the recurrent neural network, repeated calculations at different scales can be avoided. However, the event nomination boundary predicted by this method is fixed, and the accuracy of its boundary is limited by the feature extraction span, making it difficult to accurately fit the real event boundary.

Summary of the invention

In order to overcome the technical defects of the above-mentioned existing event nomination method based on recurrent neural network, the nomination boundary is fixed, the accuracy of the boundary is limited by the feature extraction span, and it is difficult to fit the real event boundary, the present invention provides an event nomination method combining boundary distribution and correction.

In order to solve the above technical problems, the technical solution of the present invention is as follows:

An event nomination method combining boundary distribution and correction, forming an event nomination network by constructing a starting point distribution network, an end point distribution network and a boundary cycle correction network; training and updating the event nomination network by constructing an event nomination network loss function; and using the event nomination network after training and updating to nominate video events.

The starting point distribution network and the end point distribution network are used for event nomination prediction;

The boundary loop correction network is used to generate bias information of predicted event nominations and correct the event nomination boundaries.

The construction process of the starting point distribution network and the end point distribution network is as follows:

Normalize the length of the existing dataset videos to determine the relative positions of the event start and end points in the video;

The relative positions of the start and end points of all events in the data set in the video are calculated, and the probability distributions of the start and end points of all events in the video on the video timeline are taken as _ws0 , _we0 ; _ws0 , _we0 represent the probability distributions of the start and end points of the event respectively, and the start point distribution network and the end point distribution network are obtained.

The process of event nomination prediction by the starting point distribution network and the end point distribution network is specifically as follows:

The video features of the sample video are obtained through a three-dimensional convolutional network. Based on the starting point distribution network and the end point distribution network, the obtained video features are calculated using a recurrent neural network to obtain the video features output at each time point in the starting point distribution network and the end point distribution network.

At each time point in the origin distribution network and the destination distribution network, K confidences are output, indicating the possibility of K fixed-length event nominations. The length of these K event nominations is:

[t-k,t+1],k∈[0,K];

The values of t and k satisfy t≥k, and the value of t changes with the length of the video; the higher the confidence, the greater the possibility that the interval is an event nomination; the sum of the confidences of the corresponding event nominations in the starting point distribution network and the end point distribution network is taken as the final confidence of the event, thereby completing the prediction of the event nomination.

The boundary cycle correction network is constructed by two layers of recurrent neural networks, wherein the recurrent neural network of the first layer is used to calculate the video features output at each time point according to the video features of the sample video; the recurrent neural network of the second layer is used to generate bias information of the predicted event nomination and correct the event nomination boundary.

Among them, the bias information of the predicted event nomination is generated, and the process of correcting the event nomination boundary is specifically as follows:

According to the predicted event nomination, the center coordinate offset Δc of the event nomination and the scale change factor Δl of the event nomination are calculated. The specific calculation formula is:

Δc＝(G _c −P _c )/P _c ;

Δl＝log(G _l /P _l );

Where: G _c represents the real event nomination center coordinates, P _c represents the predicted event nomination center coordinates; G _l represents the real event nomination scale, P _l represents the predicted event nomination scale;

The center coordinate offset Δc of the event nomination and the scale change factor Δl of the event nomination are used as supervisory signals. The second-layer recurrent neural network is trained using the L1 norm according to the supervisory signals to obtain the bias information of the predicted event nomination, which is recorded as Δc' and Δl'.

The predicted event nomination boundary is corrected by the bias information Δc' and Δl' to obtain the center position P _c 'and scale P _l 'of the corrected event nomination, specifically:

P _c ′＝P _c /(1+Δ′ _c );

According to the revised event nomination center position P _c 'and scale size P _l ', the revised start and end time are obtained, specifically:

Among them, P′ _start represents the corrected event nomination start time, and P′ _end represents the corrected event nomination end time, thus completing the correction of the predicted event nomination boundary.

The event nomination network loss function is formed by weighted superposition of the starting point distribution network loss sub-function, the end point distribution network loss sub-function and the boundary cycle correction network loss sub-function; wherein the starting point distribution network loss sub-function loss _s (c _s , t, X, y _s ) is specifically:

The endpoint distribution network loss sub-function loss _e ( _ce , t, X, _ye ) is specifically:

The boundary cycle correction network loss sub-function loss _reg (t _i ) is specifically:

为第t个时间点的第k个事件提名是否为真的监督信号，

为该事件提名在起始分布网络和终止分布网络下的置信度；Among them, X represents the entire data set;

A supervisory signal for whether the k-th event at the t-th time point is true or not,

Nominate the confidence level of the event under the starting distribution network and the ending distribution network;

Among them, K represents the number of event nominations output at each time point, which is the same as the confidence level; Δc _k and Δl _k are the supervisory signals for correcting the k-th event nomination at the t _i- th time point of the video; Δc' _k and Δl' _k are the bias information for predicting the same event nomination at the same time point;

Therefore, the loss function loss(c, t, X, y) is specifically:

loss (c, t, x, y) = α*loss _s (c _s , t, x, y _s ) + β*loss _e (c _e , t, x, y _e ) + γ*loss _reg (t _i );

Among them, α, β, and γ are the weight coefficients of the three sub-loss functions.

The loss function is used to train and update all recurrent neural networks in the event nomination network, thereby completing the training and updating of the event nomination network and obtaining the event nomination network after training and updating.

The specific process of using the event nomination network after training and updating to nominate and predict video events is as follows:

S1: Obtain video features of sample videos through a 3D convolutional network;

S2: Calculate the video features of the video using the trained and updated recurrent neural network, and obtain the video features output at each time point by the starting point distribution network, the end point distribution network, and the boundary cycle correction network;

S3: Output multiple confidences at each time point in the starting point distribution network and the end point distribution network respectively, and take the sum of the confidences of the corresponding event nominations in the starting point distribution network and the end point distribution network as the final confidence of the event, thereby completing the prediction of the event nomination;

S4: Boundary loops correct bias information for event nominations generated by the network;

S5: Sort the confidence of the events from large to small, take the first 1000 event nominations and correct them according to the corresponding bias information to obtain the final predicted event nominations.

Compared with the prior art, the technical solution of the present invention has the following beneficial effects:

The present invention provides an event nomination method combining boundary distribution and correction, which combines the distribution law of event start and end points in real videos to generate event nominations that fit the distribution of real events, and uses a cyclic correction network to correct the boundaries of event nominations, thereby obtaining event nominations that are more consistent with real events and make the boundaries of events more accurate.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG1 is a schematic diagram of the process of the present invention.

DETAILED DESCRIPTION

The drawings are for illustrative purposes only and are not to be construed as limiting the present patent;

In order to better illustrate the present embodiment, some parts in the drawings may be omitted, enlarged or reduced, and do not represent the size of the actual product;

It is understandable to those skilled in the art that some well-known structures and their descriptions may be omitted in the drawings.

The technical solution of the present invention is further described below in conjunction with the accompanying drawings and embodiments.

Example 1

As shown in FIG1 , an event nomination method combining boundary distribution and correction is used to form an event nomination network by constructing a starting point distribution network, an end point distribution network and a boundary cycle correction network; the event nomination network is trained and updated by constructing an event nomination network loss function; and the event nomination network after training and updating is used to make nomination predictions for video events;

The starting point distribution network and the end point distribution network are used for event nomination prediction;

The boundary loop correction network is used to generate bias information of predicted event nominations and correct the event nomination boundaries.

More specifically, the process of constructing the starting point distribution network and the end point distribution network is as follows:

Normalize the length of the existing dataset videos to determine the relative positions of the event start and end points in the video;

The relative positions of the start and end points of all events in the data set in the video are calculated, and the probability distributions of the start and end points of all events in the video on the video timeline are taken as _ws0 , _we0 ; _ws0 , _we0 represent the probability distributions of the start and end points of the event respectively, and the start point distribution network and the end point distribution network are obtained.

In the specific implementation process, when performing statistics on the data set, event nominations whose time intersection ratio tIOU with real events is greater than a certain threshold σ are classified as positive samples, and the start and end point distributions of all positive samples are statistically analyzed.

More specifically, the process of event nomination prediction by the starting point distribution network and the end point distribution network is as follows:

The video features of the sample video are obtained through a three-dimensional convolutional network. Based on the starting point distribution network and the end point distribution network, the obtained video features are calculated using a recurrent neural network to obtain the video features output at each time point in the starting point distribution network and the end point distribution network.

At each time point in the origin distribution network and the destination distribution network, K confidences are output, indicating the possibility of K fixed-length event nominations. The length of these K event nominations is:

[t-k,t+1],k∈[0,K];

The values of t and k satisfy t≥k, and the value of t changes with the length of the video; the higher the confidence, the greater the possibility that the interval is an event nomination; the sum of the confidences of the corresponding event nominations in the starting point distribution network and the end point distribution network is taken as the final confidence of the event, thereby completing the prediction of the event nomination.

More specifically, the boundary cycle correction network is constructed by two layers of recurrent neural networks, wherein the recurrent neural network of the first layer is used to calculate the video features output at each time point based on the video features of the sample video; the recurrent neural network of the second layer is used to generate bias information of the predicted event nomination and correct the event nomination boundary.

More specifically, the bias information of the predicted event nomination is generated and the process of correcting the event nomination boundary is as follows:

According to the predicted event nomination, the center coordinate offset Δc of the event nomination and the scale change factor Δl of the event nomination are calculated. The specific calculation formula is:

Δc＝(G _c −P _c )/P _c ;

Δl＝log(G _l /P _l );

Where: G _c represents the real event nomination center coordinates, P _c represents the predicted event nomination center coordinates; G _l represents the real event nomination scale, P _l represents the predicted event nomination scale;

The center coordinate offset Δc of the event nomination and the scale change factor Δl of the event nomination are used as supervisory signals. The second-layer recurrent neural network is trained using the L1 norm according to the supervisory signals to obtain the bias information of the predicted event nomination, which is recorded as Δc' and Δl'.

The predicted event nomination boundary is corrected by the bias information Δc' and Δl' to obtain the center position P _c 'and scale P _l 'of the corrected event nomination, specifically:

P _c ′＝P _c /(1+Δ′ _c );

According to the revised event nomination center position P _c 'and scale size P _l ', the revised start and end time are obtained, specifically:

Among them, P′ _start represents the corrected event nomination start time, and P′ _end represents the corrected event nomination end time, thus completing the correction of the predicted event nomination boundary.

More specifically, the event nomination network loss function is formed by weighted superposition of a starting point distribution network loss sub-function, an end point distribution network loss sub-function and a boundary cycle correction network loss sub-function; wherein the starting point distribution network loss sub-function loss _s (c _s , t, X, y _s ) is specifically:

The endpoint distribution network loss sub-function loss _e ( _ce , t, X, _ye ) is specifically:

The boundary cycle correction network loss sub-function loss _reg (t _i ) is specifically:

为第t个时间点的第k个事件提名是否为真的监督信号，

为该事件提名在起始分布网络和终止分布网络下的置信度；Among them, X represents the entire data set;

A supervisory signal for whether the k-th event at the t-th time point is true or not,

Nominate the confidence level of the event under the starting distribution network and the ending distribution network;

Among them, K represents the number of event nominations output at each time point, which is the same as the confidence level; Δc _k and Δl _k are the supervisory signals for correcting the k-th event nomination at the t _i- th time point of the video; Δc' _k and Δl' _k are the bias information for predicting the same event nomination at the same time point;

Therefore, the loss function loss(c, t, X, y) is specifically:

loss (c, t, x, y)=α*loss _s (c _s , t, x, y _s ) + P*loss _e (c _e , t, x, y _e ) + γ*loss _reg (t _i );

Among them, α, β, and γ are the weight coefficients of the three sub-loss functions.

More specifically, the loss function is used to train and update all recurrent neural networks in the event nomination network, thereby completing the training and updating of the event nomination network and obtaining the event nomination network after training and updating.

More specifically, the specific process of using the trained and updated event nomination network to make nomination predictions for video events is as follows:

S1: Obtain video features of sample videos through a 3D convolutional network;

S2: Calculate the video features of the video using the trained and updated recurrent neural network, and obtain the video features output at each time point by the starting point distribution network, the end point distribution network, and the boundary cycle correction network;

S3: Output multiple confidences at each time point in the starting point distribution network and the end point distribution network respectively, and take the sum of the confidences of the corresponding event nominations in the starting point distribution network and the end point distribution network as the final confidence of the event, thereby completing the prediction of the event nomination;

S4: Boundary loops correct bias information for event nominations generated by the network;

S5: Sort the confidence of the events from large to small, take the first 1000 event nominations and correct them according to the corresponding bias information to obtain the final predicted event nominations.

Example 2

On the basis of Example 1, more specifically, training and verification are performed on ActivityNet, which contains 20,000 untrimmed videos, totaling 849 hours, and about 100,000 descriptive statement labeling information. In this data set, each video contains several events and description statements for the events, and the start and end time and duration of the time in the same video are different. ActivityNet consists of three parts: a training set, a validation set, and a test set, with 10,024, 4,926, and 5,044 videos respectively. The embodiment mainly conducts experiments on the training set and the validation set.

When using a three-dimensional convolutional network [9] to extract features, a video feature is extracted every 64 frames, and the feature dimension is compressed to 500 using principal component analysis. The recurrent neural network used is a long short-term memory network LSTM with a dimension of 512, and K in the model is set to 256. In the language model for generating sentences, the upper limit of the number of words in each sentence is set to 32, and words with less than 3 occurrences are deleted when the vocabulary is established. The event nomination network is first trained to stability, and then trained in conjunction with the language model, and the learning rate is set to 5e-5.

In the specific implementation process, two indicators are usually used to evaluate the quality of event nominations: recall rate and precision rate. Recall rate evaluates how many real events are covered in the predicted event nominations, and precision rate evaluates how many of the predicted event nominations are correct. In addition, there is another indicator for comprehensive evaluation of event nominations, namely the _f1 score, which takes into account both precision and recall rate. It is calculated by recall rate and precision rate. The calculation method is as follows:

f ₁ = 2*(precision*recall/(precision+recall));

The comparison between the event nomination method combining boundary distribution and correction proposed by the present invention and the existing method on ActivityNet is shown in Table 1:

Table 1

method Recall @1000 Accuracy @1000 _f1 score @1000 SST[5] 0.716 0.533 0.571 Start and end point modeling 0.731 0.530 0.573 Boundary Regression 0.704 0.561 0.590 Start and end point distribution + boundary point regression 0.716 0.560 0.592

As shown in Table 1, @1000 indicates the event nominations ranked in the top 1000 in terms of confidence. The method used in the present invention is compared with the existing methods. SST[13] is the basis of the method used in the present invention, and it also predicts event nominations based on a recurrent neural network. However, in SST, data such as the distribution of the start and end points of event nominations are not processed, and the predicted event nomination boundaries are fixed. Therefore, by counting the start and end points of events, the recall rate is significantly improved, and after combining the regression of the event nomination boundaries, the recall rate remains basically unchanged, while the precision rate and _f1 score are greatly improved.

In the specific implementation process, the main indicators for evaluating the performance of event nomination for dense video description are BLEU-1, BLEU-2, BLEU-3, BLEU-4, Meteor, Rouge-L, and CIDEr-D, which are used to evaluate the similarity between the description sentences generated by the event nomination network and the real description sentences. Among the above indicators, Meteor mainly focuses on the performance of the event nomination network in this indicator because its measurement results are more similar to the results of manual judgment.

As shown in Table 2, compared with existing methods on the dense video description task, the method of the present invention performs better than existing methods in most indicators, especially in the Meteor indicator, which shows the effectiveness of the event nomination network.

Table 2 ActivityNet Caption validation experimental results

Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the present invention, and are not intended to limit the embodiments of the present invention. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the embodiments here. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the claims of the present invention.

[1] R.Krishna, K.Hata, F.Ren, L.Fei-Fei, and J.C.Niebles, "Dense-captioningevents in videos," in Proc.IEEE International Conference on Computer Vision, 2017, pp.706-715 .

[2] Z.Shou, D.Wang, and S.Chang, "Temporal action localization inuntrimmed videos via multi-stage CNNs," in Proc. IEEE Conference on ComputerVision and Pattern Recognition, 2016, pp.1049-1058.

[3]S.Ji, W.Xu, M.Yang, and K.Yu, "3D convolutional neural networks for human action recognition," IEEE Transactions on Pattern Analysis and MachineIntelligence, vol.35, no.1, pp.221- 231,2013.

[4] J.Gao, Z.Yang, C.Sun, K.Chen, and R.Nevatia, "TURN TAP: Temporal unitregression network for temporal action proposals," in Proc.IEEE International Conference on Computer Vision, 2017, pp. 3648-3656.

[5]Y.Chao, S.Vijayanarasimhan, B.Seybold, D.A.Ross,J.Deng,andR.Sukthankar, "Rethinking the faster R-CNN architecture for temporal actionlocalization," in Proc.IEEE Conference on Computer Vision and PatternRecognition, 2018, pp.1130-1139.

[6] S.Ren, K.He, R.Girshick, and J.Sun, "Faster R-CNN: Towards real-timeobject detection with region proposal networks," IEEE Transactions on PatternAnalysis and Machine Intelligence, vol.39, no .6, pp.1137-1149, 2017.

[7] V.Escorcia, F.Caba, J.C.Niebles, and B.Ghanem, “Daps: Deep actionproposals for action understanding,” in Proc.European Conference on ComputerVision, 2016, pp.768–784.

[8] S.Buch, V.Escorcia, C.Shen, B.Ghanem, and J.C.Niebles, "SST: Single-stream temporal action proposals," in Proc.IEEE Conference on Computer Vision and Pattern Recognition, 2017, pp.6373 -6382.

[9] S.Ji, W.Xu, M.Yang, and K.Yu, "3D convolutional neural networks for human action recognition," IEEE Transactions on Pattern Analysis and MachineIntelligence, vol.35, no.1, pp.221- 231,2013.

[10] R.Krishna, K.Hata, F.Ren, L.Fei-Fei, and J.C.Niebles, "Dense-captioning events in videos," 2017 IEEE International Conference on ComputerVision(ICCV), pp.706–715, 2017.

[11] Y. Li, T. Yao, Y. Pan, H. Chao, and T. Mei, "Jointly localizing and describing events for dense video captioning," in Proc. IEEE Conference on Computer Vision and Pattern Recognition, 2018, pp. 7492-7500.

[12] J.Wang, W.Jiang, L.Ma, W.Liu, and Y.Xu, "Bidirectional attentivefusion with context gating for dense video captioning," in Proc.IEEEConference on Computer Vision and Pattern Recognition, 2018, pp .7190-7198.

[13] S.Buch, V.Escorcia, C.Shen, B.Ghanem, and J.C.Niebles, "SST: Single-stream temporal action proposals," in Proc.IEEE Conference on Computer Vision and Pattern Recognition, 2017, pp.6373 -6382.

Claims (6)

1. An event naming method combining boundary distribution and correction is characterized in that: forming an event nomination network by constructing a starting point distribution network, an ending point distribution network and a boundary circulation correction network; training and updating the event nomination network by constructing an event nomination network loss function; performing nomination prediction on the video event by using the event nomination network after training and updating;

the starting point distribution network and the ending point distribution network are used for predicting event nomination;

the boundary circulation correction network is used for generating predicted event nomination bias information and correcting event nomination boundaries;

the construction process of the starting point distribution network and the ending point distribution network comprises the following steps:

normalizing the video length of the existing data set, and determining the relative position of an event starting point in the video;

counting the relative positions of all event starting and stopping points in the video in the data set, and taking probability distribution w of all event starting and stopping points in the video on a video time line _s0 ,w _e0 ；w _s0 ,w _e0 Respectively representing the starting point probability distribution and the ending point probability distribution of the event to obtain a starting point distribution network and an ending point distribution network;

the process of predicting event nomination by the starting point distribution network and the ending point distribution network specifically comprises the following steps:

acquiring video features of a sample video through a three-dimensional convolution network, and calculating the acquired video features by using a cyclic neural network based on a starting point distribution network and an end point distribution network to acquire video features output by each time point in the starting point distribution network and the end point distribution network;

outputting K confidence degrees at each time point in the starting point distribution network and the end point distribution network, wherein the confidence degrees represent the possibility of K event nominations with fixed lengths, and the lengths of the K event nominations are as follows:

[t-k,t+1],k∈[0,K]；

wherein the values of t and k meet the condition that t is more than or equal to k, and the value of t is changed along with the change of the video length; the higher the confidence, the greater the likelihood of being an event nomination; and taking the sum of the confidence degrees of the corresponding event nomination in the starting point distribution network and the ending point distribution network as the confidence degree of the last event so as to complete the prediction of the event nomination.

2. The event naming method combining boundary distribution and correction according to claim 1, wherein the boundary circulation correction network is constructed by two layers of circulation neural networks, wherein the circulation neural network of the first layer is used for calculating video characteristics output at each time point according to video characteristics of the sample video; the second layer of cyclic neural network is used for generating predicted event nomination bias information and correcting event nomination boundaries.

3. The method for event naming by combining boundary distribution and correction according to claim 2, wherein the generating of the bias information of the predicted event naming and the correcting of the event naming boundary are specifically:

calculating the central coordinate offset delta c of the event nomination and the scale change factor delta l of the event nomination according to the predicted event nomination, wherein the specific calculation formula is as follows:

Δc＝(G _c -P _c )/P _c ；

Δl＝log(G _l /P _l )；

wherein: g _c Representing the actual event nomination center coordinates, P _c Representing predicted event nomination center coordinates; g _l Representing the size, P, of the actual event nomination scale _l Representing a predicted event nomination scale size;

taking the central coordinate offset delta c of event nomination and the scale change factor delta L of event nomination as supervision signals, training a second layer of circulating neural network by utilizing L1 norm according to the supervision signals to obtain predicted offset information of event nomination, and recording the predicted offset information as delta c 'and delta L';

predicted by the pair of offset information Δc' and ΔlCorrecting the event nomination boundary to obtain a corrected event nomination center position P '' _c And a dimension of P' _l The method specifically comprises the following steps:

P′ _c ＝P _c /(1+Δ' _c )；

center position P 'noted from corrected event' _c And dimension size P _l The "corrected start-stop time" is specifically:

wherein P' _start Indicating the corrected event nomination starting time, P' _end And indicating the corrected event nomination termination time, and finishing the correction of the predicted event nomination boundary.

4. The event naming method combining boundary distribution and correction according to claim 3, wherein the event naming network loss function is formed by weighted superposition of a start distribution network loss sub-function, an end distribution network loss sub-function and a boundary circulation correction network loss sub-function; wherein the origin distribution network loses the subfunction loss _s (c _s ,t,X,y _s ) The method comprises the following steps:

the endpoint distribution network loss subfunction loss _e (c _e ,t,X,y _e ) The method comprises the following steps:

the boundary circulation corrects the network loss subfunction loss _reg (t _i ) The method comprises the following steps:

wherein X represents the entire dataset;

whether the kth event nominated for the kth time point is a genuine supervision signal,

Suggesting confidence levels for the event under the start distribution network and the end distribution network;

k represents the number of event nominations output by each time point and is the same as the confidence coefficient; Δc _k And Deltal _k Respectively at the t th of video _i A supervision signal for correcting the nomination of the kth event at each time point; Δc' _k And Deltal' _k The bias information predicted by nominating the same event at the same time point is obtained;

therefore, the loss function loss (c, t, X, y) is specifically:

loss(c，t，X，y)＝α*loss _s (c _s ，t，X，y _s )+β*loss _e (c _e ，t，X，y _e )+γ*loss _reg (t _i )；

wherein, alpha, beta and gamma are weight coefficients of three sub-loss functions.

5. The method for event naming by combining boundary distribution and correction according to claim 4, wherein training update is performed on all the cyclic neural networks in the event naming network by using the loss function, so as to complete training update of the event naming network and obtain the event naming network after training update.

6. The method for event naming by combining boundary distribution and correction according to claim 5, wherein the specific process of predicting the video event by using the event naming network after training update is:

s1: acquiring video characteristics of a sample video through a three-dimensional convolution network;

s2: calculating video characteristics of the video by using the training updated cyclic neural network to respectively obtain video characteristics output by each time point of a starting point distribution network, an ending point distribution network and a boundary cyclic correction network;

s3: outputting a plurality of confidence coefficients at each time point in the starting point distribution network and the end point distribution network respectively, taking the sum of the confidence coefficients of the corresponding event nomination in the starting point distribution network and the end point distribution network as the confidence coefficient of the last event, and completing the prediction of the event nomination;

s4: the boundary circulation correction network generates bias information of predicted event nomination;

s5: and sequencing the confidence degrees of the events from large to small, taking the top 1000 event nominations, and correcting the event nominations according to corresponding bias information to obtain the final predicted event nominations.

Images