CN112926445A

CN112926445A - Parabolic behavior recognition method, model training method and related device

Info

Publication number: CN112926445A
Application number: CN202110206044.XA
Authority: CN
Inventors: 翁仁亮; 钱扬
Original assignee: Beijing Aibee Technology Co Ltd
Current assignee: Beijing Aibee Technology Co Ltd
Priority date: 2021-02-24
Filing date: 2021-02-24
Publication date: 2021-06-08

Abstract

The embodiment of the application discloses a parabolic behavior identification method, a model training method and a related device, wherein the parabolic behavior identification method comprises the following steps: acquiring position information and time information corresponding to a plurality of motion pixel points on a target motion track; the target motion track is determined according to a plurality of motion areas corresponding to the motion target in the target track image and time information corresponding to the motion areas, the motion areas respectively belong to a plurality of frames of images in the target video, and the time information corresponding to the motion areas is determined according to the sequence of the images to which the motion areas belong in the target video; the time information corresponding to the motion pixel point is the time information corresponding to the motion area to which the motion pixel point belongs; determining the confidence coefficient of the target motion track corresponding to the parabolic behavior according to the position information and the time information corresponding to the multiple motion pixel points through a parabolic behavior identification model; and determining whether the moving object in the object video corresponds to the parabolic behavior according to the confidence coefficient.

Description

Parabolic behavior recognition method, model training method and related device

Technical Field

The application relates to the technical field of computers, in particular to a parabolic behavior recognition method, a model training method and a related device.

Background

In public places, abnormal throwing is an extremely dangerous behavior, for example, throwing prohibited articles upstairs of airport terminals to an isolation area may disturb the operational order of the airport if not, and may cause a serious safety accident if not. Currently, it is detected whether such parabolic behavior is present in public places, mainly based on images taken by surveillance cameras deployed in public places.

The related art mainly realizes detection and identification of parabolic behavior by the following means: generating a track image of a moving target according to an image shot by a monitoring camera, determining a moving track of the moving target in the track image, and then judging whether the related attribute of the moving track meets the attribute standard of a parabolic track by adopting a manually set rule, thereby detecting and identifying parabolic behavior.

The detection and identification mode of the parabolic behavior needs to borrow the prior knowledge of people, is difficult to cover various application environments, has certain scene limitation, and is easy to identify the parabolic behavior by mistake or neglect.

Disclosure of Invention

The embodiment of the application provides a parabolic behavior identification method, a model training method and a related device, which can accurately identify parabolic behaviors in various scenes.

In view of the above, a first aspect of the present application provides a method for identifying parabolic behavior, the method including:

acquiring position information and time information corresponding to a plurality of motion pixel points on a target motion track; the target motion track is determined according to a plurality of motion areas corresponding to the motion target in the target track image and time information corresponding to the motion areas, the motion areas respectively belong to a plurality of frames of images in the target video, and the time information corresponding to the motion areas is determined according to the sequence of the images to which the motion areas belong in the target video; the motion pixel point belongs to the motion area, and the time information corresponding to the motion pixel point is the time information corresponding to the motion area to which the motion pixel point belongs;

determining the confidence coefficient of the target motion track corresponding to the parabolic behavior according to the position information and the time information which correspond to the plurality of motion pixel points on the target motion track through a parabolic behavior identification model;

determining whether the moving object in the target video corresponds to parabolic behavior according to the confidence.

A second aspect of the present application provides a model training method, the method comprising:

acquiring a training sample set; the training sample set comprises a positive training sample and a negative training sample, the positive training sample comprises position information and time information corresponding to a plurality of sampling points on a motion track corresponding to the parabolic behavior, and the negative training sample comprises position information and time information corresponding to a plurality of sampling points on a motion track corresponding to the non-parabolic behavior;

training a parabolic behavior recognition model to be trained on the basis of the training sample set;

and determining to finish the training of the parabolic behavior recognition model when the parabolic behavior recognition model meets the training finishing condition.

A third aspect of the application provides a parabolic behavior recognition apparatus, the apparatus comprising:

the information acquisition module is used for acquiring the position information and the time information corresponding to the plurality of motion pixel points on the target motion track; the target motion track is determined according to a plurality of motion areas corresponding to the motion target in the target track image and time information corresponding to the motion areas, the motion areas respectively belong to a plurality of frames of images in the target video, and the time information corresponding to the motion areas is determined according to the sequence of the images to which the motion areas belong in the target video; the motion pixel point belongs to the motion area, and the time information corresponding to the motion pixel point is the time information corresponding to the motion area to which the motion pixel point belongs;

the confidence coefficient determining module is used for determining the confidence coefficient of the target motion track corresponding to the parabolic behavior according to the position information and the time information which correspond to the plurality of motion pixel points on the target motion track through a parabolic behavior recognition model;

and the behavior identification module is used for determining whether the moving target in the target video corresponds to the parabolic behavior according to the confidence coefficient.

A fourth aspect of the present application provides a model training apparatus, the apparatus comprising:

the training sample acquisition module is used for acquiring a training sample set; the training sample set comprises a positive training sample and a negative training sample, the positive training sample comprises position information and time information corresponding to a plurality of sampling points on a motion track corresponding to the parabolic behavior, and the negative training sample comprises position information and time information corresponding to a plurality of sampling points on a motion track corresponding to the non-parabolic behavior;

the training module is used for training a parabolic behavior recognition model to be trained on the basis of the training sample set; and determining to finish the training of the parabolic behavior recognition model when the parabolic behavior recognition model meets the training finishing condition.

A fifth aspect of the present application provides an apparatus comprising: a processor and a memory;

the memory for storing a computer program;

the processor is configured to invoke the computer program to execute the parabolic behavior recognition method according to the first aspect or the model training method according to the second aspect.

A sixth aspect of the present application provides a computer-readable storage medium for storing a computer program for executing the method for parabolic behavior recognition according to the first aspect or the method for model training according to the second aspect.

According to the technical scheme, the embodiment of the application has the following advantages:

the embodiment of the application provides a method for identifying parabolic behavior, which comprises the following steps: acquiring position information and time information corresponding to a plurality of motion pixel points on a target motion track, wherein the target motion track is determined according to a plurality of motion areas corresponding to a motion target in a target track image and time information corresponding to the motion areas, the plurality of motion areas in the target track image respectively belong to a plurality of frames of images in a target video, the time information corresponding to the motion areas is determined according to the sequence of the images to which the motion areas belong in the target video, the motion pixels belong to the motion areas, and the time information corresponding to the motion pixels is the time information corresponding to the motion areas to which the motion pixels belong; then, determining the confidence coefficient of the target motion track corresponding to the parabolic behavior according to the position information and the time information which correspond to the multiple motion pixel points on the target motion track through a parabolic behavior identification model; further, it is determined whether the moving object in the target video corresponds to a parabolic behavior according to the confidence. In the method for identifying the parabolic behavior, a pre-trained parabolic behavior identification model is adopted, and whether the motion trail corresponds to the parabolic behavior is identified according to the position information and the time information corresponding to a plurality of motion pixel points on the motion trail, so that the whole implementation process does not need manual intervention; in addition, when the parabolic behavior recognition model is trained, a large number of abundant training samples can be adopted to train the parabolic behavior recognition model, so that the parabolic behavior recognition model is suitable for various application environments and can accurately recognize parabolic behaviors in various application environments.

Drawings

Fig. 1 is a schematic flowchart of a parabolic behavior identification method according to an embodiment of the present application;

FIG. 2 is an exemplary target track image provided by an embodiment of the present application;

FIG. 3 is a schematic flow chart illustrating a model training method according to an embodiment of the present disclosure;

fig. 4 is a schematic structural diagram of a parabolic behavior recognition apparatus according to an embodiment of the present application;

FIG. 5 is a schematic structural diagram of a model training apparatus according to an embodiment of the present disclosure;

fig. 6 is a schematic structural diagram of a server according to an embodiment of the present application

Fig. 7 is a schematic structural diagram of a terminal device according to an embodiment of the present application.

Detailed Description

In order to make the technical solutions of the present application better understood, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.

The terms "first," "second," "third," "fourth," and the like in the description and in the claims of the present application and in the drawings described above, if any, are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used is interchangeable under appropriate circumstances such that the embodiments of the application described herein are capable of operation in sequences other than those illustrated or described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.

In order to reduce manual intervention in a parabolic behavior identification process and accurately identify parabolic behaviors in various application scenarios, embodiments of the present application provide a parabolic behavior identification method.

In the method for identifying the parabolic behavior, position information and time information corresponding to a plurality of motion pixel points on a target motion track are obtained firstly, the target motion track is determined according to a plurality of motion areas corresponding to a motion target in a target track image and time information corresponding to the motion areas, the plurality of motion areas in the target track image belong to a plurality of frames of images in a target video respectively, the time information corresponding to the motion areas is determined according to the sequence of the images to which the motion areas belong in the target video, the motion areas to which the motion pixel points belong, and the time information corresponding to the motion pixel points is the time information corresponding to the motion areas to which the motion pixel points belong; then, determining the confidence coefficient of the target motion track corresponding to the parabolic behavior according to the position information and the time information which correspond to the multiple motion pixel points on the target motion track through a parabolic behavior identification model; further, it is determined whether the moving object in the target video corresponds to a parabolic behavior according to the confidence.

According to the parabolic behavior identification method, a pre-trained parabolic behavior identification model is adopted, whether the motion trail corresponds to the parabolic behavior or not is identified according to the position information and the time information corresponding to the multiple motion pixel points on the motion trail, and the whole implementation process does not need manual intervention; in addition, a large number of abundant training samples can be adopted to train the parabolic behavior recognition model, so that the parabolic behavior recognition model obtained by training is suitable for various application environments, and parabolic behaviors can be accurately recognized in various application environments.

It should be noted that the parabolic behavior recognition method and the model training method provided in the embodiments of the present application may be applied to various devices with data processing capabilities, such as a terminal device, a server, and the like. The terminal device may be a computer, a tablet computer, a Personal Digital Assistant (PDA), a smart phone, or the like; the server may be an application server or a Web server, and in particular, when deployed, the server may be an independent server or a cluster server.

The parabolic behavior identification method provided by the present application is described in detail below by way of a method embodiment.

Referring to fig. 1, fig. 1 is a schematic flowchart of a parabolic behavior identification method according to an embodiment of the present application. For convenience of description, the following embodiments are described taking a server as an execution subject. As shown in fig. 1, the method for identifying parabolic behavior includes the following steps:

step 101: acquiring position information and time information corresponding to a plurality of motion pixel points on a target motion track; the target motion track is determined according to a plurality of motion areas corresponding to the motion target in the target track image and time information corresponding to the motion areas, the motion areas respectively belong to a plurality of frames of images in the target video, and the time information corresponding to the motion areas is determined according to the sequence of the images to which the motion areas belong in the target video; the motion pixel point belongs to the motion area, and the time information corresponding to the motion pixel point is the time information corresponding to the motion area to which the motion pixel point belongs.

In practical applications, a server generally needs to detect whether a parabolic behavior occurs in a scene shot by a target camera based on a target video shot by the target camera; in this case, the server generally needs to generate a target track image for representing a motion track of a moving target in the target video based on the target video captured by the target camera.

In specific implementation, the server can convert each frame of target image in a target video shot by the target camera into a corresponding moving image through a motion detection algorithm, wherein the moving image comprises a motion area corresponding to a motion target and a scene area corresponding to a scene, and the pixel value of the motion area is different from the pixel value of the static area; for example, the pixel values of the motion area and the scene area in the moving image may be 255 and 0, respectively, and the motion area and the scene area may appear white and black, respectively. Then, the server may screen out multiple frames of moving images from the moving images corresponding to the respective target images of the frames, and accordingly determine time information corresponding to the moving areas in the multiple frames of moving images according to the time sequence of the target images corresponding to the multiple frames of moving images in the target video. And generating a target track image which can represent the motion track of the motion target corresponding to the motion area according to the position of the motion area in the multi-frame motion image and the time information corresponding to the motion area in the multi-frame motion image.

Fig. 2 illustrates an exemplary target trajectory image, which includes a plurality of motion areas corresponding to a motion target, where the plurality of motion areas belong to different motion images and correspond to different time information, and the plurality of motion areas are connected in series according to the order of the time information from front to back, so as to fit the target motion trajectory corresponding to the motion target.

When the server specifically identifies whether the target motion track in the target track image corresponds to the parabolic behavior, it needs to collect a plurality of motion pixel points on the target motion track, and determine the position coordinates of each of the plurality of motion pixel points in the target track image and the time information corresponding to each of the plurality of motion pixel points. During specific implementation, after the server fits a target motion track corresponding to a moving target based on motion areas corresponding to the same moving target and different corresponding time information in a target track image, a pixel point on the target motion track can be further collected as a motion pixel point in each motion area corresponding to the moving target; for example, the server may collect pixel points intersecting the target motion trajectory on the boundary of the motion region as motion pixel points; furthermore, the server may determine the position coordinates of the moving pixel point in the target track image as the corresponding position information, and determine the time information corresponding to the moving area to which the moving pixel point belongs as the corresponding time information.

It should be noted that, in practical applications, the time information corresponding to the motion region may actually be a number corresponding to the motion region, where the number is determined according to the order in which the motion region is added to the target track image, or may be understood as the number is determined according to the order in the target video of the image to which the motion region belongs. Correspondingly, the time information corresponding to the motion pixel point is the number corresponding to the motion area to which the motion pixel point belongs.

Optionally, to avoid subsequent waste of unnecessary computation processing resources, the server may perform preliminary judgment on the target motion trajectory before determining, by using the parabolic behavior recognition model, a confidence that the target motion trajectory corresponds to the parabolic behavior, so as to judge whether the target motion trajectory possibly corresponds to the parabolic behavior. Specifically, the server may determine whether the target motion trajectory satisfies a constraint condition of a parabolic behavior, where the constraint condition of the parabolic behavior includes at least one of: the number corresponding to the starting motion pixel point on the target motion track is smaller than the number corresponding to the ending motion pixel point, and the target motion track passes through the target isolator; if the target motion track meets the constraint condition of the parabolic behavior, the target motion track is possibly corresponding to the parabolic behavior, and the subsequent steps can be continuously executed; if the target motion trajectory does not satisfy the constraint condition of the parabolic behavior, it is indicated that the target motion trajectory cannot correspond to the parabolic behavior, so that subsequent steps do not need to be executed continuously, and waste of computing and processing resources of the server can be reduced to a certain extent.

After the server fits to obtain a target motion track based on the motion area in the target track image and the number corresponding to the motion area, whether the target motion track is a parabola can be detected by adopting a curve detection algorithm; when the target motion trajectory is a parabola, the server may further determine whether the number corresponding to the start pixel point on the parabola is smaller than the number corresponding to the end pixel point on the parabola, if so, it indicates that the target motion trajectory may correspond to a parabolic behavior, otherwise, if not, it indicates that the target motion trajectory is not generated by a real parabolic behavior.

Furthermore, in real circumstances, objects thrown by parabolic action often need to traverse a particular partition, for example, throwing the object from inside the wall to outside the wall, or throwing the object from outside the wall to inside the wall; if the detected target motion track crosses the target spacer, it indicates that the target motion track may correspond to an effective parabolic behavior, whereas if the detected target motion track does not cross the target spacer, it indicates that the target motion track may correspond to an ineffective parabolic behavior.

It should be understood that, in practical application, the server may further set other parabolic behavior constraint conditions according to actual requirements, so as to perform preliminary detection and identification on the target motion trajectory through the set parabolic behavior constraint conditions, and the parabolic behavior constraint conditions are not limited in any way in this application.

Step 102: and determining the confidence coefficient of the target motion track corresponding to the parabolic behavior according to the position information and the time information which correspond to the plurality of motion pixel points on the target motion track through a parabolic behavior identification model.

The server collects a plurality of moving pixel points from a target motion track, and after position information and time information corresponding to the plurality of moving pixel points are determined, the server can generate specific feature vectors according to the position information and the time information corresponding to the plurality of moving pixel points, and input the feature vectors into a pre-trained parabolic behavior recognition model, and the parabolic behavior recognition model can correspondingly output the confidence coefficient of the target motion track corresponding to the parabolic behavior after analyzing and processing the input feature vectors.

It should be noted that the parabolic behavior recognition model in the embodiment of the present application may be any neural network classification model, for example, a classification network model including three layers of perceptrons, and the present application does not limit the specific structure of the parabolic behavior recognition model in any way. The method for training the parabolic behavior recognition model will be described in detail below by another embodiment of the method.

During specific implementation, the server can generate position feature vectors according to the position coordinates of a plurality of moving pixel points in the target track image and the time information corresponding to the plurality of moving pixel points; and then, determining the confidence coefficient of the target motion track corresponding to the parabolic behavior according to the position characteristic vector through a parabolic behavior recognition model.

For example, when the time information corresponding to the moving pixel point is a number, the server may preset that the position feature vector of the input parabolic behavior recognition model is a feature vector with a size of 1 × 2T, where T is a threshold of the number of the moving regions included in the target trajectory image, and may also be understood as a maximum number corresponding to the moving region included in the target trajectory image. The server constructs a position feature vector f with the size of 1 x 2T according to the position coordinates of each moving pixel point in the target track image; for example, the server may normalize the position coordinates of the motion pixel point with the number t in the target track image, that is, the abscissa x of the motion pixel point is divided by the width w of the target track image to obtain a normalized abscissa

Dividing the vertical coordinate y of the motion pixel point by the height h of the target track image to obtain a normalized vertical coordinate

Will be provided with

And

filling the 2t th bit and the 2t +1 th bit in the position feature vector f; the blank elements in f can be filled with 0 s. Furthermore, the server may input the position feature vector f into a pre-trained parabolic behavior recognition model to obtain a confidence level of the output of the parabolic behavior recognition model.

Of course, in practical applications, the server may also construct the location feature vector in other ways. For example, the position feature vector is preset to be a feature vector with the size of 2 × T, and the abscissa x and the ordinate y of the motion pixel point with the number of T are normalized to obtain the abscissa

And ordinate

Then, will

Fill in to the t-th bit of the first line in the position feature vector, will

Filling to the t-th bit of the total second line of the position feature vectors.

Step 103: determining whether the moving object in the target video corresponds to parabolic behavior according to the confidence.

After the server obtains the confidence coefficient output by the parabolic behavior recognition model, whether the moving target corresponding to the target moving track corresponds to the parabolic behavior or not can be determined according to the confidence coefficient. Specifically, the server may preset a confidence threshold, and if the confidence output by the parabolic behavior recognition model is higher than the confidence threshold, determine that the moving object corresponding to the target moving trajectory corresponds to the parabolic behavior; on the contrary, if the confidence coefficient output by the parabolic behavior recognition model is not higher than the confidence coefficient threshold, it is determined that the moving object corresponding to the target moving track does not correspond to the parabolic behavior.

Optionally, in a case that it is determined that the moving object in the target video corresponds to the parabolic behavior, the server may further generate alarm information, where the alarm information is used to prompt that the parabolic behavior exists in a shooting place corresponding to the target video; therefore, related staff are prompted to detect whether the parabolic behavior is dangerous or not in time. For example, assuming that the target video is shot by a target camera deployed in an airport and corresponds to a specific area such as an isolation area, when the server detects that the moving target in the target video corresponds to a parabolic behavior, an alarm message may be sent to an airport security officer to notify the airport security officer that the parabolic behavior exists in the isolation area shot by the target camera.

Optionally, the target motion trajectories in different target trajectory images may correspond to the same parabolic behavior, for example, the target motion trajectories in two target trajectory images with similar generation times may be generated by the same parabolic behavior; in order to avoid repeated alarm for the same parabolic behavior, the method provided by the embodiment of the application can further detect whether the target motion trajectories in the two target trajectory images correspond to the same parabolic behavior when it is detected that the target motion trajectories in the two target trajectory images with similar generation time both correspond to the parabolic behavior.

That is, the server may collect, based on a preset sampling standard, first sampling points on a first target motion trajectory to form a first sampling point set, and collect, on a second target motion trajectory to form a second sampling point set, when it is determined that a confidence level of the first target motion trajectory determined according to a first target trajectory image corresponding to the parabolic behavior is higher than a preset confidence level threshold, and it is determined that a confidence level of a second target motion trajectory determined according to a second target trajectory image corresponding to the parabolic behavior is higher than a preset confidence level threshold, and a time interval between the first target trajectory image and the second target trajectory image is less than a preset time length; the first sampling point and the second sampling point are concentrated into a first sampling point and a second sampling point which correspond to the same abscissa, and a sampling point pair is formed; for each sampling point pair, determining a difference value of the vertical coordinates of a first sampling point and a second sampling point as a difference value distance corresponding to the sampling point pair; calculating the average value of the difference distance corresponding to each sampling point pair as an average distance; and judging whether the average distance is smaller than a preset distance threshold value, if so, determining that the first target motion track and the second target motion track correspond to the same parabolic behavior.

For example, the server may uniformly acquire a plurality of first acquisition points according to an x coordinate on a first target motion trajectory in the first target trajectory image to form a first acquisition point set, and uniformly acquire a plurality of second acquisition points according to an x coordinate on a second target trajectory in the second target trajectory image to form a second acquisition point set. Then, forming an acquisition point pair by using the first acquisition point and the second acquisition point which have the same abscissa in the first acquisition point set and the second acquisition point set; and calculating the difference value of the vertical coordinates of the first acquisition point and the second acquisition point as the corresponding difference value distance of the acquisition point pair aiming at each acquisition point pair. And further, calculating the average value of the difference distances corresponding to the acquisition points to obtain the average distance. Judging whether the average distance is smaller than a preset distance threshold value, if so, determining that a first target motion track in the first target track image and a second target track in the second target track image actually correspond to the same parabolic behavior; conversely, if not, it may be determined that the first target motion trajectory in the first target trajectory image and the second target trajectory in the second target trajectory image actually correspond to different parabolic behaviors.

In the method for identifying the parabolic behavior, position information and time information corresponding to a plurality of motion pixels on a target motion track are obtained, the target motion track is determined according to a plurality of motion areas corresponding to a motion target in a target track image and time information corresponding to the motion areas, the motion areas in the target track image belong to a plurality of frames of images in a target video respectively, the time information corresponding to the motion areas is determined according to the sequence of the images to which the motion areas belong in the target video, the motion areas to which the motion pixels belong, and the time information corresponding to the motion pixels is the time information corresponding to the motion areas to which the motion pixels belong; then, determining the confidence coefficient of the target motion track corresponding to the parabolic behavior according to the position information and the time information which correspond to the multiple motion pixel points on the target motion track through a parabolic behavior identification model; further, it is determined whether the moving object in the target video corresponds to a parabolic behavior according to the confidence. Thus, a pre-trained parabolic behavior recognition model is adopted, whether the motion trail corresponds to the parabolic behavior is recognized according to the position information and the time information corresponding to the motion pixel points on the motion trail, and manual intervention is not needed in the overall implementation process; in addition, a large number of abundant training samples can be adopted to train the parabolic behavior recognition model, so that the parabolic behavior recognition model obtained by training is suitable for various application environments, and parabolic behaviors can be accurately recognized in various application environments.

The following describes in detail a model training method for training the above parabolic behavior recognition model according to an embodiment of the present application.

Referring to fig. 3, fig. 3 is a schematic flowchart of a model training method provided in the embodiment of the present application. For convenience of description, the following embodiments are described by taking a server as an execution subject. As shown in fig. 3, the model training method includes the following steps:

step 301: acquiring a training sample set; the training sample set comprises a positive training sample and a negative training sample, the positive training sample comprises position information and time information corresponding to a plurality of sampling points on a motion track corresponding to the parabolic behavior, and the negative training sample comprises position information and time information corresponding to a plurality of sampling points on a motion track corresponding to the non-parabolic behavior.

Before the server trains the parabolic behavior recognition model, a training sample set comprising a large number of training samples needs to be obtained. The training sample set comprises a positive training sample corresponding to the parabolic behavior and a negative training sample corresponding to the non-parabolic behavior, wherein the positive training sample comprises position information and time information corresponding to a plurality of sampling points on a motion track corresponding to the parabolic behavior, and the negative training sample comprises position information and time information corresponding to a plurality of sampling points on a motion track corresponding to the non-parabolic behavior.

In one possible implementation, the server may construct the positive training sample by: setting an initial coordinate of the parabolic track, initial time information, initial speed and acceleration corresponding to the initial coordinate, and simulating to generate the parabolic track based on the initial coordinate, the initial speed and the acceleration; then, sampling points are collected on the parabolic track according to the generated time sequence, and corresponding time information is configured for the sampling points according to the time sequence generated by the sampling points on the basis of the initial time information; and then, constructing a training sample based on the initial coordinates and the initial time information corresponding to the initial coordinates, and the acquired sampling points and the time information corresponding to the sampling points.

In consideration of the fact that real parabolic data are difficult to obtain in real life and are small in quantity, and the requirement of training a neural network model is generally difficult to meet, the method provided by the embodiment of the application automatically generates a large number of positive training samples corresponding to parabolic behaviors based on the physical laws followed by the parabolic behaviors. Specifically, an initial coordinate and an initial number, an initial velocity and an acceleration corresponding to the initial coordinate may be randomly generated in a set range, and a parabolic track may be generated based on the initial coordinate, the initial velocity and the acceleration; then, sampling uniformly or non-uniformly in time, recording the coordinates of each sampling point, and configuring a corresponding number for the sampling point. Points on the parabolic trajectory that take into account real parabolic behavior are not ideal particles and therefore a range of perturbations can be applied to the coordinates of each sample point.

According to prior knowledge, the time of an object thrown by a throwing action in a camera picture does not exceed T seconds, the average frame rate of a camera is K, T-T-K motion areas corresponding to a motion target are contained in a target track image generated according to a video shot by the camera at most, and T sampling points are collected on the motion track of the thrown object correspondingly; therefore, the number of sampling points included in the positive training samples should not exceed T. The positive training samples generated in this way fit the true parabolic data better, and theoretically, the positive training samples generated in this way can cover various parabolic cases.

In one possible implementation, the server may construct the negative training sample by: detecting whether the motion trail of the moving target is a parabola or not according to a plurality of motion areas corresponding to a plurality of moving targets in the trail image and time information corresponding to the motion areas respectively through a parabola detection algorithm; under the condition that the motion trail of the moving target is not parabolic, a plurality of moving pixel points are collected on the motion trail of the moving target, and a negative training sample is constructed on the basis of the coordinates of the moving pixel points in the trail image and the time information corresponding to the motion areas to which the moving pixel points belong.

Since there is no parabolic behavior most of the time in the real environment, negative training samples can be generated based on the motion trajectories collected by the camera corresponding to other moving objects. That is, the server may adopt a parabola detection algorithm to detect whether the motion trajectory of the moving object corresponding to the motion region is a parabola or not according to the plurality of motion regions and their respective corresponding numbers in the trajectory image generated based on the video shot by the camera; if not, a plurality of motion pixel points can be collected on the motion trail obtained based on the fitting of the plurality of motion areas, and a negative training sample is constructed by utilizing the coordinates of the plurality of motion pixel points in the trail image and the numbers corresponding to the motion areas to which the plurality of motion pixel points belong.

Considering the resolution of images taken by various camerasThe rates are different, so the coordinates (x, y) of the motion pixel points in the training sample are usually normalized, and the abscissa after normalization

Normalized ordinate

Where w and h are the width and height, respectively, of the trajectory image to which the moving pixel point belongs. Then, a position characteristic vector f with the size of 1 x 2T is constructed, and coordinates obtained by normalizing the moving pixel points with the number of T are arranged at the 2T position and the 2T +1 position in the position characteristic vector f; for other blank elements of the motion feature vector, 0 may be filled.

Step 302: and training a parabolic behavior recognition model to be trained based on the training sample set.

After the server acquires the training sample set, the server can train the pre-constructed parabolic behavior recognition model to be trained by using the positive training sample and the negative training sample in the training sample set. The parabolic behavior recognition model may be any kind of neural network classification model, for example, a classification network model including three layers of perceptrons.

During specific training, the server can input the position characteristic vector determined based on the coordinate and time information corresponding to each moving pixel point in the training sample into a parabolic behavior recognition model to be trained; the parabolic behavior recognition model analyzes and processes the input position characteristic vector and outputs a prediction confidence coefficient. Further, the server may construct a cross-entropy loss function based on the prediction confidence and the behavior type (i.e., whether the training sample is a parabolic behavior), and adjust model parameters in the trained parabolic behavior recognition model based on the cross-entropy loss function. It should be understood that when the training sample is a positive training sample, the standard confidence corresponding to the positive training sample should be 1, and the server may construct a cross entropy loss function based on the prediction confidence output by the parabolic behavior recognition model and 1; when the training sample is a negative training sample, the standard confidence corresponding to the negative training sample should be 0, and the server may construct a cross entropy loss function based on the prediction confidence output by the parabolic behavior recognition model and 0.

Step 303: and determining to finish the training of the parabolic behavior recognition model when the parabolic behavior recognition model meets the training finishing condition.

In the process of training the parabolic behavior recognition model, the server can judge whether the parabolic behavior recognition model meets the training end condition; if the parabolic behavior recognition model meets the training end condition, the completion of the training of the parabolic behavior recognition model can be confirmed, and the parabolic behavior recognition model can be put into practical application.

For example, the server may test the model performance of the parabolic behavior recognition model by using the test sample set, and if the test result obtained by the test indicates that the accuracy of the parabolic behavior recognition model reaches a preset accuracy threshold, the server may consider that the parabolic behavior recognition model has satisfied the training end condition; or, the server may also determine whether the number of iterative training for the parabolic behavior recognition model reaches a preset training number threshold, and if so, may also consider that the parabolic behavior recognition model has satisfied the training end condition. The present application does not limit the training end condition in any way.

In the model training method, a server firstly obtains a training sample set, the training sample set comprises a positive training sample and a negative training sample, the positive training sample comprises position information and time information corresponding to a plurality of sampling points on a motion track corresponding to a parabolic behavior, and the negative training sample comprises position information and time information corresponding to a plurality of sampling points on a motion track corresponding to a non-parabolic behavior; then, training a parabolic behavior recognition model to be trained based on the training sample set; and determining to finish the training of the parabolic behavior recognition model when the parabolic behavior recognition model meets the training finishing condition. Therefore, a large number of abundant training samples are adopted to train the parabolic behavior recognition model when the parabolic behavior recognition model is trained, so that the parabolic behavior recognition model obtained by training is suitable for various application environments, and parabolic behaviors can be accurately recognized in various application environments.

The embodiment of the application also provides a parabolic behavior recognition device. Referring to fig. 4, fig. 4 is a schematic structural diagram of a parabolic behavior recognition apparatus provided in an embodiment of the present application, and as shown in fig. 4, the apparatus includes:

the information acquisition module 401 is configured to acquire position information and time information corresponding to a plurality of motion pixels on a target motion trajectory; the target motion track is determined according to a plurality of motion areas corresponding to the motion target in the target track image and time information corresponding to the motion areas, the motion areas respectively belong to a plurality of frames of images in the target video, and the time information corresponding to the motion areas is determined according to the sequence of the images to which the motion areas belong in the target video; the motion pixel point belongs to the motion area, and the time information corresponding to the motion pixel point is the time information corresponding to the motion area to which the motion pixel point belongs;

a confidence determining module 402, configured to determine, through a parabolic behavior recognition model, a confidence that the target motion trajectory corresponds to a parabolic behavior according to position information and time information corresponding to each of a plurality of motion pixel points on the target motion trajectory;

a behavior identification module 403, configured to determine whether the moving object in the target video corresponds to a parabolic behavior according to the confidence.

Optionally, the confidence determining module 402 is specifically configured to:

generating a position feature vector according to the position coordinates of the plurality of moving pixel points in the target track image and the time information corresponding to the plurality of moving pixel points;

and determining the confidence degree of the target motion track corresponding to the parabolic behavior according to the position feature vector through the parabolic behavior recognition model.

Optionally, the position feature vector is a feature vector with a size of 1 × 2T, where T is determined according to a threshold value of the number of motion regions in the target trajectory image, and T is a positive integer greater than 1; the time information corresponding to the motion pixel points is a number, and the number is determined according to the sequence in which the motion areas to which the motion pixel points belong are added into the target track image; the confidence determination module 402 is specifically configured to:

for each motion pixel point, respectively carrying out normalization processing on the abscissa and the ordinate of the motion pixel point in the target track image; and respectively filling the normalized abscissa and ordinate to the position of 2t and the position of 2t +1 in the position characteristic vector according to the number t corresponding to the motion pixel point.

Optionally, in a case that it is determined that the confidence that the first target motion trajectory determined according to the first target trajectory image corresponds to the parabolic behavior is higher than a preset confidence threshold, and it is determined that the confidence that the second target motion trajectory determined according to the second target trajectory image corresponds to the parabolic behavior is higher than the preset confidence threshold, and a time interval between the first target trajectory image and the second target trajectory image is less than a preset time length, the apparatus further includes:

the same parabolic behavior judging module is used for collecting first sampling points on the first target motion track to form a first sampling point set and collecting second sampling points on the second target motion track to form a second sampling point set based on a preset sampling standard; the first sampling point and the second sampling point are concentrated into a first sampling point and a second sampling point which correspond to the same abscissa, and a sampling point pair is formed; for each sampling point pair, determining a difference value of the vertical coordinates of the first sampling point and the second sampling point as a corresponding difference distance of the sampling point pair; calculating the average value of the difference distance corresponding to each sampling point pair as an average distance; and judging whether the average distance is smaller than a preset distance threshold value, if so, determining that the first target motion track and the second target motion track correspond to the same parabolic behavior.

Optionally, the time information corresponding to the motion pixel point is a number, and the number is determined according to the order in which the motion area to which the motion pixel point belongs is added to the target track image; the device further comprises:

the parabolic behavior pre-judging module is used for judging whether the target motion track meets a parabolic behavior constraint condition; the parabolic behavior constraints include at least one of: the number corresponding to the starting motion pixel point on the target motion track is smaller than the number corresponding to the ending motion pixel point, and the target motion track passes through the target isolator; and if so, executing the parabolic behavior identification model, and determining the confidence coefficient of the target motion track corresponding to the parabolic behavior according to the position information and the number corresponding to each of the plurality of motion pixel points on the target motion track.

The device for identifying the parabolic behavior firstly acquires the position information and the time information corresponding to a plurality of motion pixels on a target motion track, wherein the target motion track is determined according to a plurality of motion areas corresponding to a motion target in a target track image and the time information corresponding to the motion areas, the plurality of motion areas in the target track image respectively belong to a plurality of frames of images in a target video, the time information corresponding to the motion areas is determined according to the sequence of the images to which the motion areas belong in the target video, the motion areas to which the motion pixels belong, and the time information corresponding to the motion pixels is the time information corresponding to the motion areas to which the motion pixels belong; then, determining the confidence coefficient of the target motion track corresponding to the parabolic behavior according to the position information and the time information which correspond to the multiple motion pixel points on the target motion track through a parabolic behavior identification model; further, it is determined whether the moving object in the target video corresponds to a parabolic behavior according to the confidence. Thus, a pre-trained parabolic behavior recognition model is adopted, whether the motion trail corresponds to the parabolic behavior is recognized according to the position information and the time information corresponding to the motion pixel points on the motion trail, and manual intervention is not needed in the overall implementation process; in addition, a large number of abundant training samples can be adopted to train the parabolic behavior recognition model, so that the parabolic behavior recognition model obtained by training is suitable for various application environments, and parabolic behaviors can be accurately recognized in various application environments.

The embodiment of the application also provides a model training device. Referring to fig. 5, fig. 5 is a block diagram of a model training apparatus according to an embodiment of the present disclosure, as shown in fig. 5, the apparatus includes:

Optionally, the apparatus further comprises:

the positive sample construction module is used for setting an initial coordinate of a parabolic track, initial time information, an initial speed and an acceleration corresponding to the initial coordinate, and generating the parabolic track based on the initial coordinate, the initial speed and the acceleration in a simulation mode; sampling points are collected on the parabolic track according to the generated time sequence, and corresponding time information is configured for the sampling points according to the time sequence generated by the sampling points on the basis of the initial time information; and constructing the positive training sample based on the initial coordinates and the initial time information corresponding to the initial coordinates, and the acquired sampling points and the time information corresponding to the sampling points.

Optionally, the apparatus further comprises:

and the negative sample construction module is used for collecting a plurality of motion pixel points on the motion trail of the motion target under the condition that the motion trail of the motion target is not parabolic, and constructing the negative training sample based on the coordinates of the motion pixel points in the trail image and the time information corresponding to the motion areas to which the motion pixel points belong.

The model training device firstly acquires a training sample set, wherein the training sample set comprises a positive training sample and a negative training sample, the positive training sample comprises position information and time information which correspond to a plurality of sampling points on a motion track corresponding to the parabolic behavior, and the negative training sample comprises position information and time information which correspond to a plurality of sampling points on a motion track corresponding to the non-parabolic behavior; then, training a parabolic behavior recognition model to be trained based on the training sample set; and determining to finish the training of the parabolic behavior recognition model when the parabolic behavior recognition model meets the training finishing condition. Therefore, a large number of abundant training samples are adopted to train the parabolic behavior recognition model when the parabolic behavior recognition model is trained, so that the parabolic behavior recognition model obtained by training is suitable for various application environments, and parabolic behaviors can be accurately recognized in various application environments.

The embodiment of the present application further provides a device for identifying parabolic behavior and training a model, where the device may specifically be a server or a terminal device, and the server and the terminal device provided in the embodiment of the present application will be described in terms of hardware implementation.

Referring to fig. 6, fig. 6 is a schematic structural diagram of a server 600 according to an embodiment of the present disclosure. The server 600 may vary significantly due to configuration or performance, and may include one or more Central Processing Units (CPUs) 622 (e.g., one or more processors) and memory 632, one or more storage media 630 (e.g., one or more mass storage devices) storing applications 642 or data 644. Memory 632 and storage medium 630 may be, among other things, transient or persistent storage. The program stored in the storage medium 630 may include one or more modules (not shown), each of which may include a series of instruction operations for the server. Still further, the central processor 622 may be configured to communicate with the storage medium 660 to execute a series of instruction operations in the storage medium 660 on the server 600.

The server 600 may also include one or more power supplies 626, one or more wired or wireless network interfaces 650, one or more input-output interfaces 658, and/or one or more operating systems 641, such as Windows Server, Mac OS XTM, UnixTM, LinuxTM, FreeBSDTM, and so forth.

The steps performed by the server in the above embodiments may be based on the server structure shown in fig. 6.

The CPU 622 is configured to execute the following steps:

Alternatively, the first and second electrodes may be,

Optionally, the CPU 622 can also be used to execute the steps of any implementation manner of the parabolic behavior recognition method and the model training method provided in the embodiments of the present application.

Referring to fig. 7, fig. 7 is a schematic structural diagram of a terminal device according to an embodiment of the present application. For convenience of explanation, only the parts related to the embodiments of the present application are shown, and details of the specific technology are not disclosed. The terminal may be any terminal device including a computer, a tablet computer, a Personal Digital Assistant (PDA), and the like, taking the terminal as the computer as an example:

fig. 7 is a block diagram illustrating a partial structure of a computer related to a terminal provided in an embodiment of the present application. Referring to fig. 7, the computer includes: radio Frequency (RF) circuit 710, memory 720, input unit 730, display unit 740, sensor 750, audio circuit 760, wireless fidelity (WiFi) module 770, processor 780, and power supply 790. Those skilled in the art will appreciate that the computer architecture shown in FIG. 7 is not intended to be limiting of computers, and may include more or fewer components than those shown, or some components in combination, or a different arrangement of components.

The memory 720 may be used to store software programs and modules, and the processor 780 performs various functional applications of the computer and data processing by operating the software programs and modules stored in the memory 720. The memory 720 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application program required by at least one function (such as a sound playing function, an image playing function, etc.), and the like; the storage data area may store data (such as audio data, a phonebook, etc.) created according to the use of the computer, etc. Further, the memory 720 may include high speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other volatile solid state storage device.

The processor 780 is a control center of the computer, connects various parts of the entire computer using various interfaces and lines, performs various functions of the computer and processes data by operating or executing software programs and/or modules stored in the memory 720 and calling data stored in the memory 720, thereby monitoring the entire computer. Optionally, processor 780 may include one or more processing units; preferably, the processor 780 may integrate an application processor, which primarily handles operating systems, user interfaces, applications, etc., and a modem processor, which primarily handles wireless communications. It will be appreciated that the modem processor described above may not be integrated into processor 780.

In the embodiment of the present application, the processor 780 included in the terminal further has the following functions:

Alternatively, the first and second electrodes may be,

Optionally, the processor 780 is further configured to perform the steps of any one implementation manner of the parabolic behavior recognition method and the model training method provided in the embodiments of the present application.

The present application further provides a computer-readable storage medium for storing a program code for executing any one of the embodiments of the parabolic behavior recognition method and the model training method described in the foregoing embodiments.

The present embodiments also provide a computer program product including instructions, which when run on a computer, cause the computer to perform any one of the embodiments of the parabolic behavior recognition method and the model training method described in the foregoing embodiments.

It is clear to those skilled in the art that, for convenience and brevity of description, the specific working processes of the above-described systems, apparatuses and units may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.

In the several embodiments provided in the present application, it should be understood that the disclosed system, apparatus and method may be implemented in other manners. For example, the above-described apparatus embodiments are merely illustrative, and for example, the division of the units is only one logical division, and other divisions may be realized in practice, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may be in an electrical, mechanical or other form.

The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.

In addition, functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.

The integrated unit, if implemented in the form of a software functional unit and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present application may be substantially implemented or contributed to by the prior art, or all or part of the technical solution may be embodied in a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present application. And the aforementioned storage medium includes: a U disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing computer programs.

It should be understood that in the present application, "at least one" means one or more, "a plurality" means two or more. "and/or" for describing an association relationship of associated objects, indicating that there may be three relationships, e.g., "a and/or B" may indicate: only A, only B and both A and B are present, wherein A and B may be singular or plural. The character "/" generally indicates that the former and latter associated objects are in an "or" relationship. "at least one of the following" or similar expressions refer to any combination of these items, including any combination of single item(s) or plural items. For example, at least one (one) of a, b, or c, may represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", wherein a, b, c may be single or plural.

The above embodiments are only used for illustrating the technical solutions of the present application, and not for limiting the same; although the present application has been described in detail with reference to the foregoing embodiments, it should be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions in the embodiments of the present application.

Claims

1. A method for identifying parabolic behavior, the method comprising:

2. The method of claim 1, wherein determining, by the parabolic behavior recognition model, the confidence level that the target motion trajectory corresponds to the parabolic behavior according to the position information and the time information corresponding to each of the plurality of motion pixels on the target motion trajectory comprises:

3. The method according to claim 2, wherein the position feature vector is a feature vector with a size of 1 x 2T, wherein T is determined according to a threshold value of the number of motion areas in the target track image, and T is a positive integer greater than 1; the time information corresponding to the motion pixel points is a number, and the number is determined according to the sequence in which the motion areas to which the motion pixel points belong are added into the target track image;

generating a position feature vector according to the position coordinates of the plurality of moving pixel points in the target track image and the time information corresponding to the plurality of moving pixel points, including:

4. The method of claim 1, wherein in a case where it is determined that the confidence that the first target motion trajectory determined from the first target trajectory image corresponds to the parabolic behavior is higher than a preset confidence threshold, and that the confidence that the second target motion trajectory determined from the second target trajectory image corresponds to the parabolic behavior is higher than the preset confidence threshold, and the time interval between the first target trajectory image and the second target trajectory image is less than a preset time duration, the method further comprises:

based on a preset sampling standard, collecting first sampling points on the first target motion track to form a first sampling point set, and collecting second sampling points on the second target motion track to form a second sampling point set;

the first sampling point and the second sampling point are concentrated into a first sampling point and a second sampling point which correspond to the same abscissa, and a sampling point pair is formed;

for each sampling point pair, determining a difference value of the vertical coordinates of the first sampling point and the second sampling point as a corresponding difference distance of the sampling point pair;

calculating the average value of the difference distance corresponding to each sampling point pair as an average distance;

and judging whether the average distance is smaller than a preset distance threshold value, if so, determining that the first target motion track and the second target motion track correspond to the same parabolic behavior.

5. The method according to claim 1, wherein the time information corresponding to the motion pixel point is a number determined according to an order in which the motion region to which the motion pixel point belongs is added to the target track image;

before determining, by the parabolic behavior recognition model, the confidence that the target motion trajectory corresponds to the parabolic behavior according to the position information and the time information corresponding to each of the plurality of motion pixels on the target motion trajectory, the method further includes:

judging whether the target motion track meets a parabolic behavior constraint condition; the parabolic behavior constraints include at least one of: the number corresponding to the starting motion pixel point on the target motion track is smaller than the number corresponding to the ending motion pixel point, and the target motion track passes through the target isolator;

and if so, executing the parabolic behavior identification model, and determining the confidence coefficient of the target motion track corresponding to the parabolic behavior according to the position information and the number corresponding to each of the plurality of motion pixel points on the target motion track.

6. A method of model training, the method comprising:

7. The method of claim 6, wherein the positive training sample is constructed by:

setting an initial coordinate of a parabolic track, initial time information, an initial speed and an acceleration corresponding to the initial coordinate, and simulating to generate the parabolic track based on the initial coordinate, the initial speed and the acceleration;

sampling points are collected on the parabolic track according to the generated time sequence, and corresponding time information is configured for the sampling points according to the time sequence generated by the sampling points on the basis of the initial time information;

and constructing the positive training sample based on the initial coordinates and the initial time information corresponding to the initial coordinates, and the acquired sampling points and the time information corresponding to the sampling points.

8. The method of claim 6, wherein the negative training sample is constructed by:

detecting whether the motion track of the moving target is a parabola or not according to a plurality of motion areas corresponding to a plurality of moving targets in the track image and time information corresponding to the motion areas respectively through a parabola detection algorithm;

under the condition that the motion trail of the motion target is not parabolic, collecting a plurality of motion pixel points on the motion trail of the motion target, and constructing the negative training sample based on the coordinates of the motion pixel points in the trail image and the time information corresponding to the motion areas to which the motion pixel points belong.

9. A parabolic behavior recognition apparatus, characterized in that the apparatus comprises:

10. A model training apparatus, the apparatus comprising:

11. An apparatus, characterized in that the apparatus comprises: a processor and a memory;

the memory for storing a computer program;

the processor, configured to invoke the computer program to perform the method for parabolic behavior recognition according to any one of claims 1 to 5 or the method for model training according to any one of claims 6 to 8.

12. A computer-readable storage medium for storing a computer program for performing the method for parabolic behavior recognition according to any one of claims 1 to 5 or the method for model training according to any one of claims 6 to 8.