JP7292492B2 - Object tracking method and device, storage medium and computer program - Google Patents

Description

The present invention relates to the field of computer vision, and more particularly to an object tracking method and apparatus, storage medium and computer program.

Currently, there is a growing demand for analyzing motion trajectories of objects by multi-object tracking technology. In the process of multi-object tracking, it is necessary to obtain the locations of multiple objects by object detection, and then perform single-object tracking for each object.

The processing time of the multiple object tracking shows a linear correlation with the number of objects in the scene. For example, if a scene contains N objects, where N is a positive integer, multiple-object tracking requires N inferences of single-object tracking, and the processing time is It increases up to N times the time required for single object tracking. The larger the value of N, the longer the time for multiple object tracking. As a result, the equipment required high computing power and took a long time.

The present invention provides an object tracking method and apparatus, a storage medium and a computer program.

A first aspect of embodiments of the present invention provides an object tracking method. The method comprises the steps of: obtaining a plurality of scene images corresponding to the same scene; performing feature extraction processing and target part detection on each scene image among the plurality of scene images; acquiring feature information and positions of a plurality of target parts in each of the scene images; and acquiring target feature information corresponding to each of the positions of the plurality of target parts among the feature information of each of the scene images. and identifying a plurality of identical objects appearing in the plurality of scene images based on the target feature information corresponding to each of the acquired positions of the plurality of target regions, wherein each scene image is , including some or all of said plurality of same objects.

In some alternative embodiments, feature extraction processing and target site detection are performed on each scene image of the plurality of scene images, and the feature information of each scene image and the plurality of scene images in each scene image are obtained. The step of obtaining the position of the target part includes: extracting a first feature map of each scene image among the plurality of scene images; detecting the target part in the first feature map of each scene image; obtaining positions of a plurality of target parts in each of the scene images, and performing feature extraction processing on a first feature map of each of the scene images to obtain a multidimensional second feature map. The step of acquiring target feature information corresponding to each of the positions of the plurality of target parts among the feature information of each of the scene images includes: Obtaining a target feature vector corresponding to each.

In some alternative embodiments, identifying a plurality of identical objects appearing in the plurality of scene images based on target feature information corresponding to each of the acquired locations of the plurality of target sites. uses a plurality of pieces of target feature information respectively corresponding to two adjacent scene images out of the plurality of scene images, and calculates the similarity between each target part in each of the adjacent two scene images. and identifying a plurality of identical objects appearing in different scene images based on the similarity between each target portion in the adjacent two scene images.

In some alternative embodiments, the adjacent two scene images are a first scene image and a second scene image, and the adjacent two scene images of the plurality of scene images. using a plurality of target feature information corresponding to each of the N target feature vectors in the first scene image to obtain the similarity between each target part in the two adjacent scene images and M target feature vectors in a second scene image; and each of the N target feature vectors in the first scene image and the M target feature vectors in the second scene image. obtaining an N×M dimensional similarity matrix based on the similarity of , represents the degree of similarity between any first target portion in the first scene image and any second target portion in the second scene image.

In some optional embodiments, identifying a plurality of identical objects appearing in the different scene images based on a similarity between each target portion in the adjacent two scene images comprises: identifying a maximum similarity value from similarities between each of the first target feature vectors of the N target feature vectors and the M target feature vectors based on a similarity matrix; and identifying a second target feature vector corresponding to the maximum similarity value among the M target feature vectors if the maximum value is greater than a predetermined threshold; making the object to which the first target part corresponding to the feature vector belongs and the object to which the second target part corresponding to the second target feature vector in the second scene image belongs the same object.

In some alternative embodiments, feature extraction processing and target site detection are performed on each scene image of the plurality of scene images, and the feature information of each scene image and the plurality of scene images in each scene image are obtained. The step of obtaining a position of a target part includes: extracting a first feature map of each scene image among the plurality of scene images via a backbone network of the feature detection model; and performing part detection of the feature detection model. performing target feature detection in a first feature map of each scene image, via a branch, to obtain locations of a plurality of target features in each scene image; and, via a feature extraction branch of the feature detection model, performing feature extraction processing on the first feature map of each scene image to obtain a multi-dimensional second feature map.

In some optional embodiments, the method includes inputting multiple sample scene images corresponding to the same scene into an initial neural network model, and performing multiple sample scene images for each sample scene image output from the initial neural network model. obtaining a sample feature vector corresponding to each target site position; and obtaining two adjacent sample scenes based on object identifiers corresponding to each of the plurality of marked target sites in each of the sample scene images. determining a first degree of similarity between the sample feature vectors corresponding to the target site locations of the same object identifier and/or the samples corresponding to the target site locations of different object identifiers in an image; determining a second degree of similarity between feature vectors; and determining the first degree of similarity and the second degree of similarity based on object identifiers corresponding to each of the plurality of marked target regions in each of the sample scene images. performing supervised training on the initial neural network model to obtain the feature detection model based on at least one of:

In some optional embodiments, at least one of the first similarity measure and the second similarity measure is determined based on an object identifier corresponding to each of the plurality of marked target portions in each of the sample scene images. supervised training the initial neural network model to obtain the feature detection model based on a first loss function, the difference between a first similarity reference value and the first similarity; setting the difference between the second similarity reference value and the second similarity as a second loss function; and based on at least one of the first loss function and the second loss function, training the initial neural network model to obtain the feature detection model;
The first similarity reference value is a similarity reference value between sample feature vectors corresponding to target portions of the same marked object identifier in the adjacent two sample scene images, and the second similarity The reference value is a similarity reference value between the sample feature vectors corresponding to the marked target portions of different object identifiers in the adjacent two sample scene images.

In some optional embodiments, the method includes determining whether a motion trajectory of at least one of the plurality of same objects appearing in the plurality of scene images matches a desired motion trajectory within a predetermined time period. The step of determining whether the

In some optional embodiments, the plurality of scene images correspond to a classroom scene, the object includes a teaching subject, and the desired motion trajectory is at least specified to the teaching subject in a teaching task. Contains one type of motion trajectory.

A second aspect of an embodiment of the present invention provides an object tracking device. The apparatus includes an acquisition module for acquiring a plurality of scene images corresponding to the same scene, performing feature extraction processing and target part detection on each scene image among the plurality of scene images, and a processing module for acquiring feature information of a scene image and positions of a plurality of target parts in each of the scene images; A plurality of identical objects appearing in the plurality of scene images are identified based on a feature information identifying module for acquiring feature information and target feature information corresponding to each of the acquired positions of the plurality of target parts. and an object identification module for doing so, wherein each scene image includes some or all of said plurality of same objects.

A third aspect of embodiments of the invention provides a computer-readable storage medium. A computer program is stored in the storage medium, and the computer program is used to execute the object tracking method according to any one of the first aspects.

A fourth aspect of an embodiment of the present invention provides an object tracking device. The object tracking device comprises a processor and a memory for storing executable instructions executable by the processor, the processor calling the executable instructions stored in the memory to perform the Arranged to implement the object tracking method according to any one of the clauses.

A fifth aspect of embodiments of the present invention provides a computer program product. The object tracking method according to any one of the first aspects is implementable when said computer program is executed by a processor.

The technical solutions according to the embodiments of the present invention can have the following advantageous effects.

In the embodiment of the present invention, after identifying a plurality of objects in each of two adjacent scene images, a single object tracking inference is performed for each object in the preceding scene image to determine the plurality of objects contained in the subsequent scene image. instead of performing single frame estimation on a single scene image to obtain target feature information corresponding to the positions of a plurality of target parts, and performing matching on the single frame estimation results to obtain adjacent two Acquire multiple same objects in the scene images one by one to achieve the purpose of multi-object tracking. Also, even if the current scene contains multiple objects, the time of the entire multi-object tracking procedure is not related to the number of objects contained in the scene image because the estimation is performed for the entire scene image. Therefore, tracking time does not increase by iterating single-object tracking inferences as the number of objects increases. This greatly saves computing resources, shortens the time of multi-object tracking, and effectively improves the detection efficiency of multi-object tracking.

It is to be understood that the general description above and the detailed description hereinafter are merely illustrative and explanatory and are not limiting of the present invention.

The drawings herein are incorporated into and constitute a part of the specification. These drawings, while illustrating embodiments consistent with the invention, are used together with the description to interpret the principles of the invention.
1 is a flowchart of an object tracking method according to one exemplary embodiment of the present invention; 4 is a flow chart of another object tracking method according to an exemplary embodiment of the present invention; 4 is a flow chart of another object tracking method according to an exemplary embodiment of the present invention; 4 is a flow chart of another object tracking method according to an exemplary embodiment of the present invention; 4 is a flow chart of another object tracking method according to an exemplary embodiment of the present invention; FIG. 4 is a structural schematic diagram of a feature detection model shown in one exemplary embodiment of the present invention; FIG. 4 is a schematic diagram of an estimation procedure for multiple object tracking according to one exemplary embodiment of the present invention; 4 is a flow chart of another object tracking method according to an exemplary embodiment of the present invention; FIG. 4 is a schematic diagram of a training scene for a feature detection model in accordance with one exemplary embodiment of the present invention; 4 is a flow chart of another object tracking method according to an exemplary embodiment of the present invention; 1 is a block diagram of an object tracking device according to one exemplary embodiment of the present invention; FIG. 1 is a structural schematic diagram for an object tracking device shown in one exemplary embodiment of the present invention; FIG.

An illustrative embodiment will now be described in detail. An illustration thereof is shown in the drawing. The following description, when referring to the drawings, like numerals in different drawings indicate the same or similar elements, unless otherwise indicated. The embodiments described in the illustrative examples below do not represent all embodiments consistent with the present invention. Rather, they are merely examples of apparatus and methods consistent with some aspects of the present invention as set forth in detail in the appended claims.

The terminology used in the invention is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. As used in the present invention and the appended claims, the singular forms "a", "said" and "the" include plural forms unless the context clearly dictates otherwise. intended to It should be understood that the term "and/or" as used herein refers to any and all possible combinations of one or more of the associated listed items.

It should be understood that although the terms first, second, third, etc. are used in the present invention to describe various types of information, these information are not limited to these terms. These terms are only used to distinguish between similar types of information. For example, first information may be referred to as second information, and similarly, second information may be referred to as first information, without departing from the scope of the present invention. This is context dependent. For example, the word "if" as used herein may be interpreted as "when" or "when" or "depending on the particular circumstances".

Embodiments of the present invention provide a multi-object tracking scheme and are applicable to terminal devices in different scenes as an example. Different scenes include, but are not limited to, classrooms, surveillance camera locations, or other indoor or outdoor scenes where multiple objects need to be tracked. The terminal device can be any device with a camera head, or the terminal device can be an external imaging device. The terminal device may collect a plurality of scene images one after the other in the same scene, or may collect the video stream as it is and use the plurality of images in the video stream as the plurality of scene images.

Further, the terminal device performs feature extraction processing and target site detection on each scene image of the acquired plurality of scene images, and extracts feature information of each scene image and a plurality of target sites in each scene image. By obtaining target feature information corresponding to each of the positions of a plurality of target parts among the feature information of each scene image based on the position of each of the scene images, a plurality of the same objects appearing in the plurality of scene images are identified. do.

For example, in a classroom, the terminal equipment can employ teaching multimedia equipment with a camera head placed in the classroom, including but not limited to teaching projectors, classroom monitoring equipment, and the like. The terminal device acquires a plurality of scene images in the classroom, performs feature extraction processing and target part detection on each scene image among the plurality of scene images, and obtains feature information of each scene image. and the positions of a plurality of target parts in each scene image. A plurality of the same objects appearing in the plurality of scene images are identified by obtaining target feature information corresponding to the positions of the plurality of target parts from among the feature information of each of the scene images; serve the purpose of tracking. Objects in the scene may include, but are not limited to, teaching subjects, such as students. Target parts may include, but are not limited to, human facial parts and human body parts.

Further for example, in a subway or railway station, one or more monitoring camera heads may be arranged to acquire multiple scene images of the subway or railway station via the monitoring camera heads. Objects in the scene may include passengers, suitcases carried by passengers, employees, and the like. By adopting the technical solution according to the embodiment of the present invention, it is possible to identify the same multiple objects appearing in multiple scene images in a scene with a large number of people, such as a subway station or a train station. serve the purpose of tracking.

By way of example, the multiple object tracking scheme according to embodiments of the present invention is also applicable to cloud servers in different scenes. The cloud server may be provided with an external camera head, and may collect multiple scene images back and forth in the same scene via the external camera head, or collect the video stream as it is and A plurality of images in the stream may be used as the plurality of scene images. Collected scene images may be sent to a cloud server via a router or gateway. The cloud server performs feature extraction processing and target site detection on each scene image, and acquires feature information of each scene image and positions of a plurality of target sites in each scene image, thereby obtaining each scene image. Among the feature information, target feature information corresponding to each of the positions of the plurality of target parts is obtained, and a plurality of identical objects appearing in the plurality of scene images are specified.

For example, an external camera head is installed in a classroom to collect multiple scene images in the classroom and send them to a cloud server via a router or gateway, and the cloud server executes the above object tracking method.

In the embodiment of the present invention, after specifying a plurality of same objects appearing in a plurality of scene images via a terminal device or a cloud server, the same objects are marked in the same recognition frame, and the marked scene image is may be output. For example, in two adjacent output scene images, the red recognition frame marks the object 1 in the scene, the green recognition frame marks the object 2 in the scene, and the blue recognition frame marks the object 3 in the scene. Better indication of multiple identical objects in the current scene, such as by marking. Alternatively, the same or different objects may be distinguished by object identifiers corresponding to recognition frames. For example, one output scene image contains three recognition frames, the corresponding object identifiers are 1, 2 and 3, respectively, and the adjacent scene image contains two recognition frames and the corresponding object When the identifiers are 1 and 3, respectively, the recognition frames with the object identifier of 1 in these two scene images correspond to the same object, and the recognition frames with the object identifier of 3 also correspond to the same object. It can be specified that the recognition frames with identifiers 1 and 3 correspond to different objects.

Furthermore, the motion trajectory of at least one of the plurality of identical objects is specified in a predetermined time period via a terminal device or a cloud server, and whether or not the motion trajectory matches the desired motion trajectory is determined. may be analyzed.

For example, if the current scene is a classroom and the object includes a teaching target, the desired motion trajectory is at least one motion trajectory specified for the teaching target in the teaching task, e.g. (The other location may be the podium, the blackboard, or the location where other classmates are located), but is not limited thereto. Alternatively, the desired motion trajectory may include existing at the same position. Teachers can better conduct teaching activities based on the motion trajectories of multiple teaching objects.

Further, for example, if the current scene is a subway station or a train station where surveillance cameras are placed, and the objects include, but are not limited to, passengers, the target motion trajectory may be a designated dangerous motion trajectory or illegal motion trajectory. Trajectories may include, but are not limited to, moving from a home position to a position where a rail is located, moving above or below a ticket gate, and the like. According to the motion trajectory of the passenger, the employee can better manage the station and avoid the occurrence of dangerous behavior or illegal boarding behavior such as free riding.

The above is merely an example description of the scenes that the present invention applies to, and other rooms or scenes that require quick action type recognition are also included in the protection scope of the present invention.

As shown in FIG. 1, FIG. 1 shows an object tracking method according to one exemplary embodiment, including the following steps.

At step 101, a plurality of scene images corresponding to the same scene are acquired.

In embodiments of the present invention, multiple scene images may be collected one behind the other for the same scene, or a video stream may be collected and multiple images in the video stream may be multiple scene images. A scene of the present invention includes, but is not limited to, any scene in which multiple object tracking needs to be performed, such as a classroom, a point of surveillance camera placement, and the like.

In step 102, feature extraction processing and target site detection are performed on each scene image of the plurality of scene images, and the feature information of each scene image and the positions of the plurality of target sites in each scene image are determined. get.

In the embodiment of the present invention, performing feature extraction on each scene image means extracting feature information from each scene image, and the feature information may include color features, texture features, shape features, and the like. Good, but not limited to them. Color features are global features and describe surface color attributes of objects corresponding to images; texture features are also global features and describe surface texture attributes of objects corresponding to images; There are representations, one for contour features and another for region features. The contour features of the image are mainly for the outer boundary of the object, and the region features of the image are related to the shape of the image region.

In the embodiment of the present invention, one target site corresponds to one object, but it is not limited to this, and multiple target sites may correspond to one object. Target parts may include, but are not limited to, human facial parts and/or human body parts. Body parts may include the entire human body or certain designated parts of the human body, such as hands, feet, and the like. The position of the target site may be indicated by at least the center position of the recognition frame of the target site. For example, when the target part includes a human face part, the position of the target part may be indicated by the center position of the human face recognition frame. The recognition frame of the target part may be implemented as, for example, a circumscribed rectangular frame of the target part.

In step 103, among the feature information of each scene image, target feature information corresponding to each of the positions of the plurality of target parts is obtained.

In an embodiment of the present invention, each scene image includes a plurality of target regions, and based on the acquired feature information of each scene image, feature extraction is performed on pixels in an area including the target region to obtain a plurality of target regions. Identify target feature information corresponding to each of the positions of the part. As an example, target feature information corresponding to each of a plurality of pixels included in each target site area may be acquired from the feature information of each scene image by convolution processing or the like.

In step 104, a plurality of identical objects appearing in the plurality of scene images are identified based on the target feature information corresponding to each of the acquired positions of the plurality of target parts. Each scene image includes some or all of the same objects.

In the above-described embodiment, target feature information corresponding to the positions of a plurality of target parts in each scene image is acquired, and matching is performed with respect to the target feature information of the plurality of scene images. Multiple identical objects appearing in a scene image can be identified.

In the above embodiment, after specifying a plurality of objects in each of two adjacent scene images, each object in the former scene image is identified as a single object among the plurality of objects included in the latter scene image. No need to do each object tracking inference. Instead, single-frame estimation is performed on a single scene image to obtain target feature information corresponding to the positions of a plurality of target parts, and single-frame estimation results of two adjacent scene images obtained are obtained. to obtain multiple identical objects in every two adjacent scene images to achieve the purpose of multiple object tracking. Even if the current scene contains multiple objects, the time of the entire multi-object tracking procedure is independent of the number of objects contained in the scene image because the estimation is performed on the entire scene image. Therefore, tracking time does not increase by iterating single-object tracking inferences as the number of objects increases. This greatly saves computing resources, shortens the time of multi-object tracking, and effectively improves the detection efficiency of multi-object tracking.

In some alternative embodiments, as shown in FIG. 2, step 102 may include the following steps.

At step 102-1, a first feature map of each scene image among the plurality of scene images is extracted.

In an embodiment of the present invention, image features of each scene image may be extracted through a pre-trained neural network model to obtain a first feature map. The neural network model may employ a model such as a Visual Geometry Group Network (VGG Net), but is not limited thereto.

In step 102-2, target part detection is performed in the first feature map of each scene image to obtain the positions of a plurality of target parts in each scene image; feature extraction processing is performed to obtain a multidimensional second feature map.

In embodiments of the present invention, the target part may include a human facial part and/or an anthropomorphic part. Human facial regions and/or human body regions are detected in the first feature map of each scene image through a region prediction network (RPN), and human facial regions and/or human body regions corresponding to the human facial regions are detected. You may identify the human body area|region corresponding to . The human face area may be marked with a human face recognition frame, and the human body area may be marked with a human body recognition frame. As an example, the center position of the human face recognition frame may be the position of the human face part. Similarly, the center position of the human body recognition frame may be the position of the human body part.

Further, feature extraction processing may be performed on the first feature map of each scene image, and multiple types of feature information included in the first feature map may be extracted via different channels. Thus, a multi-dimensional second feature map is obtained. Illustratively, the size of the second feature map may be the same as the size of the first feature map, and the dimension value of the second feature map is the predetermined number of channels corresponding to each scene image.

Accordingly, step 103 may include the following.

A target feature vector corresponding to each of the positions of the plurality of target parts in the multidimensional second feature map is obtained.

In an embodiment of the present invention, the target feature information represents feature information corresponding to each of a plurality of pixels in each of a plurality of target region regions included in the second feature map of any dimension. used for The target part may include a human facial part and/or an anthropomorphic part.

Feature information corresponding to any one pixel in a plurality of target region regions included in any dimension of the second feature map can also constitute one one-dimensional feature vector. From these feature vectors, one or more feature vectors may be selected to represent feature information of the region of the target site (ie, target feature information) to facilitate subsequent similarity calculations. In an embodiment of the present invention, a feature vector corresponding to the pixel at the target site location may be selected and used as the target feature vector corresponding to the target site location in the second feature map of that dimension. The position of the target part may include the center position of the human face recognition frame and/or the center position of the human body recognition frame.

Further, for the second feature map of at least one dimension of the multi-dimensional second feature map, features corresponding to pixels at the locations of the plurality of target sites are obtained so as to improve the accuracy of subsequent target site matching. Information may be obtained to obtain a target feature vector corresponding to each of the plurality of target site locations. Illustratively, for each dimension of the second feature map, a target feature vector corresponding to each of a plurality of target site locations can be obtained. Thus, the dimension values of the target feature vector and the dimension values of the second feature map are made the same. For example, if the dimension value of the second feature map is C, the dimension value of the target feature vector is also C.

In the above embodiment, the entire procedure of feature extraction, target site detection, and identification of target feature vectors corresponding to each of the locations of a plurality of target sites, performed sequentially on the entire scene image, is performed on a single scene image. Since it is a frame estimate, it does not matter how many or how many objects it contains. Subsequent matching is performed against the target feature vectors corresponding to each of the multiple object positions in the two adjacent scene images, so there is no need for separate single-object tracking inference. Even if the number of objects included in the scene image increases, the mapping procedure can be completed at once. Since the object tracking method of the present invention is independent of the number of objects in the scene image, there is no increase in tracking time as the number of objects increases. This greatly saves computing resources, shortens the time of multi-object tracking, and effectively improves the detection efficiency of multi-object tracking.

In some alternative embodiments, as shown in FIG. 3, step 104 may include the following steps.

In step 104-1, using the plurality of target feature information corresponding to each of the adjacent two scene images out of the plurality of scene images, each target in the adjacent two scene images is Get the similarity between parts.

In the embodiment of the present invention, among the feature information of each scene image, a plurality of target feature information corresponding to the plurality of target parts are already specified, and a plurality of target features corresponding to each of two adjacent scene images are identified. Similarity calculation is performed using the feature information, and the similarity between each target part in each two adjacent scene images can be obtained.

At step 104-2, a plurality of identical objects appearing in the different scene images are identified based on the degree of similarity between each target portion in the two adjacent scene images.

In the embodiment of the present invention, the same object appearing in different scene images may be the object to which the target part with the highest degree of similarity belongs in each of two adjacent scene images.

In the above embodiment, multiple identical objects appearing in different scene images can be identified based on the similarity between each target part in two adjacent scene images, and the purpose of multiple object tracking is achieved. , the tracking procedure is independent of the number of objects, increasing availability.

In some alternative embodiments, the adjacent two scene images are a first scene image _T0 and a second scene image _T1 .

As shown in FIG. 4, step 104-1 above may include the following steps.

Step 104-11 determines the degree of similarity between each of the N target feature vectors in the first scene image and the M target feature vectors in the second scene image.

In an embodiment of the present invention, the target feature information is used to represent feature information corresponding to each of a plurality of pixels in each of a plurality of target region regions included in the second feature map of any dimension. be done. The target part may include a human facial part and/or an anthropomorphic part.

Based on the target feature information, feature information corresponding to any one pixel in a plurality of target region regions included in any dimension of the second feature map can also constitute one one-dimensional feature vector. be. From these feature vectors, one or more feature vectors may be selected to represent the feature information of the region of the target site to facilitate subsequent similarity calculations. In an embodiment of the present invention, a feature vector corresponding to the pixel at the target site location may be selected and used as the target feature vector corresponding to the target site location in the second feature map of that dimension. The position of the target part may include the center position of the human face recognition frame and/or the center position of the human body recognition frame.

In the step of identifying similarity, the degree of similarity between each of the N target feature vectors in the first scene image and the M target feature vectors in the second scene image among the two adjacent scene images is specified. You may N and M are positive integers of 2 or more. That is, the degree of similarity between each of the plurality of target feature vectors in the first scene image and the plurality of target feature vectors in the second scene image is identified.

In one possible implementation, cosine similarity values between the target feature vectors may be determined during similarity determination. By calculating the cosine value of the included angle between any one target feature vector in the first scene image and any one target feature vector in the second scene image, their similarity is evaluated.

In step 104-12, based on the similarities between each of the N target feature vectors in the first scene image and the M target feature vectors in the second scene image, an N×M dimensional similarity matrix to get

In an embodiment of the present invention, the value of any dimension in the similarity matrix indicates the similarity between any first target portion in said first scene image and any second target portion in said second scene image. represents degrees. N and M may or may not be equal.

In the above embodiment, an N×M dimensional similarity matrix is obtained by determining the similarity between each of the N target feature vectors in the first scene image and the M target feature vectors in the second scene image. The degree of similarity between any of the first target parts in the first scene image and any of the second target parts in the second scene image may be indicated by a similarity matrix, which facilitates implementation and use. more likely.

In some alternative embodiments, a bipartite graph algorithm may be employed for step 104-2. A plurality of identical objects appearing in the different scene images are identified based on the similarity between each target portion in the two adjacent scene images under the condition that the spatial distance constraint is satisfied.

In a bipartite graph algorithm, in one bipartite graph, the left vertex is X and the right vertex is Y, the weight value _wij is given to the left and right connections X _i Y _j of each group, and the sum of all w _ij is It refers to finding the maximum matching. In an embodiment of the present invention, X _i corresponds to one of the N target feature vectors in the first scene image and Y _j corresponds to one of the M target feature vectors in the second scene image. and the weighting value w _ij corresponds to the similarity. In the present invention, when the similarity is the maximum, the N target feature vectors and the second target feature vector are matched, and the same objects in the current two adjacent scene images can finally be identified. It is necessary to

In an embodiment of the present invention, the condition for satisfying the spatial distance constraint includes that the dimension of similarity between the N target feature vectors and the M target feature vectors does not exceed N×M.

In one possible implementation, it is necessary to ensure that the similarity is maximum and also that this maximum similarity exceeds a predetermined threshold so that the accuracy of multiple object tracking is further improved.

As shown in FIG. 5, step 104-2 may include the following steps.

At step 104-21, based on the similarity matrix, identify a maximum similarity value from the similarity between each of the first target feature vectors of the N target feature vectors and the M target feature vectors. do.

In an embodiment of the present invention, the first target feature vector is any one of the N target feature vectors identified in the first scene image. A similarity between the first target feature vector and each target feature vector in the second scene image may be obtained based on a similarity matrix. One similarity maximum value may be identified from these similarities.

であり、第１目標特徴ベクトルとＭ個の第２目標特徴ベクトルとの間の類似度がそれぞれａ₁₁、ａ₁₂及びａ₁₃であるとすれば、その中の最大値（ａ₁₁と仮定する）は、特定可能である。 The similarity matrix is

, and the similarities between the _first target feature vector and the M second target feature vectors are a ₁₁ , a ₁₂ and a ₁₃ , respectively. ) is identifiable.

At step 104-22, if the maximum similarity value is greater than a predetermined threshold, a second target feature vector corresponding to the maximum similarity value among the M target feature vectors is identified.

In the embodiment of the present invention, the second target feature vector is the target feature vector corresponding to the maximum similarity value among the M target feature vectors included in the second scene image.

To further ensure the accuracy of multiple object tracking, it is necessary to ensure that the maximum similarity value is greater than a predetermined threshold.

In step 104-23, the object to which the first target portion corresponding to the first target feature vector in the first scene image belongs and the second target portion to which the second target feature vector in the second scene image belongs. be the same object.

In the embodiment of the present invention, when the maximum similarity value is greater than a predetermined threshold, the object to which the first target part corresponding to the first target feature vector of the first scene image belongs and the second scene image The object to which the second target part corresponding to the second target feature vector belongs is assumed to be the same object.

If the maximum similarity value is equal to or less than the predetermined threshold, even if the object to which the first target part corresponding to the first target feature vector in the first scene image belongs does not have the same object in the second scene image. good.

Repeat steps 104-21 to 104-23 above, where the number of repetitions is the number N of target feature vectors contained in the first scene image, and all the same objects appearing in the first scene image and the second scene image are finalized. can be specifically identified.

In the above embodiment, according to the similarity matrix, the two objects with the closest similarities between the target parts in the two adjacent scene images may be regarded as the same object, thus achieving the purpose of tracking multiple objects. and increased availability.

In some alternative embodiments, after a plurality of scene images are acquired, at least two of the plurality of scene images are input to a pre-trained feature detection model, and the feature detection model is: performing feature extraction processing and target site detection on each scene image out of the plurality of scene images via the camera, and acquiring feature information of each scene image and positions of a plurality of target sites in each scene image; and, based on the positions of the plurality of target parts in each scene image, a plurality of pieces of target feature information corresponding to the plurality of target parts may be obtained from among the feature information of each of the scene images.

As shown in FIG. 6, the configuration of the feature detection model is such that a plurality of scene images are input to the feature detection model. Feature extraction is performed on each scene image to obtain a first feature map of each scene image.

Further, performing target site detection in a first feature map of each scene image via a site detection branch of the feature detection model to obtain locations of a plurality of target sites in each scene image; through the feature extraction branch of to perform feature extraction processing on the first feature map of each scene image to obtain a multi-dimensional second feature map. The object may include a person, and the target part may include a human facial part and/or a human body part. A feature extraction branch may be formed by serially connecting at least one convolutional layer. The size of the second feature map is the same as the size of the first feature map. Thus, the positions of multiple target sites are the same in the second feature map of each dimension. The dimension value of the second feature map is the same as the predetermined number of channels corresponding to each scene image.

Furthermore, a plurality of target feature vectors corresponding to the positions of the plurality of target regions may be obtained in the second multidimensional feature map. The position of the target part may be indicated by the center position of the human face recognition frame and/or the center position of the human body recognition frame. The dimension values of the target feature vector are the same as the dimension values of the second feature map. The center position coordinates of a certain human face recognition frame are (x, y), and the size of the second feature map obtained in the feature extraction branch matches the size of the first feature map, both of which are H×W. , H and W are the length and width of the image, respectively, the dimension value of the second feature map is C, and C is the predetermined number of channels corresponding to each scene image. Since the target feature vector corresponding to the center position (x, y) of the human face recognition frame can be obtained for each channel, the dimension value of the target feature vector is C.

In an embodiment of the present invention, after extracting a plurality of target feature vectors corresponding to the positions of the plurality of target parts in the multidimensional second feature map, each of the N target feature vectors in the first scene image and obtaining a similarity matrix by identifying similarities to the M target feature vectors in a second scene image, and identifying a plurality of identical objects appearing in the different scene images based on the similarity matrix; may The identification method is the same as the method in step 104-2 above, and will not be described repeatedly here.

As shown in FIG. 7, N target feature vectors and M target feature vectors are obtained by separately inputting the feature detection models to the first scene image _T0 and the second scene image _T1 . may Furthermore, a bipartite graph algorithm is employed to identify the same object appearing in _T0 and _T1 by matching against the extracted features of the target site under the condition that the spatial distance constraint is satisfied. good too.

In the above embodiment, single-frame estimation is performed for each scene image. No matter how many objects are included in each scene image, multi-object tracking can be realized quickly, and the detection efficiency of multi-object tracking is effective. substantially improved.

In some alternative embodiments, as shown in FIG. 8, the method may further include the following steps.

In step 100-1, a plurality of sample scene images corresponding to the same scene are input to the initial neural network model, and each sample scene image output from the initial neural network model corresponds to each of the positions of a plurality of target parts. Get a sample feature vector that

In an embodiment of the present invention, a plurality of existing sample images corresponding to the same scene are adopted as input values for an initial neural network model, and a plurality of sample images are preliminarily identified by recognition frames and/or corresponding object identifiers in the plurality of sample images. marking the same object and different objects in .

In an embodiment of the present invention, the configuration of the initial neural network model may include a backbone network, a site detection branch and a feature extraction branch, also shown in FIG. If the input values include multiple sample scene images, a sample feature vector corresponding to each of multiple target site locations in each sample scene image may be obtained.

In step 100-2, based on the object identifiers corresponding to each of the plurality of marked target portions in each of the sample scene images, two adjacent sample scene images of the target portions having the same object identifier are selected. determining a first similarity between the sample feature vectors corresponding to locations and/or determining a second similarity between the sample feature vectors corresponding to locations of the target sites of different object identifiers. .

In an embodiment of the present invention, based on the sample feature vectors corresponding to each of the positions of a plurality of target regions in each sample scene image output from the initial neural network model, the same feature vectors in adjacent two sample scene images are obtained. a first degree of similarity between the sample feature vectors corresponding to the target site locations of object identifiers and/or corresponding to different target site locations of the object identifiers in the adjacent two sample scene images; A second degree of similarity between the sample feature vectors may be determined.

The first similarity value and the second similarity value may be obtained based on cosine similarity values between sample feature vectors.

In step 100-3, based on at least one of the first similarity and the second similarity based on object identifiers corresponding to each of the plurality of marked target parts in each of the sample scene images , perform supervised training on the initial neural network model to obtain the feature detection model.

In an embodiment of the present invention, the loss function may be specified by increasing the first similarity value and decreasing the second similarity value, as shown in FIG. Based on the object identifiers corresponding to each of the plurality of target parts in the adjacent two sample scene images, the network parameters of the predetermined model are adjusted by the specified loss function, and after completing supervised training, the feature Get the detection model.

In the above embodiment, supervised training is performed on an initial neural network model to obtain the feature detection model based on object identifiers corresponding to each of the plurality of marked target regions in each of the sample scene images. improves the detection performance and generalization performance of the feature detection model.

In some alternative embodiments, for step 100-3, the difference between the first similarity reference value and said first similarity may be the first loss function. The first similarity reference value is a similarity reference value between sample feature vectors corresponding to target portions of the same marked object identifier in the two sample scene images. Illustratively, the first similarity reference value is a cosine similarity value between the sample feature vectors, which may have a value of one.

A feature detection model is obtained by adjusting the network parameters of the initial neural network model to minimize the first loss function or reach a predetermined number of training times.

Alternatively, the difference between the second similarity reference value and the second similarity may be used as the second loss function. The second similarity reference value is a similarity reference value between sample feature vectors corresponding to target portions of different marked object identifiers in the two sample scene images. Illustratively, the second similarity reference value is a cosine similarity value between sample feature vectors, which may be zero.

Similarly, the feature detection model is obtained by adjusting the network parameters of the initial neural network model to minimize the second loss function or reach a predetermined number of training times.

Or, both the first loss function and the second loss function are the loss functions of the initial neural network model, and the network parameters of the initial neural network model are adjusted to minimize the two loss functions or reach a predetermined number of training times. Thus, a feature detection model may be obtained.

In some alternative embodiments, as shown in FIG. 10, the method may further include the following steps.

In step 105, it is determined whether or not the motion trajectory of at least one of the plurality of same objects appearing in the plurality of scene images within a predetermined time period matches the desired motion trajectory.

In an embodiment of the present invention, the plurality of scene images correspond to a classroom scene, the object includes a teaching target, and the desired motion trajectory is at least one type of motion specified for the teaching target in a teaching task. Includes trajectories. At least one motion trajectory specified for the teaching target in the teaching task includes, but is not limited to, walking from the current location to another location specified by the teacher. The other position may be the podium, the blackboard, or the position where other classmates are located, or the desired motion trajectory may further include that no movement occurs at the current position.

For example, in a classroom, a teaching multimedia device (including, but not limited to, a teaching projector, a classroom monitoring device, etc.) having a camera head placed in the classroom can be used to capture multiple scene images in the classroom. may be collected before or after. At least one motion trajectory of the teaching object included in the classroom scene image is identified. The teaching subject may be a student.

Furthermore, within a set time period, for example, within a time period during which the teacher teaches, the motion trajectory of each teaching object, such as each student, matches at least one type of motion trajectory specified for the teaching object in the teaching task. You may specify whether For example, whether or not you moved from your current position in front of the blackboard or to the position where other classmates are located according to the teacher's instructions, or whether you have been in the same position all the time without movement of the movement trajectory, such as your own Identify whether or not the person is sitting in a position and taking a class. The results may be displayed via a teaching multimedia device to help the teacher perform teaching tasks better.

Corresponding to the above method embodiments, the present invention further provides apparatus embodiments.

Referring to FIG. 11, FIG. 11 is an object tracking device block diagram illustrating one exemplary embodiment of the present invention. The apparatus includes an acquisition module 210 for acquiring a plurality of scene images corresponding to the same scene, performing feature extraction processing and target site detection on each scene image among the plurality of scene images, and a processing module 220 for acquiring feature information of a scene image and positions of a plurality of target regions in each of the scene images; A plurality of identical objects appearing in the plurality of scene images based on a feature information identifying module 230 for acquiring target feature information and target feature information corresponding to each of the acquired positions of the plurality of target parts. and an object identification module 240 for identifying , wherein each scene image includes some or all of said plurality of same objects.

In some alternative embodiments, the processing module comprises a first processing sub-module for extracting a first feature map for each scene image of the plurality of scene images; Target site detection is performed in one feature map to obtain positions of a plurality of target sites in each scene image, Feature extraction processing is performed on the first feature map of each scene image, and a multidimensional second feature map is obtained. a second processing sub-module for obtaining a feature map, wherein the feature information identifying module obtains a plurality of target feature vectors corresponding to the positions of the plurality of target sites in the second multidimensional feature map. a feature vector identification sub-module for

In some optional embodiments, the object identification module utilizes a plurality of target feature information respectively corresponding to two adjacent scene images of the plurality of scene images, a similarity determination sub-module for obtaining a similarity between each target part in each scene image; an object identification sub-module for identifying multiple identical objects appearing in the scene image.

In some optional embodiments, the adjacent two scene images are a first scene image and a second scene image, and the similarity determination sub-module determines N targets in the first scene image. identifying a similarity between each of the feature vectors and M target feature vectors in a second scene image; determining a similarity between each of the N target feature vectors in the first scene image and the M target feature vectors in the second scene image and obtaining an N×M dimensional similarity matrix based on the similarity with the target feature vector, where N and M are positive integers of 2 or more, and any The dimension value represents the degree of similarity between any first target portion in the first scene image and any second target portion in the second scene image.

In some optional embodiments, the object identification sub-module determines each of the first target feature vectors of the N target feature vectors and the M target feature vectors based on the similarity matrix. and if the maximum similarity value is greater than a predetermined threshold, a second target corresponding to the maximum similarity value among the M target feature vectors identifying a feature vector; an object to which a first target portion corresponding to the first target feature vector in the first scene image belongs; and a second target portion corresponding to the second target feature vector in the second scene image; to be the same object as the object to which .

In some alternative embodiments, the processing module comprises a third processing sub for extracting a first feature map for each scene image of the plurality of scene images via a backbone network of feature detection models. and, via a feature detection branch of the feature detection model, perform target feature detection in a first feature map of each scene image to obtain locations of a plurality of target features in each scene image; a fourth processing sub-module for performing feature extraction processing on the first feature map of each scene image via a feature extraction branch of the detection model to obtain a multi-dimensional second feature map.

In some alternative embodiments, the apparatus inputs a plurality of sample scene images corresponding to the same scene into a given model, and calculates the locations of a plurality of target regions in each sample scene image output from the given model. and a feature vector identification module for obtaining a plurality of feature vectors corresponding to the two adjacent sample scene images based on object identifiers corresponding to each of the plurality of marked target sites in each of the two adjacent sample scene images. determining a first degree of similarity between sample feature vectors corresponding to locations of the target portion of the same object identifier in each of the sample scene images; a similarity determination module for determining a second similarity between sample feature vectors corresponding to target site locations of object identifiers; Supervised training is performed on the predetermined model based on at least one of the second similarity and the first similarity based on the corresponding object identifiers to obtain the feature detection model. and a training module for.

In some embodiments, the difference between a first similarity reference value and the first similarity measure is a first loss function, and the difference between a second similarity reference value and the second similarity measure is a second loss function. and training the initial neural network model to obtain the feature detection model based on at least one of the first loss function and the second loss function; The first similarity reference value is a similarity reference value between sample feature vectors corresponding to target portions of the same marked object identifier in the adjacent two sample scene images, and the second similarity The reference value is a similarity reference value between the sample feature vectors corresponding to the marked target portions of different object identifiers in the adjacent two sample scene images.

In some optional embodiments, the apparatus determines whether a motion trajectory of at least one of the plurality of same objects appearing in the plurality of scene images within a predetermined time period matches a desired motion trajectory. It further comprises a motion trajectory identification module for identifying whether or not.

In some optional embodiments, the plurality of scene images correspond to a classroom scene, the object includes a teaching subject, and the desired motion trajectory is at least specified to the teaching subject in a teaching task. Contains one type of motion trajectory.

Since the device embodiment basically corresponds to the method embodiment, the relevant part can be referred to the description of the method embodiment. The apparatus embodiments described above are merely exemplary, in which the means described as separate parts may or may not be physically separated. Also, a part displayed as a means may or may not be a physical means. Moreover, these means may be located at one location or distributed over multiple network cells. Some or all of the modules can be selected according to actual needs to achieve the purpose of this embodiment. A person of ordinary skill in the art can understand and implement it without the effort worth the inventive step.

Embodiments of the invention further provide a computer-readable storage medium. A storage medium stores a computer program, and the computer program is used to execute the object tracking method according to any one of the above items.

In some alternative embodiments, embodiments of the invention provide computer program products. The computer program product includes computer readable code, and when the computer readable code is run on the device, a processor in the device executes instructions for performing the object tracking method according to any one of the embodiments described above. do.

In some alternative embodiments, embodiments of the invention further provide another computer program product. The computer program product stores computer readable instructions, and when the instructions are executed, the computer performs the operations of the object tracking method according to any one of the embodiments above.

Said computer program product may be specifically implemented in hardware, software or a combination thereof. In one preferred embodiment, the computer program product may be embodied as a computer storage medium, and in another preferred embodiment, the computer program product is a software product, such as a Software Development Kit (SDK). ), etc.

In some alternative embodiments, embodiments of the invention provide computer programs. When the computer program is executed, the computer performs the operations of the object tracking method according to any one of the embodiments above.

Embodiments of the present invention further provide an object tracking device. The object tracking device comprises a processor and a memory for storing executable instructions executable by the processor, the processor performing any one of the above by invoking the executable instructions stored in the memory. It is configured to implement an object tracking method.

FIG. 12 is a hardware structural schematic diagram of an object tracking device according to an embodiment of the present invention. The object tracking device 310 comprises a processor 311 and may further comprise an input device 312 , an output device 313 and a memory 314 . The input device 312, output device 313, memory 314 and processor 311 are connected to each other via a bus.

The memory may be random access memory (RAM), read-only memory (ROM), erasable programmable read only memory (EPROM), or portable read-only memory ( compact disc read-only memory, CD-ROM). The memory stores relevant instructions and data.

The input device inputs data and/or signals, and the output device outputs data and/or signals. The output device and the input device may be independent devices or may be one whole device.

A processor may be one or more processors, and may include, for example, one or more central processing units (CPUs). When the processor is one CPU, the CPU may be a single-core CPU or a multi-core CPU.

The memory stores program codes and data for the network appliance.

The processor calls up the program code and data in the memory to perform the steps in the above method embodiments. For details, please refer to the description in the method embodiment, which will not be repeated here.

As can be appreciated, FIG. 12 merely shows a simplified design of one type of object tracker. In practical applications, the object tracking device may include other elements as required, including, but not limited to, any number of input/output devices, processors, controllers, memories, and the like. All object tracking devices that can implement the embodiments of the present invention are within the protection scope of the present invention.

Other embodiments of the invention may readily occur to those skilled in the art after considering the specification and practicing the invention disclosed herein. The present invention is intended to cover any variations, uses or adaptations of the present invention. These modifications, uses, or adaptive changes follow the general principles of the present invention and include common knowledge or common technical means in the art that are not disclosed in the present invention. It is intended that the specification and examples be considered as exemplary only, with the true scope and spirit of the invention being given by the following claims.

The above are only preferred embodiments of the present invention and are not intended to limit the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall fall within the protection scope of the present invention.

This application claims the priority of the Chinese patent application with application number 202010352365.6 and entitled "Object Tracking Method and Apparatus, Storage Medium" filed on April 28, 2020, and is incorporated herein by reference in its entirety.

Claims (13)

An object tracking method comprising:
acquiring a plurality of scene images corresponding to the same scene;
performing feature extraction processing and target site detection on each scene image of the plurality of scene images to obtain feature information of each scene image and positions of a plurality of target sites in each scene image; ,
a step of obtaining target feature information corresponding to each of the positions of the plurality of target parts among the feature information of each of the scene images;
identifying a plurality of identical objects appearing in the plurality of scene images based on the acquired target feature information corresponding to each of the plurality of target site positions;
each scene image includes some or all of the plurality of same objects;
performing feature extraction processing and target site detection on each scene image out of the plurality of scene images, and acquiring feature information of each scene image and positions of a plurality of target sites in each scene image; ,
extracting a first feature map for each scene image of the plurality of scene images;
performing target site detection on the first feature map of each scene image, obtaining positions of a plurality of target sites on each scene image, and performing feature extraction processing on the first feature map of each scene image; , obtaining a multi-dimensional second feature map;
The step of acquiring target feature information corresponding to each of the positions of the plurality of target parts among the feature information of each scene image,
An object tracking method , comprising obtaining a target feature vector corresponding to each of the positions of the plurality of target regions in the multidimensional second feature map. The step of identifying a plurality of identical objects appearing in the plurality of scene images based on the target feature information corresponding to each of the acquired positions of the plurality of target parts,
Using a plurality of pieces of target feature information respectively corresponding to two adjacent scene images out of the plurality of scene images, obtaining a degree of similarity between each target part in each of the adjacent two scene images. and
and identifying a plurality of identical objects appearing in different scene images based on the similarity between each target portion in the adjacent two scene images. object tracking method. The two adjacent scene images are a first scene image and a second scene image,
Using a plurality of pieces of target feature information respectively corresponding to two adjacent scene images out of the plurality of scene images, obtaining a degree of similarity between each target part in each of the adjacent two scene images. to do
determining the similarity between each of the N target feature vectors in the first scene image and the M target feature vectors in the second scene image;
obtaining an N×M dimensional similarity matrix based on the similarity between each of N target feature vectors in the first scene image and M target feature vectors in the second scene image; including
N and M are positive integers of 2 or more, and the value of any dimension in the similarity matrix is any first target site in the first scene image and any one in the second scene image. 3. The object tracking method according to claim 2 , wherein the degree of similarity with the second target portion is expressed. Identifying a plurality of identical objects appearing in the different scene images based on the similarity between each target part in the two adjacent scene images;
identifying a maximum similarity value from similarities between each of the first target feature vectors of the N target feature vectors and the M target feature vectors based on the similarity matrix;
identifying a second target feature vector corresponding to the maximum similarity value among the M target feature vectors when the maximum similarity value is greater than a predetermined threshold;
The object to which the first target part corresponding to the first target feature vector in the first scene image belongs and the object to which the second target part corresponding to the second target feature vector in the second scene image belong are made the same object. 4. The object tracking method of claim 3 , comprising: performing feature extraction processing and target site detection on each scene image out of the plurality of scene images, and acquiring feature information of each scene image and positions of a plurality of target sites in each scene image; ,
extracting a first feature map for each scene image of the plurality of scene images via a backbone network of feature detection models;
performing target site detection in a first feature map of each scene image via a site detection branch of the feature detection model to obtain locations of a plurality of target sites in each scene image; performing a feature extraction process on the first feature map of each scene image via a feature extraction branch to obtain a multi-dimensional second feature map . The object tracking method according to any one of 1. A plurality of sample scene images corresponding to the same scene are input to an initial neural network model, and sample feature vectors corresponding to the positions of a plurality of target parts in each sample scene image output from the initial neural network model are obtained. and
Based on the object identifiers corresponding to each of the plurality of marked target parts in each of the sample scene images, the samples corresponding to the positions of the target parts having the same object identifier in two adjacent sample scene images. determining a first degree of similarity between feature vectors and/or a second degree of similarity between the sample feature vectors corresponding to locations of the target sites with different object identifiers;
The initial neural network model based on at least one of the first similarity and the second similarity based on object identifiers corresponding to each of the plurality of marked target parts in each of the sample scene images. 6. The object tracking method of claim 5 , further comprising performing supervised training on to obtain the feature detection model. The initial neural network model based on at least one of the first similarity and the second similarity based on object identifiers corresponding to each of the plurality of marked target parts in each of the sample scene images. supervised training of to obtain the feature detection model,
setting a difference between a first similarity reference value and the first similarity as a first loss function;
setting a difference between a second similarity reference value and the second similarity as a second loss function;
training the initial neural network model to obtain the feature detection model based on at least one of the first loss function and the second loss function;
The first similarity reference value is a similarity reference value between sample feature vectors corresponding to target portions of the same marked object identifier in the adjacent two sample scene images, and the second similarity 7. The reference value of claim 6 , wherein the reference value is a similarity reference value between sample feature vectors corresponding to marked target portions of different object identifiers in the adjacent two sample scene images. Object tracking method. The method further includes the step of determining whether or not a motion trajectory of at least one of the plurality of same objects appearing in the plurality of scene images within a predetermined time period matches a desired motion trajectory. Object tracking method according to any one of claims 1 to 7 . The plurality of scene images correspond to a classroom scene, the object includes a teaching subject, and the desired motion trajectory includes at least one motion trajectory specified for the teaching subject in a teaching task. The object tracking method according to claim 8 , wherein An object tracking device,
an acquisition module for acquiring multiple scene images corresponding to the same scene;
performing feature extraction processing and target site detection on each scene image out of the plurality of scene images, and acquiring feature information of each scene image and positions of a plurality of target sites in each scene image; a processing module;
a feature information specifying module for acquiring target feature information corresponding to each of the positions of the plurality of target parts among the feature information of each scene image;
an object identification module for identifying a plurality of identical objects appearing in the plurality of scene images based on target feature information corresponding to each of the acquired positions of the plurality of target sites;
each scene image includes some or all of the plurality of same objects;
The processing module is
a first processing sub-module for extracting a first feature map of each scene image of the plurality of scene images;
performing target site detection on the first feature map of each scene image, obtaining positions of a plurality of target sites on each scene image, and performing feature extraction processing on the first feature map of each scene image; , a second processing sub-module for obtaining a multi-dimensional second feature map;
The feature information identification module comprises a feature vector identification sub-module for obtaining a plurality of target feature vectors corresponding to the positions of the plurality of target parts in the second multidimensional feature map. Device. A computer-readable storage medium,
A computer readable storage, characterized in that a computer program is stored in the computer readable storage medium, and the computer program is used to execute the object tracking method according to any one of claims 1 to 9 . medium. An object tracking device,
a processor;
a memory for storing executable instructions that can be executed by the processor;
10. Object tracking device, characterized in that the processor is arranged to implement the object tracking method according to any one of claims 1 to 9 by invoking executable instructions stored in the memory. A computer program,
A computer program, characterized in that, when said computer program is executed by a processor, an object tracking method according to any one of claims 1 to 9 is implemented.

Images