DE112020003171T5 - Method and device for linking targets - Google Patents

Abstract

A method and apparatus for linking goals. The method includes: obtaining a box currently to be marked and a first video frame corresponding to the current box to be marked (S101); obtaining recommendation boxes with presumed link relationship to the box currently to be marked and a second video frame corresponding to each recommendation box (S 102); displaying the first video frame containing the box to be currently marked and the second video frame containing the recommendation box corresponding to each recommendation box (S 103); Determining, based on user's first linking relationship operation information, from all of the recommendation boxes, whether there is a target recommendation box in linking relationship with the box to be currently marked, and producing, in the presence of a target recommendation box in linking relationship with the box to be currently marked, a linking relationship between the box to be currently marked and the target recommendation box (S 104) to realize convenient and effective linking of the same target in each video frame.

Description

TECHNICAL AREA

The present invention relates to the technical field of target tracking and in particular to a method and an apparatus for linking targets.

STATE OF THE ART

Currently, a tracking algorithm based on target detection can automatically detect a target contained in each video frame of a video to be detected, and associate an object contained in the previous video frame related to the same target with the target. During a process of tracking and detecting individual targets in the video to be detected by the target detection-based tracking algorithm, situations of missing detection or false detection often occur. For example, the following situations may occur: for a target object that disappears and reappears in a detecting video, the target detection-based tracking algorithm recognizes the target object that disappears and reappears as a different object from the target object, i.e. that the target object identified as different before it disappears and after it reappears; or the same target appearing consecutively is identified as different targets. In view of the above situations, it is necessary to manually check, calibrate, and judge the detection results obtained by the tracking algorithm based on the target detection. Then, it becomes a problem to be urgently solved to provide a convenient and effective method for checking, calibrating, and judging and correcting missing recognition or wrong recognition in recognition results of a video to be captured.

CONTENT OF THE PRESENT INVENTION

• Gemäß einem Aspekt stellen die Ausführungsbeispiele der vorliegenden Erfindung ein Verfahren zum Verknüpfen von Zielen bereit, wobei das Verfahren umfasst:
  • Erhalten einer aktuell zu markierenden Box und eines der aktuell zu markierenden Box entsprechenden ersten Videorahmens, wobei der der aktuell zu markierenden Box entsprechende erste Videorahmen ein erkannter Videorahmen, der ein der aktuell zu markierenden Box entsprechendes Ziel enthält und der eine erste Filterbedingung erfüllt, in einem zu markierenden Video ist;
  • Erhalten von Empfehlungsboxen mit vermuteter Verknüpfungsbeziehung zu der aktuell zu markierenden Box und eines jeder Empfehlungsbox entsprechenden zweiten Videorahmens, wobei der jeder Empfehlungsbox entsprechende zweite Videorahmen ein erkannter Videorahmen, der ein der Empfehlungsbox entsprechendes Ziel enthält und der eine zweite Filterbedingung erfüllt, in dem zu markierenden Video ist;
  • Anzeigen des ersten Videorahmens, der die aktuell zu markierenden Box enthält, und des jeder Empfehlungsbox entsprechenden zweiten Videorahmens, der die Empfehlungsbox enthält, so dass ein Benutzer basierend auf dem jeder Empfehlungsbox entsprechenden zweiten Videorahmen eine Ziel-Empfehlungsbox, die in Verknüpfungsbeziehung mit der aktuell zu markierenden Box steht, aus den angezeigten Empfehlungsboxen bestimmt; Bestimmen, basierend auf einer ersten Verknüpfungsbeziehung-Operationsinformation des Benutzers, aus allen Empfehlungsboxen, ob eine Ziel-Empfehlungsbox, die in Verknüpfungsbeziehung mit der aktuell zu markierenden Box steht, vorliegt, und Herstellen, beim Vorliegen einer Ziel-Empfehlungsbox, die in Verknüpfungsbeziehung mit der aktuell zu markierenden Box steht, einer Verknüpfungsbeziehung zwischen der aktuell zu markierenden Box und der Ziel-Empfehlungsbox.
  • Optional ist vorgesehen, dass der erkannte Videorahmen, der das der aktuell zu markierenden Box entsprechende Ziel enthält und der eine erste Filterbedingung erfüllt, in einem zu markierenden Video der erkannte letzte Videorahmen, der das der aktuell zu markierenden Box entsprechende Ziel enthält, im zu markierenden Video ist; wobei der erkannte Videorahmen, der ein der Empfehlungsbox entsprechendes Ziel enthält und die zweite Filterbedingung erfüllt, in einem zu markierenden Video ein erkannter erster Videorahmen, der das der Empfehlungsbox entsprechende Ziel enthält, im zu markierenden Video ist.
  • Optional ist vorgesehen, dass der Schritt des Erhaltens der aktuell zu markierenden Box im zu markierenden Video durch eine der folgenden zwei Implementierungsformen realisiert wird:
    • erste Implementierungsform:
      • aus unmarkierten Boxen, die dem zu markierenden Video entsprechen, wird eine unmarkierte Box mit der frühesten Erfassungszeit, die durch eine einem End-Videorahmen entsprechende Zeitstempelinformation dargestellt wird, als die aktuell zu markierende Box bestimmt;
      • zweite Implementierungsform:
    • wenn erkannt wird, dass für angezeigten historisch zu markierende Boxen der Benutzer eine mit denen verknüpfte Empfehlungsbox aus den entsprechenden angezeigten Empfehlungsboxen auswählt, wird die durch den Benutzer ausgewählte Empfehlungsbox als die aktuell zu markierende Box bestimmt.
  • Optional ist vorgesehen, dass der Schritt „Erhalten von Empfehlungsboxen mit vermuteter Verknüpfungsbeziehung zu der aktuell zu markierenden Box und eines jeder Empfehlungsbox entsprechenden zweiten Videorahmens“ umfasst:
    • Durchlaufen durch vormarkierte Ergebnisse, die N Videorahmen nach dem der aktuell zu markierenden Box entsprechenden End-Videorahmen entsprechen, Bestimmen einer Erkennungsbox, deren Boxkennung eine vorgegebene Empfehlungsbedingung erfüllt, aus Erkennungsboxen, die den N Videorahmen nach dem der aktuell zu markierenden Box entsprechenden End-Videorahmen entsprechen, als eine Erkennungsbox mit vermuteter Verknüpfungsbeziehung zu der aktuell zu markierenden Box, wobei die Erkennungsbox,
    • deren Boxkennung eine vorgegebene Empfehlungsbedingung erfüllt, eine Erkennungsbox umfasst, deren Boxkennungsinformation nicht in dem der aktuell zu markierenden Box entsprechenden End-Videorahmen sowie vorherigen Videorahmen erscheint;
  • Bestimmen, für jede Empfehlungsbox mit vermuteter Verknüpfungsbeziehung zu der aktuell zu markierenden Box, eines Videorahmens, der das der Empfehlungsbox entsprechende Ziel enthält und die zweite Filterbedienung erfüllt, aus den N Videorahmen nach dem der aktuell zu markierenden Box entsprechenden End-Videorahmen als der der Empfehlungsbox entsprechende zweite Videorahmen, wobei der Videorahmen, der das der Empfehlungsbox entsprechende Ziel enthält und die zweite Filterbedienung erfüllt, umfasst: den erkannten ersten Videorahmen, der das der Empfehlungsbox entsprechende Ziel enthält, in dem zu markierenden Video.
  • Optional ist vorgesehen, dass das Verfahren vor dem Schritt „Erhalten von Empfehlungsboxen mit vermuteter Verknüpfungsbeziehung zu der aktuell zu markierenden Box und eines jeder Empfehlungsbox entsprechenden zweiten Videorahmens“ ferner umfasst:
    • Bestimmen, ob eine Anzeigefunktion zum Anzeigen der Empfehlungsbox, die der aktuell zu markierenden Box entspricht, aktiviert ist;
    • wenn aktiviert, wird der Schritt „Erhalten von Empfehlungsboxen mit vermuteter Verknüpfungsbeziehung zu der aktuell zu markierenden Box und eines jeder Empfehlungsbox entsprechenden zweiten Videorahmens“ durchgeführt;
    • wenn nicht aktiviert, werden M Videorahmen nach dem End-Videorahmen, das der aktuell zu markierenden Box entspricht, in dem zu markierenden Video als abzuspielende Videorahmen erhalten, die der aktuell zu markierenden Box entsprechen, wobei der End-Videorahmen, das der aktuell zu markierenden Box entspricht, ist: ein erkannter End-Videorahmen, der das der aktuell zu markierenden Box entsprechende Ziel enthält, in dem markierenden Video;

The present invention provides a method and apparatus for linking goals to realize convenient and effective linking of the same goal appearing in each video frame of the video. The specific technical configurations are as follows:

• According to one aspect, the embodiments of the present invention provide a method for linking goals, the method comprising:
  • obtaining a box currently to be marked and a first video frame corresponding to the box currently to be marked, wherein the first video frame corresponding to the box currently to be marked is a recognized video frame containing a target corresponding to the box currently to be marked and which satisfies a first filter condition, in one video to be tagged is;
  • Obtaining recommendation boxes with presumed linking relationship to the box currently to be marked and a second video frame corresponding to each recommendation box, wherein the second video frame corresponding to each recommendation box is a recognized video frame containing a target corresponding to the recommendation box and satisfying a second filter condition in the video to be marked is;
  • Displaying the first video frame containing the box currently to be marked and the second video frame corresponding to each recommendation box containing the recommendation box, so that a user based on the second video frame corresponding to each recommendation box selects a target recommendation box that is in linking relationship with the current to marked box is determined from the displayed recommendation boxes; determining, based on user's first linking relationship operation information, from all of the recommendation boxes, whether there is a target recommendation box in linking relationship with the box to be currently marked, and producing, when there is a target recommendation box in linking relationship with the box currently to be marked, a linking relationship between the box currently to be marked and the target recommendation box.
  • It is optionally provided that the recognized video frame, which contains the target corresponding to the box currently to be marked and which meets a first filter condition, in a video to be marked, the recognized last video frame, which contains the target corresponding to the box currently to be marked, in the to be marked video is; wherein the detected video frame, which contains a target corresponding to the recommendation box and satisfies the second filter condition, in a video to be marked is a detected first video frame, which contains the target corresponding to the recommendation box, in the video to be marked.
  • Optionally, it is provided that the step of obtaining the box currently to be marked in the video to be marked is realized by one of the following two forms of implementation:
    • first implementation form:
      • from unmarked boxes corresponding to video to be marked, determining an unmarked box having the earliest detection time represented by timestamp information corresponding to an end video frame as the current box to be marked;
      • second implementation form:
    • if it is detected that for historically displayed boxes to be marked, the user selects a recommendation box associated with them from among the corresponding displayed recommendation boxes, the recommendation box selected by the user is determined as the current box to be marked.
  • Optionally, it is provided that the step "obtaining recommendation boxes with presumed linkage relationship to the box currently to be marked and a second video frame corresponding to each recommendation box" comprises:
    • iterating through pre-marked results corresponding to N video frames after the ending video frame corresponding to the box currently to be marked, determining a recognition box whose box identifier satisfies a predetermined recommendation condition from recognition boxes corresponding to the N video frames after the ending video frame corresponding to the box currently to be marked correspond, as a recognition box with presumed linkage relationship to the box currently to be marked, wherein the recognition box,
    • whose box identifier satisfies a predetermined recommendation condition, comprises a recognition box whose box identifier information does not appear in the ending video frame corresponding to the current box to be marked and previous video frames;
  • Determine, for each recommendation box with presumed linking relation to the current box to be marked, a video frame containing the target corresponding to the recommendation box and satisfying the second filter operation, from the N video frames after the end video frame corresponding to the current box to be marked as that of the recommendation box corresponding second video frame, wherein the video frame containing the target corresponding to the recommendation box and satisfying the second filter operation comprises: the detected first video frame containing the target corresponding to the recommendation box in the video to be tagged.
  • Optionally, it is provided that the method before the step "obtaining recommendation boxes with a presumed linkage to the box currently to be marked and a second video frame corresponding to each recommendation box" further comprises:
    • determining whether a display function for displaying the recommendation box corresponding to the box currently to be marked is activated;
    • if activated, the step "obtaining recommendation boxes with presumed link relationship to the current box to be marked and a second video frame corresponding to each recommendation box" is performed;
    • if not checked, M video frames are obtained as video frames to play corresponding to the box currently to be marked after the end video frame corresponding to the box currently to be marked in the video to be marked, where the end video frame corresponding to the box currently to be marked box corresponds to is: a detected ending video frame containing the target corresponding to the current box to be marked in the marking video;

Displaying the first video frame containing the box currently to be marked and playing the video frames to be played that correspond to the box currently to be marked, so that the user selects a target recognition box, which is linked to the box currently to be marked, from the recognition boxes contained by the video frames to be played corresponding to the current box to be marked;

Determining, based on a user's second linking relationship operation information, from the recognition boxes contained by the video frames to be played corresponding to the box currently to be marked, whether there is a target recognition box that is in linking relationship with the box currently to be marked, wherein in case the presence of a target recognition box which is in linking relationship with the box to be currently marked is determined, a linking relationship is established between the box to be currently marked and the target recognition box.

• Vergrößertes Anzeigen einer ersten Empfehlungsbox und eines der ersten Empfehlungsbox entsprechenden zweiten Videorahmens, wenn ein erster Befehl zum Vergrößern, der durch den Benutzer für die erste Empfehlungsbox mit vermuteter Verknüpfungsbeziehung zu der aktuell zu markierenden Box ausgelöst wird, erkannt, wobei Auslöseoperationen des ersten Befehls zum Vergrößern umfassen: Rechtsklicken auf den zweiten Videorahmen, der der ersten Empfehlungsbox entspricht; und/oder
• Vergrößertes Anzeigen eines ersten Videorahmens, der der aktuell zu markierenden Box entspricht, wenn ein zweiter Befehl zum Vergrößern, der durch den Benutzer für den ersten Videorahmen, der der aktuell zu markierenden Box entspricht, ausgelöst wird, erkannt wird.
• Optional ist vorgesehen, dass das Verfahren nach dem Schritt „Bestimmen, basierend auf einer ersten Verknüpfungsbeziehung-Operationsinformation des Benutzers, aus allen Empfehlungsboxen, ob eine Ziel-Empfehlungsbox, die in Verknüpfungsbeziehung mit der aktuell zu markierenden Box steht, vorliegt, und Herstellen, beim Vorliegen einer Ziel-Empfehlungsbox, die in Verknüpfungsbeziehung mit der aktuell zu markierenden Box steht, einer Verknüpfungsbeziehung zwischen der aktuell zu markierenden Box und der Ziel-Empfehlungsbox“ umfasst:
  • Bestimmen der Ziel-Empfehlungsbox als eine neue aktuell zu markierende Box; Bestimmen eines Videorahmen, der ein der neue aktuell zu markierenden Box entsprechendes Ziel enthält und der die erste Filterbedingung erfüllt, aus dem zu markierenden Video als ein erster Videorahmen, der der neuen aktuell zu markierenden Box entspricht;

Optionally, it is provided that the method before the step "determine, based on a first linking relationship operation information of the user, from all recommendation boxes, whether there is a target recommendation box that has a linking relationship with the box currently to be marked, and establish, when Presence of a target recommendation box which has a linking relationship with the box currently to be marked, a linking relationship between the box currently to be marked and the target recommendation box” includes:

• Magnified display of a first recommendation box and a second video frame corresponding to the first recommendation box when a first magnification command triggered by the user for the first recommendation box with presumed linkage relationship to the box currently to be marked is detected, wherein trigger operations of the first magnification command include: right-clicking the second video frame corresponding to the first recommendation box; and or
• Magnifying display of a first video frame corresponding to the box currently to be marked when a second magnify command initiated by the user for the first video frame corresponding to the box currently to be marked is detected.
• Optionally, it is provided that the method after the step "determine, based on a first linking relationship operation information of the user, from all recommendation boxes, whether there is a target recommendation box that has a linking relationship with the box currently to be marked, and produce, when Presence of a target recommendation box which has a linking relationship with the box currently to be marked, a linking relationship between the box currently to be marked and the target recommendation box” includes:
  • determining the target recommendation box as a new box to be currently marked; determining a video frame containing a target corresponding to the new box currently to be marked and satisfying the first filter condition from the video to be marked as a first video frame corresponding to the new box currently to be marked;

Obtaining new recommendation boxes with a presumed linkage relationship to the new box currently to be marked and a second video frame corresponding to each new recommendation box, the second video frame corresponding to each new recommendation box being a recognized video frame which contains a target corresponding to the new recommendation box and which satisfies a second filter condition, is in the video to be tagged;

Displaying the first video frame containing the new box currently to be marked and the second video frame corresponding to each new recommendation box containing the new recommendation box, so that based on the second video frame corresponding to each new recommendation box the user selects a new target recommendation box to be Linking relationship with the new box currently to be marked is determined from the displayed new recommendation boxes;

Displaying the first video frame containing the box currently to be marked and the second video frame corresponding to each recommendation box and containing that recommendation box when it is detected that the user triggers a jump command specifying a jump to the previous paragraph.

• Erhalten einer Rahmennummer-Information des ersten Videorahmens, die der aktuell zu markierenden Box entspricht, und einer Boxkennungsinformation, die der aktuell zu markierenden Box entspricht;

Optionally, it is provided that before the step "displaying the first video frame containing the box currently to be marked and the second video frame corresponding to each recommendation box containing the recommendation box", the method further comprises:

• obtaining frame number information of the first video frame corresponding to the box currently to be marked and box ID information corresponding to the box currently to be marked;

• Anzeigen des ersten Videorahmens, der die aktuell zu markierenden Box enthält, der Rahmennummer-Information des ersten Videorahmens, die der aktuell zu markierenden Box entspricht und der Boxkennungsinformation, die der aktuell zu markierenden Box entspricht, in einem ersten Anzeigebereich einer ersten voreingestellten Anzeigeschnittstelle; und Anzeigen des zweiten Videorahmens, der jeder Empfehlungsbox mit vermuteter Verknüpfungsbeziehung zu der aktuell zu markierenden Box entspricht, der Rahmennummer-Information des zweiten Videorahmens und der Boxkennungsinformation, die jeder Empfehlungsbox entspricht, in einem zweiten Anzeigebereich der ersten voreingestellten Anzeigeschnittstelle.

obtaining frame number information of the second video frame corresponding to each recommendation box assumed to be linked to the box currently to be marked and box identifier information corresponding to each recommendation box; wherein the step of "displaying the first video frame containing the box currently to be marked and the second video frame corresponding to each recommendation box containing the recommendation box" comprises:

• displaying the first video frame including the box currently to be marked, the frame number information of the first video frame corresponding to the box currently to be marked, and the box ID information corresponding to the box currently to be marked in a first display area of a first preset display interface; and displaying the second video frame corresponding to each recommendation box assumed to be linked to the box currently to be marked, the frame number information of the second video frame, and the box ID information corresponding to each recommendation box in a second display area of the first preset display interface.

Optionally, it is provided that a third display area of the first default display interface further comprises a first trigger area for a jump command to jump to the previous paragraph and a second trigger area for a jump command to jump to the next paragraph, the jump command to jump to the previous paragraph being a command the displaying the first video frame, the last one too corresponding to the box to be marked before the box currently to be marked, wherein the first video frame corresponding to a last box to be marked before the current box to be marked is a detected video frame corresponding to a target corresponding to the last box to be marked before the current box to be marked and satisfies the first filter condition, is in the video to be marked; wherein the jump command to jump to the next paragraph is a command that directs the display of the first video frame corresponding to a next box to be marked after the current box to be marked, the next box to be marked after the current box to be marked being an unmarked box , whose time stamp information corresponding to the corresponding end-frame video frame represents the earliest detection time and is later than the detection time of the box currently to be marked, from the unmarked boxes corresponding to video to be marked.

• ein erstes Modul zum Erhalten, das konfiguriert ist, um eine aktuell zu markierende Box und einen der aktuell zu markierenden Box entsprechenden ersten Videorahmen zu erhalten, wobei der der aktuell zu markierenden Box entsprechende erste Videorahmen ein erkannter Videorahmen, der ein der aktuell zu markierenden Box entsprechendes Ziel enthält und der eine erste Filterbedingung erfüllt, in einem zu markierenden Video ist;
• ein zweites Modul zum Erhalten, das konfiguriert ist, um Empfehlungsboxen mit vermuteter Verknüpfungsbeziehung zu der aktuell zu markierenden Box und einen jeder Empfehlungsbox entsprechenden zweiten Videorahmen zu erhalten, wobei der jeder Empfehlungsbox entsprechende zweite Videorahmen ein erkannter Videorahmen, der ein der Empfehlungsbox entsprechendes Ziel enthält und der eine zweite Filterbedingung erfüllt, in dem zu markierenden Video ist;
• ein erstes Anzeigemodul, das konfiguriert ist, um den ersten Videorahmen, der die aktuell zu markierenden Box enthält, und den jeder Empfehlungsbox entsprechenden zweiten Videorahmen, der die Empfehlungsbox enthält, anzuzeigen, so dass ein Benutzer basierend auf dem jeder Empfehlungsbox entsprechenden zweiten Videorahmen eine Ziel-Empfehlungsbox, die in Verknüpfungsbeziehung mit der aktuell zu markierenden Box steht, aus den angezeigten Empfehlungsboxen bestimmt;
• ein erstes Bestimmungs- und Herstellungsmodul, das konfiguriert ist, um basierend auf einer ersten Verknüpfungsbeziehung-Operationsinformation des Benutzers aus allen Empfehlungsboxen zu bestimmen, ob eine Ziel-Empfehlungsbox, die in Verknüpfungsbeziehung mit der aktuell zu markierenden Box steht, vorliegt, und um beim Vorliegen einer Ziel-Empfehlungsbox, die in Verknüpfungsbeziehung mit der aktuell zu markierenden Box steht, eine Verknüpfungsbeziehung zwischen der aktuell zu markierenden Box und der Ziel-Empfehlungsbox herzustellen.

According to a second aspect, the embodiments of the present invention provide an apparatus for linking goals, the apparatus comprising:

• a first obtaining module configured to obtain a current box to be marked and a first video frame corresponding to the current box to be marked, wherein the first video frame corresponding to the current box to be marked is a recognized video frame corresponding to the current box to be marked contains a corresponding target and which satisfies a first filter condition, is in a video to be marked;
• a second obtaining module configured to obtain recommendation boxes with presumed linkage relationship to the box currently to be marked and a second video frame corresponding to each recommendation box, wherein the second video frame corresponding to each recommendation box is a detected video frame containing a target corresponding to the recommendation box and that satisfies a second filter condition is in the video to be tagged;
• a first display module configured to display the first video frame containing the box currently to be marked and the second video frame corresponding to each recommendation box containing the recommendation box so that a user based on the second video frame corresponding to each recommendation box provides a target -Recommendation box which is in link relation with the box currently to be marked determined from the displayed recommendation boxes;
• a first determining and producing module configured to determine whether there is a target recommendation box linked in linking relationship with the box currently to be marked, based on first linking relationship operation information of the user from all of the recommendation boxes, and if present a target recommendation box which has a linking relationship with the box to be currently marked to establish a linking relationship between the box to be currently marked and the target recommendation box.

It is optionally provided that the recognized video frame, which contains the target corresponding to the box currently to be marked and which meets a first filter condition, in a video to be marked, the recognized last video frame, which contains the target corresponding to the box currently to be marked, in the to be marked video is; wherein the detected video frame, which contains a target corresponding to the recommendation box and satisfies the second filter condition, in a video to be marked is a detected first video frame, which contains the target corresponding to the recommendation box, in the video to be marked.

Optionally, it is envisaged that the first obtaining module is specifically configured to select from unmarked boxes corresponding to the video to be marked an unmarked box with the earliest acquisition time represented by a timestamp information corresponding to an end video frame than the current to determine marking box;
or
to determine the recommendation box selected by the user as the current box to be marked when it is detected that for historically displayed boxes to be marked the user selects a recommendation box associated with those from among the corresponding displayed recommendation boxes.

Optionally, it is envisaged that the second module for obtaining is specifically configured to: iterate through pre-marked results corresponding to N video frames after the end video frame corresponding to the current box to be marked, determine from recognition boxes a recognition box whose box identifier satisfies a predetermined recommendation condition , which correspond to the N video frames after the end video frame corresponding to the box currently to be marked, as a recognition box with presumed linkage relationship to the box currently to be marked, wherein the recognition box whose box identifier satisfies a predetermined recommendation condition comprises a recognition box whose box identifier infor mation does not appear in the ending video frame and previous video frame corresponding to the box currently being marked;

Determine, for each recommendation box with presumed linking relation to the current box to be marked, a video frame containing the target corresponding to the recommendation box and satisfying the second filter operation, from the N video frames after the end video frame corresponding to the current box to be marked as that of the recommendation box corresponding second video frame, wherein the video frame containing the target corresponding to the recommendation box and satisfying the second filter operation comprises: the detected first video frame containing the target corresponding to the recommendation box in the video to be tagged.

• ein Beurteilungsmodul, das konfiguriert ist, um vor dem „Erhalten von Empfehlungsboxen mit vermuteter Verknüpfungsbeziehung zu der aktuell zu markierenden Box und eines jeder Empfehlungsbox entsprechenden zweiten Videorahmens“ zu beurteilen, ob eine Anzeigefunktion zum Anzeigen einer der aktuell zu markierenden Box entsprechenden Empfehlungsbox aktiviert ist,
• wobei bei einer aktivierten Anzeigefunktion der zweite Modul zum Erhalten ausgelöst wird;
• ein drittes Modul zum Erhalten, das konfiguriert ist, um bei inaktivierter Anzeigefunktion M Videorahmen nach dem End-Videorahmen, das der aktuell zu markierenden Box entspricht, in dem zu markierenden Video als abzuspielende Videorahmen zu erhalten, die der aktuell zu markierenden Box entsprechen, wobei der End-Videorahmen, das der aktuell zu markierenden Box entspricht, ist: ein erkannter End-Videorahmen, der das der aktuell zu markierenden Box entsprechende Ziel enthält, in dem markierenden Video;
• ein zweites Anzeigemodul, das konfiguriert ist, um den ersten Videorahmen, der die aktuell zu markierende Box enthält, anzuzeigen und die abzuspielenden Videorahmen, die der aktuell zu markierenden Box entsprechen, abzuspielen, so dass der Benutzer eine Ziel-Erkennungsbox, die in Verknüpfungsbeziehung mit der aktuell zu markierenden Box steht, aus den Erkennungsboxen, die von den der aktuell zu markierenden Box entsprechenden abzuspielenden Videorahmen enthalten sind;
• ein zweites Bestimmungs- und Herstellungsmodul, das konfiguriert ist, um basierend auf einer zweiten Verknüpfungsbeziehung-Operationsinformation des Benutzers aus den Erkennungsboxen, die von den der aktuell zu markierenden Box entsprechenden abzuspielenden Videorahmen enthalten sind, zu bestimmen, ob eine Ziel-Erkennungsbox, die in Verknüpfungsbeziehung mit der aktuell zu markierenden Box steht, vorliegt, wobei in Fall, dass das Vorliegen einer Ziel-Erkennungsbox, die in Verknüpfungsbeziehung mit der aktuell zu markierenden Box steht, bestimmt wird, eine Verknüpfungsbeziehung zwischen der aktuell zu markierenden Box und der Ziel-Erkennungsbox hergestellt wird.

Optionally, it is provided that the device further comprises:

• a judgment module configured to judge whether a display function for displaying a recommendation box corresponding to the box currently to be marked is activated before “obtaining recommendation boxes with presumed linkage relation to the box to be marked currently and a second video frame corresponding to each recommendation box”,
• wherein when a display function is activated, the second module is triggered to obtain;
• a third obtaining module configured, with the display function disabled, to obtain M video frames after the end video frame corresponding to the current box to be marked in the video to be marked as video frames to be played corresponding to the current box to be marked, where the ending video frame corresponding to the current box to be marked is: a detected ending video frame containing the target corresponding to the current box to be marked in the marking video;
• a second display module configured to display the first video frame containing the box currently to be marked and to play the video frames to be played that correspond to the box currently to be marked, so that the user can select a target recognition box that is in linking relationship with of the box currently to be marked, from the recognition boxes contained by the video frames to be played corresponding to the box currently to be marked;
• a second determining and making module configured to determine, based on user's second linking relationship operation information, from the recognition boxes contained by the video frames to be played corresponding to the box currently to be marked, whether a target recognition box contained in linkage relationship with the box currently to be marked, wherein in case of determining the existence of a target recognition box which has linkage relationship with the box currently to be marked, a linkage relationship between the box currently to be marked and the target recognition box will be produced.

• ein Modul zum vergrößerten Anzeigen, das konfiguriert ist, um vor dem „Bestimmen, basierend auf einer ersten Verknüpfungsbeziehung-Operationsinformation des Benutzers, aus allen Empfehlungsboxen, ob eine Ziel-Empfehlungsbox, die in Verknüpfungsbeziehung mit der aktuell zu markierenden Box steht, vorliegt, und Herstellen, beim Vorliegen einer Ziel-Empfehlungsbox, die in Verknüpfungsbeziehung mit der aktuell zu markierenden Box steht, einer Verknüpfungsbeziehung zwischen der aktuell zu markierenden Box und der Ziel-Empfehlungsbox“ folgende Vorgänge durchzuführen: vergrößertes Anzeigen einer ersten Empfehlungsbox und eines der ersten Empfehlungsbox entsprechenden zweiten Videorahmens, wenn ein erster Befehl zum Vergrößern, der durch den Benutzer für die erste Empfehlungsbox mit vermuteter Verknüpfungsbeziehung zu der aktuell zu markierenden Box ausgelöst wird, erkannt, wobei Auslöseoperationen des ersten Befehls zum Vergrößern umfassen: Rechtsklicken auf den zweiten Videorahmen, der der ersten Empfehlungsbox entspricht; und/oder

Optionally, it is provided that the device further comprises:

• an enlarged display module configured to, before "determining, based on user's first linking relationship operation information, from all of the recommendation boxes, whether there is a target recommendation box that is in linking relationship with the box currently to be marked, and Establishing, in the presence of a target recommendation box in linking relationship with the box currently to be marked, a linking relationship between the box currently to be marked and the target recommendation box” to perform the following operations: enlarged displaying a first recommendation box and a second recommendation box corresponding to the first recommendation box video frame when a first zoom-in command triggered by the user for the first recommendation box with presumed link relationship to the box currently to be marked is detected, triggering operations of the first zoom-in command comprising: right-clicking on the second video frame corresponding to the e corresponds to the first recommendation box; and or

Magnifying display of a first video frame corresponding to the box currently to be marked when a second magnify command initiated by the user for the first video frame corresponding to the box currently to be marked is detected.

• ein erstes Bestimmungsmodul, das konfiguriert ist, um nach dem „Bestimmen, basierend auf einer ersten Verknüpfungsbeziehung-Operationsinformation des Benutzers, aus allen Empfehlungsboxen, ob eine Ziel-Empfehlungsbox, die in Verknüpfungsbeziehung mit der aktuell zu markierenden Box steht, vorliegt, und Herstellen, beim Vorliegen einer Ziel-Empfehlungsbox, die in Verknüpfungsbeziehung mit der aktuell zu markierenden Box steht, einer Verknüpfungsbeziehung zwischen der aktuell zu markierenden Box und der Ziel-Empfehlungsbox“ die Ziel-Empfehlungsbox als eine neue aktuell zu markierende Box zu bestimmen.
• ein zweites Bestimmungsmodul, das konfiguriert ist, um einen Videorahmen, der ein der neue aktuell zu markierenden Box entsprechendes Ziel enthält und der die erste Filterbedingung erfüllt, aus dem zu markierenden Video als ein erster Videorahmen zu bestimmen, der der neuen aktuell zu markierenden Box entspricht;
• ein viertes Modul zum Erhalten, das konfiguriert ist, um neue Empfehlungsboxen mit vermuteter Verknüpfungsbeziehung zu der neuen aktuell zu markierenden Box und einen jeder neuen Empfehlungsbox entsprechenden zweiten Videorahmen zu erhalten, wobei der jeder neuen Empfehlungsbox entsprechende zweite Videorahmen ein erkannter Videorahmen, der ein der neuen Empfehlungsbox entsprechendes Ziel enthält und der eine zweite Filterbedingung erfüllt, in dem zu markierenden Video ist;
• ein drittes Anzeigemodul, das konfiguriert ist, um den ersten Videorahmen, der die neue aktuell zu markierenden Box enthält, und den jeder neuen Empfehlungsbox entsprechenden zweiten Videorahmen, der die neue Empfehlungsbox enthält, anzuzeigen, so dass der Benutzer basierend auf dem jeder neuen Empfehlungsbox entsprechenden zweiten Videorahmen eine neue Ziel-Empfehlungsbox, die in Verknüpfungsbeziehung mit der neuen aktuell zu markierenden Box steht, aus den angezeigten neuen Empfehlungsboxen bestimmt;
• ein viertes Anzeigemodul, das konfiguriert ist, um beim Erkennen eines Auslösens eines Sprungbefehl, der einen Sprung zum vorherigen Absatz angibt, durch den Benutzer den ersten Videorahmen, der die aktuell zu markierende Box enthält, und den zweiten Videorahmen, der jeder Empfehlungsbox entspricht und diese Empfehlungsbox enthält, anzuzeigen.

Optionally, it is provided that the device further comprises:

• a first determination module configured to, after "determining, from all the recommendation boxes, based on first linking relationship operation information of the user, whether there is a target recommendation box that is in linking relationship with the box to be currently marked, and producing, in the presence of a target recommendation box which is in a linking relationship with the box currently to be marked, a linking relationship between the currently to box to be marked and the target recommendation box” to designate the target recommendation box as a new box to be currently marked.
• a second determination module configured to determine a video frame containing a target corresponding to the new box currently to be marked and satisfying the first filter condition from the video to be marked as a first video frame corresponding to the new box currently to be marked ;
• a fourth obtaining module configured to obtain new recommendation boxes with presumed linkage relationship to the new box currently to be marked and a second video frame corresponding to each new recommendation box, wherein the second video frame corresponding to each new recommendation box is a detected video frame corresponding to one of the new recommendation box contains corresponding target and which satisfies a second filter condition is in the video to be marked;
• a third display module configured to display the first video frame containing the new box currently to be marked and the second video frame corresponding to each new recommendation box containing the new recommendation box so that the user based on the corresponding each new recommendation box second video frame, determines a new target recommendation box associated with the new box currently to be marked from the displayed new recommendation boxes;
• a fourth display module configured to display, upon the user detecting a triggering of a jump command specifying a jump to the previous paragraph, the first video frame containing the box currently to be marked and the second video frame corresponding to each recommendation box and those recommendation box contains.

• ein fünftes Modul zum Erhalten, das konfiguriert ist, um vor dem „Anzeigen des ersten Videorahmens, der die aktuell zu markierenden Box enthält, und des jeder Empfehlungsbox entsprechenden zweiten Videorahmens, der die Empfehlungsbox enthält“ eine Rahmennummer-Information des ersten Videorahmens, die der aktuell zu markierenden Box entspricht, und eine Boxkennungsinformation, die der aktuell zu markierenden Box entspricht, zu erhalten;
• ein sechstes Modul zum Erhalten, das konfiguriert ist, um eine Rahmennummer-Information des zweiten Videorahmens, der jeder Empfehlungsbox mit vermuteter Verknüpfungsbeziehung zu der aktuell zu markierenden Box entspricht, und eine Boxkennungsinformation, die jeder Empfehlungsbox entspricht, zu erhalten;
• wobei das erste Anzeigemodul spezifisch konfiguriert ist, um folgende Vorgänge durchzuführen: Anzeigen des ersten Videorahmens, der die aktuell zu markierenden Box enthält, der Rahmennummer-Information des ersten Videorahmens, die der aktuell zu markierenden Box entspricht und der Boxkennungsinformation, die der aktuell zu markierenden Box entspricht, in einem ersten Anzeigebereich einer ersten voreingestellten Anzeigeschnittstelle;
• und Anzeigen des zweiten Videorahmens, der jeder Empfehlungsbox mit vermuteter Verknüpfungsbeziehung zu der aktuell zu markierenden Box entspricht, der Rahmennummer-Information des zweiten Videorahmens und der Boxkennungsinformation, die jeder Empfehlungsbox entspricht, in einem zweiten Anzeigebereich der ersten voreingestellten Anzeigeschnittstelle.

Optionally, it is provided that the device further comprises:

• a fifth module for obtaining configured to, before "displaying the first video frame containing the box currently to be marked and the second video frame corresponding to each recommendation box containing the recommendation box" a frame number information of the first video frame that the corresponds to the box currently to be marked and obtain box identification information corresponding to the box to be marked currently;
• a sixth obtaining module configured to obtain frame number information of the second video frame corresponding to each recommendation box assumed to be linked to the box currently to be marked and box identifier information corresponding to each recommendation box;
• wherein the first display module is specifically configured to perform the following operations: displaying the first video frame containing the box currently to be marked, the frame number information of the first video frame corresponding to the box currently to be marked, and the box identifier information corresponding to the current box to be marked Box corresponds, in a first display area, to a first default display interface;
• and displaying the second video frame corresponding to each recommendation box assumed to be linked to the box currently to be marked, the frame number information of the second video frame, and the box ID information corresponding to each recommendation box in a second display area of the first preset display interface.

Optionally, it is provided that a third display area of the first default display interface further comprises a first trigger area for a jump command to jump to the previous paragraph and a second trigger area for a jump command to jump to the next paragraph, the jump command to jump to the previous paragraph being a command which instructs the display of the first video frame corresponding to a last box to be marked before the current box to be marked, the first video frame corresponding to a last box to be marked before the current box to be marked being a detected video frame corresponding to one of the last has a target corresponding to the box to be marked before the current box to be marked and satisfies the first filter condition in which is the video to be marked; wherein the jump command to jump to the next paragraph is a command that directs the display of the first video frame corresponding to a next box to be marked after the current box to be marked, the next box to be marked after the current box to be marked being an unmarked box , whose time stamp information corresponding to the corresponding end-frame video frame represents the earliest detection time and is later than the detection time of the box currently to be marked, from the unmarked boxes corresponding to video to be marked.

From the above content, it is apparent that the present embodiments of the present Invention provide a method and apparatus for linking goals. The method includes: obtaining a current box to be marked and a first video frame corresponding to the current box to be marked, the first video frame corresponding to the current box to be marked being a recognized video frame containing a target corresponding to the current box to be marked and having a first filter condition fulfilled, is in a video to be marked; Obtaining recommendation boxes with presumed linking relationship to the box currently to be marked and a second video frame corresponding to each recommendation box, wherein the second video frame corresponding to each recommendation box is a recognized video frame containing a target corresponding to the recommendation box and satisfying a second filter condition in the video to be marked is; Displaying the first video frame containing the box currently to be marked and the second video frame corresponding to each recommendation box containing the recommendation box, so that a user based on the second video frame corresponding to each recommendation box selects a target recommendation box that is in linking relationship with the current to marked box is determined from the displayed recommendation boxes; determining, based on user's first linking relationship operation information, from all of the recommendation boxes, whether there is a target recommendation box in linking relationship with the box to be currently marked, and producing, when there is a target recommendation box in linking relationship with the box currently to be marked, a linking relationship between the box currently to be marked and the target recommendation box.

By applying the embodiments of the present invention, the box currently to be marked and its corresponding first video frame, as well as recommendation boxes with presumed linking relation to the box currently to be marked and a second video frame corresponding to each recommendation box can be obtained and then displayed to show the linking of boxes, presumably related to the same goal, in marked video frames at picture level. By directly obtaining and displaying the recommendation boxes with presumed linking relationship to the current box to be marked and the second video frame corresponding to each recommendation box, the time for the user to play and watch the video to be marked is saved, with the process of finding the linking boxes is highly optimized for the user through the calculation of the recommendation boxes, increasing the marking efficiency for boxes to be marked which are in linking relationship with each other, that is, the efficiency for linking targets is increased, realizing convenient and effective linking of the same target that appears in each video frame of the video. Of course, not every product or method embodying the present invention need achieve all of the benefits described above simultaneously.

1. 1. Die aktuell zu markierende Box und ihr entsprechender erster Videorahmen, sowie Empfehlungsboxen mit vermuteter Verknüpfungsbeziehung zu der aktuell zu markierenden Box und eines jeder Empfehlungsbox entsprechenden zweiten Videorahmens können erhalten und dann angezeigt werden, um die Verknüpfung von Boxen, die sich vermutlich auf dasselbe Ziel beziehen, in zu markierten Videorahmen auf Bildebene zu ermöglichen. Dadurch, dass die Empfehlungsboxen mit vermuteter Verknüpfungsbeziehung zu der aktuell zu markierenden Box und der jeder Empfehlungsbox entsprechende zweite Videorahmen direkt erhalten und angezeigt werden, wird die Zeitdauer zum Abspielen und Ansehen des zu markierenden Videos für den Benutzer gespart, wobei der Ablauf zum Auffinden der Verknüpfungsboxen für den Benutzer durch die Berechnung der Empfehlungsboxen stark optimiert wird, wobei die Markierungseffizienz für zu markierende Boxen, die in Verknüpfungsbeziehung zueinander stehen, erhöht wird, d.h., dass die Effizienz zum Verknüpfen von Zielen erhöht wird, wobei eine bequeme und effektive Verknüpfung desselben Ziels realisiert wird, das in jedem Videorahmen des Videos erscheint.
2. 2. Dadurch, dass der der aktuell zu markierenden Box entsprechende erster Videorahmen der aktuell erkannte letzte Videorahmen, der das der aktuell zu markierenden Box entsprechende Ziel enthält, im zu markierenden Video ist, und dass der jeder Empfehlungsbox entsprechende zweite Videorahmen ein aktuell erkannter erster Videorahmen, der das der Empfehlungsbox entsprechende Ziel enthält, im zu markierenden Video ist, wird bis zu einem gewissen Grad die Zeitdauer zum Markieren und Verknüpfen von Erkennungsboxen, die dem Ziel desselben physischen Objekts entsprechen, besser eingespart, was die Effizienz zum Markieren sowie Verknüpfen von Erkennungsboxen, die dem Ziel desselben physischen Objekts entsprechen, durch den Benutzer erhöht.
3. 3. Wenn der Befehl zum Anzeigen von den der aktuell zu markierten Box entsprechenden Empfehlungsboxen aktiviert ist, werden die Empfehlungsboxen mit vermuteter Verknüpfungsbeziehung zu der aktuell zu markierenden Box und eines jeder Empfehlungsbox entsprechenden zweiten Videorahmens direkt erhalten und dann angezeigt. Wenn der Befehl zum Anzeigen von den der aktuell zu markierten Box entsprechenden Empfehlungsboxen nicht aktiviert ist, werden die aktuell zu markierende Box, der erste Videorahmen, in den sich die aktuell zu markierende Box befindet, und die abzuspielenden Videorahmen, die der aktuell zu markierenden Box entsprechen, erhalten, so dass der Benutzer eine Ziel-Empfehlungsbox, die in Verknüpfungsbeziehung mit der aktuell zu markierenden Box steht, aus den abzuspielenden Videorahmen bestimmt und beim Vorliegen der Ziel-Empfehlungsbox eine Verknüpfungsbeziehung zwischen der aktuell zu markierenden Box und der Ziel-Empfehlungsbox herstellt. Dadurch, dass die aktuell zu markierende Box, der erste Videorahmen, in den sich die aktuell zu markierende Box befindet, und die abzuspielenden Videorahmen gleichzeitig angezeigt, wird es für den Benutzer gewissenmaßen bequem, einen Referenzvergleich durchzuführen und die Ziel-Empfehlungsbox, die in Verknüpfungsbeziehung mit der aktuell zu markierenden Box steht, aus den abzuspielenden Videorahmen zu bestimmen, was die Effizienz zum Markieren durch den Benutzer bis zu einem gewissen Grad verbessert.
4. 4. Durch die N Videorahmen nach dem der aktuell zu markierenden Box entsprechenden End-Videorahmen werden durchlaufen, und daraus wird eine Erkennungsbox, deren Boxkennung die voreingestellte Empfehlungsbedingung erfüllt, und zwar eine Erkennungsbox, deren Boxkennung nicht im der aktuell zu markierenden Box entsprechenden End-Videorahmen und in den vorherigen Videorahmen erscheint, als die Empfehlungsbox mit vermuteter Verknüpfungsbeziehung zu der aktuell zu markierenden Box bestimmt, um aus den N Videorahmen nach dem der aktuell zu markierenden Videorahmen entsprechenden End-Videorahmen eine Empfehlungsbox, die am wahrscheinlichsten in vermuteter Verknüpfungsbeziehung mit der aktuell zu markierenden Box steht, aufzuzählen und dem Benutzer zu empfehlen, so dass der Benutzer sich nur auf das Ziel konzentrieren, das dieser Empfehlungsbox darin entspricht, was das Betrachtungsvolumen und die Arbeitsbelastung des Benutzer reduziert und die Effizienz zum Markieren erhöht.
5. 5. Eine Vielzahl von Hilfsfunktionen sind bereitgestellt. Beispielsweise wird eine Funktion zum viertgrößten Anzeigen der ersten Empfehlungsbox mit vermuteter Verknüpfungsbeziehung zu der aktuell zu markierenden Box und eines entsprechenden zweiten Videorahmens bereitgestellt, so dass der Benutzer nach einem sorgfältigen Vergleichen eine Verknüpfungsoperation durchführen kann, was benutzerfreundlicher ist und die Anforderungen an die beruflichen Fähigkeiten des Benutzers verringert. Mit dem Sprungfehl zum Springen zum vorherigen Absatz wird dem Benutzer eine Funktion zur Fehlerkorrektur bereitgestellt. Nachdem der Benutzer eine falsche Verknüpfungsbeziehung durchgeführt hat, kann er zu einer Schnittstelle vor der falschen Verknüpfungsbeziehung zurückkehren, was die Kosten der falschen Verknüpfungsbeziehung verringert. Außerdem ist es für einen Prüfer praktisch, die Markierungsergebnisse des Benutzers zu überprüfen sowie zu verifizieren, um die Genauigkeit der Markierungsdaten zu erhöhen. In unterschiedlichen Bereichen derselben Schnittstelle werden jeweils der erste Videorahmen der aktuell zu markierenden Box und die der aktuell zu markierenden Box entsprechende Boxkennungsinformation, der zweiten Videorahmen, der jeder Empfehlungsbox mit vermuteter Verknüpfungsbeziehung zu der aktuell zu markierenden Box entspricht und der diese Empfehlungsbox enthält, die Rahmennummer-Information des zweiten Videorahmens sowie der Boxkennungsinformation, die jeder Empfehlungsbox entspricht, angezeigt, so dass der Benutzer auf die oben genannten Informationen verweisen und diese betrachten kann und die Markierungserfahrung des Benutzers erhöht wird.

The inventive points of the embodiments of the present invention include:

1. 1. The current box to be marked and its corresponding first video frame, as well as recommendation boxes with presumed linking relationship to the current box to be marked and a second video frame corresponding to each recommendation box, can be obtained and then displayed to show the linking of boxes presumed to relate to the same target relate to allow in to marked video frames at picture level. By directly obtaining and displaying the recommendation boxes with presumed linking relationship to the current box to be marked and the second video frame corresponding to each recommendation box, the time for the user to play and watch the video to be marked is saved, with the process of finding the linking boxes is highly optimized for the user through the calculation of the recommendation boxes, increasing the marking efficiency for boxes to be marked that are in linking relationship with each other, that is, the efficiency for linking targets is increased, realizing convenient and effective linking of the same target that appears in each video frame of the video.
2. 2. Because the first video frame corresponding to the current box to be marked is the currently recognized last video frame containing the target corresponding to the current box to be marked in the video to be marked, and the second video frame corresponding to each recommendation box is a currently recognized first video frame , which contains the target corresponding to the recommendation box is in the video to be tagged, the time to tag and link detection boxes corresponding to the target of the same physical object is better saved to a certain extent, improving the efficiency of tagging and linking detection boxes , which correspond to the target of the same physical object, increased by the user.
3. 3. If the command to show the recommendation boxes corresponding to the box currently to be checked is activated, the recommendation boxes with presumed link reference hung directly to the box currently to be marked and a second video frame corresponding to each recommendation box and then displayed. If the command to display the recommendation boxes corresponding to the box currently being checked is not activated, the current box to be checked, the first video frame in which the current box to be checked is located, and the video frames to be played become those of the current box to be checked correspond, are obtained, so that the user determines a target recommendation box, which is in linking relationship with the box to be currently marked, from the video frames to be played, and in the presence of the target recommendation box establishes a linking relationship between the box to be currently marked and the target recommendation box . By displaying the box currently to be marked, the first video frame in which the box currently to be marked is located, and the video frames to be played at the same time, it becomes somewhat convenient for the user to carry out a reference comparison and the target recommendation box that are in linking relationship with the current box to be marked to determine from the video frames to be played, which improves the efficiency for the user to mark to some extent.
4. 4. The N video frames following the end video frame corresponding to the current box to be marked are traversed, and this results in a detection box whose box ID meets the preset recommendation condition, namely a detection box whose box ID is not in the end video frame corresponding to the current box to be marked Video frames and in the previous video frames appears as the recommendation box with presumed linking relationship to the box currently to be marked, in order to select from the N video frames after the end video frame corresponding to the video frame currently to be marked a recommendation box that is most likely in presumed linking relationship with the current box to be marked is to be enumerated and recommended to the user, so that the user concentrates only on the goal corresponding to this recommendation box therein, which reduces the viewing volume and the user's workload and increases the efficiency for marking.
5. 5. A variety of auxiliary functions are provided. For example, a function of displaying the fourth-largest first recommendation box with presumed linking relationship with the box currently to be marked and a corresponding second video frame is provided, so that the user can perform linking operation after carefully comparing, which is more user-friendly and meets the professional skill requirements of the user user reduced. With the jump error to jump to the previous paragraph, an error correction function is provided to the user. After the user performs a wrong linkage, he can return to an interface before the wrong linkage, reducing the cost of the wrong linkage. In addition, it is convenient for a reviewer to check and verify the user's tagging results to increase the accuracy of the tagging data. In different areas of the same interface, the first video frame of the box currently to be marked and the box identification information corresponding to the box currently to be marked, the second video frame corresponding to each recommendation box with presumed linkage relationship to the box currently to be marked and containing this recommendation box, the frame number -Information of the second video frame as well as the box identification information corresponding to each recommendation box are displayed, so that the user can refer to and view the above information and enhance the user's marking experience.

character list

• 1 zeigt ein schematisches Ablaufdiagramm eines Verfahrens zum Verknüpfen von Zielen gemäß einem Ausführungsbeispiel der vorliegenden Erfindung;
• 2 zeigt ein weiteres schematisches Ablaufdiagramm eines Verfahrens zum Verknüpfen von Zielen gemäß einem Ausführungsbeispiel der vorliegenden Erfindung;
• 3 zeigt ein Beispieldiagramm einer ersten voreingestellten Anzeigeschnittstelle;
• 4 zeigt eine schematische Strukturansicht einer Vorrichtung zum Verknüpfen von Zielen gemäß einem Ausführungsbeispiel der vorliegenden Erfindung.

In order to indicate the exemplary embodiments of the present invention or the technical configurations of the prior art more clearly, the drawings to be used in the description of the exemplary embodiments or the prior art are briefly presented below. Obviously, the drawings described below represent only some exemplary embodiments of the present invention. The person skilled in the art can also obtain further drawings on the basis of these drawings without exercising any inventive work.

• 1 shows a schematic flowchart of a method for linking goals according to an embodiment of the present invention;
• 2 shows a further schematic flow diagram of a method for linking goals according to an embodiment of the present invention;
• 3 Figure 12 shows an example diagram of a first default display interface;
• 4 12 is a schematic structural view of a target linking apparatus according to an embodiment of the present invention.

DETAILED DESCRIPTION

The technical configurations in embodiments of the present invention will be described clearly and fully hereinafter with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention and do not cover all embodiments. All other embodiments that can be achieved by those skilled in the art based on the embodiments in the present invention without any inventive work should fall within the scope of the present invention.

It is to be noted that the terms "comprising" and "having" and their variations in the embodiments and drawings of the present invention are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device having a set of steps or items is not limited to the listed steps or items, but also optionally includes steps or items not listed, or optionally includes others inherent steps or units in that process, procedure, product or device.

The present invention provides a method and apparatus for linking goals to realize convenient and effective linking of the same goal appearing in each video frame of the video. The embodiments of the present invention will be described in detail below.

• S101: Erhalten einer aktuell zu markierenden Box und eines der aktuell zu markierenden Box entsprechenden ersten Videorahmens.

1 shows a schematic flowchart of a method for linking goals according to an embodiment of the present invention. The procedure can include the following steps:

• S101: Obtain a box currently to be marked and a first video frame corresponding to the current box to be marked.

In this case, the first video frame corresponding to the box currently to be marked is a recognized video frame which contains a target corresponding to the box currently to be marked and which satisfies a first filter condition in a video to be marked.

The method for linking targets provided by the embodiments of the present invention can be applied to any type of electronic device, wherein the electronic device can be a server or a terminal. In one form of implementation, a preset marking tool can be mounted on the electronic device, so that the method for linking targets provided by the embodiments of the present invention can be implemented by the preset marking tool.

The electronic device may first receive a video to be tagged and pre-tagged results corresponding to the video to be tagged. The video to be tagged can be any type of video. For example, the video to be marked may be a road video that, in the case that a test vehicle is driving in the target scene, is captured by an image capturing device arranged on the test vehicle while the test vehicle is driving for the surroundings.

In one case, the electronic device may display a display interface that allows the user to select the video to be tagged in order to tag it. Icons corresponding to each video to be tagged are displayed on the display interface. The user can select a video to be tagged that needs to be tagged with a mouse, stylus, or finger, respectively, so that the electronic device can obtain the video to be tagged and its corresponding pre-tagged result.

The pre-marked result corresponding to the video to be marked can be a result obtained by tracking and recognizing individual targets in the video to be marked using a target detection-based tracking algorithm, or a result obtained by historical manual marking of the individual targets in the video to be tagged is received by the user. The pre-tagged results corresponding to the video to be tagged may include: a detected position information of the detection box of the target included in each video frame of the video to be tagged, a linking relationship between targets of the same physical object detected from different video frames, and one of the detection box of the marker information corresponding to individual targets; and semantic information corresponding to point cloud data corresponding to each target.

Thereby, the linking relationship between targets of the same physical object recognized from different video frames can be established box identification information about the marker box corresponding to the destination is displayed. For example, the marked boxes corresponding to targets recognized from different video frames of the same physical object may have the same box identifier information. Here, the box ID information of the recognition boxes corresponding to the targets of the same physical object are the same, while the box ID information of the recognition boxes corresponding to the targets of different physical objects are unequal.

The mark information corresponding to a recognition box is information indicating whether the recognition box is marked.

In the embodiments of the present invention, a target's detection box may be referred to as a marked box after it is marked by the user, while a target's recognition box before it is marked by the user may be referred to as an unmarked box.

The tracking algorithm based on target detection can be a deep learning based target tracking model or a traditional target tracking algorithm, all of which are possible. The embodiments of the present invention do not limit the specific type of target recognition-based tracking algorithm.

In one implementation, it is necessary to perform manual verification and recognition of the pre-tagged results corresponding to the video to be tagged to ensure the accuracy of the pre-tagged results corresponding to the video to be tagged. After the electronic device has received the video to be tagged and its pre-tagged results, there are two options. In one option, the electronic device can first run through the pre-marked results corresponding to the video to be marked, i.e. the detection boxes corresponding to the video to be marked, whereby from the unmarked recognition boxes corresponding to the video to be marked, i.e. the unmarked boxes, a current to box to be marked is determined, and wherein a first video frame corresponding to the box currently to be marked is determined from the video to be marked. In the other possibility, the electronic device can receive a selection operation, which is triggered by the user for the unmarked recognition boxes corresponding to the video to be marked, i.e. the unmarked boxes, in order to select an unmarked recognition box selected by the user as the current box to be marked determine. In a further implementation, the electronic device may determine the current box to be marked from the pre-marked results corresponding to the video to be marked, where the first video frame corresponding to the current box to be marked is determined from the video to be marked and then to the electronic device is sent, wherein the electronic device receives the current box to be marked and the first video frame corresponding to the current box to be marked. In this case, the first video frame corresponding to the box currently to be marked comprises a target corresponding to the box currently to be marked.

Given the characteristics of target tracking results, the same target often appears in multiple video frames, with a physical target appearing once at the same time. In order to reduce to some extent the workload for the user to mark, the detected video frame containing the target corresponding to the current box to be marked and which satisfies a first filter condition, in the video to be marked, a detected last video frame, i.e., a end video frame containing the target corresponding to the current box to be marked must be in the video to be marked. That is, the first video frame corresponding to the current box to be marked may be an end video frame containing the target corresponding to the current box to be marked in the video to be marked. In one case, to ensure that the user can better mark the current box to be marked, it is determined from other video frames whether there is a target corresponding to the same physical object as the target corresponding to the current box to be marked. The recognized video frame, which contains the target corresponding to the box currently to be marked and which satisfies a first filter condition, can be a recognized video frame in the video to be marked, which contains the target corresponding to the box currently to be marked and the target of which has a sharpness that exceeds a preset sharpness threshold , be in the video to be tagged.

S102: Obtain recommendation boxes with presumed link relationship to the box currently to be marked and a second video frame corresponding to each recommendation box.

In this case, the second video frame corresponding to each recommendation box is a recognized video frame, which contains a target corresponding to the recommendation box and which satisfies a second filter condition, in the video to be marked. Given the characteristics of target tracking results, the same target often appears in multiple video frames. To mark the workload through to reduce the user to a certain extent, the detected video frame containing the target corresponding to the recommendation box and which satisfies the second filter condition in the video to be marked a currently recognized first video frame containing the target corresponding to the recommendation box in the to be marked be video. In one case, to ensure that the user can better mark the current box to be marked, it is determined from other video frames whether there is a target corresponding to the same physical object as the target corresponding to the current box to be marked. The recognized video frame, which contains the target corresponding to the recommendation box and which satisfies a second filter condition, in the video to be marked can be a recognized video frame, which contains the target corresponding to the recommendation box and whose target has a sharpness that exceeds a preset sharpness threshold value, in the video to be marked .

In the present step, the electronic device for the current box to be marked from among recognition boxes corresponding to several video frames after the end video frame corresponding to the current box to be marked in the video to be marked, a recognition box with presumed link relationship to the current box to be marked as one of the determine the recommendation box corresponding to the box currently to be marked and determine a second video frame from the video to be marked which corresponds to each recommendation box.

S103: Displaying the first video frame containing the box currently to be marked and the second video frame corresponding to each recommendation box containing the recommendation box, so that a user based on the second video frame corresponding to each recommendation box selects a target recommendation box that is in linking relationship with the box currently to be marked is determined from the displayed recommendation boxes.

In the present step, the electronic device may display the first video frame containing the box currently to be marked in different areas of the first preset display interface, and display the second video frame corresponding to each recommendation box corresponding to the box currently to be marked and containing this recommendation box. Thus, the user can intuitively compare the target in the first video frame corresponding to the current box to be marked with the target in the second video frame corresponding to each recommendation box that corresponds to that recommendation box and determine whether the recommendation box is in link relationship with the current box to be marked, i.e., whether the target corresponding to the recommendation box and the target corresponding to the box currently being marked refer to the same physical object.

In one case, the electronic device may display the second video frame corresponding to each recommendation box corresponding to the current box to be marked in the form of an image list in the first preset display interface for the user to drag to view. Accordingly, the first preset display interface can also display a selection area without a link box, so that when it is determined in the recommendation box corresponding to the currently selected box that there is no recommendation box with a link relationship, the user performs a corresponding operation.

S104: Determining, based on user's first linking relationship operation information, from all of the recommendation boxes, whether there is a target recommendation box in linking relationship with the box to be currently marked, and, if there is a target recommendation box, establishing the linking relationship with the box currently to be marked, a linking relationship between the box currently to be marked and the target recommendation box.

In the present step, the electronic device obtains the linking relationship operation information triggered by the user based on the first video frame containing the current box to be marked and the second video frame corresponding to each recommendation box corresponding to the current box to be marked . The electronic device determines whether there is a target recommendation box linked with the current box to be marked based on the first linking relationship operation information from all of the recommendation boxes, and if there is a target recommendation box linked with the current one, delivers box to be marked, there is a linking relationship between the box currently to be marked and the target recommendation box. Here, the above process for establishing a linking relationship between the box currently to be marked and the target recommendation box may be that the box identification information corresponding to the box currently to be marked and the box identification information corresponding to the target recommendation box are modified into the same information will.

In a case where the electronic device determines that the user selects at least one recommendation box based on the first association relationship operation information selects the recommendation boxes corresponding to the box currently to be marked, the electronic device determines that in all recommendation boxes corresponding to the box currently to be marked, there is a target recommendation box that is in linking relationship with the box currently to be marked, the electronic device using the user determines the selected recommendation box as the target recommendation box and establishes a linking relationship between the box currently to be marked and the target recommendation box, that is, the target corresponding to the box currently to be marked and the target corresponding to the target recommendation box refer to the same physical object, ie allowing a link between the target corresponding to the box currently being marked and the target corresponding to the target recommendation box.

Then, to ensure the regularity of the user's tagging process, to reduce to some extent the time for the user to tag and link detection boxes corresponding to the targets of the same physical object, and the efficiency of tagging and linking the targets of the same physical object Increasing recognition boxes by the user, the target recommendation box selected by the user, which is in linking relationship with the box currently to be marked, can continue to be used as a new box currently to be marked, to turn the next marking process for the new box currently to be marked to perform.

In another case where the electronic device determines based on the first linking relationship operation information that the user does not select a recommendation box from the recommendation boxes corresponding to the box currently to be marked, that is, the user triggers the selection area without a link box, the electronic device determines that in all of the recommendation boxes corresponding to the box currently to be marked there is no target recommendation box which is in a linking relationship with the box currently to be marked.

Subsequently, in order to ensure the correctness of the user's marking process and to increase the efficiency for marking by the user, the electronic device can directly determine a new current-to-be-marked box from the unmarked boxes corresponding to the current box to be marked in order to carry out the next marking process for the to carry out a new box that is currently to be marked. By applying the embodiments of the present invention, the box currently to be marked and its corresponding first video frame, as well as recommendation boxes with presumed linking relation to the box currently to be marked and a second video frame corresponding to each recommendation box can be obtained and then displayed to show the linking of boxes, presumably related to the same goal, in marked video frames at picture level. By directly obtaining and displaying the recommendation boxes with presumed linking relationship to the current box to be marked and the second video frame corresponding to each recommendation box, the time for the user to play and watch the video to be marked is saved, with the process of finding the linking boxes is highly optimized for the user through the calculation of the recommendation boxes, increasing the marking efficiency for boxes to be marked which are in linking relationship with each other, that is, the efficiency for linking targets is increased, realizing convenient and effective linking of the same target that appears in each video frame of the video.

• erste Implementierungsform:
  • aus unmarkierten Boxen, die dem zu markierenden Video entsprechen, wird eine unmarkierte Box mit der frühesten Erfassungszeit, die durch eine einem End-Videorahmen entsprechende Zeitstempelinformation dargestellt wird, als die aktuell zu markierende Box bestimmt.

In a further exemplary embodiment of the present invention, step S101 can be implemented by one of the following two forms of implementation:

• first implementation form:
  • from unmarked boxes corresponding to video to be marked, an unmarked box with the earliest detection time represented by timestamp information corresponding to an end video frame is determined as the current box to be marked.

In one case, the possibility is considered that the user initiates the tagging process of the video to be tagged for the first time in the present tagging task. Accordingly, upon receipt of the video to be tagged and its corresponding pretagged results, the electronic device may first iterate through the pretagged results corresponding to the video to be tagged, namely the detection boxes corresponding to the video to be tagged, wherein the electronic device selects the untagged ones corresponding to the video to be tagged Recognition boxes, ie the unmarked boxes, determine a box currently to be marked and from the video to be marked a first video frame corresponding to the box currently to be marked can be determined. Given the characteristics of target tracking results, the same target often appears in multiple video frames. In order to reduce the user's tagging workload to a certain extent and improve the regulatory meas bility of the user's marking process, the electronic device can from the unmarked detection boxes corresponding to the video to be marked, ie the unmarked boxes, an unmarked box with the earliest detection time represented by a time stamp information corresponding to an end video frame than the current determine the box to be marked.

Here, the end video frame corresponding to the unmarked box may be a last video frame of the video frames containing the target corresponding to the unmarked box in the video to be marked.

In another case, the possibility is considered that it is not the first time that the user triggers the tagging process of the video to be tagged in the present tagging task. In the event that the user does not select a recommendation box that has a linking relationship with the boxes to be marked historically from the correspondingly displayed recommendation boxes during the previous marking process for the displayed boxes to be marked historically, the electronic device can directly select from the corresponding video to be marked unmarked boxes designate an unmarked box with the earliest detection time represented by timestamp information corresponding to an end video frame as the current box to be marked.

Second form of implementation:

if it is detected that for historically displayed boxes to be marked, the user selects a recommendation box associated with them from among the corresponding displayed recommendation boxes, the recommendation box selected by the user is determined as the current box to be marked.

In the present form of implementation, the possibility is considered that it is not the first time that the user triggers the tagging process of the video to be tagged in the present tagging task. In the case that the user selects a recommendation box that has a linking relationship with the boxes to be marked historically from the correspondingly displayed recommendation boxes during the previous marking process for the displayed boxes to be marked historically, the recognition boxes for the same target can be marked for convenience of the tagging process and to ensure the tagging efficiency of the users. If it is detected that for historically displayed boxes to be marked, the user selects an associated recommendation box from the corresponding displayed recommendation boxes, the recommendation box selected by the user is determined as the current box to be marked. Then, the following process of linking targets is performed.

• 011: Durchlaufen durch vormarkierte Ergebnisse, die N Videorahmen nach dem der aktuell zu markierenden Box entsprechenden End-Videorahmen entsprechen, Bestimmen einer Erkennungsbox, deren Boxkennung eine vorgegebene Empfehlungsbedingung erfüllt, aus Erkennungsboxen, die den N Videorahmen nach dem der aktuell zu markierenden Box entsprechenden End-Videorahmen entsprechen, als eine Erkennungsbox mit vermuteter Verknüpfungsbeziehung zu der aktuell zu markierenden Box.

In a further exemplary embodiment of the present invention, it is provided that step S102 can include the following steps 011 to 012:

• 011: iterate through pre-marked results corresponding to N video frames after the end video frame corresponding to the current box to be marked, determining a recognition box whose box identifier satisfies a predetermined recommendation condition from recognition boxes corresponding to the N video frames after the end corresponding to the current box to be marked - video frames correspond, as a recognition box with presumed link relationship to the current box to be marked.

The detection box whose box identifier satisfies a predetermined recommendation condition includes a detection box whose box identifier information does not appear in the end video frame corresponding to the box currently to be marked and in the previous video frame.
012: Determine, for each recommendation box with presumed linking relation to the current box to be marked, a video frame containing the target corresponding to the recommendation box and satisfying the second filter operation, from the N video frames after the end video frame corresponding to the current box to be marked as the second video frames corresponding to the recommendation box.

In this case, the video frame containing the target corresponding to the recommendation box and the second filter operation fulfills the recognized first video frame containing the target corresponding to the recommendation box in the video to be marked.

In the embodiments of the present invention, it is considered that due to lack of linkage, the target of the same physical object may be detected as another target, that the box identification information of recognition boxes corresponding to different targets is different, that there is only one same physical object at the same time There is time in a video frame and that the computational effort of the electronic device should be reduced. The process of obtaining, by the electronic device, the recommendation box currently to be marked may include: iterating through pre-marked results, the N video frames after the ending video frame corresponding to the box currently to be marked, determining a newly appearing recognition box, ie, a recognition box whose box identifier satisfies a predetermined recommendation condition, from recognition boxes corresponding to the N video frames after the ending video frame corresponding to the box currently to be marked , as a recognition box with presumed link relationship to the box currently to be marked. The detection box whose box ID satisfies a predetermined recommendation condition includes a detection box whose box ID information does not appear in the ending video frame corresponding to the current box to be marked and previous video frames.

For each recommendation box with presumed link relationship to the box currently to be marked, a video frame containing the target corresponding to the recommendation box and satisfying the second filter operation is selected from the N video frames after the end video frame corresponding to the box currently to be marked as the second corresponding to the recommendation box Video frame determined, i.e. the detected first video frame containing the target corresponding to the recommendation box is determined in the video to be marked as the second video frame corresponding to the recommendation box. For example, in the N video frames after the end video frame corresponding to the box currently to be marked, there is an NPth, N-P+1st, N-P+2nd, N-P+3rd and N-P+ 4th video frames containing the target corresponding to recommendation box A, which corresponds to the box to be currently marked. Now the first video frame, i.e., the N-Pth video frame containing the target corresponding to recommendation box, in the video to be marked is determined as the second video frame corresponding to recommendation box A.

Where N is a positive integer and N is a default value based on experience. Considering the factor of avoiding the omission of the linking relationship between the detection boxes, the factors of user workload and work efficiency, and the factor that the linking relationship detection boxes whose interval of video frames is too large for training the subsequent deep learning-based target tracking model are of little importance, the value of N should not be too small or too large. In one case, the range of values of N may be [20, 30].

In the exemplary embodiments of the present invention, from the N video frames after the end video frame corresponding to the video frame currently to be marked, a recommendation box that is most likely to have a presumed link relationship with the box currently to be marked is enumerated and recommended to the user, so that the user can can only focus on the target that corresponds to this recommendation box in it, which reduces the viewing volume and user's workload and increases the efficiency for tagging.

• 021: Bestimmen, ob eine Anzeigefunktion zum Anzeigen der Empfehlungsbox, die der aktuell zu markierenden Box entspricht, aktiviert ist; wenn ja, wird der Schritt S102 durchgeführt; wenn nein, wird der Schritt 022 durchgeführt;
• 022: Erhalten von M Videorahmen nach dem End-Videorahmen, das der aktuell zu markierenden Box entspricht, in dem zu markierenden Video als abzuspielende Videorahmen, die der aktuell zu markierenden Box entsprechen.

In a further exemplary embodiment of the present invention, it is provided that the method can include the following steps 021 to 024 before step S102:

• 021: determining whether a display function for displaying the recommendation box corresponding to the box to be currently marked is activated; if yes, step S102 is performed; if no, step 022 is performed;
• 022: Obtain M video frames after the end video frame corresponding to the current box to be marked in the video to be marked as video frames to be played corresponding to the current box to be marked.

Here, the end video frame corresponding to the box currently to be marked is a recognized end video frame containing the target corresponding to the box currently to be marked in the marking video.
023: Displaying the first video frame containing the box currently to be marked, and playing the video frames to be played corresponding to the box currently to be marked, so that the user can select a target recognition box that is in link relationship with the box currently to be marked the recognition boxes contained by the video frames to be played corresponding to the current box to be marked;
024: Determining, based on user's second linking relationship operation information, from the recognition boxes contained by the video frames to be played corresponding to the box currently to be marked, whether there is a target recognition box in linking relationship with the box currently to be marked , wherein in case of the presence of a target recognition box which has a linking relationship with the box to be currently marked, it is determined that a linking relationship is established between the box to be currently marked and the target recognition box.

In the embodiments of the present invention, when judging that the display function of displaying the recommendation box corresponding to the box currently to be marked is activated, the electronic device may provide the user with the display function of displaying the recommendation box corresponding to the box currently to be marked. Ie, the electronic Apparatus may perform the steps of obtaining a recommendation box with presumed linkage relationship to the box currently to be marked and obtaining the second video frame corresponding to each recommendation box and the subsequent flow. When judging that the display function of displaying the recommendation box corresponding to the current box to be marked is not activated, the M video frames after the end video frame corresponding to the current box to be marked in the video to be marked can be directly as the video frames to be played, the corresponding to the box currently being marked. In the first display area of the second default display interface, the first video frame containing the current box to be marked is displayed. And in the second display area of the second preset display interface, the video frames to be played corresponding to the current box to be marked are played frame by frame. The user can view the target in each video frame of the video frames to be played and check whether there is a target related to the same physical object of a target corresponding to the current box to be marked, i.e. whether in the detection boxes there is a detection box that is in linking relationship with the box currently being marked. A corresponding operation is then triggered.

The electronic device obtains the second association relationship operation information based on the corresponding operation initiated by the user. In a case where it is determined based on the second linking relationship operation information that the user selects a recognition box from the recognition boxes corresponding to the box to be currently marked in the video to be played, it is determined that in the recognition boxes in the video to be played a target there is a recognition box which is in linking relationship with the box currently to be marked, wherein the recognition box selected by the user is determined as the target recognition box and a linking relationship is established between the box currently to be marked and the target recognition box, i.e. that the target corresponding to the box currently to be marked and the target corresponding to the target detection box refer to the same physical object, i.e. that a link between the target corresponding to the box currently to be marked and the target corresponding to the Target detection box corresponds, is enabled.

Then, to ensure the regularity of the user's tagging process, to reduce to some extent the time for the user to tag and link detection boxes corresponding to the targets of the same physical object, and the efficiency of tagging and linking the targets of the same physical object Increasing recognition boxes by the user, the target recognition box selected by the user, which is in linking relationship with the current box to be marked, can continue to be used as a new current box to be marked, to turn the next marking process for the new current box to be marked to perform.

In another case where the electronic device determines based on the second linking relationship operation information that the user does not select a recognition box from the recognition boxes corresponding to the box currently to be marked in the video to be played, the electronic device determines that in the the currently to detection boxes corresponding to the box to be marked, there is no target detection box in the video to be played which is in link relation with the box to be marked currently.

Subsequently, in order to ensure the correctness of the user's marking process and to increase the efficiency for marking by the user, the electronic device can directly determine a new current-to-be-marked box from the unmarked boxes corresponding to the current box to be marked in order to carry out the next marking process for the to carry out a new box that is currently to be marked.

• vergrößertes Anzeigen einer ersten Empfehlungsbox und eines der ersten Empfehlungsbox entsprechenden zweiten Videorahmens, wenn ein erster Befehl zum Vergrößern, der durch den Benutzer für die erste Empfehlungsbox mit vermuteter Verknüpfungsbeziehung zu der aktuell zu markierenden Box ausgelöst wird, erkannt wird.

In a further exemplary embodiment of the present invention, it is provided that in order to ensure a better user experience and increase the accuracy of the marking results by the user, the method before step S104 can include the following steps:

• enlarged display of a first recommendation box and a second video frame corresponding to the first recommendation box when a first enlargement command triggered by the user for the first recommendation box with presumed linking relationship to the box to be marked currently is detected.

Here, triggering operations of the first zoom-in command include: right-clicking on the second video frame corresponding to the first recommendation box. and/or displaying enlarged a first video frame corresponding to the current box to be marked when a second enlargement command issued by the user for the first video frame corresponding to the box currently being marked is detected.

The embodiments of the present invention provide an enlarged display function. When a zoom-in command triggered by the user for the first recommendation box with presumed link relationship to the box currently to be marked is recognized, i.e. when a right-click on the second video frame corresponding to the first recommendation box is recognized by the user, the first recommendation box and the second video frame corresponding to the first recommendation box are enlarged so that the user can carefully compare the target corresponding to the recommendation box with the target corresponding to the box currently being marked to determine whether the two targets relate to the same physical object. If a second zoom-in command issued by the user for the first video frame corresponding to the box currently being marked is recognized, the first video frame corresponding to the box currently being marked is displayed enlarged. The triggering operation of the second zoom display command may be: the user keeps pressing the left button or the right button of the mouse to scroll the mouse in the display area of the first video frame corresponding to the current box to be marked; alternatively, the user slides two fingers across the display area of the first video frame corresponding to the current box to be marked.

Then the user can drag the screen by holding down the "Shift" key and the left button of the mouse or by holding down the middle button of the mouse. That is, the user can drag the first video frame corresponding to the current box to be marked or the second video frame corresponding to the first recommendation box.

In one case, the display position referred to above may be the above-mentioned first display area of the first default display interface.

• S201: Bestimmen der Ziel-Empfehlungsbox als eine neue aktuell zu markierende Box.
• S202: Bestimmen eines Videorahmen, der ein der neue aktuell zu markierenden Box entsprechendes Ziel enthält und der die erste Filterbedingung erfüllt, aus dem zu markierenden Video als ein erster Videorahmen, der der neuen aktuell zu markierenden Box entspricht.
• S203: Erhalten von neuen Empfehlungsboxen mit vermuteter Verknüpfungsbeziehung zu der neuen aktuell zu markierenden Box und eines jeder neuen Empfehlungsbox entsprechenden zweiten Videorahmens.

In a further embodiment of the present invention it is provided that the method as in 2 shown, after step S104 may include the following steps:

• S201: Determine the target recommendation box as a new box to be currently marked.
• S202: Determine a video frame containing a target corresponding to the new box currently to be marked and satisfying the first filter condition from the video to be marked as a first video frame corresponding to the new box currently to be marked.
• S203: Obtain new recommendation boxes with presumed link relationship to the new box currently to be marked and a second video frame corresponding to each new recommendation box.

In this case, the second video frame corresponding to each new recommendation box is a recognized video frame, which contains a target corresponding to the new recommendation box and which satisfies a second filter condition, in the video to be marked.
S204: Displaying the first video frame containing the new box to be currently marked and the second video frame corresponding to each new recommendation box containing the new recommendation box, so that based on the second video frame corresponding to each new recommendation box the user selects a new target recommendation box, which is in linking relationship with the new box currently to be marked is determined from the displayed new recommendation boxes.
S205: Displaying the first video frame containing the box currently to be marked and the second video frame corresponding to each recommendation box and containing that recommendation box when it is detected that the user triggers a jump command that instructs a jump to the previous paragraph.

In the present embodiment, after the electronic device establishes the linking relationship between the current box to be marked and the recommendation box, the target recommendation box can be directly determined as a new current box to be marked. And steps S202 to S204 are performed. The process for determining the first video frame corresponding to the new box currently to be marked in step S202 can be referred to the above-described process for determining the first video frame corresponding to the box currently to be marked. Regarding the process for obtaining new recommendation boxes with presumed linking relationship to the new box currently to be marked and a second video frame corresponding to each new recommendation box in step S203, the process for obtaining recommendation boxes with presumed linking relationship to the box currently to be marked and one described above can be referred to second video frame corresponding to each recommendation box. Regarding the process of displaying the first video frame containing the new box currently to be marked and the second video frame corresponding to each new recommendation box containing the new recommendation box in step S204, the process described above for displaying the first video frame containing the contains the current box to be marked, and the second video frame corresponding to each recommendation box and containing the recommendation box. This is not repeated here.

In order to reduce the cost of erroneous concatenation operations, the embodiments of the present invention provide an error correction function. If the user determines that the selected target suggestion box is incorrect for the box currently being marked, the user can trigger a jump command to jump to the previous paragraph. If the electronic device detects that the user initiates a jump command that instructs a jump to the previous paragraph, the first video frame containing the box currently to be marked and the second video frame corresponding to each recommendation box and containing this recommendation box are displayed .

In one form of implementation, during the process of establishing a linking relationship between the box currently to be marked and the target recommendation box, the electronic device may perform the following operation: the box identification information corresponding to the target recommendation box and the box identification information corresponding to the box currently to be marked corresponds to be modified into the same identification information. In order to provide error correction information to the user, after establishing the linking relationship between the box currently to be marked and the target recommendation box, the electronic device still stores the information before establishing the linking relationship between the box currently to be marked and the target recommendation box. For example, the corresponding box identification information is stored before establishing the linking relationship between the box to be currently marked and the target recommendation box. Accordingly, upon detecting that the user initiates a jump command directing a jump to the previous paragraph, the electronic device may display the first video frame containing the current box to be marked and the second video frame corresponding to each recommendation box and containing that recommendation box, show again.

• 031: Erhalten einer Rahmennummer-Information des ersten Videorahmens, die der aktuell zu markierenden Box entspricht, und einer Boxkennungsinformation, die der aktuell zu markierenden Box entspricht.
• 032: Erhalten einer Rahmennummer-Information des zweiten Videorahmens, der jeder Empfehlungsbox mit vermuteter Verknüpfungsbeziehung zu der aktuell zu markierenden Box entspricht, und einer Boxkennungsinformation, die jeder Empfehlungsbox entspricht. Der Schritt S103 umfassen folgende Schritte 041 bis 042:
• 041: Anzeigen des ersten Videorahmens, der die aktuell zu markierenden Box enthält, der Rahmennummer-Information des ersten Videorahmens, die der aktuell zu markierenden Box entspricht und der Boxkennungsinformation, die der aktuell zu markierenden Box entspricht, in einem ersten Anzeigebereich einer ersten voreingestellten Anzeigeschnittstelle.
• 042: Anzeigen des zweiten Videorahmens, der jeder Empfehlungsbox mit vermuteter Verknüpfungsbeziehung zu der aktuell zu markierenden Box entspricht, der Rahmennummer-Information des zweiten Videorahmens und der Boxkennungsinformation, die jeder Empfehlungsbox entspricht, in einem zweiten Anzeigebereich der ersten voreingestellten Anzeigeschnittstelle.

In a further exemplary embodiment of the present invention, it is provided that the method can include the following steps 031 to 032 before step S103:

• 031: Obtain frame number information of the first video frame corresponding to the box to be currently marked and box ID information corresponding to the box to be currently marked.
• 032: Obtaining frame number information of the second video frame corresponding to each recommendation box having presumed link relation to the box to be marked currently, and box ID information corresponding to each recommendation box. Step S103 includes the following steps 041 to 042:
• 041: Displaying the first video frame containing the box currently to be marked, the frame number information of the first video frame corresponding to the box currently to be marked and the box identifier information corresponding to the box currently to be marked in a first display area of a first preset display interface .
• 042: Displaying the second video frame corresponding to each recommendation box with presumed linkage relation to the box to be currently marked, the frame number information of the second video frame and the box identifier information corresponding to each recommendation box in a second display area of the first preset display interface.

In order to better support the user's marking work, in the exemplary embodiments of the present invention, after "displaying the first video frame containing the box currently to be marked and the second video frame corresponding to each recommendation box containing the recommendation box", the electronic device can further display a Obtain frame number information of the first video frame corresponding to the box currently to be marked and box identifier information corresponding to the box currently to be marked. Here, the frame number information may be a box ID and the like that provides information for uniquely identifying an ID box corresponding to a recognized target. The frame number information of the first video frame corresponding to the box currently to be marked is the position information of the first video frame corresponding to the box currently to be marked in the video to be marked, ie what frame the first video frame is. The electronic device obtains frame number information of the second video frame corresponding to each recommendation box with presumed linking relation to the box to be currently marked, and box identification information corresponding to each recommendation box. In this case, as each recommendation box with a suspected linking relationship to the box currently to be marked, a recommendation box corresponding to the box currently to be marked, frame number information of the second video frame, of each recommendation box with a suspected linking relationship to the currently-to-be-marked box corresponds, or position information of the second video frame corresponding to each recommendation box with presumed linkage relationship to the current-to-be-marked box in the video to be marked (ie, which frame the second video frame is).

The electronic device displays the first video frame containing the current box to be marked, the frame number information of the first video frame corresponding to the current box to be marked, and the box ID information corresponding to the current box to be marked in a first display area of the first preset Display interface and displays the second video frame corresponding to each recommendation box with presumed linkage relationship to the box currently to be marked, the frame number information of the second video frame and the box identification information corresponding to each recommendation box in a second display area of the first preset display interface. Here, the display order of the second video frame corresponding to each recommendation box corresponding to the current box to be marked may be: displayed according to the order of the second video frames in the video to be masked, represented by the frame number information of the second video frames, corresponding to each recommendation box corresponding to the box currently being marked.

As in 3 1, an example diagram of a structure of a first default display interface is shown. The first display area of the first preset display interface can be referred to as the main image display field, which is located on the left side of the first preset display interface, and on which the first video frame containing the current box to be marked, the frame number information of the first video frame that the current corresponds to the box to be marked and the box ID information corresponding to the box to be marked currently is displayed. The second display area of the first preset display interface can be called a recommendation box selection field, on which the second video frame corresponding to each recommendation box corresponding to the box to be marked, the box identifier information corresponding to each recommendation box, and the frame number information of the second video frame corresponding to each corresponds to the recommendation box are displayed. Here, the second video frame corresponding to each recommendation box corresponding to the box to be marked is displayed in the form of a preview image, ie, in the form of a thumbnail.

As in 3 shown, in the second display area at the end of the second video frame corresponding to the recommendation box, an “unassigned box”, ie, the above-mentioned selection area without a link box, is displayed. When the user selects an area corresponding to the “unassigned box”, the electronic device determines that in the recommendation box corresponding to the box to be currently marked, there is no recommendation box that has a linking relationship with the box to be currently marked.

Using the example of 3 the user determines that the recommendation box with a corresponding box information box, i.e. with an ID of "8", is linked to the box to be currently marked with a corresponding box information box, i.e. with an ID of "7". The user can click on the area where the second video frame corresponding to the recommendation box with ID "8" is located to select the recommendation box with ID "8". The electronic device then establishes a linking relationship between the box currently to be marked and the recommendation box with ID "8" selected by the user, for example the ID of the recommendation box with ID "8" being changed from "8" to "7".

In another embodiment of the present invention, the third display area of the first preset display interface may further include a first trigger area for a skip command to skip to the previous paragraph The skip command to skip to the previous paragraph refers to a command that displays the first video frame of a last box to be marked before the current box to be marked. The first video frame corresponding to a last box to mark before the current box to mark refers to a detected video frame that has a target corresponding to the last box to mark before the current box to mark and satisfies the first filter condition in which to marking video. The jump command for jumping to the next paragraph refers to a command that directs the display of the first video frame corresponding to a next box to be marked after the current box to be marked. The next box to be marked after the current box to be marked refers to an unmarked box whose time stamp information corresponding to the corresponding end frame video frame represents the earliest detection time and is later than the detection time of the current box to be marked, from among the unmarked corresponding to the video to be marked boxing

In one case, the third display area may be below the second display area. As in 3 shown, introduces the “jumping to the previous paragraph” represents the first trigger area for the jump command to jump to the previous paragraph, while “jump to the next paragraph” represents the second trigger area for the jump command to jump to the next paragraph.

• ein erstes Modul 410 zum Erhalten, das konfiguriert ist, um eine aktuell zu markierende Box und einen der aktuell zu markierenden Box entsprechenden ersten Videorahmen zu erhalten, wobei der der aktuell zu markierenden Box entsprechende erste Videorahmen ein erkannter Videorahmen, der ein der aktuell zu markierenden Box entsprechendes Ziel enthält und der eine erste Filterbedingung erfüllt, in einem zu markierenden Video ist;
• ein zweites Modul 420 zum Erhalten, das konfiguriert ist, um Empfehlungsboxen mit vermuteter Verknüpfungsbeziehung zu der aktuell zu markierenden Box und einen jeder Empfehlungsbox entsprechenden zweiten Videorahmen zu erhalten, wobei der jeder Empfehlungsbox entsprechende zweite Videorahmen ein erkannter Videorahmen, der ein der Empfehlungsbox entsprechendes Ziel enthält und der eine zweite Filterbedingung erfüllt, in dem zu markierenden Video ist;
• ein erstes Anzeigemodul 430, das konfiguriert ist, um den ersten Videorahmen, der die aktuell zu markierenden Box enthält, und den jeder Empfehlungsbox entsprechenden zweiten Videorahmen, der die Empfehlungsbox enthält, anzuzeigen, so dass ein Benutzer basierend auf dem jeder Empfehlungsbox entsprechenden zweiten Videorahmen eine Ziel-Empfehlungsbox, die in Verknüpfungsbeziehung mit der aktuell zu markierenden Box steht, aus den angezeigten Empfehlungsboxen bestimmt;
• ein erstes Bestimmungs- und Herstellungsmodul 440, das konfiguriert ist, um basierend auf einer ersten Verknüpfungsbeziehung-Operationsinformation des Benutzers aus allen Empfehlungsboxen zu bestimmen, ob eine Ziel-Empfehlungsbox, die in Verknüpfungsbeziehung mit der aktuell zu markierenden Box steht, vorliegt, und um beim Vorliegen einer Ziel-Empfehlungsbox, die in Verknüpfungsbeziehung mit der aktuell zu markierenden Box steht, eine Verknüpfungsbeziehung zwischen der aktuell zu markierenden Box und der Ziel-Empfehlungsbox herzustellen.

According to the above embodiments of the method, the following embodiments of the present invention provide an apparatus for linking goals, wherein, as in 4 shown, the device includes:

• a first obtaining module 410 configured to obtain a box currently to be marked and a first video frame corresponding to the current box to be marked, wherein the first video frame corresponding to the current box to be marked is a recognized video frame that is one of the current to be marked Box contains corresponding target and which satisfies a first filter condition is in a video to be tagged;
• a second obtaining module 420 configured to obtain recommendation boxes with suspected association relationship to the box currently to be marked and a second video frame corresponding to each recommendation box, wherein the second video frame corresponding to each recommendation box is a detected video frame containing a target corresponding to the recommendation box and which satisfies a second filter condition, is in the video to be marked;
• a first display module 430 configured to display the first video frame containing the current box to be marked and the second video frame corresponding to each recommendation box containing the recommendation box so that a user based on the second video frame corresponding to each recommendation box can display a target recommendation box which is in linking relationship with the box to be currently marked, determined from the displayed recommendation boxes;
• a first determining and producing module 440 configured to determine whether there is a target recommendation box in linking relationship with the box to be currently marked based on first linking relationship operation information of the user from all of the recommendation boxes, and to at presence of a target recommendation box which is in linking relationship with the box to be currently marked, establishing a linking relationship between the box to be currently marked and the target recommendation box.

By applying the embodiments of the present invention, the box currently to be marked and its corresponding first video frame, as well as recommendation boxes with presumed linking relation to the box currently to be marked and a second video frame corresponding to each recommendation box can be obtained and then displayed to show the linking of boxes, which presumably relate to the same target, in video frames to be tagged at the picture level. By directly obtaining and displaying the recommendation boxes with presumed linking relationship to the current box to be marked and the second video frame corresponding to each recommendation box, the time for the user to play and watch the video to be marked is saved, with the process of finding the linking boxes is highly optimized for the user through the calculation of the recommendation boxes, increasing the marking efficiency for boxes to be marked which are in linking relationship with each other, that is, the efficiency for linking targets is increased, realizing convenient and effective linking of the same target that appears in each video frame of the video.

In a further exemplary embodiment of the present invention, it is provided that the recognized video frame, which contains the target corresponding to the box currently to be marked and which satisfies a first filter condition, is the recognized last video frame in a video to be marked, which corresponds to the box currently to be marked destination contains is in the video to be tagged;
wherein the detected video frame, which contains a target corresponding to the recommendation box and satisfies the second filter condition, in a video to be marked is a detected first video frame, which contains the target corresponding to the recommendation box, in the video to be marked.
In a further exemplary embodiment of the present invention, it is provided that the first module 410 is specifically configured to obtain
to determine, from among unmarked boxes corresponding to video to be marked, an unmarked box having the earliest detection time represented by timestamp information corresponding to an end video frame as the current box to be marked;
or to designate the recommendation box selected by the user as the current box to be marked when it is detected that for historically displayed boxes to be marked the user selects a recommendation box associated with those from among the corresponding displayed recommendation boxes.

In another embodiment of the present invention, it is contemplated that the second module 420 is specifically configured to obtain to: iterate through pre-marked results corresponding to N video frames after the ending video frame corresponding to the current box to be marked, determine a recognition box whose box ID satisfies a predetermined recommendation condition, from recognition boxes that correspond to the N video frames after the end video frame corresponding to the box currently to be marked, as a recognition box with a presumed linkage relationship to the box currently to be marked, wherein the recognition box whose box identifier meets a predetermined recommendation condition, a includes detection box whose box identifier information does not appear in the ending video frame corresponding to the current box to be marked and previous video frames;

Determine, for each recommendation box with presumed linking relation to the current box to be marked, a video frame containing the target corresponding to the recommendation box and satisfying the second filter operation, from the N video frames after the end video frame corresponding to the current box to be marked as that of the recommendation box corresponding second video frame, wherein the video frame containing the target corresponding to the recommendation box and satisfying the second filter operation comprises: the detected first video frame containing the target corresponding to the recommendation box in the video to be tagged.

In a further exemplary embodiment of the present invention, it is provided that the device further comprises: an assessment module (not shown in the figure), which is configured to corresponding second video frame” to judge whether a display function for displaying a recommendation box corresponding to the box to be currently marked is activated;
with an activated display function, the second module 420 is triggered to obtain;
a third receiving module (not shown in the figure) configured to receive, with the display function inactivated, M video frames after the end video frame corresponding to the current box to be marked, in the video to be marked, as video frames to be played, which the correspond to the current box to be marked, wherein the ending video frame corresponding to the current box to be marked is: a detected ending video frame containing the target corresponding to the current box to be marked in the marking video;
a second display module (not shown in the figure) configured to display the first video frame containing the box currently to be marked and to play the video frames to be played that correspond to the box currently to be marked, so that the user has a target - recognition box which is in link relation with the box currently to be marked, from the recognition boxes contained by the video frames to be played corresponding to the box currently to be marked;
a second determination and production module (not shown in the figure) configured to determine, based on a user's second linking relationship operation information, from the recognition boxes contained by the video frames to be played corresponding to the box currently to be marked, whether a target recognition box in linking relationship with the box currently to be marked exists, wherein in case of the presence of a target recognition box in linking relationship with the box to be marked currently, it is determined that a linking relationship between the currently box to be marked and the target recognition box.

In another embodiment of the present invention, it is provided that the device further comprises: an enlarged display module (not shown in the figure) configured to all recommendation boxes whether there is a target recommendation box that is in linking relationship with the box currently to be marked, and establishing, if there is a target recommendation box that is in linking relationship with the box currently to be marked, a linking relationship between the currently to be marked Box and the target recommendation box" to perform the following operations: enlarged displaying a first recommendation box and a second video frame corresponding to the first recommendation box when a first enlargement command issued by the user for the first recommendation box with presumed link relationship to the box to be currently marked is detected, wherein triggering operations of the first zoom-in command include: right-clicking the second video frame corresponding to the first recommendation box;
and/or displaying enlarged a first video frame corresponding to the current box to be marked when a second enlarge command initiated by the user for the first video frame corresponding to the current box to be marked is recognized.
In a further embodiment of the present invention it is provided that the device further comprises: a first determination module (not shown in the figure) configured to after "determining, based on a user's first linking relationship operation information, from all of the recommendation boxes whether there is a target recommendation box that is in linking relationship with the box to be currently marked, and establishing, when there is a target recommendation box that is in linking relationship with the box currently to be marked, a linking relationship between the box currently to be marked and the target recommendation box” to designate the target recommendation box as a new box currently to be marked.
a second determination module (not shown in the figure) configured to determine a video frame, which contains a target corresponding to the new box currently to be marked and which satisfies the first filter condition, from the video to be marked as a first video frame that corresponds to the new current box to be marked;
a fourth receiving module (not shown in the figure) configured to receive new recommendation boxes with presumed linkage relationship to the new box currently to be marked and a second video frame corresponding to each new recommendation box, the second video frame corresponding to each new recommendation box detected video frame that contains a target corresponding to the new recommendation box and that satisfies a second filter condition is in the video to be tagged;
a third display module (not shown in the figure) configured to display the first video frame containing the new box currently to be marked and the second video frame corresponding to each new recommendation box containing the new recommendation box, so that the user based on the second video frame corresponding to each new recommendation box, determining a new target recommendation box associated with the new current box to be marked from the displayed new recommendation boxes;
a fourth display module (not shown in the figure) configured to display, upon detection by the user of a jump command specifying a jump to the previous paragraph, the first video frame containing the box currently to be marked and the second video frame , corresponding to each recommendation box and containing that recommendation box.

• ein fünftes Modul zum Erhalten (in der Figur nicht gezeigt), das konfiguriert ist, um vor dem „Anzeigen des ersten Videorahmens, der die aktuell zu markierenden Box enthält, und
• des jeder Empfehlungsbox entsprechenden zweiten Videorahmens, der die Empfehlungsbox enthält“ eine Rahmennummer-Information des ersten Videorahmens, die der aktuell zu markierenden Box entspricht, und eine Boxkennungsinformation, die der aktuell zu markierenden Box entspricht, zu erhalten;
• ein sechstes Modul zum Erhalten (in der Figur nicht gezeigt), das konfiguriert ist, um eine Rahmennummer-Information des zweiten Videorahmens, der jeder Empfehlungsbox mit vermuteter Verknüpfungsbeziehung zu der aktuell zu markierenden Box entspricht, und eine Boxkennungsinformation, die jeder Empfehlungsbox entspricht, zu erhalten;
• wobei das erste Anzeigemodul 430 spezifisch konfiguriert ist, um folgende Vorgänge durchzuführen: Anzeigen des ersten Videorahmens, der die aktuell zu markierenden Box enthält, der Rahmennummer-Information des ersten Videorahmens, die der aktuell zu markierenden Box entspricht und der Boxkennungsinformation, die der aktuell zu markierenden Box entspricht, in einem ersten Anzeigebereich einer ersten voreingestellten Anzeigeschnittstelle;
• und Anzeigen des zweiten Videorahmens, der jeder Empfehlungsbox mit vermuteter Verknüpfungsbeziehung zu der aktuell zu markierenden Box entspricht, der Rahmennummer-Information des zweiten Videorahmens und der Boxkennungsinformation, die jeder Empfehlungsbox entspricht, in einem zweiten Anzeigebereich der ersten voreingestellten Anzeigeschnittstelle.

In a further exemplary embodiment of the present invention, it is provided that the device further comprises:

• a fifth receiving module (not shown in the figure) configured to, prior to "displaying the first video frame containing the box currently to be marked, and
• of the second video frame corresponding to each recommendation box containing the recommendation box” to obtain frame number information of the first video frame corresponding to the box to be marked currently and box ID information corresponding to the box to be marked currently;
• a sixth obtaining module (not shown in the figure) configured to receive frame number information of the second video frame corresponding to each recommendation box with presumed linking relation to the box currently to be marked, and box identifier information corresponding to each recommendation box receive;
• wherein the first display module 430 is specifically configured to perform the following operations: displaying the first video frame containing the current box to be marked, the frame number information of the first video frame corresponding to the current box to be marked, and the box identification information corresponding to the current to corresponding to the box marked in a first display area of a first default display interface;
• and displaying the second video frame corresponding to each recommendation box assumed to be linked to the box currently to be marked, the frame number information of the second video frame, and the box ID information corresponding to each recommendation box in a second display area of the first preset display interface.

In an embodiment of the present invention, it is provided that a third display area of the first default display interface further comprises a first trigger area for a skip command to skip to the previous paragraph and a second trigger area for a skip command to skip to the next paragraph, the skip command to skip to the previous paragraph Paragraph is a command that directs the display of the first video frame corresponding to a last box to be marked before the current box to be marked, where the first video frame corresponding to a last box to be marked before the current box to be marked is a recognized video frame that has a target corresponding to the last box to be marked before the current box to be marked and satisfies the first filter condition is in the video to be marked; wherein the jump command to jump to the next paragraph is a command that directs the display of the first video frame corresponding to a next box to be marked after the current box to be marked, the next box to be marked after the current box to be marked being an unmarked box , whose timestem corresponding to the corresponding end frame video frame pelinformation represents the earliest capture time and is later than the capture time of the current box to be marked from unmarked boxes corresponding to the video to be marked.

The above exemplary embodiments of the device and the system correspond to the exemplary embodiments of the method and have the same technical effects as the exemplary embodiments of the method. For a specific description, reference is made to the exemplary embodiments of the method. The embodiments of the device are obtained based on the embodiments of the method. For a specific description, reference is made to the portion of the method embodiments not repeated here. Those of ordinary skill in the art can understand that the figure represents only a schematic view of an embodiment, the modules or processes in the figure are not strictly necessary to implement the present invention.

A person of ordinary skill in the art can understand that the modules of the device in the embodiments can be distributed according to the description of the embodiments in the device in the embodiments, or that they can also undergo a corresponding change and thus in one or more devices other than the device according to the present embodiment can be arranged. The modules of the above exemplary embodiment can be combined into one module or further divided into a number of sub-modules.

Finally, it should be noted that the above embodiments are for explanation rather than limitation of the technical aspects of the present invention. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should appreciate that they can still modify the technical configurations of the above embodiments or make equivalent substitutions for some or all of the technical features thereof. However, these modifications or substitutions do not cause the essence of the respective technical configurations to depart from the spirit and scope of the technical configurations of the embodiments of the present invention.

Claims (10)

A method of linking goals, characterized in that the method comprises: obtaining a box currently to be marked and a first video frame corresponding to the current box to be marked, wherein the first video frame corresponding to the current box to be marked is a recognized video frame which is one of the currently to contains a target corresponding to the marked box and which satisfies a first filter condition, is in a video to be marked; Obtaining recommendation boxes with presumed linking relationship to the box currently to be marked and a second video frame corresponding to each recommendation box, wherein the second video frame corresponding to each recommendation box is a recognized video frame containing a target corresponding to the recommendation box and satisfying a second filter condition in the video to be marked is; Displaying the first video frame containing the box currently to be marked and the second video frame corresponding to each recommendation box containing the recommendation box, so that a user based on the second video frame corresponding to each recommendation box selects a target recommendation box that is in linking relationship with the current to marked box is determined from the displayed recommendation boxes; determining, based on user's first linking relationship operation information, from all of the recommendation boxes, whether there is a target recommendation box in linking relationship with the box to be currently marked, and producing, when there is a target recommendation box in linking relationship with the box currently to be marked, a linking relationship between the box currently to be marked and the target recommendation box. procedure after claim 1 , characterized in that the recognized video frame, which contains the target corresponding to the current box to be marked and which satisfies a first filter condition, in a video to be marked, the recognized last video frame, which contains the target corresponding to the current box to be marked, in the to be marked video is; wherein the detected video frame, which contains a target corresponding to the recommendation box and satisfies the second filter condition, in a video to be marked is a detected first video frame, which contains the target corresponding to the recommendation box, in the video to be marked. procedure after claim 1 , characterized in that the step of obtaining the current box to be marked in the video to be marked is realized by one of the following two implementation forms: first implementation form: from unmarked boxes corresponding to the to be marked correspond to video, an unmarked box with the earliest capture time represented by timestamp information corresponding to an end video frame is determined as the current box to be marked; second form of implementation: if it is recognized that for displayed historical boxes to be marked, the user selects an associated recommendation box from the corresponding displayed recommendation boxes, the recommendation box selected by the user is determined as the current box to be marked. procedure after claim 1 , characterized in that the step "obtaining recommendation boxes with presumed linkage relationship to the box currently to be marked and a second video frame corresponding to each recommendation box" comprises: traversing through pre-marked results corresponding to N video frames after the end video frame corresponding to the box currently to be marked , Determining a recognition box whose box identifier satisfies a predetermined recommendation condition from recognition boxes that correspond to the N video frames after the end video frame corresponding to the box currently to be marked as a recognition box with a presumed linkage relationship to the box currently to be marked, the recognition box whose box identifier satisfies a predetermined recommendation condition, comprises a recognition box whose box identifier information does not appear in the ending video frame corresponding to the current box to be marked and previous video frames; Determine, for each recommendation box with presumed linking relation to the current box to be marked, a video frame containing the target corresponding to the recommendation box and satisfying the second filter operation, from the N video frames after the end video frame corresponding to the current box to be marked as that of the recommendation box corresponding second video frame, wherein the video frame containing the target corresponding to the recommendation box and satisfying the second filter operation comprises: the detected first video frame containing the target corresponding to the recommendation box in the video to be tagged. procedure after claim 1 , characterized in that the method before the step "obtaining recommendation boxes with presumed linkage relationship to the box currently to be marked and a second video frame corresponding to each recommendation box" further comprises: determining whether a display function for displaying the recommendation box, that of the box currently to be marked corresponds, is activated; if activated, the step "obtaining recommendation boxes with presumed link relationship to the current box to be marked and a second video frame corresponding to each recommendation box" is performed; if not checked, M video frames are obtained as video frames to play corresponding to the box currently to be marked after the end video frame corresponding to the box currently to be marked in the video to be marked, where the end video frame corresponding to the box currently to be marked box corresponds to is: a detected ending video frame containing the target corresponding to the current box to be boxed in the marking video; Displaying the first video frame containing the box currently to be marked and playing the video frames to be played that correspond to the box currently to be marked, so that the user selects a target recognition box, which is linked to the box currently to be marked, from the recognition boxes contained by the video frames to be played corresponding to the current box to be marked; Determining, based on a user's second linking relationship operation information, from the recognition boxes contained by the video frames to be played corresponding to the box currently to be marked, whether there is a target recognition box that is in linking relationship with the box currently to be marked, wherein in case the presence of a target recognition box which is in linking relationship with the box to be currently marked is determined, a linking relationship is established between the box to be currently marked and the target recognition box. Procedure according to one of Claims 1 until 5 , characterized in that the method before the step "determining, based on a first linking relationship operation information of the user, from all the recommendation boxes, whether there is a target recommendation box that is in linking relationship with the box currently to be marked, and producing, at Presence of a target recommendation box which is in linking relationship with the box currently to be marked, a linking relationship between the box currently to be marked and the target recommendation box” includes: Enlarged displaying a first recommendation box and a second video frame corresponding to the first recommendation box when a first Zoom-in command triggered by the user for the first recommendation box with presumed link relationship to the current box to be marked is recognized, wherein triggering operations of the first zoom-in command include: right-clicking on the second video frame corresponding to the first recommendation box; and/or displaying enlarged a first video frame corresponding to the box currently to be marked when a second enlarge command initiated by the user for the first video frame corresponding to the box currently to be marked is detected. procedure after claim 6 , characterized in that the method after the step "determining, based on a first linking relationship operation information of the user, from all recommendation boxes, whether there is a target recommendation box, which is in linking relationship with the box currently to be marked, and producing, when existence of a target recommendation box which is in linking relationship with the box currently to be marked, a linking relationship between the box currently to be marked and the target recommendation box” comprises: determining the target recommendation box as a new box currently to be marked; determining a video frame containing a target corresponding to the new box currently to be marked and satisfying the first filter condition from the video to be marked as a first video frame corresponding to the new box currently to be marked; Obtaining new recommendation boxes with a presumed linkage relationship to the new box currently to be marked and a second video frame corresponding to each new recommendation box, the second video frame corresponding to each new recommendation box being a recognized video frame which contains a target corresponding to the new recommendation box and which satisfies a second filter condition, is in the video to be tagged; Displaying the first video frame containing the new box currently to be marked and the second video frame corresponding to each new recommendation box containing the new recommendation box, so that based on the second video frame corresponding to each new recommendation box the user selects a new target recommendation box to be Linking relationship with the new box currently to be marked is determined from the displayed new recommendation boxes; Displaying the first video frame containing the box currently to be marked and the second video frame corresponding to each recommendation box and containing that recommendation box when it is detected that the user triggers a jump command specifying a jump to the previous paragraph. Procedure according to one of Claims 1 until 6 , characterized in that before the step of "displaying the first video frame containing the box currently to be marked and the second video frame corresponding to each recommendation box containing the recommendation box", the method further comprises: obtaining frame number information of the first video frame that corresponds to the current box to be marked, and box identification information corresponding to the current box to be marked; obtaining frame number information of the second video frame corresponding to each recommendation box assumed to be linked to the box currently to be marked and box identification information corresponding to each recommendation box; wherein the step of "displaying the first video frame containing the box currently to be marked and the second video frame corresponding to each recommendation box containing the recommendation box" comprises: displaying the first video frame containing the box currently to be marked, the frame number information of the first video frame corresponding to the box currently to be marked and the box identifier information corresponding to the box currently to be marked in a first display area of a first preset display interface; and displaying the second video frame corresponding to each recommendation box assumed to be linked to the box currently to be marked, the frame number information of the second video frame, and the box ID information corresponding to each recommendation box in a second display area of the first preset display interface. Procedure according to one of Claims 1 until 6 , characterized in that a third display area of the first default display interface further comprises a first trigger area for a skip command to skip to the previous paragraph and a second trigger area for a skip command to skip to the next paragraph, the skip command to skip to the previous paragraph being a command, which instructs the display of the first video frame corresponding to a last box to be marked before the current box to be marked, the first video frame corresponding to a last box to be marked before the current box to be marked being a detected video frame corresponding to one of the last has a target corresponding to the box to be marked before the current box to be marked and satisfies the first filter condition in which is the video to be marked; wherein the jump command to jump to the next paragraph is a command that directs the display of the first video frame corresponding to a next box to be marked after the current box to be marked, the next box to be marked after the current box to be marked being an unmarked box , whose time corresponding to the corresponding end frame video frame stamp information represents the earliest capture time and is later than the capture time of the current box to be marked from unmarked boxes corresponding to the video to be marked. A device for linking goals, characterized in that the device comprises: a first receiving module configured to receive a box currently to be marked and a first video frame corresponding to the box currently to be marked, that of the box currently to be marked corresponding first video frame is a detected video frame that contains a target corresponding to the current box to be marked and that satisfies a first filter condition in a video to be marked; a second obtaining module configured to obtain recommendation boxes with presumed linkage relationship to the box currently to be marked and a second video frame corresponding to each recommendation box, wherein the second video frame corresponding to each recommendation box is a detected video frame containing a target corresponding to the recommendation box and that satisfies a second filter condition is in the video to be tagged; a first display module configured to display the first video frame containing the box currently to be marked and the second video frame corresponding to each recommendation box containing the recommendation box so that a user based on the second video frame corresponding to each recommendation box provides a target -Recommendation box which is in link relation with the box currently to be marked determined from the displayed recommendation boxes; a first determining and producing module configured to determine whether there is a target recommendation box linked in linking relationship with the box currently to be marked, based on first linking relationship operation information of the user from all of the recommendation boxes, and if present a target recommendation box which has a linking relationship with the box to be currently marked to establish a linking relationship between the box to be currently marked and the target recommendation box.

Images