CN111738053B - Tracking object determination method and device and handheld camera - Google Patents

Tracking object determination method and device and handheld camera Download PDF

Info

Publication number
CN111738053B
CN111738053B CN202010297067.1A CN202010297067A CN111738053B CN 111738053 B CN111738053 B CN 111738053B CN 202010297067 A CN202010297067 A CN 202010297067A CN 111738053 B CN111738053 B CN 111738053B
Authority
CN
China
Prior art keywords
information
tracking
image frame
recognition
matching
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202010297067.1A
Other languages
Chinese (zh)
Other versions
CN111738053A (en
Inventor
霍磊
梁峰
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shanghai Moxiang Network Technology Co ltd
Original Assignee
Shanghai Moxiang Network Technology Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shanghai Moxiang Network Technology Co ltd filed Critical Shanghai Moxiang Network Technology Co ltd
Priority to CN202010297067.1A priority Critical patent/CN111738053B/en
Priority to PCT/CN2020/099830 priority patent/WO2021208253A1/en
Publication of CN111738053A publication Critical patent/CN111738053A/en
Application granted granted Critical
Publication of CN111738053B publication Critical patent/CN111738053B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/10—Terrestrial scenes
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/22—Matching criteria, e.g. proximity measures
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G06T7/246—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/10—Image acquisition
    • G06V10/12—Details of acquisition arrangements; Constructional details thereof
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10016—Video; Image sequence

Abstract

The embodiment of the application provides a method and equipment for determining a tracked object and a handheld camera, wherein the method for determining the tracked object comprises the following steps: performing image recognition on the current shot image frame to obtain recognition information for identifying a recognition object in the current shot image frame; matching the recognition object with the tracking object according to the recognition information and tracking information for identifying the tracking object in the shot image frame to obtain matching information; and updating the tracking information according to the matching information. Therefore, in the multi-target tracking shooting process, the current shooting image frame can be compared with the previous image frame, the matching relation between the identification object in the current shooting image frame and the tracking object in the shot image frame is searched, the tracking information of the tracking object is updated, the tracking state of the tracking object is updated, and the tracking object can be managed and maintained in real time and accurately.

Description

Tracking object determination method and device and handheld camera
Technical Field
The embodiment of the application relates to the technical field of image recognition, in particular to a method and equipment for determining a tracked object and a handheld camera.
Background
Computer vision refers to a simulation of biological vision using a computer and associated equipment. The main task of the system is to process the acquired pictures or videos to obtain three-dimensional information of corresponding scenes. The target detection and tracking is an important branch in the field of computer vision, and is widely applied to the fields of military guidance, visual navigation, robots, intelligent transportation, public safety and the like.
With the development of vision processing technology and artificial intelligence technology, the handheld intelligent camera can also track the target to be shot by applying target detection and tracking technology. However, compared with an industrial camera, the field of view of a handheld intelligent camera is relatively limited, and the position needs to be continuously converted to obtain a panoramic image; moreover, the multi-target tracking management method of the handheld intelligent camera is imperfect, so that when the number of the targets to be tracked is large or the types of the targets to be tracked are large, the tracking information for tracking the targets to be photographed cannot be updated in time, and the situations of tracking failure and recognition failure occur.
Disclosure of Invention
In view of the above, an object to be tracked is determined by a method, a device and a handheld camera, so as to overcome a defect that tracking information for tracking a target to be photographed cannot be updated in time in the prior art.
The embodiment of the application provides a method for determining a tracked object, which comprises the following steps: performing image recognition on the current shot image frame to obtain recognition information for identifying a recognition object in the current shot image frame; matching the recognition object with the tracking object according to the recognition information and tracking information for identifying the tracking object in the shot image frame to obtain matching information; and updating the tracking information according to the matching information.
An embodiment of the present application provides a tracking object determining apparatus, including: the device comprises a memory, a processor and a video collector, wherein the video collector is used for collecting a target to be tracked in a target area; the memory is used for storing program codes; the processor, invoking the program code, when executed, is configured to: performing image recognition on the current shot image frame to obtain recognition information for identifying a recognition object in the current shot image frame; matching the recognition object with the tracking object according to the recognition information and tracking information for identifying the tracking object in the shot image frame to obtain matching information; and updating the tracking information according to the matching information.
An embodiment of the present application provides a handheld camera, including the tracking object determining device in the above embodiment, further including: the carrier is fixedly connected with the video collector and used for carrying at least one part of the video collector.
In the embodiment of the application, firstly, image recognition is carried out on a current shooting image frame to obtain recognition information used for identifying a recognition object in the current shooting image frame; then matching the recognition object with the tracking object according to the recognition information and the tracking information for identifying the tracking object in the shot image frame to obtain matching information; and updating the tracking information according to the matching information. In the multi-target tracking shooting process, the current shooting image frame can be compared with the previous image frame, the tracking information of the tracking object is updated by searching the matching relation between the identification object in the current shooting image frame and the tracking object in the shot image frame, and the tracking state of the tracking object is updated, so that the tracking object can be managed and maintained in a more real-time and accurate manner.
Drawings
Some specific embodiments of the present application will be described in detail hereinafter by way of illustration and not limitation with reference to the accompanying drawings. The same reference numbers in the drawings identify the same or similar elements or components. Those skilled in the art will appreciate that the drawings are not necessarily drawn to scale. In the drawings:
fig. 1 is a schematic flowchart of a method for determining a tracked object according to an embodiment of the present application;
fig. 2 is a schematic flowchart of a tracked object determining method provided in the second embodiment of the present application;
fig. 3 is a schematic flowchart of a method for determining a tracked object according to a third embodiment of the present application;
fig. 4 is a block diagram of a tracked object determining apparatus according to a fourth embodiment of the present application;
fig. 5-7 are schematic structural block diagrams of a handheld camera provided in the fifth embodiment of the present application.
Detailed Description
The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. As used in this specification and the appended claims, the singular forms "a", "an", and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It should also be understood that the term "and/or" as used herein refers to and includes any and all possible combinations of one or more of the associated listed items.
It should be understood that the terms "first," "second," and the like as used in the description and in the claims, do not denote any order, quantity, or importance, but rather are used to distinguish one element from another. Also, the use of the terms "a" or "an" and the like do not denote a limitation of quantity, but rather denote the presence of at least one.
In recent years, target detection and tracking systems are one direction in which the field of computer vision has been rapidly developed in recent years. With the development of visual processing technology and artificial intelligence technology, the handheld intelligent camera can be used for tracking a target to be shot, and performing object recognition and scene recognition according to the target, so that a user can classify and manage shot photos or videos and perform subsequent automatic processing conveniently. However, compared with an industrial camera, the field of view of a camera of a household handheld intelligent camera is often limited, a panoramic image needs to be obtained by constantly changing positions, once the number of targets is large, the types of the targets are large and the targets are scattered, the small field of view cannot well support the requirement of real-time multi-target tracking, when a plurality of types and/or a plurality of types of objects occur, tracking information of the targets to be shot cannot be updated and tracked in time, and error conditions such as tracking failure and recognition failure are caused.
In view of the deficiencies in the above technical solutions, the method for determining a tracking object in the technical solution provided in the embodiments of the present application improves user experience.
The following further describes specific implementation of the embodiments of the present invention with reference to the drawings.
Example one
An embodiment of the present application provides a method for determining a tracked object, as shown in fig. 1, where fig. 1 is a schematic flowchart of the method for determining a tracked object provided in the embodiment of the present application.
The tracking object determining method of the embodiment includes:
s101, carrying out image recognition on the current shooting image frame, and obtaining recognition information used for identifying a recognition object in the current shooting image frame.
In this embodiment, the identification information is used to identify an identification result corresponding to an identification object identified by using a preset image identification algorithm, and the specific information content and recording manner included in the identification information are not limited, and the type of the image identification algorithm is not limited.
For example, the identification information may include color feature information, depth feature information, position feature information, and the like of the identification object. As another example, the image recognition algorithm may be RCNN, SSD, YOLO, or the like.
And S102, matching the recognition object with the tracking object according to the recognition information and the tracking information for identifying the tracking object in the shot image frame to obtain matching information.
In the present embodiment, the captured image frame is one or more consecutive image frames prior to the current captured image frame.
In this embodiment, the tracking target is an object obtained by identifying and marking at least one captured image frame by using a preset image identification algorithm; the tracking information is used for identifying the recognition result of a tracking target by a preset image recognition algorithm and the tracking state of the tracking target in one or more continuous shot image frames before the current shot image frame.
In this embodiment, the matching information is at least used to indicate whether the recognition object and the tracking object are matched. When the identification object is matched with the tracking object, the identification object and the tracking object are the same object, or the identification object and the tracking object are the same object, the possibility is extremely high, and the tracking object can be considered to appear in the current shooting image frame; when the recognition object does not match the tracking object, indicating that the recognition object and the tracking object are likely not the same object, it may be considered that the tracking object does not appear in the currently captured image frame.
In this embodiment, the identification information and the tracking information are identification results obtained by identifying the image frame by using a preset image identification algorithm, so that the identification object and the tracking object can be matched according to the identification information and the tracking information, and the adopted matching algorithm is not limited in this embodiment.
And S103, updating the tracking information according to the matching information.
In this embodiment, after the image recognition of the current captured image frame is completed, the tracking state corresponding to the tracking target may change, and it may be determined whether the tracking target appears in the current captured image frame according to the matching information, so that the tracking information corresponding to the tracking target may be updated according to the matching information, so that the tracking information updated by the tracking target may be used in an image frame after the current captured image frame.
As can be seen from the above embodiments, the method for determining a tracked object of the present application performs image recognition on a current captured image frame to obtain recognition information for identifying a recognition object in the current captured image frame; then matching the recognition object with the tracking object according to the recognition information and the tracking information for identifying the tracking object in the shot image frame to obtain matching information; and updating the tracking information according to the matching information. Therefore, in the multi-target tracking shooting process, the current shooting image frame can be compared with the last image frame, the tracking state of the tracking target is updated, and the management of the tracking target is accurately carried out in real time.
Example two
A second embodiment of the present application provides a method for determining a tracked object, as shown in fig. 2, and fig. 2 is a schematic flowchart of the method for determining a tracked object provided in the second embodiment of the present application.
The tracking object determining method of the embodiment includes:
s201, carrying out image recognition on the current shooting image frame, and obtaining recognition information used for identifying a recognition object in the current shooting image frame.
In this embodiment, step S201 is the same as step S101 in the first embodiment, and is not described herein again.
S202, matching the recognition object and the tracking object according to the recognition information and the tracking information used for identifying the tracking object in the shot image frame to obtain matching information.
In this embodiment, in order to obtain more accurate matching information, step S202 may include:
s202a, obtaining similarity information for identifying a degree of similarity between the recognition object and the tracking object and position information for identifying a positional relationship between the recognition object and the tracking object, based on the recognition information and the tracking information.
The similarity information is used to indicate the similarity between the recognition object and the tracking object, and the calculation method of the similarity is not limited.
For example, the image similarity algorithm may be a histogram matching algorithm, a manhattan distance algorithm, a chebyshev distance algorithm, a cosine distance algorithm, a pearson correlation coefficient algorithm, a hamming distance algorithm, a jaccard distance algorithm, a braekhitis distance algorithm, a mahalanobis distance algorithm, a JS divergence algorithm, or the like, and the obtained similarity may be at least one of a similarity of color features, a similarity of feature points, and a similarity of shapes of the recognition object and the tracking object.
Optionally, in order to obtain a more accurate similarity calculation result, when the histogram matching algorithm is used to calculate the similarity between the identification object and the tracking object, the color histograms of the identification object and the tracking object may be extracted first; then, the distance (such as the Papanicolaou distance, the intersection distance of the histograms and the like) between the two color histograms is calculated, and the similarity information of the identification object and the tracking object is obtained according to the distance. The color histogram is used to describe the proportion of different colors in the whole image, and does not care the spatial position of each color.
In this embodiment, the calculation method of the position relationship between the recognition object and the tracking object is not limited.
Optionally, in order to determine the position relationship between the recognition object and the tracking object more accurately, the position information includes intersection comparison sub-information and distance sub-information, where the intersection comparison sub-information is used to identify an intersection ratio corresponding to the recognition object and the tracking object, and the distance sub-information is used to identify a distance between the recognition object and the tracking object.
The cross-over ratio may measure the degree of overlap of the recognition object and the tracking object. For example, when the intersection ratio is a value between 0 and 1, the intersection ratio represents the overlapping degree of the identification object and the tracking object, and the higher the value of the intersection ratio, the higher the overlapping degree of the identification object and the tracking object is; when the intersection ratio is 0, the identification object and the tracking object are not overlapped; when the cross-over ratio is 1, the recognition object and the tracking object are completely overlapped.
Alternatively, the intersection region area and the union region area of the identification object and the tracking object in the image frame may be obtained first, and then the ratio of the intersection region area to the union region area may be calculated, thereby obtaining the corresponding intersection ratio of the identification object and the tracking object.
Alternatively, the distance between the recognition object and the tracking object may be a distance between a center point of the tracking object in the image frame and a center point of the recognition object in the image frame.
S202b, matching the recognition object and the tracking object according to the similarity information and the position information to obtain matching information.
In this embodiment, in order to more accurately match the tracked object, the identification object and the tracked object may be matched together according to the similarity information and the position information.
In this embodiment, step S202 may further include: and matching the identification object and the tracking object by sequentially using a greedy algorithm and a Hungary algorithm according to the identification information and the tracking information to obtain matching information.
Wherein the greedy algorithm always makes the selection that seems best at the present time when matching. In the matching process, only one identification object and the tracking object are considered to be matched in each step, or only one tracking object and the identification object are considered to be matched in each step, and each step is required to ensure that a local optimal solution can be obtained. Until all data is enumerated. The Hungarian algorithm is a combinatorial optimization algorithm for solving a task allocation problem in polynomial time.
The greedy algorithm and the Hungarian algorithm are sequentially used for matching the identification object and the tracking object, so that the similarity and the position information of the tracking object and the identification object can be comprehensively considered, and the accuracy of matching the identification object and the tracking object can be further improved by adopting the two algorithms.
And S203, updating the tracking information according to the matching information.
In this embodiment, since the identification object and the tracking object may be successfully or unsuccessfully matched, in order to effectively utilize the matching information and improve the effect of tracking and shooting multiple targets, step S203 may further include:
s203a, when the identified object matches with the tracking object, updating the tracking information of the tracking object.
And S203b, when the identification object does not match with the tracking object, determining the identification object as a new tracking object.
As can be seen from the above embodiments, the method for determining the tracked object of the present application can comprehensively consider the similarity information and the location information between the identified object and the tracked object, and/or match the identified object and the tracked object by using the greedy algorithm and the hungarian algorithm, so that the identified object and the tracked object can be more accurately matched; and a subsequent processing scheme is set for two different matching results of the identification object and the tracking object, so that the tracking object can be more effectively managed.
EXAMPLE III
A third embodiment of the present application provides a method for determining a tracked object, as shown in fig. 3, and fig. 3 is a schematic flowchart of the method for determining a tracked object provided in the third embodiment of the present application.
The tracking object determining method of the embodiment includes:
s301, image recognition is carried out on the current shooting image frame, and recognition information used for identifying a recognition object in the current shooting image frame is obtained.
In this embodiment, step S301 is the same as step S101 in the first embodiment, and is not described herein again.
S302, matching the recognition object with the tracking object according to the recognition information and the tracking information for identifying the tracking object in the shot image frame to obtain matching information.
In this embodiment, step S302 is the same as step S102 in the first embodiment or step S202 in the second embodiment, and is not repeated here.
And S303, updating the tracking information according to the matching information.
In the embodiment, in order to more accurately determine the tracking target of the multi-target shooting, the overlapping condition of the tracking target and other targets in the image frame can be further considered when the tracking information is updated. Specifically, step S303 may include:
s303a, obtaining the overlay information according to the identification information and the tracking information.
The overlapping information is used for identifying the image overlapping degree of the tracking object and other objects in the previous captured image frame of the current captured image frame and the image overlapping degree of the identification object and other objects of the current captured image frame. When the image overlapping degree is high, the image overlapping degree indicates that the image is overlapped with a tracking object or a recognition object and other objects in the image frame, or the image is close to the other objects and cannot be distinguished, and the image is not suitable for being used as a tracking target of a shooting device to carry out tracking shooting.
S303b, updating the tracking information according to the matching information and the overlapping information.
Alternatively, when the recognition object does not match the tracking object and the image overlap degree is less than the overlap degree threshold, it indicates that the recognition object is likely to be suitable for tracking shooting as a tracking target of the shooting device, and therefore the recognition object may be determined as a new tracking object. Wherein, the overlapping threshold value can be set according to specific requirements.
Optionally, in the multi-target tracking shooting process, the state of one or more tracked objects in a shot image frame may change, for example, disappear in the image frame or overlap with other objects, so as to facilitate management of the tracked objects, the tracking information may further include state identifiers corresponding to the tracked objects, where the type of the state identifier is greater than 2, and the tracked objects may be classified and managed. Correspondingly, the state identifier corresponding to the tracking object may also be switched according to the real-time shooting condition, and step S303b may include: and updating the state identifier corresponding to the tracking object according to the matching information and the overlapping information.
Optionally, in order to manage the tracked object more accurately and comprehensively and reduce consumption of computing resources and storage resources, the tracking information may further include appearance duration information for identifying a continuous appearance duration value of the tracked object in the captured image frame, or disappearance duration information for identifying a continuous disappearance duration value of the tracked object in the captured image frame. Correspondingly, updating the tracking information according to the matching information and the overlapping information comprises:
and a substep S1 of updating the appearance time length information or the disappearance time length information according to the matching information.
And a substep S2, updating the state identifier corresponding to the tracking object according to the updated appearance time length information or the updated disappearance time length information and the overlapping information.
The updated appearance duration information is used for identifying the continuous appearance duration value of the tracking object in the shot image frame; the updated disappearance duration information is used for identifying the continuous disappearance duration value of the tracking object in the shot image frame; the overlap information is used to identify a value of a duration of continuous overlap of the tracked object in the captured image frames.
Optionally, if the continuous appearance duration value, the continuous disappearance duration value, and the continuous overlapping duration value of the tracked object satisfy the preset condition, the state identifier corresponding to the tracked object may be updated. The preset conditions can be set according to specific requirements.
Optionally, the state identifier corresponding to the tracked object may be one of a first identifier, a second identifier, a third identifier, a fourth identifier, and a fifth identifier. The concrete description is as follows:
when the state identifier corresponding to the tracked object is the first identifier, the tracked object is in a tracked shooting state, and the shooting device performs tracking shooting on the tracked object in a continuous tracked state, for example, the shooting angle is adjusted according to the motion condition of the tracked object.
When the state identifier corresponding to the tracking object is the second identifier, the tracking object is in a long-time lost state, the tracking object in the long-time lost state is not recognized in all L continuous captured image frames before the current captured image frame, but is in a continuously tracked state in an L +1 th captured image frame before the current captured image frame, that is, the state identifier corresponding to the tracking object in an L +1 th captured image frame before the current captured image frame is the first identifier, wherein L is greater than or equal to 0.
When the state identifier corresponding to the tracking object is the third identifier, the tracking object is in an overlapped state, the tracking object in the overlapped state is overlapped with other objects in X continuous captured image frames before the current captured image frame, or is too close to the other objects to be distinguished, but is in a continuously tracked state in the X +1 th captured image frame before the current captured image frame, that is, the state identifier corresponding to the tracking object in the X +1 th captured image frame before the current captured image frame is the first identifier, wherein X is greater than or equal to 0.
When the state identifier corresponding to the tracking object is the fourth identifier, the tracking object is in a short-time lost state, the tracking object in the short-time lost state is not recognized in Y continuous captured image frames before the current captured image frame, but is in a continuously tracked state in Y +1 captured image frames before the current captured image frame, that is, the state identifier corresponding to the tracking object in the Y +1 captured image frame before the current captured image frame is the first identifier, wherein Y is greater than or equal to 0, and Y is less than the value of L.
When the state identifier corresponding to the tracking object is the fifth identifier, the tracking object is in a new found state, and the corresponding state identifier is never determined as the first identifier, that is, the tracking object with the state identifier of the fifth identifier is not subjected to tracking shooting in the current shooting image frame and the previous Z continuous image frames, wherein Z is an integer greater than or equal to 0.
Optionally, when the state identifier corresponding to the tracked object is the first identifier, if the continuous disappearance duration value is greater than the first disappearance threshold, the state identifier corresponding to the tracked object is updated to be the second identifier.
The first fade threshold may be represented by a time value or a number of frames of consecutive image frames, which may be set according to specific needs. For example, the first disappearance threshold may be represented by 10 consecutive image frames.
When the state identifier corresponding to the tracked object is the first identifier, if the continuous disappearance duration value of the tracked object in the photographed image frame is greater than the first disappearance threshold, it indicates that the tracked object may be outside the photographed image frame, and the probability of being photographed subsequently is relatively low, the state identifier of the tracked object may be updated to be the second identifier.
Optionally, when the state identifier corresponding to the tracked object is the second identifier, if the continuous occurrence duration value is greater than the first occurrence threshold, the state identifier corresponding to the tracked object is updated to be the first identifier.
The first occurrence threshold may be represented by a time value or a number of frames of successive image frames, which may be set according to specific needs. For example, the first occurrence threshold may be represented by 5 consecutive image frames.
When the state identifier corresponding to the tracked object is the second identifier, if the duration value of the tracked object continuously appearing in the captured image frame is greater than the first appearance threshold value, which indicates that the tracked object has reappeared in the captured image frame, the tracked object may be switched to the continuously tracked state again, and the state identifier of the tracked object may be updated to be the first identifier.
Optionally, when the state identifier corresponding to the tracked object is the second identifier, if the continuous disappearance duration value is greater than the second disappearance threshold, the tracked object is cancelled.
The second fade-out threshold may be represented by a time value or a number of frames of consecutive image frames, which may be set according to specific needs. For example, the second disappearance threshold may be represented by 5 consecutive image frames.
When the state identifier corresponding to the tracked object is the second identifier, if the duration value of continuous disappearance of the tracked object is greater than the second disappearance threshold, it indicates that the tracked object may be outside the shot picture, the probability of subsequent shooting is low, the target may not be used as the tracked object, and the state identifier or the tracking information corresponding to the target is deleted, so as to reduce occupation of computing resources and storage resources.
Optionally, when the state identifier corresponding to the tracked object is the first identifier, if the continuous overlapping duration value is greater than the first overlapping threshold, the state identifier corresponding to the tracked object is updated to be the third identifier.
The first overlap threshold may be represented by a time value or a number of frames of consecutive image frames, which may be set according to specific needs. For example, the first overlap threshold may be represented by 8 consecutive image frames.
When the state identifier corresponding to the tracked object is the first identifier, if the continuous overlapping duration value of the tracked object in the captured image frame is greater than the first overlapping threshold value, it indicates that the tracked object is continuously overlapped with other objects or is too close to other objects to be distinguished, and it is temporarily difficult for the imaging device to perform tracking imaging, so that the state identifier of the tracked object can be updated to be the third identifier.
Optionally, when the state identifier corresponding to the tracked object is the second identifier, if the sum of the continuous overlapping duration value and the continuous disappearing duration value is greater than the second overlapping threshold, the tracked object is cancelled.
The second overlap threshold is greater than the first overlap threshold and may be represented by a time-length value or a number of frames of successive image frames, which may be set according to particular needs. For example, the first overlap threshold may be represented by 5 consecutive image frames and the second overlap threshold may be represented by 10 consecutive image frames.
When the state identifier corresponding to the tracked object is the second identifier, if the sum of the continuous overlapping duration value and the continuous disappearing duration value corresponding to the tracked object is greater than the second overlapping threshold value, it indicates that the tracked object is difficult to be tracked and shot by the shooting device in the subsequent time, the target can not be used as the tracked object any more, and the state identifier or the tracking information corresponding to the target is deleted, so that the occupation of computing resources and storage resources is reduced.
Optionally, when the state identifier corresponding to the tracked object is the third identifier, if the sum of the continuous overlapping duration value and the continuous disappearing duration value is greater than the third overlapping threshold, the tracked object is cancelled.
The third overlap threshold may be represented by a time-length value or a number of frames of consecutive image frames, and the third overlap threshold is greater than the first overlap threshold, and the third overlap threshold may be the same as or different from the second overlap threshold, and may be set according to specific requirements. For example, the first overlap threshold may be represented by 5 consecutive image frames and the third overlap threshold may be represented by 8 consecutive image frames.
When the state identifier corresponding to the tracked object is the third identifier, if the sum of the continuous overlapping duration value and the continuous disappearing duration value corresponding to the tracked object is greater than the third overlapping threshold, it indicates that the tracked object is difficult to be tracked and shot by the shooting device in the subsequent time, the target is no longer taken as the tracked object, and the state identifier or the tracking information corresponding to the target is deleted, so that the occupation of computing resources and storage resources is reduced.
Optionally, when the state identifier corresponding to the tracked object is the first identifier, if the continuous disappearance duration value is greater than the third disappearance threshold, the state identifier corresponding to the tracked object is updated to be the fourth identifier.
The third fade threshold is less than the first fade threshold, and may be represented by a time value or a number of frames of consecutive image frames, which may be set according to specific needs. For example, the first fade threshold may be represented by 8 consecutive image frames, and the third fade threshold may be represented by 5 consecutive image frames.
When the state identifier corresponding to the tracked object is the first identifier, if the continuous disappearance duration value of the tracked object in the photographed image frame is greater than the third disappearance threshold, it indicates that the tracked object may be outside the photographed image frame, and it is uncertain whether the tracked object may be photographed continuously or not, so that the state identifier of the tracked object can be updated to be the fourth identifier.
Optionally, when the state identifier corresponding to the tracked object is the fourth identifier, if the continuous occurrence duration value is greater than the second occurrence threshold, the state identifier corresponding to the tracked object is updated to be the first identifier.
The second occurrence threshold may be represented by a time value or a number of frames of consecutive image frames, and may be the same as or different from the first occurrence threshold, and may be set according to specific requirements. For example, the second occurrence threshold may be represented by 3 consecutive image frames.
When the state identifier corresponding to the tracked object is the fourth identifier, if the duration value of the tracked object continuously appearing in the captured image frame is greater than the second appearance threshold value, which indicates that the tracked object has reappeared in the captured image frame, the tracked object may be switched to the continuously tracked state again, and the state identifier of the tracked object may be updated to be the first identifier.
Optionally, when the state identifier corresponding to the tracked object is the fourth identifier, if the continuous disappearance duration value is greater than the fourth disappearance threshold, the state identifier corresponding to the tracked object is updated to be the second identifier.
The fourth disappearance threshold is smaller than the first disappearance threshold, and may be represented by a time value or a number of frames of consecutive image frames, and may be set according to specific needs.
The fourth fade-out threshold may be equal to the difference between the first fade-out threshold and the second fade-out threshold. For example, a first fade threshold may be represented by 8 consecutive image frames, a third fade threshold may be represented by 5 consecutive image frames, and a fourth fade threshold may be represented by 3 consecutive image frames.
When the state identifier corresponding to the tracked object is the fourth identifier, if the duration value of the continuous disappearance of the tracked object in the photographed image frame is greater than the fourth disappearance threshold, it indicates that the tracked object has disappeared in the photographed image frame for a long time, and the probability of being photographed subsequently is relatively low, the state identifier of the tracked object can be updated to be the second identifier.
Optionally, when the state identifier corresponding to the tracked object is the fourth identifier, if the continuous overlapping duration value is greater than the fourth overlapping threshold, the state identifier corresponding to the tracked object is updated to be the third identifier.
The fourth overlap threshold may be represented by a time value or a number of frames of consecutive image frames, which may be set according to specific needs. For example, the fourth overlay threshold may be represented by 5 consecutive image frames.
When the state identifier corresponding to the tracked object is the fourth identifier, if the continuous overlapping duration value corresponding to the tracked object is greater than the fourth overlapping threshold value, it indicates that the tracked object is continuously overlapped with other objects or is very close to other objects and cannot be distinguished, and the tracked object is temporarily difficult to be tracked and shot by the shooting device, so that the state identifier of the tracked object can be updated to be the third identifier.
Optionally, when the state identifier corresponding to the tracking target is the fifth identifier, if the continuous occurrence duration value is greater than the third occurrence threshold, the state identifier corresponding to the tracking target is updated to be the first identifier.
The third occurrence threshold may be represented by a time value or a number of frames of successive image frames, which may be set according to specific needs. For example, the third occurrence threshold may be represented by 5 consecutive image frames.
When the state identifier corresponding to the tracking target is the fifth identifier, if the duration value of the continuous appearance of the tracking target in the photographed image frame is greater than the third appearance threshold value, which indicates that the tracking target is likely to continuously appear in the photographed image subsequently, the tracking target may be switched to the continuous tracked state, that is, the state identifier of the tracking target may be updated to be the first identifier.
Optionally, when the state identifier corresponding to the tracked target is the fifth identifier, if the continuous overlapping duration value is greater than the fifth overlapping threshold, the tracked target is cancelled.
The fifth overlap threshold may be represented by a time value or a number of frames of consecutive image frames, which may be set according to specific needs. For example, the fifth overlay threshold may be represented by 5 consecutive image frames.
When the state identifier corresponding to the tracking target is the fifth identifier, if the continuous overlapping duration value of the tracking target in the shot image frame is greater than the fifth overlapping threshold value, it indicates that the tracking target is difficult to be tracked and shot by the shooting device in the subsequent time, the target can no longer be taken as the tracking target, and the state identifier or the tracking information corresponding to the target is deleted, so as to reduce the occupation of computing resources and storage resources.
As can be seen from the above embodiments, the tracking object determining method further considers the overlapping degree of the tracking object or the recognition object with other objects in the image frame in the process of updating the tracking information, which is beneficial to improving the accuracy of real-time tracking; in the multi-target tracking process, the tracked objects are classified, and different states are distinguished by adopting the state identifiers, so that the tracked objects are convenient to manage, and the consumption of computing resources and storage resources is reduced.
Example four
A fourth embodiment of the present application provides a device for determining a tracked object, as shown in fig. 4, where fig. 4 is a block diagram of a structure of the device for determining a tracked object provided in the embodiment of the present application.
The tracked object determining apparatus of the present embodiment includes: the tracking system comprises a memory 401, a processor 402 and a video collector 403, wherein the video collector 403 is used for collecting an object to be tracked in a target area; the memory 401 is used to store program codes; a processor 402, calling program code, which when executed, is configured to: performing image recognition on the current shot image frame to obtain recognition information for identifying a recognition object in the current shot image frame; matching the recognition object with the tracking object according to the recognition information and the tracking information for identifying the tracking object in the shot image frame to obtain matching information; and updating the tracking information according to the matching information.
Optionally, the processor 402 calls the program code, and when the program code is executed, the processor performs the following operations: matching the recognition object with the tracking object according to the recognition information and tracking information for identifying the tracking object in the photographed image frame, and obtaining matching information includes: according to the identification information and the tracking information, obtaining similarity information for identifying the similarity degree between the identification object and the tracking object and position information for identifying the position relation between the identification object and the tracking object; and matching the identification object with the tracking object according to the similarity information and the position information to obtain matching information.
Optionally, the processor 402 calls the program code, and when the program code is executed, the processor performs the following operations: the position information comprises intersection ratio sub information and distance sub information, wherein the intersection ratio sub information is used for identifying intersection ratios corresponding to the recognition object and the tracking object, and the distance sub information is used for identifying the distance between the recognition object and the tracking object.
Optionally, the processor 402 calls the program code, and when the program code is executed, the processor performs the following operations: matching the recognition object with the tracking object according to the recognition information and tracking information for identifying the tracking object in the photographed image frame, and obtaining matching information includes: and matching the identification object and the tracking object by sequentially using a greedy algorithm and a Hungary algorithm according to the identification information and the tracking information to obtain matching information.
Optionally, the processor 402 calls the program code, and when the program code is executed, the processor performs the following operations: updating the tracking information based on the matching information includes: and when the identification object does not match with the tracking object, determining the identification object as a new tracking object.
Optionally, the processor 402 calls the program code, and when the program code is executed, the processor performs the following operations: updating the tracking information based on the matching information includes: acquiring overlapping information according to the identification information and the tracking information, wherein the overlapping information is used for identifying the image overlapping degree of the tracking object and other objects in a previous captured image frame of the current captured image frame and the image overlapping degree of the identification object and other objects of the current captured image frame; and updating the tracking information according to the matching information and the overlapping information.
Optionally, the processor 402 calls the program code, and when the program code is executed, the processor performs the following operations: updating the tracking information based on the matching information and the overlapping information includes: and when the identification object does not match the tracking object and the image overlapping degree is less than the overlapping degree threshold value, determining the identification object as a new tracking object.
Optionally, the processor 402 calls the program code, and when the program code is executed, the processor performs the following operations: the tracking information comprises a state identifier corresponding to the tracking object, wherein the type of the state identifier is more than 2; correspondingly, updating the tracking information according to the matching information and the overlapping information comprises: and updating the state identifier corresponding to the tracking object according to the matching information and the overlapping information.
Optionally, the processor 402 calls the program code, and when the program code is executed, the processor performs the following operations: the tracking information also comprises appearance duration information used for identifying a continuous appearance duration value of the tracking object in the shot image frame, or disappearance duration information used for identifying a continuous disappearance duration value of the tracking object in the shot image frame; correspondingly, updating the tracking information according to the matching information and the overlapping information comprises: updating the appearing time length information or the disappearing time length information according to the matching information; and updating the state identifier corresponding to the tracking object according to the updated appearance time length information or the updated disappearance time length information and the overlapping information.
In this embodiment, please refer to the first to third embodiments for detailed technical content.
As can be seen from the above embodiments, the tracked object determining device of the present application performs image recognition on a current captured image frame to obtain recognition information for identifying a recognition object in the current captured image frame; then matching the recognition object with the tracking object according to the recognition information and the tracking information for identifying the tracking object in the shot image frame to obtain matching information; and updating the tracking information according to the matching information. In the multi-target tracking shooting process, the current shooting image frame can be compared with the previous image frame, the tracking information of the tracking object is updated by searching the matching relation between the identification object in the current shooting image frame and the tracking object in the shot image frame, the tracking state of the tracking object is updated, and the tracking object can be managed and maintained in a more real-time and accurate manner.
EXAMPLE five
An embodiment of the present application provides a handheld camera, and as shown in fig. 5, fig. 5 is a block diagram of a structure of the handheld camera provided in the embodiment of the present application.
The handheld camera of this embodiment includes the tracking object determining apparatus of the fifth embodiment, and the handheld camera further includes: the carrier is fixedly connected with the video collector and used for carrying at least one part of the video collector.
Optionally, the carrier includes, but is not limited to, a handheld pan/tilt head 1.
Optionally, the handheld pan/tilt head 1 is a handheld three-axis pan/tilt head.
Optionally, the video collector includes, but is not limited to, a camera for a handheld three-axis pan-tilt.
The basic construction of the hand-held camera will be briefly described below.
The handheld cloud platform 1 of the embodiment of the invention comprises: the camera device 12 may include a three-axis pan-tilt camera in the present embodiment, and two or more axes of pan-tilt cameras in other embodiments.
The handle 11 is provided with a display 13 for displaying the contents of the shooting by the shooting device 12. The present invention does not limit the type of the display 13.
Through setting up display screen 13 at the handle 11 of handheld cloud platform 1, this display screen can show the shooting content of taking device 12 to realize that the user can browse the picture or the video that taking device 12 was taken through this display screen 13 fast, thereby improve handheld cloud platform 1 and user's interactivity and interest, satisfy user's diversified demand.
In one embodiment, the handle 11 is further provided with an operation function portion for controlling the photographing device 12, by which the operation of the photographing device 12 can be controlled, for example, the opening and closing of the photographing device 12, the photographing of the photographing device 12, the change of the posture of the pan-tilt portion of the photographing device 12, and the like, so as to facilitate the user's quick operation of the photographing device 12. The operation function part can be in the form of a key, a knob or a touch screen.
In one embodiment, the operation function portion includes a shooting button 14 for controlling shooting by the shooting device 12, a power/function button 15 for controlling on/off and other functions of the shooting device 12, and a gimbal key 16 for controlling movement of the pan/tilt head. Of course, the operation function portion may further include other control keys, such as an image storage key, an image playing control key, and the like, which may be set according to actual requirements.
In one embodiment, the operation function portion and the display 13 are disposed on the same surface of the handle 11, and the operation function portion and the display 13 are both disposed on the front surface of the handle 11, which is ergonomic and makes the overall appearance of the handheld tripod head 1 more reasonable and beautiful.
Further, the side of the handle 11 is provided with a function operating key a for facilitating the user to quickly and intelligently form a piece by one key. When the camera is started, the orange side key on the right side of the camera body is clicked to start the function, a video is automatically shot at intervals, N sections (N is more than or equal to 2) are shot totally, after a mobile device such as a mobile phone is connected, the function of 'one-key film forming' is selected, the shooting sections are intelligently screened by the system and matched with a proper template, and wonderful works are quickly generated.
In an alternative embodiment, the handle 11 is also provided with a catch 17 for plugging in a memory element. In this embodiment, the card slot 17 is provided on the side surface of the handle 11 adjacent to the display 13, and the image captured by the imaging device 12 can be stored in the memory card by inserting the memory card into the card slot 17. In addition, the card slot 17 is arranged on the side part, so that the use of other functions is not influenced, and the user experience is better.
In one embodiment, a power supply battery for supplying power to the handle 11 and the camera 12 may be disposed inside the handle 11. The power supply battery can adopt a lithium battery, and has large capacity and small volume so as to realize the miniaturization design of the handheld cloud deck 1.
In one embodiment, the handle 11 is further provided with a charging/USB interface 18. In the present embodiment, the charging interface/USB interface 18 is disposed at the bottom of the handle 11, so as to facilitate connection with an external power source or a storage device, thereby charging the power supply battery or performing data transmission.
In one embodiment, the handle 11 is further provided with a sound pickup hole 19 for receiving an audio signal, and the sound pickup hole 19 is communicated with a microphone. Pickup hole 19 may include one or more. An indicator light 20 for displaying status is also included. The user may interact audibly with the display screen 13 through the sound pickup hole 19. In addition, the indicator light 20 can reach the warning effect, and the user can obtain the electric quantity condition and the current executive function condition of handheld cloud platform 1 through the indicator light 20. In addition, the sound collecting hole 19 and the indicator light 20 can be arranged on the front surface of the handle 11, so that the use habit and the operation convenience of a user are better met.
In one embodiment, the camera 12 includes a pan-tilt support and a camera mounted on the pan-tilt support. The camera may be a camera, or may be an image pickup device composed of a lens and an image sensor (such as a CMOS or CCD), and may be specifically selected as needed. The camera may be integrated on the pan-tilt support, so that the camera device 12 is a pan-tilt camera; the camera can also be an external shooting device which can be detachably connected or clamped and carried on the tripod head bracket.
In one embodiment, the pan-tilt support is a three-axis pan-tilt support and the camera 12 is a three-axis pan-tilt camera. The three-axis pan-tilt support comprises a yaw shaft assembly 22, a rolling shaft assembly 23 movably connected with the yaw shaft assembly 22, and a pitching shaft assembly 24 movably connected with the rolling shaft assembly 23, and the shooting device is mounted on the pitching shaft assembly 24. The yaw shaft assembly 22 rotates the camera 12 in the yaw direction. Of course, in other examples, the holder may be a two-axis holder, a four-axis holder, or the like, which may be specifically selected as needed.
In one embodiment, a mounting portion is provided at one end of the connecting arm connected to the yaw axle assembly, which may be provided in the handle, and the yaw axle assembly drives the camera 12 to rotate in the yaw direction.
In an alternative embodiment, the handle 11 is provided with an adapter 26 for coupling with the mobile device 2 (e.g., a mobile phone), and the adapter 26 is detachably connected with the handle 11. The adaptor 26 protrudes from the side of the handle for connecting the mobile device 2, and when the adaptor 26 is connected to the mobile device 2, the handheld tripod head 1 is butted with the adaptor 26 and is supported on the end of the mobile device 2.
Set up the adaptor 26 that is used for being connected with mobile device 2 at handle 11, and then with handle 11 and mobile device 2 interconnect, handle 11 can regard as a base of mobile device 2, and the user can come together to hold cloud platform 1 and pick up the operation through the other end that grips mobile device 2, connects convenient and fast, and the product aesthetic property is strong. In addition, after the handle 11 is coupled with the mobile device 2 through the adaptor 26, the communication connection between the handheld tripod head 1 and the mobile device 2 can be realized, and data transmission can be performed between the shooting device 12 and the mobile device 2.
In one embodiment, the adaptor 26 is removably attached to the handle 11, i.e., mechanical connection or disconnection between the adaptor 26 and the handle 11 is possible. Further, the adaptor 26 is provided with an electrical contact and the handle 11 is provided with an electrical contact mating portion for mating with the electrical contact.
In this way, the adapter 26 can be removed from the handle 11 when the handheld head 1 does not need to be connected to the mobile device 2. When the handheld cloud platform 1 needs to be connected with the mobile device 2, the adaptor 26 is mounted on the handle 11, the mechanical connection between the adaptor 26 and the handle 11 is completed, and meanwhile, the electrical connection between the electrical contact part and the electrical contact matching part is guaranteed through the connection between the electrical contact part and the electrical contact matching part, so that data transmission between the shooting device 12 and the mobile device 2 can be achieved through the adaptor 26.
In one embodiment, the handle 11 has a receiving groove 27 at a side thereof, and the adaptor 26 is slidably engaged in the receiving groove 27. When the adaptor 26 is received in the receiving slot 27, a portion of the adaptor 26 protrudes from the receiving slot 27, and a portion of the adaptor 26 protruding from the receiving slot 27 is used for connecting with the mobile device 2.
In one embodiment, as shown in fig. 5, when the adaptor 26 is inserted into the receiving slot 27 from the adaptor 26, the adaptor is flush with the receiving slot 27, and the adaptor 26 is received in the receiving slot 27 of the handle 11.
Therefore, when the handheld tripod head 1 needs to be connected with the mobile device 2, the adaptor 26 can be inserted into the receiving slot 27 from the adaptor, so that the adaptor 26 protrudes from the receiving slot 27, so that the mobile device 2 and the handle 11 can be connected with each other
After the mobile device 2 is used or when the mobile device 2 needs to be pulled out, the adaptor 26 may be taken out from the receiving groove 27 of the handle 11, and then reversely put into the receiving groove 27 from the adaptor 26, so that the adaptor 26 may be received in the handle 11. The adaptor 26 is flush with the receiving groove 27 of the handle 11, so that when the adaptor 26 is received in the handle 11, the surface of the handle 11 is smooth, and the adaptor 26 is more convenient to carry when received in the handle 11.
In one embodiment, the receiving slot 27 is semi-open on one side surface of the handle 11, which facilitates the sliding engagement of the adaptor 26 with the receiving slot 27. Of course, in other examples, the adaptor 26 may be detachably connected to the receiving slot 27 of the handle 11 by a snap connection, a plug connection, or the like.
In one embodiment, the receiving slot 27 is formed on the side of the handle 11, and the cover 28 is clamped to cover the receiving slot 27 when the switch function is not used, so that the user can operate the switch conveniently without affecting the overall appearance of the front and side of the handle.
In one embodiment, the electrical contact and the electrical contact mating portion may be electrically connected by way of contact. For example, the electrical contacts may be selected as pogo pins, electrical sockets, and electrical contacts. Of course, in other examples, the electrical contact portion and the electrical contact mating portion may be directly connected by surface-to-surface contact.
A1, a tracked object determining method, comprising:
performing image recognition on the current shot image frame to obtain recognition information for identifying a recognition object in the current shot image frame;
matching the recognition object with the tracking object according to the recognition information and tracking information for identifying the tracking object in the shot image frame to obtain matching information;
and updating the tracking information according to the matching information.
A2, the method for determining a tracked object according to a1, wherein the matching the recognized object with the tracked object according to the recognition information and tracking information for identifying the tracked object in the captured image frame, and the obtaining matching information includes:
according to the identification information and the tracking information, obtaining similarity information used for identifying the similarity degree between the identification object and the tracking object and position information used for identifying the position relation between the identification object and the tracking object;
and matching the identification object with the tracking object according to the similarity information and the position information to obtain matching information.
A3, the method for determining a tracking object according to a2, wherein the position information includes intersection ratio sub information for identifying an intersection ratio corresponding to the recognition object and the tracking object, and distance sub information for identifying a distance between the recognition object and the tracking object.
A4, the method for determining a tracked object according to a1, wherein the matching the recognized object with the tracked object according to the recognition information and tracking information for identifying the tracked object in the captured image frame, and the obtaining matching information includes:
and matching the identification object and the tracking object by sequentially using a greedy algorithm and a Hungarian algorithm according to the identification information and the tracking information to obtain matching information.
A5, the method for determining tracked object according to a1, wherein the updating the tracking information according to the matching information includes:
when the identification object does not match the tracking object, determining the identification object as a new tracking object.
A6, the method for determining tracked object according to a1, wherein the updating the tracking information according to the matching information includes:
acquiring overlapping information according to the identification information and the tracking information, wherein the overlapping information is used for identifying the image overlapping degree of the tracking object and other objects in a previous captured image frame of the current captured image frame and the image overlapping degree of the identification object and other objects of the current captured image frame;
and updating the tracking information according to the matching information and the overlapping information.
A7, the method for determining a tracked object according to a6, wherein the updating the tracking information according to the matching information and the overlapping information includes:
when the identified object does not match the tracked object and the image overlap is less than an overlap threshold, determining the identified object as a new tracked object.
A8, the method for determining the tracked object according to A6, wherein the tracking information includes the status identifier corresponding to the tracked object, and the type of the status identifier is greater than 2; correspondingly, the updating the tracking information according to the matching information and the overlapping information includes:
and updating the state identifier corresponding to the tracking object according to the matching information and the overlapping information.
A9, the tracked object determining method according to A8, wherein the tracking information further includes appearance duration information for identifying a value of a duration during which the tracking target continuously appears in the captured image frames, or disappearance duration information for identifying a value of a duration during which the tracking target continuously disappears in the captured image frames; correspondingly, the updating the tracking information according to the matching information and the overlapping information includes:
updating the appearing time length information or the disappearing time length information according to the matching information;
and updating the state identifier corresponding to the tracking object according to the updated appearance time length information or the updated disappearance time length information and the overlapping information.
A10, a tracked object determining apparatus, comprising: the device comprises a memory, a processor and a video collector, wherein the video collector is used for collecting a target to be tracked in a target area; the memory is used for storing program codes; the processor, invoking the program code, when executed, is configured to:
performing image recognition on the current shot image frame to obtain recognition information for identifying a recognition object in the current shot image frame;
matching the recognition object with the tracking object according to the recognition information and tracking information for identifying the tracking object in the shot image frame to obtain matching information;
and updating the tracking information according to the matching information.
A11, the tracked object determining apparatus according to a10, wherein said matching the recognized object with the tracked object according to the recognition information and tracking information for identifying the tracked object in the captured image frame, and obtaining matching information comprises:
according to the identification information and the tracking information, obtaining similarity information used for identifying the similarity degree between the identification object and the tracking object and position information used for identifying the position relation between the identification object and the tracking object;
and matching the identification object with the tracking object according to the similarity information and the position information to obtain matching information.
A12, the tracked object determination device according to a11, wherein the position information includes intersection ratio sub information for identifying an intersection ratio corresponding to the recognition object and the tracked object, and distance sub information for identifying a distance between the recognition object and the tracked object.
A13, the tracked object determining apparatus according to a10, wherein said matching the recognized object with the tracked object according to the recognition information and tracking information for identifying the tracked object in the captured image frame, and obtaining matching information comprises:
and matching the identification object and the tracking object by sequentially using a greedy algorithm and a Hungarian algorithm according to the identification information and the tracking information to obtain matching information.
A14, the tracked object determining apparatus according to a10, wherein said updating the tracking information according to the matching information comprises:
when the identification object does not match the tracking object, determining the identification object as a new tracking object.
A15, the tracked object determining apparatus according to a10, wherein said updating the tracking information according to the matching information comprises:
acquiring overlapping information according to the identification information and the tracking information, wherein the overlapping information is used for identifying the image overlapping degree of the tracking object and other objects in a previous captured image frame of the current captured image frame and the image overlapping degree of the identification object and other objects of the current captured image frame;
and updating the tracking information according to the matching information and the overlapping information.
A16, the tracked object determining apparatus according to a15, wherein said updating the tracking information according to the matching information and the overlapping information comprises:
when the identified object does not match the tracked object and the image overlap is less than an overlap threshold, determining the identified object as a new tracked object.
A17, the tracked object determination device according to A15, wherein the tracking information includes a state identifier corresponding to the tracked object, and the type of the state identifier is greater than 2; correspondingly, the updating the tracking information according to the matching information and the overlapping information includes:
and updating the state identifier corresponding to the tracking object according to the matching information and the overlapping information.
A18, the tracked object determining apparatus according to a17, wherein the tracking information further includes appearance duration information for identifying a value of a duration at which the tracking target appears continuously in the captured image frames, or disappearance duration information for identifying a value of a duration at which the tracking target disappears continuously in the captured image frames; correspondingly, the updating the tracking information according to the matching information and the overlapping information includes:
updating the appearing time length information or the disappearing time length information according to the matching information;
and updating the state identifier corresponding to the tracking object according to the updated appearance time length information or the updated disappearance time length information and the overlapping information.
A19, a hand-held camera, comprising the tracked object determining apparatus according to a10-18, characterized by further comprising: the carrier is fixedly connected with the video collector and used for carrying at least one part of the video collector.
A20, the hand-held camera according to a19, wherein the carrier comprises but is not limited to a hand-held pan-tilt head.
A21, the hand-held camera according to A20, wherein the hand-held pan-tilt is a hand-held tri-axial pan-tilt.
A22, the hand-held camera according to A21, wherein the video collector includes but is not limited to a camera for a hand-held three-axis pan-tilt head.
It should be noted that, according to the implementation requirement, each component/step described in the embodiment of the present invention may be divided into more components/steps, and two or more components/steps or partial operations of the components/steps may also be combined into a new component/step to achieve the purpose of the embodiment of the present invention.
The above-described method according to an embodiment of the present invention may be implemented in hardware, firmware, or as software or computer code storable in a recording medium such as a CD ROM, a RAM, a floppy disk, a hard disk, or a magneto-optical disk, or as computer code originally stored in a remote recording medium or a non-transitory machine-readable medium downloaded through a network and to be stored in a local recording medium, so that the method described herein may be stored in such software processing on a recording medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware such as an ASIC or FPGA. It is understood that the computer, processor, microprocessor controller or programmable hardware includes memory components (e.g., RAM, ROM, flash memory, etc.) that can store or receive software or computer code that, when accessed and executed by the computer, processor or hardware, implements the object tracking shot methods described herein. Further, when a general-purpose computer accesses code for implementing the methods illustrated herein, execution of the code transforms the general-purpose computer into a special-purpose computer for performing the methods illustrated herein.
Those of ordinary skill in the art will appreciate that the various illustrative elements and method steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, or combinations of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present embodiments.
The above embodiments are only for illustrating the embodiments of the present invention and not for limiting the embodiments of the present invention, and those skilled in the art can make various changes and modifications without departing from the spirit and scope of the embodiments of the present invention, so that all equivalent technical solutions also belong to the scope of the embodiments of the present invention, and the scope of the embodiments of the present invention should be defined by the claims.

Claims (14)

1. A tracked object determination method, comprising:
performing image recognition on the current shot image frame to obtain recognition information for identifying a recognition object in the current shot image frame;
obtaining similarity information for identifying a degree of similarity between the recognition object and the tracking object, position information for identifying a positional relationship between the recognition object and the tracking object, and superimposition information, from the recognition information and tracking information for identifying a tracking object in the captured image frame; wherein the overlap information is used to identify an image overlap degree of the tracking object with other objects in a previous captured image frame of the current captured image frame, and an image overlap degree of the recognition object with other objects of the current captured image frame;
matching the identification object with the tracking object according to the similarity information and the position information to obtain matching information;
updating the tracking information according to the matching information and the overlapping information; wherein when the identified object does not match the tracked object and the image overlap is less than an overlap threshold, the identified object is determined to be the new tracked object.
2. The tracked object determining method according to claim 1, wherein the position information includes intersection ratio sub-information for identifying an intersection ratio corresponding to the identified object and the tracked object, and distance sub-information for identifying a distance between the identified object and the tracked object.
3. The tracked object determining method according to claim 1, wherein said matching the identified object with the tracked object based on the identification information and tracking information for identifying the tracked object in the captured image frame, and obtaining matching information comprises:
and matching the identification object and the tracking object by sequentially using a greedy algorithm and a Hungarian algorithm according to the identification information and the tracking information to obtain matching information.
4. The method according to claim 1, wherein the tracking information includes a status identifier corresponding to the tracking object, wherein the type of the status identifier is greater than 2; correspondingly, the updating the tracking information according to the matching information and the overlapping information includes:
and updating the state identifier corresponding to the tracking object according to the matching information and the overlapping information.
5. The tracked object determining method according to claim 4, wherein the tracking information further includes appearance duration information for identifying a value of a duration of continuous appearance of the tracked object in the captured image frame or disappearance duration information for identifying a value of a duration of continuous disappearance of the tracked object in the captured image frame; correspondingly, the updating the tracking information according to the matching information and the overlapping information includes:
updating the appearing time length information or the disappearing time length information according to the matching information;
and updating the state identifier corresponding to the tracking object according to the updated appearance time length information or the updated disappearance time length information and the overlapping information.
6. A tracked object determination apparatus, characterized by comprising: the device comprises a memory, a processor and a video collector, wherein the video collector is used for collecting a target to be tracked in a target area; the memory is used for storing programs; the processor, invoking the program, when the program is executed, is configured to:
performing image recognition on the current shot image frame to obtain recognition information for identifying a recognition object in the current shot image frame;
obtaining similarity information for identifying a degree of similarity between the recognition object and the tracking object, position information for identifying a positional relationship between the recognition object and the tracking object, and superimposition information, from the recognition information and tracking information for identifying a tracking object in the captured image frame; wherein the overlap information is used to identify an image overlap degree of the tracking object with other objects in a previous captured image frame of the current captured image frame, and an image overlap degree of the recognition object with other objects of the current captured image frame;
matching the identification object with the tracking object according to the similarity information and the position information to obtain matching information;
updating the tracking information according to the matching information and the overlapping information; wherein when the identified object does not match the tracked object and the image overlap is less than an overlap threshold, the identified object is determined to be the new tracked object.
7. The tracking object determination device according to claim 6, wherein the position information includes intersection ratio sub information for identifying an intersection ratio corresponding to the recognition object and the tracking object, and distance sub information for identifying a distance between the recognition object and the tracking object.
8. The tracked object determining apparatus according to claim 6, wherein said matching the recognition object with the tracked object based on the recognition information and tracking information for identifying the tracked object in the captured image frame, and obtaining matching information comprises:
and matching the identification object and the tracking object by sequentially using a greedy algorithm and a Hungarian algorithm according to the identification information and the tracking information to obtain matching information.
9. The tracked object determination apparatus according to claim 6, wherein the tracking information includes a status identifier corresponding to the tracked object, wherein the category of the status identifier is greater than 2; correspondingly, the updating the tracking information according to the matching information and the overlapping information includes:
and updating the state identifier corresponding to the tracking object according to the matching information and the overlapping information.
10. The tracked object determination apparatus according to claim 9, wherein said tracking information further includes appearance duration information for identifying a value of a duration of continuous appearance of said tracked object in a captured image frame or disappearance duration information for identifying a value of a duration of continuous disappearance of said tracked object in a captured image frame; correspondingly, the updating the tracking information according to the matching information and the overlapping information includes:
updating the appearing time length information or the disappearing time length information according to the matching information;
and updating the state identifier corresponding to the tracking object according to the updated appearance time length information or the updated disappearance time length information and the overlapping information.
11. A handheld camera characterized by comprising the tracked object determination apparatus according to any one of claims 6 to 10, characterized by further comprising: the carrier is fixedly connected with the video collector and used for carrying at least one part of the video collector.
12. The hand-held camera of claim 11, wherein the carrier comprises, but is not limited to, a hand-held pan-tilt.
13. The hand-held camera of claim 12, wherein the hand-held pan-tilt head is a hand-held tri-axial pan-tilt head.
14. The handheld camera of claim 13, wherein the video collector comprises but is not limited to a handheld camera for a three-axis pan-tilt.
CN202010297067.1A 2020-04-15 2020-04-15 Tracking object determination method and device and handheld camera Active CN111738053B (en)

Priority Applications (2)

Application Number Priority Date Filing Date Title
CN202010297067.1A CN111738053B (en) 2020-04-15 2020-04-15 Tracking object determination method and device and handheld camera
PCT/CN2020/099830 WO2021208253A1 (en) 2020-04-15 2020-07-02 Tracking object determination method and device, and handheld camera

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202010297067.1A CN111738053B (en) 2020-04-15 2020-04-15 Tracking object determination method and device and handheld camera

Publications (2)

Publication Number Publication Date
CN111738053A CN111738053A (en) 2020-10-02
CN111738053B true CN111738053B (en) 2022-04-01

Family

ID=72646725

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202010297067.1A Active CN111738053B (en) 2020-04-15 2020-04-15 Tracking object determination method and device and handheld camera

Country Status (2)

Country Link
CN (1) CN111738053B (en)
WO (1) WO2021208253A1 (en)

Families Citing this family (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112291480B (en) * 2020-12-03 2022-06-21 维沃移动通信有限公司 Tracking focusing method, tracking focusing device, electronic device and readable storage medium
CN113544740A (en) * 2020-12-31 2021-10-22 商汤国际私人有限公司 Method and device for identifying operation event
CN115643485B (en) * 2021-11-25 2023-10-24 荣耀终端有限公司 Shooting method and electronic equipment
CN114333294B (en) * 2021-11-30 2022-12-13 上海电科智能系统股份有限公司 Multi-element multi-object perception identification tracking method based on non-full coverage
CN114708533A (en) * 2022-03-24 2022-07-05 阿里云计算有限公司 Target tracking method, device, equipment and storage medium
CN117011736A (en) * 2022-04-28 2023-11-07 北京字跳网络技术有限公司 Multi-target tracking method, device, equipment and readable storage medium

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108269269A (en) * 2016-12-30 2018-07-10 纳恩博（北京）科技有限公司 Method for tracking target and device
CN110245630A (en) * 2019-06-18 2019-09-17 广东中安金狮科创有限公司 Monitoring data processing method, device and readable storage medium storing program for executing
CN110472608A (en) * 2019-08-21 2019-11-19 石翊鹏 Image recognition tracking processing method and system
CN110516578A (en) * 2019-08-20 2019-11-29 开放智能机器（上海）有限公司 A kind of monitoring system based on recognition of face and target following
CN110619658A (en) * 2019-09-16 2019-12-27 北京地平线机器人技术研发有限公司 Object tracking method, object tracking device and electronic equipment

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR101821989B1 (en) * 2016-08-11 2018-01-25 이노뎁 주식회사 Method of providing detection of moving objects in the CCTV video data by reconstructive video processing

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108269269A (en) * 2016-12-30 2018-07-10 纳恩博（北京）科技有限公司 Method for tracking target and device
CN110245630A (en) * 2019-06-18 2019-09-17 广东中安金狮科创有限公司 Monitoring data processing method, device and readable storage medium storing program for executing
CN110516578A (en) * 2019-08-20 2019-11-29 开放智能机器（上海）有限公司 A kind of monitoring system based on recognition of face and target following
CN110472608A (en) * 2019-08-21 2019-11-19 石翊鹏 Image recognition tracking processing method and system
CN110619658A (en) * 2019-09-16 2019-12-27 北京地平线机器人技术研发有限公司 Object tracking method, object tracking device and electronic equipment

Also Published As

Publication number Publication date
CN111738053A (en) 2020-10-02
WO2021208253A1 (en) 2021-10-21

Similar Documents

Publication Publication Date Title
CN111738053B (en) Tracking object determination method and device and handheld camera
CN109947886B (en) Image processing method, image processing device, electronic equipment and storage medium
CN111983635B (en) Pose determination method and device, electronic equipment and storage medium
CN105138126A (en) Unmanned aerial vehicle shooting control method and device and electronic device
CN105279898A (en) Alarm method and device
CN111325796A (en) Method and apparatus for determining pose of vision device
CN110853095B (en) Camera positioning method and device, electronic equipment and storage medium
CN105758319B (en) The method and apparatus for measuring target object height by mobile terminal
CN111767782B (en) Tracking target determining method and device and handheld camera
CN110781821B (en) Target detection method and device based on unmanned aerial vehicle, electronic equipment and storage medium
CN110650379A (en) Video abstract generation method and device, electronic equipment and storage medium
CN110035218A (en) A kind of image processing method, image processing apparatus and photographing device
CN111724412A (en) Method and device for determining motion trail and computer storage medium
CN112945207B (en) Target positioning method and device, electronic equipment and storage medium
CN112330717B (en) Target tracking method and device, electronic equipment and storage medium
CN111753606A (en) Intelligent model upgrading method and device
CN115407355B (en) Library position map verification method and device and terminal equipment
CN109829393B (en) Moving object detection method and device and storage medium
CN111832338A (en) Object detection method and device, electronic equipment and storage medium
CN111539283B (en) Face tracking method and face tracking equipment
CN111479061B (en) Tracking state determination method and device and handheld camera
CN111508001A (en) Method and device for retrieving tracking target and handheld camera
CN114519794A (en) Feature point matching method and device, electronic equipment and storage medium
CN114387622A (en) Animal weight recognition method and device, electronic equipment and storage medium
CN114549578A (en) Target tracking method, device and storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant