KR102621875B1 - Object Tracking System - Google Patents

Abstract

An object tracking system is disclosed. An object tracking system according to an embodiment of the present invention receives images from first and second cameras installed at different locations and capture images, detects objects included in the received images, and detects each of the detected objects. An object detection unit that assigns a unique ID to an object, a path tracking unit that tracks the movement path of each detected object, and an image in which the ID and information about the movement path are merged with the images received from the first and second cameras. and an image storage unit for storing images of the detected object, images received from the first and second cameras, a movement path of the tracked object, and the tracked object appears in the images corresponding to the first and second cameras. It includes a display unit that displays the time interval.

Description

Object Tracking System

The present invention relates to an object tracking system, and more specifically, to an object tracking system for detecting an object from a plurality of images and intuitively providing a movement path of the detected object.

As the spread of surveillance cameras accelerates, the number of cameras that need to be monitored and processed at once is increasing, and the demand for search operations for missing persons, criminals, or vehicles is increasing.

When trying to search for a specific object from a large number of cameras, searching manually is inefficient and has a high probability of user error.

Accordingly, there is an increasing demand for a method that allows users to reduce their work burden and intuitively display the object they want to track.

The purpose of the present invention is to provide an object tracking system that can intuitively display objects detected in an image and their movement paths and provide a user-friendly display UI.

An object tracking system according to an embodiment of the present invention receives images from first and second cameras installed at different locations and capture images, detects objects included in the received images, and detects each of the detected objects. An object detection unit that assigns a unique ID to an object, a path tracking unit that tracks the movement path of each detected object, and an image in which the ID and information about the movement path are merged with the images received from the first and second cameras. and an image storage unit for storing images of the detected object, images received from the first and second cameras, a movement path of the tracked object, and the tracked object appears in the images corresponding to the first and second cameras. It includes a display unit that displays the time interval.

In addition, it further includes a receiver for receiving a selection signal for selecting an object included in the image, wherein the display unit selects an object corresponding to the selection signal as the tracking target object and shows a movement path of the selected object. It can be displayed.

Additionally, the object detection unit may extract a display area including the detected object, and the display unit may display the display area and the ID assigned to the detected object.

Additionally, the image storage unit may extract and store images of the detected objects from frames included in the images received from the first and second cameras, respectively, and the display unit may further display the stored images.

Additionally, the display unit may display a movement path of an object corresponding to the selection signal among images of the detected object.

The present invention can provide an object tracking system that can intuitively display objects detected in an image and the movement path of the detected objects and provide a user-friendly display UI.

1 is a diagram schematically showing the configuration of an object tracking system according to an embodiment of the present invention.
Figure 2 is a diagram schematically showing the configuration of an object tracking system according to another embodiment of the present invention.
Figure 3 is a diagram illustrating the operation of an object detection unit according to an embodiment of the present invention.
4 and 5 are diagrams illustrating a screen configuration displayed by a display unit according to an embodiment of the present invention.

Hereinafter, the present invention will be described in detail with reference to the attached drawings. In addition, in this specification and drawings, the same symbols are used for components having substantially the same configuration, thereby omitting redundant description.

The terms used herein are only used to describe specific embodiments and are not intended to limit the invention. Singular expressions include plural expressions unless the context clearly dictates otherwise. In this specification, terms such as “include” or “have” are intended to designate the presence of features, numbers, steps, operations, components, parts, or combinations thereof described in the specification, but are not intended to indicate the presence of one or more other features. It should be understood that this does not exclude in advance the possibility of the existence or addition of elements, numbers, steps, operations, components, parts, or combinations thereof. Terms such as first, second, etc. may be used to describe various components, but the components should not be limited by these terms. The above terms are used only for the purpose of distinguishing one component from another.

1 is a diagram schematically showing the configuration of an object tracking system according to an embodiment of the present invention.

Referring to FIG. 1, the object tracking system 100 according to an embodiment of the present invention includes an object detection unit 110, a path tracking unit 120, an image storage unit 130, and a display unit 140. The object detection unit 110 receives images from first and second cameras (Camera 1 and Camera 2) installed at different locations and capture images, detects objects included in the received images, and detects each detected object. A unique ID (Identification) is assigned to.

The process by which the object detection unit 110 detects an object can be broadly understood as going through two steps. First, the object detector 110 may perform first detection to extract an image of an object determined to be extracted from images received from the first and second cameras (Camera 1 and Camera 2).

Here, the object to be extracted is the object to be tracked by the object tracking system 100 and is not limited to any one type. However, in this specification, the case where a person is the subject of extraction and tracking will be explained as an example.

In the first detection, an image of an object determined to be a person can be detected among various objects included in a received image. Accordingly, the object detection unit 110 can determine which object includes a human body structure among at least one object included in the image and extract an image including an object determined to be a human.

The first detection may be performed on each of a plurality of frames constituting the images received from the first and second cameras (Camera 1 and Camera 2).

Subsequently, the object detector 110 may perform secondary detection to extract features of the object extracted as an image. The feature can be understood as visual information that can distinguish one object from another object, and a unique ID can be assigned to the object from which the feature is extracted.

Here, the feature may be the color of the object, and if the object is a person, the object detector 110 extracts the feature using information about the color of tops, bottoms, and/or shoes, and applies the extracted feature to the corresponding object. It can be saved in conjunction with the ID.

Additionally, the object detector 110 may store the image extracted in the first detection as a representative image for the object from which features have been extracted. The representative image may include information about features extracted from the object and ID information of the object.

Additionally, the object detector 110 may include a first detection module (not shown) that performs the first detection and a second detection module (not shown) that performs the second detection.

The path tracking unit 120 tracks the movement path of each detected object. As described above, the object detector 110 may extract an object from each frame included in the image received from the camera. The object first detected in the image is given a unique ID along with the representative image and feature information, and by comparing the object image and its features extracted from subsequent frames with the previously stored representative image and its feature information, a plurality of It can be determined whether object images extracted from the frame are identical to each other.

The path tracking unit 120 may determine the location of a plurality of images that are determined to be the same object within the image and store location information of each object according to changes in time.

The first and second cameras (Camera 1, Camera 2) are installed at a fixed location, and the time for storing the location information is from the time the object appears in the image to the time the object disappears from the image. You can.

Meanwhile, even when an object that disappeared from the image reappears, the object detection unit 110 compares a pre-stored image with the extracted image of the object to determine whether it is an object to which an ID has already been assigned, and performs path tracking. Unit 120 may store location information of the object and pre-stored location information together.

The image storage unit 130 stores an image in which information on the ID and the movement path are merged with the images received from the first and second cameras (Camera 1 and Camera 2), and an image of the detected object. Each object detected by the object detector 110 is assigned an ID, and among the objects that appear in each frame included in the image, all objects that are determined to be the same object have the same ID.

In addition, the location information of each object is stored according to the passage of time, and the image stored in the image storage unit 130 may be stored in accordance with the passage of time, that is, the flow of frames. .

For example, an image stored in the image storage unit 130 may include a shape surrounding a detected object, and an ID assigned to the object may be displayed on one side of the shape.

The object image stored in the image storage unit 130 may be an area displayed by the above shape, and the image of an object assigned the same ID may be stored in the same ID field included in the image storage unit 130. You can.

Additionally, the movement path of each object may be displayed as a solid line over time, and the movement paths of different objects may be displayed in different colors.

The display unit 140 displays images received from the first and second cameras (Camera 1 and Camera 2).

The display unit 140 can display images received from the first and second cameras (Camera 1 and Camera 2) in separate areas, and the images stored by the image storage unit 130 are displayed on the display unit 140. It can be displayed through At this time, the display unit 140 can display images captured by the first and second cameras (Camera 1, Camera 2) in real time, or displays images previously stored in the image storage unit 130 in non-real time. It can also be displayed.

Additionally, the display unit 140 displays the movement path of the tracking object and the time section in which the tracking object appears in the images corresponding to the first and second cameras (Camera 1 and Camera 2).

Even when the image is displayed in real time, it can be understood that the ID given to the object detection unit 110 and the location information obtained from the path tracking unit 120 are included in the image.

Figure 2 is a diagram schematically showing the configuration of an object tracking system according to another embodiment of the present invention.

Referring to FIG. 2, the object tracking system 200 according to another embodiment of the present invention includes an object detection unit 210, a path tracking unit 220, an image storage unit 230, a display unit 240, and a reception unit 250. Includes.

The object detection unit 210, the path tracking unit 220, the image storage unit 230, and the display unit 240 are the object detection unit 110, the path tracking unit 120, and the image storage unit ( Since it performs substantially the same function as the display unit 130) and the display unit 140, detailed description will be omitted only for overlapping contents.

The receiving unit 250 receives a selection signal for selecting an object included in an image displayed through the display unit 240. Additionally, the display unit 240 displays the movement path of the object corresponding to the selection signal and images captured by the first and second cameras (Camera 1 and Camera 2).

The selection signal may be generated through an input device such as a mouse or keyboard that is linked to the display unit 240 and transmitted to the receiver 250. If the display unit 240 includes a touch panel, the selection signal may be transmitted to the user. It can be input through touch input.

The selection signal input through the mouse or touch panel can be generated by clicking or touching a specific object on the screen included in the display unit 240, and the selection signal input through the keyboard inputs the ID assigned to the object. It can be created by doing.

The display unit 240 displays images captured by the first and second cameras (Camera 1 and Camera 2), and it can be understood that no merged or processed information is displayed in the image initially displayed on the display unit 240. .

Alternatively, shapes surrounding the object detected by the object detector 110 may be displayed, and the shapes corresponding to different objects may be displayed in different colors. Additionally, the ID assigned to the object may be displayed on one side of the shape.

Meanwhile, when the selection signal is input and a specific object is selected, the display unit 240 may display a movement path corresponding to the selected object. The movement path can be understood as a combination of location information generated by the path tracking unit 220, and as explained with reference to FIG. 1, the movement paths of objects with different IDs are displayed as solid lines with different colors. It can be.

The figure displaying the detected object corresponds to the display area where the object is displayed, and the object detector 210 can extract the display area through the first detection process described with reference to FIG. 1. And, the selection signal input to the receiver 250 corresponds to a signal for selecting the display area.

Figure 3 is a diagram illustrating the operation of an object detection unit according to an embodiment of the present invention.

3(a) and 3(b) exemplarily show specific frames of images captured by a first camera and a second camera at different viewpoints. Referring to Figures 3(a) and 3(b), it can be seen that the object extracted from each image is a person and the extracted object is displayed as a rectangle surrounding it.

The first and second cameras are fixedly installed at different positions and capture images from different directions and angles. In FIG. 3(a), which shows the image captured by the first camera, the first object (OBJ 1) and the second object (OBJ 2) were detected, and in FIG. 3(b) showing the image captured by the second camera, the first to fourth objects (OBJ 1 to OBJ 4) were detected.

Hereinafter, the case where the image in FIG. 3(a) was acquired temporally earlier than the image in FIG. 3(b) will be described as an example. Additionally, object detection, path tracking, image storage, and image display operations will be described using the components of the object tracking system 200 described with reference to FIG. 2.

The object detection unit 210, which receives the image captured by the first camera, extracts images or representative areas corresponding to the first object (OBJ 1) and the second object (OBJ 2) that are determined to be people. Here, the first object (OBJ 1) can be understood as being extracted before the second object (OBJ 2), and the second object (OBJ 2) is different from the first object (OBJ 1) extracted first. It may have characteristics.

Alternatively, since the image shown in FIG. 3(a) represents one frame included in the image captured by the first camera, two or more objects simultaneously included in the same frame are different objects, that is, have different characteristics. It is an independent object that has

The object detection unit 210 assigns IDs to the first and second objects (OBJ 1 and OBJ 2), and the rectangular area representing the display area of the object in FIG. 3(a) is designated as a representative image of the object. You can.

The path tracking unit 220 considers changes in the positions of the first and second objects (OBJ 1 and OBJ 2) over time detected in frames subsequent to the frame shown in FIG. 3(a) and determines the location of each object. The movement path is tracked, and the image storage unit 230 stores an image in which the IDs of the first and second objects OBJ 1 and OBJ 2 and information on the movement paths corresponding to each object are merged.

Meanwhile, when the object detector 210 receives an image from the second camera, it extracts an object included in the received image. As shown in FIG. 3(b), first to fourth objects (OBJ 1 to OBJ 4) are extracted from the image of the second camera.

The object detection unit 210 assigns IDs to the first to fourth objects (OBJ 1 to OBJ 4), and the rectangular area representing the display area of the object in FIG. 3(b) is designated as a representative image of the object. You can.

The path tracking unit 220 considers changes in the positions of the first to fourth objects (OBJ 1 to OBJ 4) over time detected in frames subsequent to the frame shown in FIG. 3(b) and determines the location of each object. The movement path is tracked, and the image storage unit 230 stores an image in which the IDs of the first to fourth objects OBJ 1 to OBJ 4 and information on the movement path corresponding to each object are merged.

At this time, the movement path of the first object (OBJ 1) obtained from the first camera image and the movement path of the first object (OBJ 1) obtained from the second camera image are not merged with or referenced to each other.

As described above, the first and second cameras are fixedly installed in different positions and capture images from different directions and angles, so the location information of the extracted object and the resulting movement path are stored in each camera. Acquired and managed independently.

4 and 5 are diagrams illustrating a screen configuration displayed by a display unit according to an embodiment of the present invention.

FIGS. 4 and 5 illustrate a user interface (UI) provided to the user by the display units 140 and 240 included in the object tracking systems 100 and 200 described with reference to FIGS. 1 and 2. It is expressed as

When explaining the user interface shown in FIG. 4 from the top, a timeline of the image captured by the first camera (Video A) and the image captured by the second camera (Video B) is provided in the first area A. As described with reference to the previous drawings, the first camera and the second camera are fixedly installed at different positions and provide images taken from different directions and at different angles.

Referring to the first area (A), the timeline is provided for each of the images captured by the first and second cameras, and a button is provided to move the image to the user's desired viewpoint.

The second area is a split screen for playing the video taken with the first camera (Video A, hereinafter 'first video') and the video taken with the second camera (Video B, hereinafter 'second video'). to provide. Additionally, it provides a split screen (Trajectory Visualization) that displays the movement path of the object selected by the user.

In the third area (C), a window (Re-ID results @ Video A, Re-ID results @ Video B) indicating the section where the object selected by the user appears in the first and second images is provided, and the first A thumbnail displaying the object extracted from the video and the second video, that is, a window displaying the image or display area of the extracted object (Thumbnail @ Video A, Thumbnail @ Video B) may be provided.

The image or display area represents the detected object and may be stored in the image storage units 130 and 230. In addition, the movement path for the object selected by the user among the images displayed in the third area (C) or the plurality of detected objects shown in the display area may be displayed through the second area (B). .

Referring to FIG. 5, the first image (Video A) captured through the first camera and the second image (Video B) captured through the second camera are exemplarily displayed in the second area (B). As shown and explained with reference to FIG. 4, a section in which a specific object appears in the first image (Video A) and the second image (Video B) may be displayed in the third area (C).

The first image (Video A) and the second image (Video B) shown in FIG. 5 represent first and second objects (OBJ 1 and OBJ 2) and first to fourth objects (OBJ 1 to OBJ 4), respectively. Included, which is the same as the example shown in FIG. 3.

In addition, the path tracking image shown between the first image (Video A) and the second image (Video B) shows the case where the user selects the first object (OBJ 1) as an example, and the first object (OBJ 1) is shown as an example. Displays the movement path of (OBJ 1).

The selection of an object whose movement path is to be tracked can be made by inputting a selection signal by the user, and the selection signal input is one of the objects displayed in the first image (Video A) or the second image (Video B). This can be done through actions such as mouse click or touch to select an object. Additionally, when using a keyboard, this can be done by entering the ID assigned to the corresponding object.

Meanwhile, in the third area (C), the object selected by the user, that is, the section in which the first object (OBJ 1) in FIG. 5 appears in the first image (Video A) and the second image (Video B) Displays .

The section is displayed in the form of a bar as shown in FIG. 5. Referring to FIG. 5, the first object (OBJ 1) is larger than the second image (Video B). ) can be understood as appearing longer in .

Meanwhile, in FIG. 5, the image showing the movement path of the first object (OBJ 1) is either the first image (Video A) or the second image (Video B) according to the user's selection. It can be displayed using, and Figure 5 shows a case where the user selects the second video (Video B).

Selecting either the first image (Video A) or the second image (Video B) is accomplished by selecting one of the bar areas displayed in the third area (C). You can. Alternatively, the tracking target object may be selected by selecting an area where the first image (Video A) is displayed or selecting an area where the second image (Video B) is displayed.

So far, the present invention has been examined focusing on its preferred embodiments. A person skilled in the art to which the present invention pertains will understand that the present invention may be implemented in a modified form without departing from the essential characteristics of the present invention. Therefore, the disclosed embodiments should be considered from an illustrative rather than a restrictive perspective. The scope of the present invention is indicated in the claims rather than the foregoing description, and all differences within the equivalent scope should be construed as being included in the present invention.

100, 200: object tracking system 110, 210: object detection unit
120, 220: path tracking unit 130, 230: image storage unit
140, 240: display unit 250: receiving unit

Claims (5)

an object detection unit that receives images from each of a first camera and a second camera installed at different locations and captures images, detects objects included in the received images, and assigns a unique ID to the detected objects;
a path tracking unit that tracks the movement path of the object detected in the plurality of frames of the image received from the first camera and the movement path of the object detected in the plurality of frames of the image received from the second camera;
An image in which information on the ID and movement path of an object detected in the image received from the first camera are merged with the image received from the first camera, an image of the detected object, and an image received from the second camera an image storage unit that stores an image in which information on the ID and movement path of an object detected in an image received from a second camera are merged and an image of the detected object; and
a display unit that displays an image received from the first camera, an image received from the second camera, and a time section in which a tracking target object appears in the image;
Including,
An image received from the first camera and an image received from the second camera are displayed in the first area of the display unit, respectively,
A path tracking image showing the movement path of the tracking object, which is a selected object among objects included in the image received from the first camera and the image received from the second camera displayed in the first area of the display unit, Displayed in a second area of the display unit different from area 1,
The movement path of the tracking object obtained from the first camera image and the movement path of the tracking object obtained from the second camera image are independent and are not merged or referenced with each other,
The movement path of the tracking object displayed in the path tracking image is the movement path of the object tracked in the image in which the tracking object is selected among the image received from the first camera and the image received from the second camera, Object tracking system. According to paragraph 1,
An object tracking system further comprising a receiver that receives a selection signal for selecting an object included in the image received from the first camera and the image received from the second camera. According to paragraph 2,
The object detection unit extracts a display area including the detected object,
The display unit displays the display area and the ID on an image received from the first camera and an image received from the second camera,
The object to be tracked is an object included in a display area corresponding to the selection signal among display areas displayed in the image received from the first camera and the image received from the second camera. According to paragraph 2,
The image storage unit stores images of objects extracted from each frame included in images received from the first camera and the second camera,
The display unit displays images of objects stored in the image storage unit in an area different from the first area where the image received from the first camera and the image received from the second camera are displayed. According to clause 4,
The object to be tracked is an object included in the image of the object corresponding to the selection signal among the extracted object images.

Images