KR101960667B1 - Suspect Tracking Apparatus and Method In Stored Images - Google Patents

Abstract

The present invention relates to an apparatus of tracking a suspect in stored images and a method thereof. According to the present invention, the method comprises the steps of: storing a plurality of image files photographed by a plurality of monitoring cameras; selecting a first monitoring camera among the plurality of monitoring cameras; playing back a first image file corresponding to the selected first monitoring camera; selecting a target object corresponding to a suspect in a playback screen of the first image file; extracting object features by learning the target object in the first image file; setting a search range including a search target monitoring camera among the plurality of monitoring cameras and a search time; displaying an analysis progress situation screen while a suspect tracking analysis using the object features extracted from an image file included in the set search range is performed; and providing a suspect tracking result including a plurality of thumbnail images in which similarity with the selected target object is displayed to each search target monitoring camera.

Description

[0001] Suspect Tracking Apparatus and Method In Stored Images [

The present invention relates to an apparatus and method for tracking a suspect in a stored image.

Recently, a CCTV (Closed Circuit Television) camera with a built-in DVR (Digital Video Recorder) has been installed in the entrance and boundary areas of the facility to capture the entrance and exit of objects, and the management (work) And a CCTV system in which the video is automatically stored in a video recording and storing device is installed and utilized.

Such a CCTV system often takes a long time to search and does not help to arrest a criminal or a vehicle due to a mistake of a monitoring person. In a video shot by a CCTV camera stored in a video storage device, It is not easy to search for cars or cars. Limited number of CCTV systems, limited monitoring personnel, and increasing crimes are increasingly unable to accurately and quickly locate more CCTV records in real time, which are virtually impossible, have a lot of constraints, There is still a need for monitoring personnel.

When the first suspect is confirmed in the stored CCTV image, there is an increasing demand for a system capable of tracking a suspect in an automatically recorded image based on the detected suspect.

Korean Registered Patent No. 1,092,472 (Registration date: 2011-12-05)

Therefore, the technical problem to be solved by the present invention is to reduce the burden on the monitoring personnel by providing visual and numerical information for tracking suspects by automatically tracking the suspects based on the selected suspects, The present invention relates to a device and method for tracking a suspect in a stored image.

According to an aspect of the present invention, there is provided a method for tracking a suspect in a stored image, the method comprising: storing a plurality of image files photographed by a plurality of surveillance cameras; Selecting a target object corresponding to a suspect on a playback screen of the first video file, selecting a target object corresponding to the target from the first video file, The method comprising the steps of: extracting an object feature by learning an object; setting a search range including a search camera and a search time among the plurality of surveillance cameras; Displaying the analysis progress screen while proceeding with the suspect tracking analysis using the step And providing a suspect tracking result including a plurality of thumbnail images of similarity to the selected target object for each surveillance target camera.

The object feature may be extracted by learning the target object using a plurality of image frames belonging to a section determined based on a point in time when the target object is selected in the first image file.

The method further comprises: selecting a plurality of thumbnail images provided for each of the search target surveillance cameras; arranging a plurality of thumbnail images selected from the user in chronological order and arranging a plurality of thumbnail images selected from the user And a screen showing the suspect's movement line information connected to the surveillance camera position on the map.

The method includes the steps of reinforcing object features by performing reinforcement learning on the target object using a plurality of image frames selected from an image file of a surveillance camera set in the search target surveillance camera, And analyzing the suspect tracking analysis.

The analysis progress screen includes a first user interface for setting a search target surveillance camera by adjusting a search radius based on the position of the first surveillance camera and a second user interface for setting the search time range , Whether the analysis progresses for each slide in which the image file is divided by the predetermined time unit, and whether or not the target object and the object having the predetermined similarity or more are included.

Whether or not an object having a predetermined degree of similarity with the target object is included may be represented in a different color depending on the similarity level.

The method may further comprise: adjusting a search radius through the first user interface and a search time range through the second user interface, and adjusting the search radius of the image file included in the search range adjusted according to the adjusted search radius and the search time range. And displaying the analysis progress screen while proceeding with the suspect tracking analysis using the extracted object features.

The reinforcement learning for the target object may be performed using an image frame including an object whose similarity to the selected target object is equal to or greater than a predetermined reference in the image file of the search target surveillance camera.

The reinforcement learning for the target object is performed using a plurality of image frames belonging to a section determined based on an image frame including an object having the highest similarity to the selected target object in the image file of the search target surveillance camera .

According to an aspect of the present invention, there is provided a susceptor tracking apparatus for a stored image, the apparatus comprising: a storage unit for storing a plurality of image files photographed by a plurality of surveillance cameras; A playback unit for playing back a first image file corresponding to the camera, a display unit for displaying a playback image of the first image file, a target object corresponding to the suspect in the playback image of the first image file, A user input unit configured to set a search range including a search target surveillance camera and a search time among cameras, and a search unit configured to extract an object feature by learning the target object from the first image file, While analyzing the suspect tracking using the extracted object features, the analysis progress screen is displayed And displaying a suspect tracking result including a plurality of thumbnail images showing similarity with the selected target object on the display unit for each search target surveillance camera.

According to the present invention, there is an advantage that the burden of the monitoring personnel can be reduced by automatically tracking the suspect to identify the copper line, and provide visual and numerical information for tracking suspects.

In particular, according to the present invention, when an approximate occurrence time and an occurrence position of an incident are estimated on the basis of the contents of the incident, the image file of the surveillance camera installed at the event occurrence location is reproduced from the vicinity of the occurrence time, Can extract the result of tracking the suspect in a large number of stored image files at a high speed with relatively few samples.

1 is a block diagram illustrating a configuration of a suspect tracking apparatus in a stored image according to an embodiment of the present invention.
2 is a flowchart illustrating an operation of a suspect tracking apparatus in a stored image according to an exemplary embodiment of the present invention.
3 to 7 are views illustrating screens of a suspect tracking program in a stored image according to an embodiment of the present invention.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings so that those skilled in the art can easily carry out the present invention.

1 is a block diagram illustrating a configuration of a suspect tracking apparatus in a stored image according to an embodiment of the present invention.

Referring to FIG. 1, a suspect tracking apparatus 100 using a surveillance camera according to the present invention can receive an image photographed from a plurality of surveillance cameras 1, 2,..., N and display it on a screen.

The surveillance cameras 1, 2, ..., N may be installed at predetermined positions in advance in the surveillance target area. The surveillance cameras 1, 2, ..., N may be implemented as a network camera connected to the object tracking device 100 through a wireless or wired network and capable of transmitting images photographed by the surveillance camera in real time. According to the embodiment, the surveillance cameras 1, 2, ..., N may store the photographed images as image files and transmit the image files requested by the user or stored at predetermined time points to the object tracking device 100 Do.

The object tracking apparatus 100 can register and store information about the surveillance cameras 1, 2, ..., N. [ The surveillance camera specific information may include ID and installation location information uniquely assigned to each camera.

The object tracking apparatus 100 may include an image input unit 110, a playback unit 120, a control unit 130, a display unit 140, a storage unit 150, and a user input unit 160.

The video input unit 110 performs a function for receiving images captured and transmitted from the surveillance cameras 1, 2, ..., N. [ The video input unit 110 may provide a network interface function for receiving video data transmitted from the surveillance cameras 1, 2, ..., N. The image data input through the image input unit 110 may be stored in the storage unit 150 as an image file.

The playback unit 120 can play back the images captured by the surveillance cameras 1, 2, ..., N and transmitted to the display unit 140 in real time. Of course, the playback unit 120 may play the image file selected by the user on the display unit 140 from among the image files stored in the storage unit 150. [

The control unit 130 controls the overall operation of the object tracking apparatus 100. In particular, when the control unit 130 selects a target object corresponding to the suspect from the user on the playback image of the video file, the control unit 130 can learn the target object from the video file to extract the object characteristic. The control unit 130 sets a search range including the search target surveillance camera and the search time among the plurality of surveillance cameras 1, 2, ..., N and acquires the extracted object features for the image files included in the set search range Suspect tracking analysis can be performed. The control unit 130 may display an analysis progress screen and may provide a suspect tracking result including a plurality of thumbnail images of similarity to the selected target object for each search target surveillance camera. Here, the thumbnail image is an image in which an object having a degree of similarity to the target object with a certain degree or more is reduced, and the similarity between the target object and the object recognized in the corresponding frame may be included in the thumbnail image.

For this, the controller 130 performs image processing on the reproduction screen of the image file, recognizes an object such as a person in the image, and calculates the similarity between the object selected from the user and the object recognized in the image have.

The display unit 140 displays various information and data related to the operation of the object tracking apparatus 100 under the control of the control unit 130. The display unit 140 may be implemented as a monitor or the like. Specifically, the display unit 140 displays the images photographed by the surveillance cameras 1, 2, ..., N on the screen, and displays a map on which the surveillance cameras 1, 2, ..., can do.

The storage unit 150 stores various kinds of information and data related to the operation of the object tracking apparatus 100 and provides the information and the data according to a request of the control unit 130. For this, the storage unit 150 may be implemented as various storage media such as a ROM, a RAM, a hard disk, a flash memory, and a solid state disk (SSD).

The storage unit 150 stores the identification information and the installation location information assigned to each of the surveillance cameras 1, 2, ..., N registered in the object tracking apparatus 100 and provides the identification information and the installation location information according to a request from the control unit 130 . Also, the storage unit 150 may store a plurality of image files for image data captured and transmitted from the surveillance cameras 1, 2, ..., N. [

The user input unit 160 may be implemented as a user input means such as a mouse, a keyboard, or a touch pad. The user input unit 160 receives a command related to the operation of the object tracking apparatus 100 from a user and transmits the command to the control unit 130.

FIG. 2 is a flow chart for explaining an operation of a suspect tracking apparatus in a stored image according to an embodiment of the present invention. FIGS. 3 to 7 illustrate a screen of a suspect tracking program in a stored image according to an embodiment of the present invention. Fig.

Referring to FIG. 2, the suspect tracking apparatus 100 may store a plurality of image files photographed from a plurality of surveillance cameras 1, 2,..., N (S205).

Next, the suspect tracking device 100 displays a surveillance camera selection screen F1 as shown in FIG. 3 on the display unit 140 when the user selects the suspect tracking menu from the stored video, and selects a surveillance camera from the user (S210).

In step S210, the user can select one of the plurality of surveillance cameras 1, 2, ..., N. In accordance with the embodiment, the user can input a speed, a search start time, Can be input through the input unit 160. In FIG. 3, 'Camera 1' among the plurality of surveillance cameras 1, 2,..., N is selected.

If the user then selects the play button B1, the playback unit 120 can play the image file corresponding to the 'camera 1' selected in step S210 in the playback screen area P1 (S215).

Next, the suspect tracking device 100 can select a target object corresponding to the suspect in the playback image of the video file reproduced in the playback screen area P1 (S220). The control unit 130 performs image processing on the reproduction screen of the reproduced image file to perform an object recognition operation for recognizing an object such as a person and may cause a predetermined marker such as a rectangle to be displayed around the recognized object. In Fig. 3, three persons are recognized on the playback screen. The object recognition operation can be implemented such that the user watches the reproduction screen and starts to be pressed by pressing the object selection button B2. Of course, it is also possible to implement an object recognition operation during reproduction of a video file selected from a user. If the user clicks one of the recognized objects on the playback screen through a mouse or the like, the target object corresponding to the suspect can be selected. Also, if one of the objects recognized on the playback screen is clicked through a mouse or the like and then the object selection button B2 is pressed, it is possible to process the target object as being finally selected.

Next, the control unit 130 may extract the object feature by learning the target object selected from the image file corresponding to 'camera 1' (S225). In step S225, the controller 130 learns the target object by using a plurality of image frames belonging to a section determined based on a time point at which the target object is selected in the image file corresponding to 'camera 1' . For example, an object feature can be extracted by learning a target object using a plurality of image frames belonging to a section of a predetermined time before the target object is selected. For example, if the predetermined time is 5 seconds and 30 frames per second, object characteristics can be extracted by learning the target object using 5 seconds 30 frames, that is, 150 frames.

Then, the suspect tracking device 100 can set a search range including the search target surveillance camera and the search time among the plurality of surveillance cameras through the analysis progress screen as illustrated in FIG. 4 (S230).

The analysis progress screen includes a first user interface 21 for setting a search target surveillance camera by adjusting a search radius based on the position of the 'camera 1' selected in step S210 from the user, And a second user interface 22.

When the user sets the search time range with the search target surveillance camera through the first user interface 21 and the second user interface 22 via the user input unit 160 and then presses the analysis start button B3, 130 may display an analysis progress status screen while proceeding with the suspect tracking analysis using the extracted object features with respect to the image file included in the set search range (S235). In FIG. 4, 'Camera 2', 'Camera 3', and 'Camera 4' are included in the search range according to the search radius based on 'Camera 1'. The control unit 130 indicates that suspect tracking analysis is being performed in parallel on the image files of 'camera 1', 'camera 2', 'camera 3', and 'camera 4'.

As shown in FIG. 4, the analysis progress screen is a screen for displaying an image file corresponding to each of 'Camera 1', 'Camera 2', 'Camera 3' and 'Camera 4' And a time line chart 23 indicating whether or not the slides are divided for each slide and whether or not an object having more than a predetermined degree of similarity with the target object is included. For example, whether or not an object having a predetermined degree of similarity or more with the target object is included can be represented in a different color according to the similarity level. For example, the slide S1 includes an object having a similarity of 90% or more to the target object, and the slide S2 includes an object having a similarity of 80% or more to the target object. On the other hand, the slide S3 indicates that the analysis is currently in progress, and the slide S4 indicates that the analysis has not been performed yet.

On the other hand, the analysis progress screen may display the entire analysis progress rate as illustrated in FIG. 4, and may include a button (B6) for selecting and removing a part of the cameras belonging to the search radius or a part of the cameras not belonging to the search radius And a button B5 for adding as an object. For example, in the case of 'Camera 4', all the analyzed slides show that only objects having a degree of similarity of 70% or less with the target object are included, so that it is possible to exclude from the search range, .

If the user selects a specific slide in the analysis progress screen shown in FIG. 4, the control unit 130 displays a tracking detail window P2 including a plurality of thumbnail images indicating the degree of similarity between the object recognized in the slide and the target object, May be popped up on the screen of the display unit 140 as illustrated in FIG. If a specific thumbnail image is selected in the tracking detail window P2, the window P3 including the detailed image showing the corresponding thumbnail image may be popped up.

On the other hand, if the analysis suspend button B7 is selected from the user during the analysis, the control unit 130 can suspend analysis for suspect tracking. When the user resets the search time range with the search target surveillance camera through the first user interface 21 and the second user interface 22 or adds or deletes the search target camera through the buttons B5 and B6, The analysis can be resumed for the image file corresponding to the adjusted search range.

Meanwhile, when analysis of a plurality of image files corresponding to the set search range is completed, the control unit 130 classifies suspect tracking results, which are composed of a plurality of thumbnail images indicating similarity with the selected target object, A screen F2 as shown in FIG. 6B may be configured and provided through the display unit 140 (S240). In FIG. 6, a thumbnail image having a bold square is selected from the user.

The user can select a plurality of thumbnail images from among a plurality of thumbnail images provided for each search target surveillance camera on the screen F2 (S245).

When the user then presses the target selection completion button B8, the control unit 130 can provide a screen F3 arranged by arranging a plurality of thumbnail images selected by the user in chronological order, as illustrated in FIG. 7 (S250 ). In addition, the controller 130 may provide the screen F4 showing the suspector copper line information connected with the surveillance camera positions corresponding to the plurality of thumbnail images selected by the user, as illustrated in FIG. 7 S255). Of course, it is possible to simultaneously display the screen F3 and the screen F4 by performing the step S245 and the step S250 at the same time, or display only the remaining screen except for one of the screen F3 and the screen F4 .

If the user presses the target re-select button B9 after the step S245 and / or S250 according to the embodiment, the process returns to the step S240 to return a plurality of thumbnail images from the user on the screen F2 You may be selected.

When the user selects the save result button B10, the control unit 130 may store the suspect tracking result including the screen F3 and the screen F4 in the storage unit 150. [

On the other hand, when the user selects the reinforcement learning button B11, reinforcement learning is performed on the target object using a plurality of image frames selected from the video file of the surveillance camera set as the search target surveillance camera, The analysis can be resumed (S260). For example, the control unit 130 may perform reinforcement learning by extracting a plurality of image frames including a target object from the image files of the camera 1, the camera 2, and the camera 3. In the first stage, object features are extracted using only one image file of the camera. In the reinforcement learning, object features can be extracted more accurately because the target object is extracted by using the image files of the other surveillance cameras from which the target object is extracted. The control unit 130 can re-execute the suspect tracking analysis by repeating steps S230 to S255 using the object characteristics reinforced in step S260.

Embodiments of the present invention include a computer-readable medium having program instructions for performing various computer-implemented operations. The medium records a program for executing the above-described method. The medium may include program instructions, data files, data structures, etc., alone or in combination. Examples of such media include magnetic media such as hard disks, floppy disks and magnetic tape, optical recording media such as CD and DVD, programmed instructions such as floptical disk and magneto-optical media, ROM, RAM, And a hardware device configured to store and execute the program. Examples of program instructions include machine language code such as those produced by a compiler, as well as high-level language code that can be executed by a computer using an interpreter or the like.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is to be understood that the invention is not limited to the disclosed exemplary embodiments, Of the right.

Claims (18)

Storing a plurality of image files photographed by a plurality of surveillance cameras,
Selecting a first surveillance camera among the plurality of surveillance cameras,
Reproducing the first image file corresponding to the selected first surveillance camera,
Receiving a target object corresponding to a suspect on a playback screen of the first video file,
Extracting object features by learning the target object from the first image file,
Setting a search range including a search target surveillance camera and a search time among the plurality of surveillance cameras;
Displaying an analysis progress status screen while proceeding with a suspect tracking analysis using the extracted object features for an image file included in the set search range, and
A step of providing a suspect tracking result including a plurality of thumbnail images showing the degree of similarity with the selected target object for each surveillance target surveillance camera
/ RTI >
The analysis progress screen,
A time line chart (23) for indicating whether the analysis progress is performed for each slide in which the image file is divided by the predetermined time unit, and whether or not the target object and the object having the predetermined similarity or more are included,
A first user interface for setting a search target surveillance camera by adjusting a search radius based on the position of the first surveillance camera, a second user interface for setting the search time range,
A button B6 for selecting a part of the surveillance cameras belonging to the search radius and excluded from the analysis target, and a button B5 for selecting a part of the surveillance cameras not belonging to the search radius to be included in the analysis target Suspect tracking method in video. The method of claim 1,
And extracting object features by learning the target object using a plurality of image frames belonging to a section determined based on a point in time when the target object is selected in the first image file. 3. The method of claim 2,
Receiving a plurality of thumbnail images from among a plurality of thumbnail images provided for each search target surveillance camera,
Providing at least one of a screen on which the selected plurality of thumbnail images are arranged in chronological order and a screen on which the suspector copper line information connecting surveillance camera positions corresponding to the selected plurality of thumbnail images is displayed on a map
The method comprising the steps of: 3. The method of claim 2,
Performing reinforcement learning on the target object by using a plurality of image frames selected from an image file of a surveillance camera set in the search target surveillance camera to reinforce object features;
The step of retrying the suspect tracking analysis using the reinforced object feature
The method comprising the steps of: delete The method of claim 1,
Wherein the presence or absence of an object having a predetermined degree of similarity with the target object is represented by a different color according to the similarity level. The method of claim 1,
Adjusting a search radius through the first user interface and a search time range through the second user interface, and
Displaying an analysis progress status screen while proceeding with the suspect tracking analysis using the extracted object features for an image file included in the search range adjusted according to the adjusted search radius and the search time range;
The method comprising the steps of: 5. The method of claim 4,
Wherein the reinforcement learning for the target object is performed using an image frame including an object whose similarity to the selected target object is equal to or greater than a predetermined reference in the image file of the search target surveillance camera. 5. The method of claim 4,
Performing reinforcement learning on the target object by using a plurality of image frames belonging to an interval determined on the basis of the image frame including the object having the highest similarity to the selected target object in the image file of the search target surveillance camera Suspect tracking method in stored image. A storage unit for storing a plurality of image files photographed by the plurality of surveillance cameras,
A playback unit for playing back a first video file corresponding to a first surveillance camera among the plurality of surveillance cameras,
A display unit for displaying a reproduction screen of the first image file,
A user input unit for selecting a target object corresponding to a suspect in a playback image of the first image file and setting a search range including a search target surveillance camera and a search time among the plurality of surveillance cameras;
Extracting object features by learning the target object from the first image file, analyzing the suspect object tracking using the extracted object features for the image files included in the set search range, and displaying an analysis progress status screen on the display unit And displaying a suspect tracking result including a plurality of thumbnail images showing similarity with the selected target object on the display unit for each search target surveillance camera,
/ RTI >
The analysis progress screen,
A time line chart (23) for indicating whether the analysis progress is performed for each slide in which the image file is divided by the predetermined time unit, and whether or not the target object and the object having the predetermined similarity or more are included,
A first user interface for setting a search target surveillance camera by adjusting a search radius based on the position of the first surveillance camera, a second user interface for setting the search time range,
A button B6 for selecting a part of the surveillance cameras belonging to the search radius and excluded from the analysis target, and a button B5 for selecting a part of the surveillance cameras not belonging to the search radius to be included in the analysis target Suspect tracking device in video. 11. The method of claim 10,
Wherein,
Wherein the object feature is extracted by learning the target object using a plurality of image frames belonging to a section determined based on a point in time when the target object is selected in the first image file. 12. The method of claim 11,
Wherein the user input unit comprises:
A plurality of thumbnail images provided for each of the search target surveillance cameras,
Wherein,
A display screen displaying at least one of a screen on which the selected plurality of thumbnail images are arranged in chronological order and a screen on which the surveillance camera information corresponding to the selected plurality of thumbnail images is connected to the map on the map, Suspect tracking device. 12. The method of claim 11,
Wherein,
The object feature is reinforced by performing reinforcement learning on the target object using a plurality of image frames selected from the image file of the surveillance camera set in the search target surveillance camera, and the suspect tracker analysis is performed using the enhanced object feature A suspect tracking device in a retrospective stored image. delete 11. The method of claim 10,
Wherein the presence or absence of an object having a predetermined degree of similarity with the target object is represented by a different color according to the degree of similarity. 11. The method of claim 10,
Wherein,
Wherein the control unit adjusts the search radius through the first user interface and the search time range through the second user interface and searches the extracted object for the image file included in the search range adjusted according to the adjusted search radius and the search time range, A suspect tracking system in a stored image that displays an analysis progress screen while carrying out a suspect tracking analysis using features. The method of claim 13,
Wherein the reinforcement learning for the target object is performed using an image frame including an object whose similarity to the selected target object is equal to or higher than a predetermined reference in the image file of the search target surveillance camera. The method of claim 13,
Performing reinforcement learning on the target object by using a plurality of image frames belonging to an interval determined on the basis of the image frame including the object having the highest similarity to the selected target object in the image file of the search target surveillance camera Suspect tracking device in stored video.

Images