WO2024071619A1

WO2024071619A1 - Electronic device and method for semi-automatically searching for cctv videos

Info

Publication number: WO2024071619A1
Application number: PCT/KR2023/010682
Authority: WO
Inventors: 양창모; 김동칠; 서경은; 오세호
Original assignee: 한국전자기술연구원
Priority date: 2022-09-30
Filing date: 2023-07-24
Publication date: 2024-04-04
Also published as: KR20240045469A

Abstract

An electronic device for semi-automatically searching for CCTV videos is provided. The electronic device comprises: an input unit that receives an input of certain information from a user; a communication unit that transmits and receives data to and from a plurality of CCTVs; a memory storing a program for semi-automatically searching for CCTV videos; and a processor that, in response to executing the program stored in the memory, when a selection input for a CCTV to be played among the plurality of CCTVs is received from a user, searches for CCTV videos on the basis of input time information for the at least one selected CCTV, generates one CCTV video playlist on the basis of moving object portion information in the found CCTV videos, and plays a CCTV video on the basis of the generated CCTV video playlist.

Description

Electronic device and method for semi-automatic CCTV image search

The present invention relates to an electronic device and method for semi-automatic CCTV image search.

Conventionally, when an administrator wants to manually search CCTV images stored in a VMS (Video Management System), he or she selects the CCTV and recording time to be played, searches and selects the images to be played, and then the user plays the selected images one by one. The main method used was to search for scenes of moving objects, such as the person or object to be found.

This prior art has the disadvantage of having to manually play and check even scene images that do not include moving objects such as people or objects in the CCTV images, so it takes a lot of time to play and search the images.

In addition, when there are multiple CCTVs, the images stored in each of the multiple CCTVs must be selected and played one by one, which also has the disadvantage of consuming a lot of time to play and search the images.

An embodiment of the present invention performs a manual search through video playback for scenes in which moving objects, such as people or objects, appear in CCTV images stored in the VMS, by extracting and splicing partial information about the moving objects to create a single video playlist. An electronic device and method for semi-automatic CCTV image search that can improve convenience for video playback and shorten search time are provided.

However, the technical challenge that this embodiment aims to achieve is not limited to the technical challenges described above, and other technical challenges may exist.

As a technical means for achieving the above-mentioned technical problem, a semi-automatic CCTV image search method according to the first aspect of the present invention includes the steps of receiving a selection input for a playback target CCTV from a plurality of CCTVs from a user; Performing a CCTV image search based on input time information for the selected at least one CCTV; Creating a CCTV video playlist based on the moving object part information in the retrieved CCTV video; and playing CCTV images based on the generated CCTV image playlist.

In some embodiments of the present invention, receiving a selection input for a CCTV to be played among a plurality of CCTVs from the user includes searching for a CCTV corresponding to information on a specific area of interest; And it may include receiving a selection input for a CCTV to be played from the user among the list of searched CCTVs.

In some embodiments of the present invention, the step of generating one CCTV image playlist based on the moving object part information in the retrieved CCTV image includes the first moving object part information detected for each CCTV image and the subsequent detection thereof. The single CCTV image playlist can be created by combining the second moving object partial information.

In some embodiments of the present invention, the step of generating one CCTV image playlist based on the moving object partial information in the retrieved CCTV images includes arranging the detected moving object partial information in a plurality of CCTV images in time series order; , the single CCTV image playlist can be created by combining the sorted moving object partial information.

Some embodiments of the present invention include receiving an input for image information search from the user; And it may further include providing video information including video capture time, CCTV ID and location information, and video file name for the moving object portion information of the CCTV video playlist in response to the input.

In addition, the electronic device for semi-automatic CCTV image search according to the second aspect of the present invention includes an input unit that receives predetermined information from the user, a communication unit that transmits and receives data to and from a plurality of CCTVs, a memory storing a program for semi-automatic CCTV image search, and Upon executing the program stored in the memory, upon receiving a selection input for a playback target CCTV among the plurality of CCTVs from the user, performing a CCTV image search based on the input time information for the selected at least one CCTV, It includes a processor that generates one CCTV image playlist based on the moving object part information in the searched CCTV image, and plays the CCTV image based on the generated CCTV image playlist.

In some embodiments of the present invention, the processor searches for CCTVs corresponding to specific area of interest information input through an input unit, and receives a selection input for a CCTV to be played from the user among the list of searched CCTVs. .

In some embodiments of the present invention, the processor may generate the one CCTV image play list by combining the first moving object partial information detected for each CCTV image and the second moving object partial information detected subsequently. there is.

In some embodiments of the present invention, the processor may arrange the moving object partial information detected in a plurality of CCTV images in time series order and combine the sorted moving object partial information to generate the one CCTV image playlist. there is.

In some embodiments of the present invention, when the processor receives an input for video information search from the user, the processor responds to the input by providing video capture time, CCTV ID, and location for the motion object part information of the CCTV video playlist. Video information including information and video file name can be provided.

In addition to this, another method for implementing the present invention, another system, and a computer-readable recording medium recording a computer program for executing the method may be further provided.

According to an embodiment of the present invention described above, the time for a user to search for a moving object in a CCTV image can be dramatically shortened.

In addition, it is inconvenient to manually select and play video files for each CCTV, and rather than checking all parts where no moving object appears, it is possible to play back the video for multiple detected moving object parts as if they were one video. It has the advantage of being able to quickly track and respond to incidents such as crimes by allowing it to be played back as it is.

The effects of the present invention are not limited to the effects mentioned above, and other effects not mentioned will be clearly understood by those skilled in the art from the description below.

1 is a block diagram showing the configuration of an electronic device according to an embodiment of the present invention.

Figure 2 is a flow chart of a semi-automatic CCTV image search method according to an embodiment of the present invention.

Figure 3 is a diagram for explaining extracting partial moving object information from CCTV images.

The advantages and features of the present invention and methods for achieving them will become clear by referring to the embodiments described in detail below along with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below and may be implemented in various different forms. The present embodiments are merely provided to ensure that the disclosure of the present invention is complete and to provide a general understanding of the technical field to which the present invention pertains. It is provided to fully inform the skilled person of the scope of the present invention, and the present invention is only defined by the scope of the claims.

The terminology used herein is for describing embodiments and is not intended to limit the invention. As used herein, singular forms also include plural forms, unless specifically stated otherwise in the context. As used in the specification, “comprises” and/or “comprising” does not exclude the presence or addition of one or more other elements in addition to the mentioned elements. Like reference numerals refer to like elements throughout the specification, and “and/or” includes each and every combination of one or more of the referenced elements. Although “first”, “second”, etc. are used to describe various components, these components are of course not limited by these terms. These terms are merely used to distinguish one component from another. Therefore, it goes without saying that the first component mentioned below may also be a second component within the technical spirit of the present invention.

Unless otherwise defined, all terms (including technical and scientific terms) used in this specification may be used with meanings commonly understood by those skilled in the art to which the present invention pertains. Additionally, terms defined in commonly used dictionaries are not interpreted ideally or excessively unless clearly specifically defined.

Hereinafter, an electronic device 100 for semi-automatic CCTV image search according to an embodiment of the present invention will be described with reference to FIG. 1.

Figure 1 is a block diagram showing the configuration of an electronic device 100 according to an embodiment of the present invention.

The electronic device 100 according to an embodiment of the present invention includes an input unit 110, a communication unit 120, a display unit 130, a memory 140, and a processor 150. At this time, the electronic device in the present invention may be a VMS.

The input unit 110 generates input data in response to user input of the electronic device 100. At this time, user input includes input of selection for CCTV, input of time information for CCTV video search, and control inputs such as video playback, pause, video information request, and termination based on the generated CCTV video playlist. can do.

The input unit 110 includes at least one input means. The input unit 110 includes a keyboard, key pad, dome switch, touch panel, touch key, mouse, menu button, etc. may include.

The communication unit 120 transmits and receives data to and from a plurality of CCTVs, and also communicates with external devices such as servers and data collection devices to transmit and receive data. This communication unit 120 may include both a wired communication module and a wireless communication module. The wired communication module can be implemented as a power line communication device, telephone line communication device, home cable (MoCA), Ethernet, IEEE1294, integrated wired home network, and RS-485 control device. In addition, wireless communication modules include WLAN (wireless LAN), Bluetooth, HDR WPAN, UWB, ZigBee, Impulse Radio, 60GHz WPAN, Binary-CDMA, wireless USB technology and wireless HDMI technology, as well as 5G (5th generation communication) and LTE-A. It may be composed of modules to implement functions such as (long term evolution-advanced), LTE (long term evolution), and Wi-Fi (wireless fidelity).

The display unit 130 displays display data according to the operation of the electronic device 100. As an example, the display unit 130 may display a CCTV list, moving object part information, a CCTV image playlist, and CCTV images based on the CCTV image and CCTV image playlist on the screen.

The display unit 130 includes a liquid crystal display (LCD), a light emitting diode (LED) display, an organic light emitting diode (OLED) display, and a micro electro mechanical systems (MEMS) display. and electronic paper displays. The display unit 130 may be combined with the input unit 110 and implemented as a touch screen.

The memory 140 stores programs for semi-automatic CCTV image search. Here, the memory 140 is a general term for non-volatile storage devices and volatile storage devices that continue to retain stored information even when power is not supplied. For example, memory 140 may include compact flash (CF) cards, secure digital (SD) cards, memory sticks, solid-state drives (SSD), and micro SD. This includes NAND flash memory such as cards, magnetic computer storage devices such as hard disk drives (HDD), and optical disc drives such as CD-ROM, DVD-ROM, etc. You can.

The processor 150 may control at least one other component (eg, hardware or software component) of the electronic device 100 by executing software such as a program, and may perform various data processing or calculations.

When the processor 150 receives a selection input for a CCTV to be played among a plurality of CCTVs from the user, it performs a CCTV image search based on the input time information for at least one selected CCTV. Then, the processor 150 creates a CCTV image playlist based on the moving object part information in the searched CCTV image, and plays the CCTV image based on the generated CCTV image playlist.

Meanwhile, in one embodiment of the present invention, the processor 150 may use at least one of machine learning, neural network, or deep learning algorithms as an artificial intelligence algorithm for semi-automatic CCTV image search. there is. For example, as an artificial intelligence algorithm, at least one of machine learning, neural network, or deep learning algorithm may be used. Examples of neural network networks include Convolutional Neural Network (CNN) and Deep Neural Network (DNN). Network) and RNN (Recurrent Neural Network).

Hereinafter, a semi-automatic CCTV image search method performed by the electronic device 100 according to an embodiment of the present invention will be described.

Figure 2 is a flow chart of a semi-automatic CCTV image search method according to an embodiment of the present invention. Figure 3 is a diagram for explaining extracting partial moving object information from CCTV images.

First, as the program for semi-automatic CCTV image search starts, the processor 150 receives an input from the user to select a CCTV to be played among a plurality of CCTVs (S110).

In one embodiment, the user can directly select specific CCTVs from the entire CCTV list output through the display unit.

In one embodiment, as information on a specific area of interest is pre-designated or input by a user, the processor 150 searches CCTV corresponding to the information on the specific area of interest. Accordingly, the user who has checked the searched CCTV list can directly select the CCTVs to be played.

In one embodiment, the processor 150 may display and provide location information of CCTVs through a GIS map, and the user may also manually select CCTVs displayed on the GIS map.

Next, the processor 150 performs a CCTV image search based on the input time information for at least one selected CCTV (S120).

In one embodiment, the user may specify the same start time and end time for a plurality of CCTVs selected with time information, or may specify different start times and end times for each. In response to these start times and end times, the processor 150 performs a CCTV image search for each CCTV.

Next, the processor 150 generates one CCTV image playlist based on the moving object part information in the searched CCTV image (S130).

In one embodiment, the processor 150 may analyze each searched CCTV image and generate information about the part where the moving object appears as moving object partial information.

In another embodiment, when the electronic device 100 and the VMS are each independently configured, the processor 150 receives information on the appearance and disappearance of a moving object previously analyzed by the VMS and provides information about the corresponding part to the moving object part. It can be created with information.

At this time, in one embodiment of the present invention, the moving object partial information may be identification information such as index information for the image rather than the image itself.

Meanwhile, in one embodiment of the present invention, various algorithms can be applied to the method for determining and detecting whether an object in a CCTV image is moving, and is not specified and implemented using any one method. For example, it is possible to determine and detect whether an object is moving by checking the same area, a changed area, or a change value greater than a threshold on a frame-by-frame basis within one CCTV image.

Once the moving object partial information is generated in this way, the processor 150 combines the first moving object partial information detected for each CCTV image and the second moving object partial information subsequently detected to create one CCTV image playlist. can be created.

Referring to FIG. 3 as an example, when the first and second CCTV images corresponding to time information (12:00 to 13:00) are searched for in the first CCTV and the second CCTV, the first moving object portion is first selected from the first CCTV image. The information and the subsequent second moving object partial information can be combined to create one CCTV image playlist. Afterwards, the same process is performed for the second CCTV image and added to the CCTV image playlist. At this time, the order of the motion object part information for each CCTV in the CCTV video playlist can be initially determined based on the order of the user's CCTV selection input, and of course can be changed according to the user's input afterwards.

In another embodiment, the processor 150 may sort the moving object partial information detected in a plurality of CCTV images in chronological order and combine the sorted moving object partial information to create one CCTV image playlist. there is.

As an example, 1-1 motion object partial information and 1-2 motion object partial information are detected in the first CCTV, and 2-1 motion object partial information and 1-2 motion object partial information are detected in the second CCTV. When detected, by sorting them in time order, one CCTV image playlist arranged in the following order can be created: 1st-1st, 2nd-1st, 2-2nd, 1-2nd.

In another embodiment, the processor 150 sorts the moving object partial information detected in a plurality of CCTV images by the same object, and sorts the group units sorted by the same object in time series order to generate one CCTV image playlist. can do. To determine whether an object is the same, the processor 150 may use a pre-trained artificial intelligence algorithm such as deep learning or machine learning.

At this time, the learning data for learning the artificial intelligence algorithm may consist of an image corresponding to the moving object partial information as input data, and the correct answer label for this may consist of the same object or a different object as output data. In the case of learning data, if time information for the same object in the first CCTV image and the second CCTV image overlaps, a predetermined weight may be additionally assigned to the corresponding learning data.

As an example, a predetermined weight may be determined based on the number of overlapping sections and the length of the overlapping sections of time information for the same object in the first CCTV image and the second CCTV image. In other words, even if the number of overlapping sections in which the same object appears in each CCTV image is the same, if the length of the overlapping section is longer, a greater weight can be assigned to the corresponding CCTV image.

A CCTV image playlist created according to this process is not created by encoding multiple images in which actual moving objects are detected into a single image, but is created based on low-capacity data such as index information. , It is possible to play CCTV video for the corresponding part through the CCTV video playlist. Through this, the user has the advantage of being able to check moving objects with only one CCTV image generated based on one CCTV image playlist, without having to check all images from numerous CCTVs one by one.

Next, the processor 150 waits for the user's input (S140), and when receiving the user's CCTV video playback input through the input unit 110 (S150-play), the processor 150 uses the CCTV video playlist based on the generated CCTV video playlist. Play the video (S160).

At this time, the processor 150 can continuously play partial images corresponding to the moving object partial information on the CCTV image playlist using the Random Jump function. Accordingly, the user can watch the video as if a single file is being played, rather than multiple partial videos being played individually.

Meanwhile, when the previous state of the CCTV video played based on the CCTV video playlist corresponds to 'stop', the processor 150 can play the CCTV video from the first time based on the CCTV video playlist.

Alternatively, when the previous state of the CCTV video played based on the CCTV video playlist corresponds to 'pause', the processor 150 can play the CCTV video that continues after the paused time based on the CCTV video playlist. there is.

When this playback process is completed, the processor 150 returns to the user input waiting stage (S140).

In addition, when the processor 150 receives the user's 'pause' input through the input unit 110 (S150-pause), the processor 150 pauses the CCTV image being played (S170) and returns to the user input waiting stage. Do it (S140).

In addition, when the processor 150 receives the user's 'information search' input through the input unit 110 (S150-information search), the processor 150 provides video capture time, CCTV ID, and location for the moving object part information of the CCTV video playlist. Video information including information and video file name is provided (S180), and the process returns to the user input waiting stage (S140).

Additionally, when the processor 150 receives the user's 'end' input through the input unit 110 (S150-end), the processor 150 ends the semi-automatic CCTV image search method (S190).

Meanwhile, in the above description, steps S110 to S190 may be further divided into additional steps or combined into fewer steps, depending on the implementation of the present invention. Additionally, some steps may be omitted or the order between steps may be changed as needed. In addition, even if other omitted content, the content described in FIG. 1 and the content described in FIG. 2 are mutually applicable.

The embodiments of the present invention described above may be implemented as a program (or application) and stored in a medium in order to be executed in conjunction with a server, which is hardware.

The above-mentioned program is C, C++, JAVA, machine language, etc. that can be read by the processor (CPU) of the computer through the device interface of the computer in order for the computer to read the program and execute the methods implemented in the program. It may include code coded in a computer language. These codes may include functional codes related to functions that define the necessary functions for executing the methods, and include control codes related to execution procedures necessary for the computer's processor to execute the functions according to predetermined procedures. can do. In addition, these codes may further include memory reference-related codes that indicate at which location (address address) in the computer's internal or external memory additional information or media required for the computer's processor to execute the above functions should be referenced. there is. In addition, if the computer's processor needs to communicate with any other remote computer or server in order to execute the above functions, the code uses the computer's communication module to determine how to communicate with any other remote computer or server. It may further include communication-related codes regarding whether communication should be performed and what information or media should be transmitted and received during communication.

The storage medium refers to a medium that stores data semi-permanently and can be read by a device, rather than a medium that stores data for a short period of time, such as a register, cache, or memory. Specifically, examples of the storage medium include ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical data storage device, etc., but are not limited thereto. That is, the program may be stored in various recording media on various servers that the computer can access or on various recording media on the user's computer. Additionally, the medium may be distributed to computer systems connected to a network, and computer-readable code may be stored in a distributed manner.

The steps of the method or algorithm described in connection with embodiments of the present invention may be implemented directly in hardware, implemented as a software module executed by hardware, or a combination thereof. The software module may be RAM (Random Access Memory), ROM (Read Only Memory), EPROM (Erasable Programmable ROM), EEPROM (Electrically Erasable Programmable ROM), Flash Memory, hard disk, removable disk, CD-ROM, or It may reside on any type of computer-readable recording medium well known in the art to which the present invention pertains.

Above, embodiments of the present invention have been described with reference to the attached drawings, but those skilled in the art will understand that the present invention can be implemented in other specific forms without changing its technical idea or essential features. You will be able to understand it. Therefore, the embodiments described above should be understood in all respects as illustrative and not restrictive.

Claims

In a method performed by a computer,

Receiving a selection input for a playback target CCTV from a plurality of CCTVs from a user;

Performing a CCTV image search based on input time information for the selected at least one CCTV;

Creating a CCTV video playlist based on the moving object part information in the retrieved CCTV video; and

Comprising the step of playing CCTV video based on the generated CCTV video playlist,

Semi-automatic CCTV video search method.
According to paragraph 1,

The step of receiving a selection input for a CCTV to be played among a plurality of CCTVs from the user,

Searching CCTV corresponding to information on a specific area of interest; and

Comprising the step of receiving a selection input for a playback target CCTV from the user among the list of searched CCTVs,

Semi-automatic CCTV video search method.
According to paragraph 1,

The step of generating a CCTV video playlist based on the moving object part information in the retrieved CCTV video is,

Generating the one CCTV image playlist by combining the first moving object partial information detected for each CCTV image and the second moving object partial information subsequently detected,

Semi-automatic CCTV video search method.
According to paragraph 1,

The step of generating a CCTV video playlist based on the moving object part information in the retrieved CCTV video is,

Sorting the moving object partial information detected in a plurality of CCTV images in time series order and combining the sorted moving object partial information to generate the one CCTV image playlist,

Semi-automatic CCTV video search method.
According to paragraph 1,

Receiving input for image information search from the user; and

Further comprising providing video information including video capture time, CCTV ID and location information, and video file name for the moving object portion information of the CCTV video playlist in response to the input,

Semi-automatic CCTV video search method.
An input unit that receives certain information from the user,

A communication department that transmits and receives data to multiple CCTVs,

Memory containing a program for semi-automatic CCTV image search and

Upon executing the program stored in the memory, upon receiving a selection input for a playback target CCTV among the plurality of CCTVs from the user, performing a CCTV image search based on the input time information for the selected at least one CCTV, A processor that generates a CCTV video playlist based on the moving object part information in the retrieved CCTV video, and plays the CCTV video based on the generated CCTV video playlist,

Electronic device for semi-automatic CCTV video search.
According to clause 6,

The processor searches for CCTV corresponding to information on a specific area of interest input through an input unit, and receives a selection input for a CCTV to be played from the user among the list of searched CCTVs,

Electronic device for semi-automatic CCTV video search.
According to clause 6,

The processor generates the one CCTV image playlist by combining the first moving object partial information detected for each CCTV image and the second moving object partial information subsequently detected.

Electronic device for semi-automatic CCTV video search.
According to clause 6,

The processor sorts the moving object partial information detected in a plurality of CCTV images in time series order and combines the sorted moving object partial information to generate the one CCTV image playlist,

Electronic device for semi-automatic CCTV video search.
According to clause 6,

When the processor receives an input for video information search from the user, the processor provides video information including video capture time, CCTV ID and location information, and video file name for the moving object part information of the CCTV video playlist in response to the input. To provide,

Electronic device for semi-automatic CCTV video search.