KR20230032202A - Effective image searching method and system thereof for searching event situation - Google Patents

Abstract

The present invention relates to an efficient video search method for finding an event situation and a system thereof. The present invention can quickly and accurately search for videos when a dispute occurs among all customers who use safety/security services using videos. In addition, it is possible to provide a function for monitoring convenience which reduces the physical time for monitoring when searching for a video at the point of dispute and allows a searcher to quickly find the desired video.

Description

Efficient image search method and system for finding event situation

The present invention relates to an efficient video search method and system for finding an event situation.

In recent years, cases in which CCTV cameras for safety/security are installed inside or outside a building or on a street for various purposes such as crime prevention, security, and store management are rapidly increasing. There are three main functions of CCTV cameras: real-time video control, video storage, and video playback function to view video in the event of a dispute such as theft/fire/violence. Among them, the most important function is the video playback function, which consumes a lot of time to find the video at the time when the problem occurred. In addition, video monitoring time increases when there are multiple cameras, and when multiple cameras are monitored at the same time or double speed playback is performed to shorten the search time, the customer cannot check the desired video and monitors again. For this reason, many developers are suggesting various methods to drastically reduce the customer's video monitoring time and to provide an efficient video search service that can be monitored without missing important video.

The present invention aims to address the aforementioned needs and/or problems.

In addition, an object of the present invention is to provide an efficient image retrieval method and system for finding an event situation that dramatically reduces the physical time to check the image and quickly finds a camera image at the time of a dispute.

According to one aspect of an efficient video search method for finding an event situation according to the present invention for achieving the above object, an image storage server stores a camera-captured image; requesting, by a client, an image search for a predetermined time to find a specific event situation in an image stored in the image storage server; Receiving, by the control server, a video search request for a predetermined time to find a specific event situation from the client; Extracting, by an artificial intelligence analysis server, an event generated during a predetermined time in the video storage server according to a request from the control server; and the image processing server divides and reproduces the image for a predetermined time including the event extracted from the artificial intelligence analysis server into time-division images as many as the preset number of split screens, and reproduces the reproduced time-division images through the control program of the client. Simultaneously outputting to a channel, wherein the image processing server divides the image for a predetermined time including the event extracted from the artificial intelligence analysis server into time-division images as many as the preset number of split screens, and reproduces the reproduced image. In the step of simultaneously outputting in multiple channels through the control program of the client, the image processing server may include playing back images for a predetermined time before and after the event occurs at low speed when an event occurs in the reproduced image.

In the step of the artificial intelligence analysis server extracting events generated during a predetermined time in the video storage server according to a request from the control server, the events extracted from the artificial intelligence analysis server include intrusion, damage, fire, wandering, and abandonment. , a specific person identification, and a specific object identification event.

The image processing server divides and reproduces the image for a predetermined time including the event extracted from the artificial intelligence analysis server into time-division images as many as the preset number of split screens, and reproduces the reproduced image through a multi-channel control program of the client. In the step of simultaneously outputting to , the image processing server may include setting the number of split screens for monitoring the image for the predetermined time to 1 split or N splits.

The image processing server may include shortening the image monitoring time by dividing the image search time by search time/N when the number of split screens for monitoring the image for the predetermined time is set to N divisions.

When the number of split screens for monitoring the video for the predetermined time is set to N splits, the image processing server highlights the image of the channel where the event occurred among the images played through multiple channels as many as the number of N split screens. It may include a step of displaying by.

When the number of split screens for monitoring the video for the predetermined time is set to 1 split or N splits, the image processing server designates forward speed or reverse speed according to the number of split screens divided into 1 split or N splits. It may include starting video playback.

When the image processing server starts playing an image by designating forward speed or reverse speed according to the number of split screens divided into 1 division or N division, the video in the section where no event occurs is displayed at a speed exceeding 1x speed. High-speed reproduction and low-speed reproduction of an image for a predetermined time before and after the event occurs at a speed equal to or less than 1x speed may be included.

When the image processing server designates reverse speed according to the number of divided screens divided into 1 division or N divisions, performing decoding by searching the image files stored in the image storage server in a reverse direction in units of group of pictures (GOP). ; and arranging the decoded images in reverse order by the image processing server and performing reverse speed-up processing by performing video frame sampling.

On the other hand, according to an aspect of an efficient video search system for finding an event situation according to the present invention for achieving the above object, an image storage server for storing a camera-captured image; a client requesting an image search for a predetermined time to find a specific event situation in the image stored in the image storage server; a control server that receives a video search request for a predetermined period of time to find a specific event situation from the client; An artificial intelligence analysis server for extracting events generated during a predetermined time in the video storage server according to a request from the control server; And the video for a predetermined time including the event extracted from the artificial intelligence analysis server is divided into time-division video as many as the preset number of split screens, and the reproduced video is simultaneously output in multiple channels through the control program of the client. An image processing server may be included, and the image processing server may reproduce images for a predetermined time before and after an event occurs at a low speed when an event occurs in a reproduced image.

The event extracted from the artificial intelligence analysis server may include at least one event of intrusion, damage, fire, loitering, abandonment, specific person identification, and specific object identification events.

The image processing server may set the number of split screens for monitoring the image for the predetermined time to 1 split or N splits.

When the number of split screens for monitoring the video for the predetermined time is set to N divisions, the image processing server can shorten the image monitoring time by dividing the image search time by search time/N.

When the number of split screens for monitoring the video for the predetermined time is set to N splits, the image processing server highlights the image of the channel where the event occurred among the images played through multiple channels as many as the number of N split screens. can be displayed.

When the number of split screens for monitoring the video for the predetermined time is set to 1 split or N splits, the image processing server designates forward speed or reverse speed according to the number of split screens divided into 1 split or N splits. You can start playing the video.

When the image processing server starts playing an image by designating forward speed or reverse speed according to the number of split screens divided into 1 division or N division, the video in the section where no event occurs is displayed at a speed exceeding 1x speed. High-speed playback, and video for a predetermined time before and after the event occurs can be played back at a speed of 1x speed or less.

When reverse speed is specified according to the number of split screens divided into the 1-division or the N-division, the image processing server performs decoding by searching the image file stored in the image storage server in a reverse direction in units of Group Of Pictures (GOP), Reverse double speed processing may be performed by arranging the decoded images in reverse order and performing video frame sampling.

Effects according to an embodiment of the present invention are described as follows.

According to the present invention, the present invention can quickly and accurately search for an image when a dispute occurs among all customers who use a safety/security service using an image.

In addition, it is possible to provide a function for reducing the physical time for monitoring when searching for a video at a dispute point and for convenient monitoring that allows a searcher to quickly find a desired video.

BRIEF DESCRIPTION OF THE DRAWINGS The accompanying drawings, which are included as part of the detailed description to aid understanding of the present invention, provide examples of the present invention and, together with the detailed description, describe the technical features of the present invention.
1 is a diagram illustrating an efficient image search method for finding an event situation according to an embodiment of the present invention.
2 is a diagram illustrating a smart image search time division processing system according to an embodiment of the present invention.
3 is a diagram illustrating an example of a forward speed processing method for smart image search according to an embodiment of the present invention.
4 is a diagram illustrating an example of a reverse speed processing method for smart image search according to an embodiment of the present invention.

Hereinafter, the embodiments disclosed in the present invention will be described in detail with reference to the accompanying drawings, but the same or similar components are given the same reference numerals regardless of reference numerals, and redundant description thereof will be omitted. The suffixes "module" and "unit" for components used in the following description are given or used together in consideration of ease of writing the specification, and do not have meanings or roles that are distinct from each other by themselves. In addition, in describing the embodiments disclosed in the present invention, if it is determined that a detailed description of related known technologies may obscure the gist of the embodiments disclosed in the present invention, the detailed description will be omitted. In addition, the accompanying drawings are only for easy understanding of the embodiments disclosed in the present invention, the technical idea disclosed in the present invention is not limited by the accompanying drawings, and all changes included in the spirit and technical scope of the present invention , it should be understood to include equivalents or substitutes.

Terms including ordinal numbers, such as first and second, may be used to describe various components, but the components are not limited by the terms. These terms are only used for the purpose of distinguishing one component from another.

It is understood that when an element is referred to as being "connected" or "connected" to another element, it may be directly connected or connected to the other element, but other elements may exist in the middle. It should be. On the other hand, when an element is referred to as “directly connected” or “directly connected” to another element, it should be understood that no other element exists in the middle.

Singular expressions include plural expressions unless the context clearly dictates otherwise.

In this application, terms such as "comprise" or "have" are intended to designate that there is a feature, number, step, operation, component, part, or combination thereof described in the specification, but one or more other features It should be understood that the presence or addition of numbers, steps, operations, components, parts, or combinations thereof is not precluded.

1 is a diagram illustrating an efficient image search method for finding an event situation according to an embodiment of the present invention.

As shown, the image processing server may first set a search section for efficient image search for finding an event situation requested by a client. Here, the search period may mean a period from a start point to an end point for image search.

Subsequently, after setting the search section, the image processing server may set artificial intelligence (AI) events such as intrusion/damage/fire/roaming/abandonment/identification of a specific person/identification of a specific object. Here, the artificial intelligence (AI) event may be an event that can be detected by a CCTV camera based on artificial intelligence, and other types of AI events can be additionally set. For example, an event for identifying a specific person or a specific object by finding a specific point in a movie image may also be included.

Subsequently, the image processing server may set the number of split screens (IF) for efficiently monitoring the image in the set image search section. At this time, if the number of split screens (IF) is divided by 1, the same monitoring time as the search requested time may be required by performing video playback at the search time. If the number of split screens (IF) is divided into N, search time / The monitoring time can be reduced by performing image segmentation processing by N.

Subsequently, the image processing server may start reproduction of images at double speed/reverse speed for corresponding images according to the number of 1-split or N-split screens. That is, the image processing server may perform forward speed or reverse speed processing on the corresponding video according to the number of 1-division or N-division screens, which will be described in more detail below.

Subsequently, the image processing server may check whether a preset AI event such as intrusion/damage/fire/roaming/abandonment/identification of a specific person/identification of a specific object has occurred in the video reproduced according to the number of 1-split or N-split screens.

As a result of the check, the video processing server operates at a low speed (0 < M ≤1), double/reverse speed can be reproduced. At this time, preferably, the video for N seconds before/after the event occurrence point can be reproduced at double speed/reverse speed at low speed (0<M≤1). However, if an AI event such as intrusion/damage/fire/roaming/abandonment/identification of a specific person/identification of a specific object does not occur in the 1-split or N-split screen video being reproduced, the video processing server does not generate the 1-split or N-split screen. Depending on the number of times, the step of starting to reproduce the image at double speed/reverse speed may be continuously performed.

Subsequently, the image processing server checks whether the number of split screens (IF) of the video being played is the end of the searched image (EOF, End Of File), and if the number of split screens (IF) is the end of the video (EOF), the smart You can end the video search. However, if the number of split screens (IF) is not the end of the video (EOF), the step of starting video playback at double speed/reverse speed according to the number of 1 split screen or N split screens may be continuously performed.

2 is a diagram illustrating a smart image search time division processing system according to an embodiment of the present invention.

As shown, the client 100 may request a smart image search to the control server 200 to check a specific event situation. For example, a search request for a 4-hour video of a specific section can be made in an image file captured by a CCTV camera.

The control server 200 informs the video storage server 300 where the video file is stored that a smart video search request has occurred, and the AI Analyzer Server 400 is an artificial intelligence-based video storage server. AI event information may be extracted from (300) and transmitted to the image processing server (500).

The image processing server 500 may process images through AI event processing, meta data processing, image time division process, and double speed playback/reverse playback process. Here, the AI event processing may be a process of controlling the video playback speed to a high speed or a low speed depending on whether an AI event such as intrusion/damage/fire/roaming/abandonment/identification of a specific person/search of a specific object has occurred in the searched image. Meta data processing may be a process for providing information about search images and AI events as meta data. In addition, the video time division process may be a process for dividing and processing a 4 hour video requested by the client 100 into 4 1 hour videos, and the double speed playback/reverse playback process divides a 4 hour video into 1 hour video in the video time division process. It may be a process for double-speed reproduction/reverse reproduction of an image divided into four images.

The client 100 divides a 4-hour video into 4 1-hour videos in the video time-division process and double-speed playback/reverse playback process of the video processing server 500 and displays the reproduced video in multiple channels through a control program. . That is, 0 to 60 minutes, 60 to 120 minutes, 120 to 180 minutes, and 180 to 240 minutes of video can be output according to the screens divided by the number of 4 split screens. In this way, the video search time for finding a specific event situation can be shortened by simultaneously outputting multi-channel time-divided video through the control program of the client.

3 is a diagram showing an example of a forward speed processing method for smart image search according to an embodiment of the present invention, and FIG. 4 shows an example of a reverse speed processing method for smart image search according to an embodiment of the present invention. it is a drawing

As shown, the video stored in the video storage server (NAS) may be decoded in a decoder through a demuxing process in which video and audio are separated in a DeMuxer.

The video frame sampler may perform sampling of a raw image decoded by a decoder according to a designated speed. Here, sampling is to extract data from an image at regular intervals. To this end, a video frame sampler is an input parameter for specifying double speed, event occurrence time, and forward/backward forward/backward playback time of event occurrence. ) can be set.

Frame sampling according to the double speed setting of the video frame sampler is performed by setting the double speed to 2x speed, 4x speed, and 8x speed, for example, as illustrated in FIG. You can perform frame sampling. Here, each double speed section may include a normal section and nx speed sections.

An encoder may encode all raw images in an event occurrence section in which an event occurs without sampling.

A muxer can mux a raw image encoded in an encoder according to a container format.

A packetizer may packetize data muxed by a muxer.

On the other hand, in the case of reverse search, as shown in FIG. 4, the DeMuxer searches files stored in the video storage server (NAS) in units of GOP (Group Of Picture) in reverse, and decodes them in the decoder. In addition, reverse double speed processing can be performed by arranging the decoded raw images of GOP (Group Of Picture) units in reverse order and then performing video frame sampling. Since the method of performing video frame sampling by arranging the decoded raw images in GOP (Group Of Picture) units in reverse order is the same as the forward speed processing method, detailed description thereof is omitted. I'm going to do it.

When a customer uses a safety/security solution using built-in and cloud video, various disputes may occur, and the video may be searched for as short as several tens of minutes or as long as several days to find the cause. At this time, in the worst case, the same monitoring time as the search request time may be required to find the video desired by the customer. In the present invention, the customer's video monitoring is based on AI events (intrusion/fire/roaming/abandonment/damage/identification of a specific person/identification of a specific object, etc.), and meaningless videos without an event are reproduced at high speed at a speed exceeding 1x speed. And preferably, it can be reproduced at a high speed of 2 to 64 times, and a meaningful video where an event has occurred is reproduced at a low speed at a speed of 1x speed or less, so that a desired image can be efficiently searched. In addition, by dividing the time the customer wants to search by N and simultaneously playing the search target images on one screen, the search time can be shortened by N times, thereby dramatically reducing the customer image search time.

That is, in the present invention, when searching for a video to check the event status, the time for watching meaningless video is reduced by replaying the section where the event does not occur at double speed, and the section where the event occurs is reproduced at a speed of 1x speed or less at low speed to avoid disputes. You can accurately monitor the generated video. In addition, when searching for video, search time can be shortened through video monitoring through time division as much as the number of split screens, and when monitoring multi-channel video as many as the number of split screens, it provides an efficient video monitoring function by highlighting the video of the conflicting channel. can do.

The above-described present invention can be implemented as computer readable code on a medium on which a program is recorded. The computer-readable medium includes all types of recording devices in which data that can be read by a computer system is stored. Examples of computer-readable media include Hard Disk Drive (HDD), Solid State Disk (SSD), Silicon Disk Drive (SDD), ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical data storage device, etc. , and also includes those implemented in the form of a carrier wave (eg, transmission over the Internet). Accordingly, the above detailed description should not be construed as limiting in all respects and should be considered illustrative. The scope of the present invention should be determined by reasonable interpretation of the appended claims, and all changes within the equivalent scope of the present invention are included in the scope of the present invention.

100: client
200: control server
300: video storage server
400: AI analysis server
500: image processing server

Claims (16)

As an efficient image search method for finding an event situation,
An image storage server storing an image captured by a camera;
requesting, by a client, an image search for a predetermined time to find a specific event situation in an image stored in the image storage server;
Receiving, by the control server, a video search request for a predetermined time to find a specific event situation from the client;
Extracting, by an artificial intelligence analysis server, an event generated during a predetermined time in the video storage server according to a request from the control server; and
The video processing server divides and reproduces the video for a predetermined time including the event extracted from the artificial intelligence analysis server into time-division video as many as the preset number of split screens, and reproduces the multi-channel video through the control program of the client. Including the step of outputting at the same time as
The image processing server divides and reproduces the image for a predetermined time including the event extracted from the artificial intelligence analysis server into time-division images as many as the preset number of split screens, and reproduces the reproduced image through a multi-channel control program of the client. In the step of simultaneously outputting to
Efficient video search method for finding an event situation comprising the step of playing, by the video processing server, video for a predetermined time before and after the event occurs at low speed when an event occurs in the reproduced video.
The method of claim 1,
In the step of the artificial intelligence analysis server extracting an event generated for a predetermined time in the video storage server according to a request from the control server,
An efficient image search method for finding an event situation, characterized in that the event extracted from the artificial intelligence analysis server includes at least one of intrusion, damage, fire, loitering, abandonment, specific person identification, and specific object identification events. .
The method of claim 1,
The image processing server divides and reproduces the image for a predetermined time including the event extracted from the artificial intelligence analysis server into time-division images as many as the preset number of split screens, and reproduces the reproduced image through a multi-channel control program of the client. In the step of simultaneously outputting to
Efficient video search method for finding an event situation, comprising the step of setting, by the video processing server, the number of split screens for monitoring the video for the predetermined time to 1 split or N splits.
The method of claim 3,
The image processing server shortens the image monitoring time by dividing the image search time by search time / N when the number of split screens for monitoring the image for the predetermined time is set to N divisions. Efficient video search method to find event situation.
The method of claim 4,
When the number of split screens for monitoring the video for the predetermined time is set to N splits, the image processing server highlights the image of the channel where the event occurred among the images played through multiple channels as many as the number of N split screens. An efficient image retrieval method for finding an event situation comprising the step of displaying an image by
The method of claim 3,
When the number of split screens for monitoring the video for the predetermined time is set to 1 split or N splits, the image processing server designates forward speed or reverse speed according to the number of split screens divided into 1 split or N splits. An efficient video search method for finding an event situation, comprising the step of starting video playback.
The method of claim 6,
When the image processing server starts playing an image by designating forward speed or reverse speed according to the number of split screens divided into 1 division or N division, the video in the section where no event occurs is displayed at a speed exceeding 1x speed. An efficient image retrieval method for finding an event situation, comprising: reproducing at high speed and reproducing at low speed an image for a predetermined time before and after the event occurs at a speed of 1x speed or less.
The method of claim 7,
When the image processing server designates reverse speed according to the number of divided screens divided into 1 division or N divisions, performing decoding by searching the image files stored in the image storage server in a reverse direction in units of group of pictures (GOP). ; and
The image processing server performs reverse speed processing by arranging the decoded images in reverse order and performing video frame sampling.
As an efficient image search system for finding event situations,
An image storage server in which camera-photographed images are stored;
a client requesting an image search for a predetermined time to find a specific event situation in the image stored in the image storage server;
a control server that receives a video search request for a predetermined period of time to find a specific event situation from the client;
An artificial intelligence analysis server for extracting events generated during a predetermined time in the video storage server according to a request from the control server; and
Video for a predetermined time including the event extracted from the artificial intelligence analysis server is divided into time-division video as many as the preset number of split screens and reproduced, and the reproduced video is simultaneously output in multiple channels through the control program of the client. including a processing server;
The video processing server is an efficient video search system for finding an event situation in which the video for a predetermined time before and after the event occurs at a low speed when an event occurs in the video being played.
The method of claim 9,
An efficient video search system for finding an event situation, characterized in that the event extracted from the artificial intelligence analysis server includes at least one of intrusion, damage, fire, loitering, abandonment, specific person identification, and specific object identification events. .
The method of claim 9,
The image processing server,
An efficient video retrieval system for finding an event situation, characterized in that the number of split screens for monitoring the video for the predetermined time is set to 1 split or N splits.
The method of claim 11,
The image processing server,
An efficient video search system for finding an event situation, characterized in that the video monitoring time is shortened by dividing the video search time by search time/N when the number of split screens for monitoring the video for the predetermined time is set to N divisions. .
The method of claim 12,
The image processing server,
When the number of split screens for monitoring the video for the predetermined time is set to N splits, among the images played through multiple channels as many as the number of N split screens, the video of the channel where the event occurred is highlighted and displayed. Efficient video retrieval system for finding event situations with
The method of claim 11,
The image processing server,
When the number of split screens for monitoring the video for the predetermined time is set to 1 split or N splits, forward speed or reverse speed is specified according to the number of split screens divided into 1 split or N splits to start video playback. Efficient image search system for finding an event situation, characterized in that.
The method of claim 14,
The image processing server,
When video playback is started by specifying the forward speed or reverse speed according to the number of split screens divided into 1-division or N-division, the video in the section where no event has occurred is reproduced at high speed at a speed exceeding 1x speed, and the event An efficient video retrieval system for finding an event situation, characterized in that the video for a predetermined time before and after the occurrence of is reproduced at a low speed at a speed of 1x speed or less.
The method of claim 15
The image processing server,
When the reverse speed is designated according to the number of divided screens divided into 1 division or N divisions, video files stored in the video storage server are searched in a reverse direction in units of GOP (Group Of Picture) to perform decoding, and the decoded video is displayed in reverse order. Efficient image retrieval system for finding an event situation, characterized in that reverse speed processing is performed by performing video frame sampling by arranging as.

Images