KR20230032976A - System and method for monitoring video using augmented reality - Google Patents

Abstract

A video monitoring system is provided that includes: a plurality of cameras for generating real-time image data; a control server; and a video monitoring server for receiving tracking target information from the control server, analyzing real-time video data received from the plurality of cameras based on the tracking target information, specifying a tracking target, and predicting a movement path according to a change in the position of the specified tracking target. Therefore, it is possible to recognize the position and movement direction along the movement of the tracking target.

Description

Video surveillance system and method using augmented reality {SYSTEM AND METHOD FOR MONITORING VIDEO USING AUGMENTED REALITY}

The present invention relates to a video surveillance system and method.

Thanks to the development of image processing technology and data processing technology, so-called augmented reality technology, a technology that adds virtual information to a real screen, is appearing one after another. For example, when a player appears during a sports broadcast, the player's team or career is shown, and when buying cosmetics or clothes through online shopping, the user is shown a virtual image of makeup or clothes in advance.

This augmented reality technology can also be applied to a closed-circuit television (CCTV). Regarding this, a CCTV image providing system to which the augmented reality technology is applied is disclosed in Korean Patent Publication No. 10-2016-0132571 (November 21, 2016).

However, the conventional CCTV image providing system can only display the information of the person possessing the tag, and the information is limited from the point of view of the manager who operates the actual CCTV system. That is, from a practical point of view, information on the surrounding environment, such as the escape route of the tag holder and the arrangement of surrounding buildings or cars, is also very important, but the conventional CCTV system has a problem in that it cannot provide sufficient information in this regard.

Accordingly, a technology capable of recognizing the location and moving direction of a tracked object along the movement is required.

A technical problem to be achieved by the present invention is to provide a video surveillance system and method using augmented reality capable of recognizing the location and moving direction of a tracked object along the movement.

According to one embodiment, a video surveillance system is provided. The video monitoring system receives a plurality of cameras generating real-time video data, a control server, and tracking target information from the control server, and analyzes the real-time video data received from the plurality of cameras based on the tracking target information. It includes a video monitoring server that specifies a tracked subject and predicts a moving path according to a location change of the specified tracked subject.

The video monitoring server specifies a tracking target corresponding to the tracking target information using a communication unit that receives the tracking target information from the control server and a pre-stored artificial intelligence algorithm, and moves according to the location change of the specified tracking target. It may include an artificial intelligence analysis unit that predicts a path, and an overlay unit that displays the movement path of the specified tracking target on real-time image data.

The video surveillance server may further include a control unit for controlling movement of the plurality of cameras based on the movement path predicted through the artificial intelligence analysis unit.

The artificial intelligence analysis unit may further include an attribute detection unit that detects human attribute information and vehicle attribute information from real-time image data using a pre-stored artificial intelligence algorithm.

The person attribute information may include gender, clothes, and wear information, and the vehicle attribute information may include vehicle type and vehicle number information.

According to one embodiment, a video surveillance method is provided. The video surveillance method includes the steps of receiving tracking target information from a control server, specifying a tracking target by analyzing real-time video data received from a plurality of cameras based on the tracking target information, and position change of the specified tracking target. and predicting a movement path according to

The method may further include displaying the movement path of the specified tracking target on real-time image data after the step of predicting the movement path according to the location change of the specified tracking target.

It is possible to predict the movement path according to the movement of the target to be tracked, and to control the movement of the camera according to the predicted movement route, thereby increasing the efficiency and accuracy of video monitoring of the target to be tracked.

It can be used for social safety nets such as facility management, disaster and fire monitoring, parking lot management, illegal dumping of garbage, monitoring of cultural assets, and monitoring of the elderly living alone.

Seamless CCTV surveillance is possible even with a small number of people, so labor costs can be reduced.

Since it is possible to track criminals by using GPS data, there is an effect of preventing crime.

1 is a block diagram of a video surveillance system according to an embodiment.
2 is a block diagram of a video surveillance server according to an embodiment.
3 is a block diagram of an artificial intelligence analysis unit according to an embodiment.
4 to 8 are diagrams for explaining the operation of the video surveillance server.
9 is a flowchart of a video monitoring method according to an exemplary embodiment.
10 is a block diagram of a video surveillance server according to an embodiment.

Hereinafter, with reference to the accompanying drawings, embodiments of the present invention will be described in detail so that those skilled in the art can easily carry out the present invention. However, the present invention may be embodied in many different forms and is not limited to the embodiments described herein. And in order to clearly explain the present invention in the drawings, parts irrelevant to the description are omitted, and similar reference numerals are attached to similar parts throughout the specification.

Throughout the specification, when a certain component is said to "include", it means that it may further include other components without excluding other components unless otherwise stated.

The terms used in the embodiments of the present invention have been selected from general terms that are currently widely used as much as possible while considering the functions in the present invention, but these may vary depending on the intention or precedent of a person skilled in the art, the emergence of new technologies, etc. . In addition, in a specific case, there is also a term arbitrarily selected by the applicant, and in this case, the meaning will be described in detail in the description of the corresponding embodiment. Therefore, the term used in the present embodiments should be defined based on the meaning of the term and the overall content of the present embodiment, not a simple name of the term.

In an embodiment of the present invention, terms including ordinal numbers such as first and second may be used to describe various components, but the components are not limited by the terms. These terms are only used for the purpose of distinguishing one component from another. For example, a first element may be termed a second element, and similarly, a second element may be termed a first element, without departing from the scope of the present invention. The terms and/or include any combination of a plurality of related recited items or any of a plurality of related recited items.

Also, in the embodiments of the present invention, singular expressions include plural expressions unless the context clearly indicates otherwise.

In addition, in the embodiments of the present invention, terms such as "include" or "have" are intended to designate that there is a feature, number, step, operation, component, part, or combination thereof described in the specification, but one or other features, numbers, steps, operations, components, parts, or combinations thereof, or any combination thereof, is not precluded from being excluded in advance.

Also, in an embodiment of the present invention, a 'module' or 'unit' performs at least one function or operation, and may be implemented as hardware or software, or a combination of hardware and software. In addition, a plurality of 'modules' or a plurality of 'units' may be integrated into at least one module and implemented by at least one processor, except for 'modules' or 'units' that need to be implemented with specific hardware.

1 is a block diagram of a video surveillance system according to an embodiment. 2 is a block diagram of a video surveillance server according to an embodiment. 3 is a block diagram of an artificial intelligence analysis unit according to an embodiment. 4 to 8 are diagrams for explaining the operation of the video surveillance server.

Referring to FIG. 1 , a video monitoring system according to an embodiment includes a plurality of cameras 100, a control server 200, a video monitoring server 300, and a 3D map image providing server 400.

Each of the plurality of cameras 100, as an example, is disposed in a preset area and can generate real-time image data.

As an example, each of the plurality of cameras 100 may include a communication module for transmitting and receiving data to and from the control server 200 and the video monitoring server 300 using a wired or wireless network.

As an example, each of the plurality of cameras 100 may be a CCTV camera capable of driving a pan-tilt-zoom (PTZ).

Each of the plurality of cameras 100 may transmit camera information including GPS coordinates and pan-tilt-zoom values to the control server 200 and the video monitoring server 300, as an example.

As an embodiment, each of the plurality of cameras 100 may include an electronic anklet recognition module, and when the electronic anklet wearer approaches the shooting area, the camera 100 in the area generates real-time image data to control the server ( 200) and the video monitoring server 300.

As an example, the control server 200 may receive tracking target information (eg, criminal's sex, clothes, impression information, vehicle type, license plate number information) from related agency servers (the National Police Agency, the Fire and Safety Agency, etc.).

As an example, the control server 200 may transmit tracking target information to the video monitoring server 300 .

As an embodiment, the video surveillance server 300 receives tracking target information from the control server 200 and analyzes real-time image data received from a plurality of cameras 100 based on the tracking target information to specify a tracking target. And, it is possible to predict the movement path according to the location change of the specified tracking target.

Referring to FIG. 2 , a video monitoring server 300 according to an embodiment may include a communication unit 310, an artificial intelligence analysis unit 320, an overlay unit 330, and a control unit 340.

As an embodiment, the communication unit 310 may include a communication module that transmits and receives data with the plurality of cameras 100, the control server 200, and the 3D map image providing server 400 using a wired or wireless network. .

As an example, the communication unit 310 may receive tracking target information from the control server 200 and receive 3D map image data from the 3D map image providing server 400 .

As an example, the artificial intelligence analysis unit 320 may specify a tracking target corresponding to the tracking target information using a pre-stored artificial intelligence algorithm, and predict a moving path according to a location change of the specified tracking target.

As an example, the pre-stored algorithm may be a machine learning algorithm or a deep learning algorithm.

Referring to FIG. 3 , the artificial intelligence analysis unit 320 according to an embodiment may include an attribute detection unit 321, a metadata generation unit 322, a movement path estimation unit 323, and an event detection unit 324. can

Referring to FIGS. 4 and 5 , the attribute detector 321 may detect human attribute information and vehicle attribute information from real-time image data using a pre-stored artificial intelligence algorithm as an example.

As an example, the person attribute information may include gender, clothing (top, bottom, hat), and impression information.

The vehicle attribute information, as an example, may include vehicle type and vehicle number information.

As an example, the metadata generation unit 322 may generate metadata (XVCP, XML Communication Protocol) by classifying human attribute information and vehicle attribute information.

As an embodiment, the movement path prediction unit 323 can specify a tracking target corresponding to the tracking target information in real-time image data of the camera 100 corresponding to the location coordinate information of the tracking target received from the control server 200. can

Referring to FIG. 6, as an embodiment, the movement path predictor 323 extracts a plurality of position coordinate values according to the position change of the tracking target using a pre-stored artificial intelligence algorithm, and extracts a plurality of position coordinate values. Based on this, it is possible to predict the moving path of the tracking target. As an example, the overlay unit 330 may display the moving path of the tracking target on real-time image data.

Referring to FIG. 7 , the event detection unit 324, as an embodiment, uses a pre-stored artificial intelligence algorithm to detect people's wandering, crowding, fighting, falling, abandonment, intrusion, vehicle stop, vehicle congestion, It is possible to detect whether any one event occurs among vehicle parking, stolen vehicle, fire, flame, smoke, and directional movement.

As an example, the event detection unit 324 provides the control server 200 with event occurrence information (e.g., human wandering, crowding, fighting, falling, abandonment, intrusion, vehicle stop, vehicle congestion, vehicle parking) when an event is detected. , stolen vehicle, fire, flame, smoke, and directional movement) may transmit an event notification message.

As an example, the overlay unit 330 may display tracking target information on real-time image data including a specified tracking target.

Referring to FIG. 8 , the overlay unit 330, as an embodiment, converts tracking target information and 3D map image data received from the 3D map image providing server 400 to real-time image data including a specified tracking target. Augmented reality image data may be generated by synthesis. As an example, the 3D map image data may include images captured in all directions along the road.

As an example, the control unit 340 may control the movement of the plurality of cameras 100 based on the movement path predicted through the artificial intelligence analysis unit 320 .

As an embodiment, the control unit 340 generates a control signal for changing the pan-tilt-zoom value of the corresponding camera 100 based on the movement path of the tracking target predicted through the movement path predictor 323, This can be transmitted to the corresponding camera 100 and controlled to include the tracking target in the image data.

The 3D map image providing server 400 may transmit 3D map image data to the video monitoring server 300 as an example.

9 is a flowchart of a video monitoring method according to an exemplary embodiment.

Referring to FIG. 9 , the video monitoring method of the video monitoring server 300 according to an embodiment includes receiving tracking target information from the control server 200 (S100), and a plurality of cameras 100 based on the tracking target information. ), analyzing the real-time image data received from the target to be tracked (S200), predicting the movement path according to the location change of the specified target to be tracked (S300), and the movement path of the target to be tracked specified in the real-time image data It may include a step of displaying (S400).

Steps S100 to S400 are the same as the operation contents of the video monitoring server 300 described above, so detailed descriptions are omitted.

10 is a block diagram of a video surveillance server according to an embodiment.

Referring to FIG. 10 , a video surveillance server according to an embodiment may be implemented as a computer system, for example, a computer readable medium. The computer system 1000 includes at least one of a processor 1010, a memory 1030, a user interface input device 1060, a user interface output device 1070, and a storage device 1080 communicating through a bus 1020. can include Computer system 1000 may also include a network interface 1090 coupled to a network. The processor 1010 may be a central processing unit (CPU) or a semiconductor device that executes instructions stored in the memory 1030 or the storage device 1080 . The memory 1030 and the storage device 1080 may include various types of volatile or non-volatile storage media. For example, the memory 1030 may include a read only memory (ROM) 1031 and a random access memory (RAM) 1032 . An embodiment of the present description may be implemented as a computer-implemented method or as a non-transitory computer-readable medium in which computer-executable instructions are stored. In one embodiment, when executed by a processor, the computer readable instructions may perform a method according to at least one aspect of the present disclosure.

The video surveillance server according to an embodiment includes a processor 1010 and a memory 1030, and the processor 1010 executes a program stored in the memory 1030 to receive tracking target information from the control server 200. Step, specifying a tracking target by analyzing real-time image data received from a plurality of cameras 100 based on the tracking target information, predicting a moving path according to the location change of the specified tracking target, A step of displaying a movement path of a specified tracking target may be performed.

Operation of the processor 1010 is the same as that of the video monitoring server 300 described above, so a detailed description thereof will be omitted.

Although the embodiments of the present invention have been described in detail above, the scope of the present invention is not limited thereto, and various modifications and improvements made by those skilled in the art using the basic concept of the present invention defined in the following claims are also included in the scope of the present invention. that fall within the scope of the right.

Those skilled in the art related to this embodiment will be able to understand that it can be implemented in a modified form within a range that does not deviate from the essential characteristics of the above description. Therefore, the disclosed methods are to be considered in an illustrative rather than a limiting sense. The scope of the present invention is shown in the claims rather than the foregoing description, and all differences within the equivalent scope will be construed as being included in the present invention.

Claims (7)

A plurality of cameras generating real-time image data;
control server, and
Receive tracking target information from the control server, analyze real-time image data received from the plurality of cameras based on the tracking target information to specify a tracking target, and predict a moving path according to the location change of the specified tracking target video surveillance server
A video surveillance system that includes a. In paragraph 1,
The video surveillance server,
A communication unit for receiving the tracking target information from the control server;
An artificial intelligence analysis unit that specifies a tracking target corresponding to the tracking target information using a pre-stored artificial intelligence algorithm and predicts a moving path according to a location change of the specified tracking target; and
A video surveillance system comprising an overlay unit displaying the specified moving path of the tracking target on real-time video data. In paragraph 2,
The video surveillance server,
Further comprising a control unit for controlling the movement of the plurality of cameras based on the movement path predicted through the artificial intelligence analysis unit, the video surveillance system. In paragraph 3,
The artificial intelligence analysis unit,
A video surveillance system comprising an attribute detector for detecting human attribute information and vehicle attribute information from real-time image data using pre-stored artificial intelligence algorithms. In paragraph 4,
The person attribute information includes gender, clothes, and impression information,
Wherein the vehicle property information includes vehicle type and vehicle number information. Receiving tracking target information from a control server;
Specifying a tracking target by analyzing real-time image data received from a plurality of cameras based on the tracking target information; and
Predicting the movement path according to the location change of the specified tracking target
A video surveillance method comprising a. In paragraph 6,
After the step of predicting the movement path according to the location change of the specified tracking target,
The video monitoring method further comprising the step of displaying a movement path of the specified tracking target on real-time video data.

Images