KR20230058960A - Distributed collaboration-based system and method for analyzing video stream thereof - Google Patents

Abstract

A distributed collaboration-based system for real-time analysis of video streams is provided. The system receives a first setting message including video analysis request information from a user terminal, and generates a first analysis result obtained by analyzing an image collected from at least one video collecting device based on the first setting message, A first image processing device that is provided to a user terminal and generates a second setting message for requesting additional analysis of the first analysis result according to the first setting message; and the second setting message based on the second setting message. and a second image processing device generating a second analysis result obtained by performing image analysis on at least one of first analysis result and the collected image and providing the result to the user terminal.

Description

Distributed Collaboration-Based System and Real-time Analysis Method of Video Stream thereof

The present invention relates to a distributed collaboration-based system and a real-time analysis method for a video stream thereof.

An image monitoring system is a system for monitoring corresponding areas in various places such as roads, parks, shopping malls, hospitals, companies, and factories using cameras.

A conventional video monitoring system simply transmits an image captured by a camera to a collection server and analyzes the transmitted image from the server.

At this time, video transmission from the camera to the server requires the use of considerable bandwidth, and such video transmission generates network traffic and delay. In addition, the server had to analyze a large amount of images, so there was a lot of burden.

Registered Patent Publication No. 10-2128945 (2020.06.25)

The problem to be solved by the present invention is to enable real-time analysis through a branching collaboration structure between an edge computing device that performs primary simple image analysis on collected images and a high-performance computing device that performs secondary advanced image analysis. It is to provide a distributed collaboration-based system and a method for real-time analysis of a video stream thereof, capable of reducing analysis costs and reducing analysis costs.

However, the problem to be solved by the present invention is not limited to the above problem, and other problems may exist.

Distributed collaboration-based system for real-time analysis of a video stream according to a first aspect of the present invention for solving the above problems receives a first setting message including video analysis request information from a user terminal, and in the first setting message A first analysis result obtained by analyzing an image collected from at least one image collection device is generated based on the image collection device and provided to the user terminal, and a request for additional analysis of the first analysis result according to the first setting message is generated. 2 A first image processing device generating a setting message; and generating a second analysis result obtained by performing image analysis on at least one of the first analysis result and the collected image based on the second setting message, and generating the user and a second image processing device provided to the terminal.

In some embodiments of the present invention, the first image processing device may have lower-cost computing resources than the second image processing device.

In some embodiments of the present invention, the first image processing device may be an edge node, and the second image processing device may be a cloud server.

In some embodiments of the present invention, the first image processing device calculates the accuracy of the first analysis result, and when there is a first analysis result having a calculated accuracy less than a preset accuracy, a first analysis result less than the preset accuracy is present. A second setting message for the analysis result may be generated.

In some embodiments of the present invention, the first image processing device generates at least one snapshot of an image corresponding to a first analysis result having less than the preset accuracy, and sends the generated snapshot to the second setting message. Together, they can be transmitted to the second image processing device.

In some embodiments of the present invention, when the first image processing device cannot analyze the behavior pattern of the object recognized through the first analysis result, the image stream corresponding to the first analysis result including the recognized object may be transmitted to the second image processing device together with the second setting message.

In some embodiments of the present invention, the first setting message may include analysis target information, analysis result providing information, generation information of the second setting message, and image type information to be provided to the second image processing device. .

In some embodiments of the present invention, the second setting message may include source information of the collected images and images included in the first analysis result.

In some embodiments of the present invention, the first image processing device transmits the second setting message to the second image processing device so that the image transmission channel is formed between the first and second image processing devices, while the image transmission channel is formed. An image storage queue for temporarily storing images collected from the collection device may be included.

In some embodiments of the present invention, the second image processing device includes a plurality of second image processing devices that generate second analysis results according to a plurality of different functions, and the first image processing device includes each of the functions. An image and a second setting message may be selectively transmitted to a corresponding second image analysis device for image analysis according to the above.

In some embodiments of the present invention, the second image processing device is composed of a plurality of second image processing devices that generate second analysis results according to a plurality of different functions, and the plurality of second image processing devices are connected. A second image processing device that manages addresses, provided functions, and device state information, receives a second setting message from the first image processing device, and is matched based on the provided functions and device state information of the second image processing device. An image processing management terminal configured to select and transmit a third setting message to the selected second image processing device may be further included.

In addition, a method for analyzing a video stream in real time in a distributed collaboration based system according to a second aspect of the present invention includes receiving, in a first video processing device, a first setting message including video analysis information from a user terminal; generating, by the first image processing device, a first analysis result obtained by analyzing an image collected from at least one image collecting device based on the first setting message; generating, by the first image processing device, a second setting message for requesting an additional analysis of a first analysis result according to the first setting message; generating a second analysis result by performing image analysis on at least one of a first analysis result and the collected image based on the second setting message, in a second image processing device; and providing at least one of the first analysis result and the second analysis result to a user terminal from the first and second image processing devices.

A computer program according to another aspect of the present invention for solving the above problems is combined with a computer that is hardware to execute a real-time analysis method of a video stream in a distributed collaboration-based system, and is stored in a computer-readable recording medium.

Other specific details of the invention are included in the detailed description and drawings.

According to one embodiment of the present invention described above, a first image processing device having a small computing power and a second image processing device having a large computing power are configured to cost-effectively analyze image streams received from a plurality of image collecting devices. can do.

Since the existing video stream analysis method analyzes video by connecting a high-cost video analysis server 1:1 per video channel, the cost of using a high-cost video analysis server occurs even in a situation where video analysis is not required. However, according to an embodiment of the present invention, the first image processing device analyzes a situation requiring image analysis, requests advanced video analysis to the second image processing device, and transmits and processes an image stream, which results in high cost of video processing. There is an advantage in that the analysis server can be used for analyzing multiple images.

In addition, there is an advantage in that a very flexible update is possible in terms of maintenance such as changing an image analysis algorithm and adding a new analysis server. That is, it is possible to respond to various image analysis requests of users through an image processing management terminal that receives information on the second image processing device in real time while flexibly adding or changing the second image processing device.

In addition, when the present invention is applied based on the existing video analysis business structure, it is possible to provide a new video analysis business. That is, operators specializing in image analysis using the present invention can enter the existing image collection infrastructure market, and users can flexibly use various image analysis services and obtain opportunities to use them at low cost. there is.

In addition, masking of the image is performed through the first image processing device, and snapshots or image streams can be selectively transmitted, so that personal sensitive information can be primarily protected, and access information of the image collection device (CCTV Addr, Id, Passwd, etc.) is not exposed to the outside, so security for the video collection infrastructure is also possible.

The effects of the present invention are not limited to the effects mentioned above, and other effects not mentioned will be clearly understood by those skilled in the art from the description below.

1 is a block diagram of a distributed collaboration-based system according to one embodiment of the present invention.
2 is a block diagram of a first image processing device according to an embodiment of the present invention.
3 is a block diagram of a second image processing device according to an embodiment of the present invention.
4 is a diagram illustrating an example of an interlocking structure of first and second image processing devices.
5 is a diagram illustrating another embodiment of an interlocking structure of first and second image processing devices.
6 is a flowchart of a real-time analysis method of a video stream in a distributed collaboration based system according to an embodiment of the present invention.

Advantages and features of the present invention, and methods of achieving them, will become clear with reference to the detailed description of the following embodiments taken in conjunction with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below, but may be implemented in various different forms, only these embodiments are intended to complete the disclosure of the present invention, and are common in the art to which the present invention belongs. It is provided to fully inform the person skilled in the art of the scope of the invention, and the invention is only defined by the scope of the claims.

Terminology used herein is for describing the embodiments and is not intended to limit the present invention. In this specification, singular forms also include plural forms unless specifically stated otherwise in a phrase. As used herein, "comprises" and/or "comprising" does not exclude the presence or addition of one or more other elements other than the recited elements. Like reference numerals throughout the specification refer to like elements, and “and/or” includes each and every combination of one or more of the recited elements. Although "first", "second", etc. are used to describe various components, these components are not limited by these terms, of course. These terms are only used to distinguish one component from another. Accordingly, it goes without saying that the first element mentioned below may also be the second element within the technical spirit of the present invention.

Unless otherwise defined, all terms (including technical and scientific terms) used in this specification may be used with meanings commonly understood by those skilled in the art to which the present invention belongs. In addition, terms defined in commonly used dictionaries are not interpreted ideally or excessively unless explicitly specifically defined.

The present invention relates to a distributed collaboration based system (1) and a real-time analysis method for a video stream thereof.

It is very important to analyze image data acquired from an image collection device. With the development of deep learning-based video analysis technology, not only crime prevention, crime prevention, parking control, entrance and exit management, and visitor management in the city center, but also operation of unmanned stores, automatic extraction of defective products in the production process, and Image analysis technology is being used in various areas throughout society, such as detecting damage to bridge piers, wind turbine blades, and high-voltage transmission towers.

In addition, with the development of wired and wireless communication infrastructure and the widespread use of communication-based video data transmission (IP cameras, etc.) technology, it is possible to build a video data transmission infrastructure at a lower cost. In addition, the number of installed CCTVs has increased significantly in areas such as city crime prevention and traffic monitoring. Accordingly, various issues have arisen when the control personnel directly analyze the video data.

In particular, a situation in which one control agent has to monitor images of dozens of cameras is occurring in a control center (eg, a local government control center). In order to solve this problem, a system that analyzes and detects only images corresponding to predefined specific conditions and informs the control personnel has been introduced.

On the other hand, image analysis technology has evolved into a technology that processes already stored images in batches using high-performance image analysis computing devices, and has recently developed in computer vision, deep learning technology, and parallel computer chip (GPU, etc.) technology. BACKGROUND OF THE INVENTION [0002] Techniques for analyzing video with respect to video streams are being developed.

At this time, real-time video analysis technology, which is relatively expensive compared to batch-type video analysis technology, can obtain analysis results in real time, so it is widely used in the areas of urban crime prevention, crime prevention, and traffic monitoring, which are directly related to life and property. However, there is a problem that is not easily spread despite being directly related to human life and property as it requires high cost.

For example, in the case of crime prevention CCTV control centers in local governments, in local governments with sufficient budgets, dozens of CCTVs are directly monitored by control personnel, and VMS (Video Management System) is introduced to monitor crime prevention and safety in the city. In the case of VMS, the unit price per CCTV input channel is calculated, and hundreds of thousands of won are spent per 1CH per month. In addition, in order to connect hundreds of channels of CCTV to the VMS of the control center 24 hours a day, communication costs are paid for each channel.

It is difficult to introduce such an expensive real-time CCTV control solution in small local governments that do not have a relatively sufficient budget. In addition, even if a CCTV infrastructure is installed one-time with a huge budget, the cost for operating a 24-hour controller or operating a VMS is incurred every month.

The high cost of real-time video analysis stems from the configuration of the entire system that collects, delivers, and analyzes real-time video streams. A video collection device (for example, CCTV) creates a video image as a video stream and transmits it to a video stream collection and analysis system using a coaxial cable or a network. At this time, the image analysis system should secure computing capacity for analyzing the image in proportion to the number of image channels collected from the image collection device. That is, it is a structure in which cost is incurred according to the number of video channels to be analyzed. The number of CCTVs, such as crime prevention and city control, is continuously increasing, so the analysis cost is also continuously rising.

However, most of the data of video streams collected in various applications such as city crime prevention, traffic monitoring, and facility monitoring are captured images that do not require analysis. For example, in the case of facility monitoring, the occurrence of an abnormal state in which an unauthorized person enters a specific facility may be very rare. Even in the case of urban crime prevention, it is highly likely that a normal situation will continue for most everyday situations that do not require expensive video analysis. In this situation, if the video analysis is performed with a structure in which the abnormal case is first detected and the expensive secondary video analysis is performed only when the abnormal case is detected, the video analysis performance is improved by sharing the high-cost video analysis system. This will allow significant cost savings while maintaining the same.

To this end, the distributed collaboration-based system 1 and its video stream real-time analysis method according to an embodiment of the present invention include an edge computing device that performs primary simple video analysis on collected video, and secondary advanced video It is possible to enable real-time analysis and lower analysis cost through a distributed collaboration structure between high-performance computing devices that perform analysis.

Hereinafter, with reference to FIGS. 1 to 5, a distributed collaboration-based system (1, hereinafter referred to as a distributed collaboration-based system) for real-time analysis of a video stream according to an embodiment of the present invention will be described.

1 is a block diagram of a distributed collaboration based system 1 according to an embodiment of the present invention.

A distributed collaboration based system 1 according to an embodiment of the present invention includes a first image processing device 100 and a second image processing device 200 .

The first image processing device 100 receives a first setting message including image analysis information from the user terminal 20 . That is, setting information about the image analysis function to be performed by the first image processing device 100 and a result processing method is received through the user terminal 20 .

The first image processing device 100 analyzes the image collected from at least one image collecting device 10 based on the first setting message, generates a first analysis result, and provides the result to the user terminal 20 . In addition, when it is set as a case in which precise analysis of the first analysis result is required, a second setting message is generated and transmitted to the second image processing device 200 .

The second image processing device 200 generates a second analysis result by performing image analysis on at least one of the first analysis result and the image collected from the image collecting device 10 based on the second setting message. Then, the generated second analysis result is provided to the user terminal 20 .

In one embodiment, the first image processing device 100 has lower-cost computing resources than the second image processing device 200 . That is, the first image processing device 100 performs relatively simpler image analysis than the second image processing device 200 by performing a lightweight image analysis algorithm.

For example, when the first image processing device 100 mounts a deep learning-based artificial intelligence inference lightweight model (YoloV3-Tiny) on an inexpensive computing board such as a 128-core NVIDIA Maxwell GPU, about 80 objects are processed at 20 fps. more than can be recognized. Here, the object to be recognized is an object of daily life such as a person, bicycle, car, dog, cat, etc., and it is possible to determine the degree to which a person, animal, or car has entered a specific area where CCTV is installed.

If three or more people enter a specific area, the user terminal 20 is notified of this, and when a scenario is applied to generate a second analysis result for analyzing the behavior of three or more people, the first image processing device ( 100), after detecting the object, transfers the obtained image stream to the second image processing device 200 according to the user's setting scenario.

The second image processing device 200 may analyze the behavior patterns of three people in the collected images in real time and inform the user of the result.

The first image processing device 100 may be an edge node, and the second image processing device 200 may be a cloud server. That is, the first image processing device 100 performs preprocessing such as data purification, sampling, object recognition and extraction on the image collected from the image collection device 10, and the result is a second image processing device serving as a cloud server. forwarded to (200).

At this time, the function of the edge node can be set in various ways according to the setting message. For example, the first analysis result, which is a simple analysis, can be generated and provided to the user terminal 20 without transmitting to the cloud server. 1 It may be set to perform only the basic function of generating analysis results and transfer all core tasks to the second image processing device 200, which is a cloud server.

The second image processing device 200 mainly performs deep learning, such as deep learning, and related analysis, inference, etc., and is generated by comprehensively performing task(s) handed over from the first image processing device 100. 2 Analysis results may be provided to the user terminal 20 , and core processing may be performed, such as distributing a portion of a task to a specific first image processing device 100 , if necessary.

Meanwhile, an image transmitted from the first image processing device 100 to the second image processing device may be classified by applying various scenarios.

As an embodiment, the first image processing device 100 may calculate the accuracy of the first analysis result. In addition, when there is a first analysis result having a calculated accuracy less than a preset accuracy, a second setting message for the first analysis result having a calculated accuracy less than the preset accuracy may be generated.

For example, when the accuracy of the result analyzed by the first image processing device 100 is less than 60% (the accuracy of the result determined by a human being is less than 60%), the image corresponding to the first analysis result is more accurate. At least one snapshot of may be created, and the created snapshot may be transmitted to the second image processing device 200 together with a second setting message. In this case, if the first image processing device 100 can recognize an object with an accuracy of 90% or more in most cases, high efficiency in analysis cost and communication cost can be expected.

In another embodiment, when it is impossible to analyze the behavior pattern of the object recognized through the first analysis result, the first image processing device 100 converts the image stream corresponding to the first analysis result including the recognized object into a second analysis result. It can be transmitted to the second image processing device 200 together with the setting message.

For example, when the analysis result of the first image processing device 100 determines that there are three people, an image stream other than a snapshot is transmitted to the second image processing device 200 to determine whether it is a violent situation. can In this case, it is possible to analyze a behavior pattern that is difficult to determine with a snapshot through the second image processing device 200, and perform advanced image analysis that cannot be processed by the first image processing device 100 by the second image processing device ( 200) can be effectively dealt with.

In another embodiment, when the first image processing device 100 transmits an image to the second image processing device 200, masking of the image (eg, mosaic, blur processing, etc.) It can be delivered after processing. This has the advantage of being able to respond to legal, business, and personal information protection issues.

In addition, when transmitting an image to the second image processing device 200, the first image processing device 100 may include a function of reducing the size of an image to reduce communication costs.

The user terminal 20 transmits a first setting message to the first image processing device 100 so that a desired service function is performed. The first setting message may include analysis target information, analysis result providing information, whether a second setting message is generated or not, and image type information to be provided to the second image processing device 200 . That is, the first setting message includes information such as what to analyze, where to transmit the analysis result, whether to perform secondary image analysis, and in what form to deliver the image to the second image processing device 200. can do.

As an embodiment, when it is necessary to generate a second analysis result according to the first analysis result, the first image processing device 100 converts access information (one-time generation and encryption information) of the corresponding image to the second image processing device 200. ) can be transmitted. Accordingly, an image transmitted from the first image processing device 100 may be transmitted to the second image processing device 200 without leakage of connection information of the image collection device 10 itself. At this time, whether to process masking of the video, whether to process a snapshot image or video stream, etc. may also be defined. Through this structure, personal information can be protected, and various advanced image analysis operators (second image processing providing device service providers) can be linked.

Meanwhile, in one embodiment of the present invention, the image collecting device 10 may be connected to the first image processing device 100 through a coaxial cable or a network cable, and the first image processing device 100 may be connected to a field in a general field. It may be equipped with a video recorder (eg, DVR or NVR) function located in the . That is, real-time image analysis and control to perform a simple image analysis function by the first image processing device 100 alone is possible, and can be implemented in a form that can be flexibly interlocked with the second image processing device 200 .

2 is a block diagram of the first image processing device 100 according to an embodiment of the present invention.

In an embodiment of the present invention, the first image processing device 100 includes an image receiver 110, a setting message interpreter 120, an image analyzer 130, an analysis result transmitter 140, and a setting message transmitter 150. ), an image transmitter 160, an image storage queue 170, an image player 180, and an image storage 190.

The image receiver 110 receives an image from at least one image collecting device 10 .

The setting message interpreter 120 receives a first setting message including video analysis request information from the user terminal 20 and analyzes the request information according to the contents of the corresponding message.

The video analyzer 130 generates a first analysis result obtained by analyzing the collected video based on the first setting message, and the analysis result transmitter 140 provides the first analysis result to the user terminal 20 . In addition, an image collected through the image player 180 or a result of the first analysis may be output.

The image analyzer 140 is a function driven by a CPU and a GPU of a low-performance processing device, and deep learning inference technology may be applied. Deep learning inference technology is a technology that lightweights and loads an image analysis model that has been trained, and when images are collected, compares them with the loaded model to derive specific results. Various technologies suitable for low-end computing environments are being developed ( TVM compiler, etc.).

The image analyzer 140 may apply traditional machine vision technology (SVM, etc.) as well as the latest deep learning technology, and performs a function of deriving a relatively simple analysis result in a low-end computing environment.

In addition, when additional analysis is required, a second setting message is generated and transmitted to the second image processing device 200 through the setting message transmitter 150 . At this time, when the image is transmitted to the second image processing device 200, the setting message transmitter 150 also includes a function of encrypting and transmitting one-time access information for receiving the image in the second image processing device 200. .

The image transmitter 160 transmits the second analysis result or the collected image to the second image processing device 200 . At this time, the video transmitter 160 performs a function of processing an image masking process, snapshot image or video stream transmission, and the like.

After the first analysis is completed, the image storage queue 170 transmits a second setting message for the second analysis to the second image processing device 200 so as to transmit information between the first and second image processing devices 100 and 200. While the image transfer channel is formed, the image collected from the image collection device 10 is temporarily stored. The image temporarily stored in the image storage queue 170 is delivered when preparation for secondary analysis is completed in the second image processing device 200 .

In addition, the first image processing device 100 may include an image storage 190 that stores images like a general video recorder (DVR or NVR), and allows a user to search for stored images and play them through an image player. You can also provide an interface.

3 is a block diagram of a second image processing device 200 according to an embodiment of the present invention.

In one embodiment of the present invention, the second image processing device 200 includes a setting message interpreter 210, an image collector 220, an image analyzer 230, and an analysis result transmitter 240.

The setting message analyzer 210 receives the second setting message transmitted from the first image processing device 100 and interprets the requested information according to the contents of the corresponding message. In this case, the second setting message includes source information of the collected image and the image corresponding to the first analysis result. That is, when the first image processing device 100 transmits an image, transmission information of the corresponding image may be included, and when an image of the image collection device 10 is directly received, an IP address, ID, password, etc. may be included. Meta information (resolution, size, FPS, etc.) of the video stream may also be included. In addition, when a snapshot is received, a URL for obtaining a corresponding image or raw data reception information (format, data length, encryption method, etc.) may be included.

The image collector 220 receives an image corresponding to the first analysis result transmitted from the first image processing device 100 or an image collected from the image collecting device 10 .

The video analyzer 230 generates a second analysis result by performing a previously provided video analysis algorithm on the incoming video, and the analysis result transmitter 240 provides the second analysis result to the user terminal 20 . In this case, the second setting message may include result value destination information of the user terminal 20 for delivering the second analysis result.

4 is a diagram illustrating an embodiment of an interlocking structure of the first and second image processing devices 100 and 200 .

In an embodiment of the present invention, at least one first image processing device 100 and at least one second image processing device 200 may be provided, and according to an embodiment, a plurality of different second image processing devices 200 may be provided. A plurality may be provided to generate the second analysis result according to the function of.

Accordingly, the first image processing device 100 may selectively transmit an image and a second setting message to a corresponding second image analysis device for image analysis according to each function.

FIG. 4 illustrates a collaboration structure of a plurality of first image processing devices 100 and a plurality of second image processing devices 200, when the subject or purpose to be analyzed is different for each second image processing device 200. , the first image processing device 100 may transmit the second setting message and the image stream to a plurality of second image processing devices 200 for analysis suitable for each target.

For example, assuming that the second image processing device 200 1 is a device for advanced analysis of a vehicle and the second image processing device 200 2 is a device for advanced analysis of a human shape, advanced vehicle analysis and human shape are assumed. In order to perform advanced analysis, one first image processing device 100 may perform image analysis in cooperation with two second image processing devices 200 .

As such, each second image processing device 200 is equipped with an advanced image analysis function specialized for a specific purpose function, and receives an analysis request from the first image processing device 100 to perform analysis for a defined goal and time. form can be applied.

5 is a diagram illustrating another embodiment of an interlocking structure of the first and second image processing devices 100 and 200 .

The distributed collaboration based system 1 according to an embodiment of the present invention may further include an image processing management device 300 .

The image processing management device 300 collects and manages access addresses of the plurality of second image processing devices 200, provided functions (advanced image analysis function), and device status information (available, unavailable, resource status, etc.) perform a function

When performing image analysis settings with the first image processing device 100, the user searches for advanced image analysis functions that can be provided from the image processing management device 300 and sets the desired function to be analyzed through the first setting message. can The user selects an advanced image analysis function and transmits a first setting message to the first image processing device 100, and after the first image processing device 100 completes the first image analysis, a second image analysis request is made. A second setting message is generated and transmitted to the image processing management device 300 .

The image processing management device 300 receives the second setting message and selects the matching second image processing device 200 based on the provided function of the second image processing device 200 and device state information. Then, the third setting message is delivered to the selected second image processing device 200 .

Through this structure, an embodiment of the present invention enables users to conveniently use advanced image analysis services specialized in each field.

According to the embodiment shown in FIG. 5 , flexible collaboration with a plurality of second image processing devices 200 is possible and cost-effective image analysis is possible. That is, there is an advantage in that computing resources of the second image processing device 200 in an idle state can be shared and used by using the high-cost high-end image analysis service only when an event occurs. In addition, when a request for a specific image analysis increases, a flexible response is possible by adding the second image processing device 200 performing the corresponding function in the structure shown in FIG. 5 . In contrast, in the existing P2P analysis structure, since the analysis system for all channels must be upgraded in order to add a new analysis function, a lot of cost is consumed in changing the system.

6 is a flowchart of a real-time analysis method of a video stream in a distributed collaboration based system 1 according to an embodiment of the present invention.

Meanwhile, each step shown in FIG. 6 may be understood as being performed by the above-described distributed collaboration based system 1, but is not necessarily limited thereto.

In the method for analyzing a video stream in real time according to an embodiment of the present invention, first, the first video processing device 100 receives a first setting message including video analysis information from the user terminal 20 (S110).

Next, based on the first setting message, a first analysis result obtained by analyzing the image collected from at least one image collecting device 10 is generated (S120), and the first analysis result is additionally analyzed according to the first setting message. A second setting message for the request is generated (S130).

Next, the second image processing device 200 generates a second analysis result by performing image analysis on at least one of the first analysis result and the collected image based on the second setting message (S140).

Thereafter, the first image processing device 100 or the second image processing device 200 provides at least one of the first analysis result and the second analysis result to the user terminal 20 (S150).

Meanwhile, in the above description, steps S110 to S150 may be further divided into additional steps or combined into fewer steps according to an embodiment of the present invention. Also, some steps may be omitted if necessary, and the order of steps may be changed. Meanwhile, the contents of the distributed collaboration based system 1 of FIGS. 1 to 5 may also be applied to the contents of FIG. 6 .

The method for real-time analysis of a video stream in a distributed collaboration-based system according to an embodiment of the present invention described above may be implemented as a program (or application) to be executed in combination with a computer, which is hardware, and stored in a medium.

The above-mentioned program is C, C++, JAVA, Ruby, C, C++, JAVA, Ruby, which the processor (CPU) of the computer can read through the device interface of the computer so that the computer reads the program and executes the methods implemented as a program. It may include a code coded in a computer language such as machine language. These codes may include functional codes related to functions defining necessary functions for executing the methods, and include control codes related to execution procedures necessary for the processor of the computer to execute the functions according to a predetermined procedure. can do. In addition, these codes may further include memory reference related code for determining where (address address) of the computer's internal or external memory the additional information or media required for the computer's processor to execute the functions should be referenced. there is. In addition, when the processor of the computer needs to communicate with any other remote computer or server in order to execute the functions, the code uses the communication module of the computer to determine how to communicate with any other remote computer or server. It may further include communication-related codes for whether to communicate, what kind of information or media to transmit/receive during communication, and the like.

The storage medium is not a medium that stores data for a short moment, such as a register, cache, or memory, but a medium that stores data semi-permanently and is readable by a device. Specifically, examples of the storage medium include ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical data storage device, etc., but are not limited thereto. That is, the program may be stored in various recording media on various servers accessible by the computer or various recording media on the user's computer. In addition, the medium may be distributed to computer systems connected through a network, and computer readable codes may be stored in a distributed manner.

The above description of the present invention is for illustrative purposes, and those skilled in the art can understand that it can be easily modified into other specific forms without changing the technical spirit or essential features of the present invention. will be. Therefore, the embodiments described above should be understood as illustrative in all respects and not limiting. For example, each component described as a single type may be implemented in a distributed manner, and similarly, components described as distributed may be implemented in a combined form.

The scope of the present invention is indicated by the following claims rather than the detailed description above, and all changes or modifications derived from the meaning and scope of the claims and equivalent concepts should be construed as being included in the scope of the present invention. do.

1: Distributed collaboration based system
10: video acquisition device
20: user terminal
100: first image processing device
200: second image processing device
300: image processing management device

Claims (12)

In a distributed collaboration-based system for real-time analysis of video streams,
A first setting message including video analysis request information is received from a user terminal, and a first analysis result obtained by analyzing an image collected from at least one video collecting device based on the first setting message is generated and transmitted to the user terminal. a first image processing device configured to provide and generate a second setting message for requesting additional analysis of the first analysis result according to the first setting message;
A second image processing device generating a second analysis result obtained by performing image analysis on at least one of the first analysis result and the collected image based on the second setting message and providing the result to the user terminal,
Distributed collaboration-based system for real-time analysis of video streams.
According to claim 1,
Characterized in that the first image processing device has lower-cost computing resources than the second image processing device.
Distributed collaboration-based system for real-time analysis of video streams.
According to claim 2,
The first image processing device is an edge node, and the second image processing device is a cloud server.
Distributed collaboration-based system for real-time analysis of video streams.
According to claim 1,
The first image processing device calculates the accuracy of the first analysis result, and when there is a first analysis result having a calculated accuracy less than a preset accuracy, a second setting message for the first analysis result less than the preset accuracy. which generates,
Distributed collaboration-based system for real-time analysis of video streams.
According to claim 4,
The first image processing device generates at least one snapshot of an image corresponding to a first analysis result having less than the preset accuracy, and transmits the created snapshot to a second image processing device together with the second setting message. to do,
Distributed collaboration-based system for real-time analysis of video streams.
According to claim 1,
When it is impossible to analyze the behavior pattern of the object recognized through the first analysis result, the first image processing device transmits an image stream corresponding to the first analysis result including the recognized object together with the second setting message. To transmit to the second image processing device,
Distributed collaboration-based system for real-time analysis of video streams.
According to claim 1,
Wherein the first setting message includes analysis target information, analysis result providing information, whether or not the second setting message is generated, and image type information to be provided to a second image processing device;
Distributed collaboration-based system for real-time analysis of video streams.
According to claim 1,
Wherein the second setting message includes source information of the collected image and the image included in the first analysis result;
Distributed collaboration-based system for real-time analysis of video streams.
According to claim 1,
The first image processing device transmits the second setting message to the second image processing device to temporarily store an image collected from the image collecting device while an image transmission channel is formed between the first and second image processing devices. Including an image storage queue to store,
Distributed collaboration-based system for real-time analysis of video streams.
According to claim 1,
The second image processing device is composed of a plurality of second image processing devices generating second analysis results according to a plurality of different functions;
The first image processing device selectively transmits an image and a second setting message to a corresponding second image analysis device for image analysis according to each function,
Distributed collaboration-based system for real-time analysis of video streams.
According to claim 1,
The second image processing device is composed of a plurality of second image processing devices generating second analysis results according to a plurality of different functions;
It manages access addresses, provided functions, and device state information of the plurality of second image processing devices, receives a second setting message from the first image processing device, and provides functions and device state information of the second image processing device. Further comprising an image processing management device that selects a second image processing device that is matched based on the selected image processing device and transmits a third setting message to the selected second image processing device.
Distributed collaboration-based system for real-time analysis of video streams.
A video stream real-time analysis method in a distributed collaboration-based system,
Receiving, in a first image processing device, a first setting message including image analysis information from a user terminal;
generating, by the first image processing device, a first analysis result obtained by analyzing an image collected from at least one image collecting device based on the first setting message;
generating, by the first image processing device, a second setting message for requesting an additional analysis of a first analysis result according to the first setting message;
generating a second analysis result by performing image analysis on at least one of a first analysis result and the collected image based on the second setting message, in a second image processing device; and
In the first and second image processing devices, providing at least one of the first analysis result and the second analysis result to a user terminal,
Video stream real-time analysis method.

Images