KR20190051175A - Intelleigent cooperating server, system and method of analyzing based on cooperation thereof - Google Patents

Abstract

According to the present invention, an intelligent collaboration server comprises: a communication module which transmits/receives data to/from an integrated control server and an image analysis server; a memory which stores a program for analysis based on collaboration; and a processor which executes the program stored in the memory. The processor is as follows: according to execution of the program, receiving analyzed first image analysis information captured by one or more edge cameras and detailed camera information from the integrated control server; when receiving second image analysis information, in which images captured by the edge camera is transmitted from the integrated control server and analyzed, from the image analysis server, analyzing one or more between the first and second image analysis information; and based on the analysis result, generating control information for controlling the one or more edge cameras through the detailed camera information. Accordingly, the present invention enables collaboration between edge cameras and the image analysis server.

Description

TECHNICAL FIELD [0001] The present invention relates to an intelligent collaboration server, an intelligent collaboration server, a system, and a collaborative analysis method based on the intelligent collaboration server,

The present invention relates to an intelligent collaboration server, a system, and a collaboration-based analysis method thereof.

1 is a view for explaining an intelligent image analysis system according to the prior art.

The intelligent image analysis system according to the prior art includes one or more edge cameras 10, an integrated control server 20 and an image analysis server 30. [

This intelligent image analysis system performs server based image analysis and edge camera based image analysis according to the analysis subject.

At this time, the server-based image analysis method means a centralized method in which an image captured by the edge camera 10 is received by the image analysis server 30 and image analysis is performed. Such a server-based image analysis method is easy to improve the algorithm and has high accuracy of analysis, but there is a problem that excessive server construction cost is incurred, and it is heavily influenced by the surrounding infrastructure such as network and platform. In addition, if an error occurs in the server itself, there is a problem that the image analysis of the entire system is impossible.

The image analysis method based on an edge camera means a method of performing image analysis at the terminal end by embedding an image analysis algorithm in the edge camera 10. This image analysis method has an advantage that image analysis can be performed even when the server is not operated, but the range and performance of spirituality analysis are limited rather than the server-based image analysis method, and it is difficult to rapidly improve the analysis algorithm.

Therefore, there is a need for techniques that can complement each other's disadvantages by using both the server-based image analysis method and the edge camera-based image analysis method.

The embodiment of the present invention is based on the hybrid method in which a server-based image analysis method and an edge camera-based image analysis method are combined to perform image analysis through image analysis of a multi-processing method in an environment requiring accuracy and real- The present invention provides an intelligent collaborative server and system capable of ensuring the reliability of the information and adaptively cooperating with and controlling the cooperation of the situation information, and a collaboration-based analysis method thereof.

It should be understood, however, that the technical scope of the present invention is not limited to the above-described technical problems, and other technical problems may exist.

According to a first aspect of the present invention, there is provided an intelligent collaboration server comprising: a communication module for transmitting / receiving data to / from an integrated control server and an image analysis server; a memory for storing a program for analyzing collaboration-

And a processor for executing the program stored in the memory. At this time, as the program is executed, the processor receives the first image analysis information and the camera detail information captured and analyzed by the one or more edge cameras from the integrated control server through the communication module, Analyzing at least one of the first and second image analysis information when the second image analysis information is received from the image analysis server and receiving the analyzed image from the edge camera, And generates control information for controlling the at least one edge camera through the camera detailed information.

The processor may analyze one or more of the first and second image analysis information based on an event-based analysis technique, a learning-based analysis technique, and a rule-based analysis technique.

Wherein the processor analyzes the at least one of the first and second image analysis information to generate situation analysis information according to the event-based analysis technique, and when the situation analysis information is matched with an event included in a previously stored event list , And generate control information corresponding to the event.

Wherein the processor analyzes the first and second image analysis information based on a previously learned algorithm according to the learning analysis technique and controls the PTZ setting value of the edge camera based on the analyzed result Wherein the previously learned algorithm sets coordinate information, speed, and bounding box size information of an object included in image analysis information prepared in advance as input, and outputs control information for controlling the corresponding edge camera as an output Can be trained.

Wherein the processor analyzes the at least one of the first image analysis information and the second image analysis information according to the rule-based analysis technique to determine that the rule matches at least one rule included in the rule stored in advance, Control information can be generated.

The processor may further include control information for zooming in or zooming out the edge camera according to the rule when the bounding box for detecting the object included in the first and second image analysis information is less than or equal to a predetermined threshold size, Lt; / RTI >

Wherein the processor analyzes the first and second image analysis information according to a result of analyzing the first and second image analysis information having the highest reliability according to the event-based analysis technique, the learning-based analysis technique, and the rule- Control information can be generated.

The processor may generate the control information in the form of metadata including at least one of a camera ID, a camera PTZ setting information, and a frame reanalysis request, and may be transmitted to at least one of the edge camera and the image analysis server.

According to a second aspect of the present invention, there is provided an intelligent collaboration system including at least one edge camera for capturing an image and generating first image analysis information for analyzing the captured image,

An image analysis server for receiving the image from the edge camera and receiving the image, the first image analysis information and the camera detail information, the image analysis server receiving the image from the integrated control server and generating second image analysis information obtained by analyzing the image, And an intelligent collaboration server for analyzing at least one of the first and second image analysis information and generating control information for controlling the at least one edge camera through the camera detail information based on the analysis result.

A third aspect of the present invention is a method for analyzing a collaboration based on an intelligent collaboration server, the method comprising: analyzing at least one of first image analysis information analyzed by one or more edge cameras and second image analysis information analyzed by an image analysis server Receiving; Analyzing at least one of the first and second image analysis information based on an event-based analysis technique, a learning-based analysis technique, and a rule-based analysis technique, and analyzing the at least one edge And generating control information for controlling the camera. At this time, the edge camera transmits the captured image together with the first image analysis information and its own camera detailed information to the integrated control server, and the image analysis server receives the captured image from the integrated control server, And generates second image analysis information.

According to any one of the above-described objects of the present invention, collaboration between an edge camera and an image analysis server can be performed.

It is possible to secure the reliability of the image analysis in an environment requiring accuracy and real time of image analysis such as defense field and security field, and it is possible to adapt to various environments adaptively.

1 is a view for explaining an intelligent image analysis system according to the prior art.
2 is a schematic diagram for explaining an intelligent collaboration system according to an embodiment of the present invention.
3 is a block diagram of an intelligent collaboration system in accordance with an embodiment of the present invention.
4 is a block diagram of an intelligent collaboration server according to an embodiment of the present invention.
5 is a flowchart of a collaboration-based analysis method according to an embodiment of the present invention.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings, which will be readily apparent to those skilled in the art. The present invention may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. In order to clearly explain the present invention in the drawings, parts not related to the description are omitted.

Whenever a component is referred to as " including " an element throughout the specification, it is to be understood that the element may include other elements, not the exclusion of any other element, unless the context clearly dictates otherwise.

2 is a schematic diagram for explaining an intelligent collaboration system 1 according to an embodiment of the present invention. 3 is a block diagram of an intelligent collaboration system 1 in accordance with an embodiment of the present invention. 4 is a block diagram of an intelligent collaboration server 100 according to an embodiment of the present invention.

The intelligent collaboration system 1 according to an exemplary embodiment of the present invention includes an intelligent collaboration server 100, an integrated control server 200, an integrated control viewer 210, an image analysis server 300 and one or more edge cameras 400, . On the other hand, each server can be linked based on REST / JSON.

One or more edge cameras 400 may be provided and take images. The edge camera 400 is embedded with an image analysis algorithm, and analyzes the photographed image to generate first image analysis information.

The edge camera 400 transmits the photographed image, the analyzed first image analysis information, and its own camera detailed information to the integrated control server 200.

The first image analysis information may include, for example, a camera ID, a frame number, a type of anomaly symptom, X and Y coordinates of the detected object, and area and height information of the detected object area.

In addition, the camera detailed information may include a camera ID, a position of a camera, information on a camera PTZ (Pan, Tilt, Zoon), and a type of a camera.

The integrated control server 200 outputs an image photographed by the edge camera 400 through the integrated control viewer and stores and manages the first image analysis information and the second image analysis information to be described later.

To this end, the integrated control server 200 receives the first image analysis information and the camera detail information of each of the edge cameras 400 from the edge camera 400, and outputs the first image analysis information and the camera detail information to the VMS (Video Mansgement System) metadata.

The image analysis server 300 receives the image and the VMS metadata from the integrated control server 200 and generates second image analysis information by analyzing the image based on the camera detailed information included in the VMS metadata.

And transmits the second image analysis information to the integrated control server 200 and the intelligent collaboration server 100.

At this time, the second image analysis information includes, for example, a camera ID, a frame number, an abnormal symptom type, an abnormal symptom region, a weather type, the number of detected objects, an object ID, The area width and height information, the ratio of the detected object, the color, the shape, and the moving speed.

For reference, the first image analysis information analyzed by the edge camera 400 generates analysis information having lower performance than the second image analysis information analyzed through the image analysis server 300, but is not limited thereto.

The intelligent collaboration server 100 analyzes at least one of the first image analysis information included in the VMS metadata received from the integrated control server 200 and the second image analysis information received from the image analysis server 300, And generates collaborative interworking information based on the result of the analysis.

In this case, the cooperative linking information includes control information for controlling the at least one edge camera 400 through the camera detailed information based on the analysis result. For example, the control information may include at least one of a camera ID, a camera PTZ setting information, and a frame reanalysis request, and the control information may be generated in a metadata form and transmitted to the edge camera 400 and the image analysis server 300 More than one can be delivered.

Hereinafter, the intelligent collaboration server 100 according to an embodiment of the present invention will be described in more detail with reference to FIG.

The intelligent collaboration server 100 includes a communication module 110, a memory 120, and a processor 130. Meanwhile, the integrated management server 200 and the image analysis server 300 may be implemented as a communication module, a memory, and a processor in the same manner as the intelligent collaboration server 100 according to an exemplary embodiment of the present invention.

The communication module 110 transmits and receives data to and from the integrated control server 200 and the image analysis server 300. Here, the communication module 110 may include both a wired communication module and a wireless communication module. The wired communication module may be implemented by a power line communication device, a telephone line communication device, a cable home (MoCA), an Ethernet, an IEEE1294, an integrated wired home network, and an RS-485 control device. In addition, the wireless communication module can be implemented by the UNB, WLAN, Bluetooth, HDR WPAN, UWB, ZigBee, Impulse Radio, 60 GHz WPAN, Binary-CDMA, wireless USB technology and wireless HDMI technology.

In the memory 120, a program for analysis based on collaboration is stored. Herein, the memory 120 is collectively referred to as a nonvolatile storage device and a volatile storage device which keep the stored information even when power is not supplied.

For example, the memory 120 may be a compact flash (CF) card, a secure digital (SD) card, a memory stick, a solid-state drive (SSD) A magnetic computer storage device such as a NAND flash memory, a hard disk drive (HDD) and the like, and an optical disc drive such as a CD-ROM, a DVD-ROM, etc. .

The processor 130 executes the program stored in the memory 120. [

Specifically, the processor 130 receives the first image analysis information and the camera detail information captured and analyzed by the at least one edge camera 400 from the integrated control server 200 through the communication module 110 in the form of VMS metadata do.

And receives the image photographed by the edge camera 400 from the integrated control server 200 and receives the analyzed second image analysis information from the image analysis server 300.

In addition, the processor 130 may acquire collaboration control information directly input by an administrator and generate collaboration interworking information together with or separately from the received information.

Next, the processor 130 analyzes one or more of the first and second image analysis information based on the event-based analysis technique, the learning-based analysis technique, and the rule-based analysis technique.

At this time, the processor 130 may analyze the at least one of the first and second image analysis information according to the event analysis technique to generate the situation analysis information. When the situation analysis information is matched with the event included in the event list stored in advance, control information corresponding to the event can be generated.

For example, the processor 130 may generate the situation analysis information indicating that the object is determined to be 'roaming' as a result of analyzing at least one of the first and second image analysis information, If there is an event, the processor 130 transmits object tracking information for tracking the roaming object to the edge camera 400 through the integrated control system, and the edge camera 400 can track the object in real time .

The processor 130 analyzes the first and second image analysis information based on the previously learned algorithm according to the learning analysis technique and controls the PTZ setting values of the edge camera 400 based on the analyzed results Lt; / RTI >

At this time, the previously learned algorithm sets the coordinate information, the velocity, and the bounding box size information of the object included in the image analysis information prepared in advance as input, and sets the control information for controlling the corresponding edge camera 400 as output .

In addition, when the processor 130 determines that one or more rules included in the rule stored in advance as a result of analyzing one or more of the first and second image analysis information according to the rule-based analysis technique, It is possible to generate control information.

For example, when the bounding box for detecting an object included in the first and second image analysis information is equal to or smaller than a preset threshold size, that is, when it is determined that an object is detected at a long distance, The control information for zooming in the edge camera 400 can be generated.

Similarly, when the bounding box is larger than the critical size, that is, when it is determined that an object is detected at a close range, control information for zooming out the edge camera 400 may be generated according to the rule.

Meanwhile, when the processor 130 analyzes the first and second image analysis information according to the event analysis technique, the learning analysis technique, and the rule-based analysis technique, if there are conflicting results or there are different results , And the control information can be generated based on the result of the probability-based analysis technique having the highest reliability among the above analysis techniques.

When the result is selected according to one of the analysis techniques and control information is generated according to the selected analysis method, the processor 130 cooperates with at least one of the edge camera 400 and the image analysis server 300 together with the control information And transmits the generated collaboration information to the image analysis server 300 and the integrated control server 200. [ At this time, the collaboration interworking information may include camera ID and PTZ setting information as camera control information, and may include re-analysis request information of a specific frame.

After transmitting the collaboration interworking information to the image analysis server 300 and the integrated control server 200, the processor 130 may receive feedback information from each server through the communication module 110. [

When the first and second image analysis information are received from the integrated control server 200 or the image analysis server 300, the above process is repeatedly performed. If the first and second image analysis information are not received, Thereby keeping the intelligent collaboration server 100 in an inactive state.

3 may be implemented in hardware such as software or an FPGA (Field Programmable Gate Array) or ASIC (Application Specific Integrated Circuit), and may perform predetermined roles can do.

However, 'components' are not meant to be limited to software or hardware, and each component may be configured to reside on an addressable storage medium and configured to play one or more processors.

Thus, by way of example, an element may comprise components such as software components, object-oriented software components, class components and task components, processes, functions, attributes, procedures, Routines, segments of program code, drivers, firmware, microcode, circuitry, data, databases, data structures, tables, arrays, and variables.

The components and functions provided within those components may be combined into a smaller number of components or further separated into additional components.

Hereinafter, a method of analyzing a collaboration based on the intelligent collaboration server 100 according to an embodiment of the present invention will be described with reference to FIG.

5 is a flowchart of a collaboration-based analysis method according to an embodiment of the present invention.

In the collaboration-based analysis method according to an embodiment of the present invention, an edge camera 400 captures an image (S101), and generates first image analysis information based on the captured image (S103).

Then, the edge camera 400 transmits the photographed image, the camera detail information, and the first image analysis information to the integrated control server 200 (S105).

The integrated control server 200 generates VMS metadata including the first image analysis information analyzed by the edge camera 400, the camera detail information, and the state information of the edge camera 400 (S107) The metadata is transmitted to the image analysis server 300 (S109), and the VMS metadata is transmitted to the intelligent collaboration server 100 (S111).

The image analysis server 300 receives the image, generates second image analysis information using the camera detail information (S113), and transmits the second image analysis information to the intelligent collaboration server 100 and the integrated management server (S115, S117).

The intelligent collaboration server 100 receives the first image analysis information included in the VMS metadata and the second image analysis information from the image analysis server 300 and generates an event based analysis technique, Based analysis technique based on at least one of the first and second image analysis information (S119).

Next, based on the analysis result, control information for controlling one or more edge cameras 400 is generated through the camera detailed information, and the generated control information is generated as collaboration linked information (S121) To the server (S123, S125).

The integrated management server transmits camera control information to the edge camera 400 (S127), and the edge camera 400 performs PTZ control based on the control information (S129).

On the other hand, in the above description, steps S101 to S127 may be further divided into further steps or combined into fewer steps, according to an embodiment of the present invention. Also, some of the steps may be omitted as necessary, and the order between the steps may be changed. In addition, the contents already described with respect to the intelligent collaboration system 1 and the server 100 in FIGS. 2 to 4 apply to the collaboration-based analysis method of FIG. 5 even if other contents are omitted.

On the other hand, an embodiment of the present invention may also be embodied in the form of a computer program stored in a medium executed by a computer or a recording medium including instructions executable by the computer. Computer readable media can be any available media that can be accessed by a computer and includes both volatile and nonvolatile media, removable and non-removable media. In addition, the computer-readable medium may include both computer storage media and communication media. Computer storage media includes both volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Communication media typically includes any information delivery media, including computer readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave, or other transport mechanism.

While the methods and systems of the present invention have been described in connection with specific embodiments, some or all of those elements or operations may be implemented using a computer system having a general purpose hardware architecture.

It will be understood by those skilled in the art that the foregoing description of the present invention is for illustrative purposes only and that those of ordinary skill in the art can readily understand that various changes and modifications may be made without departing from the spirit or essential characteristics of the present invention. will be. It is therefore to be understood that the above-described embodiments are illustrative in all aspects and not restrictive. For example, each component described as a single entity may be distributed and implemented, and components described as being distributed may also be implemented in a combined form.

The scope of the present invention is defined by the appended claims rather than the detailed description and all changes or modifications derived from the meaning and scope of the claims and their equivalents are to be construed as being included within the scope of the present invention do.

1: Intelligent collaboration system
100: Intelligent Collaboration Server
200: Integrated control server
300: image analysis server
400: Edge camera

Claims (10)

In an intelligent collaboration server,
A communication module for transmitting and receiving data with the integrated control server and the image analysis server,
A program for analysis of collaborative based on stored memory and
And a processor for executing a program stored in the memory,
Wherein the processor is configured to receive the first image analysis information and camera detail information captured and analyzed by at least one edge camera from the integrated control server through the communication module as the program is executed, Wherein the image analysis server analyzes at least one of the first and second image analysis information when the second image analysis information is received from the image analysis server and receives the image captured by the edge camera and analyzes the second image analysis information, And generates control information for controlling the at least one edge camera through the detailed information. The method according to claim 1,
Wherein the processor analyzes one or more of the first and second image analysis information based on an event-based analysis technique, a learning-based analysis technique, and a rule-based analysis technique. 3. The method of claim 2,
Wherein the processor analyzes the at least one of the first and second image analysis information to generate situation analysis information according to the event-based analysis technique, and when the situation analysis information is matched with an event included in a previously stored event list And generates control information corresponding to the event. 3. The method of claim 2,
Wherein the processor analyzes the first and second image analysis information based on a previously learned algorithm according to the learning analysis technique and controls the PTZ setting value of the edge camera based on the analyzed result Lt; / RTI >
The pre-learned algorithm is set by inputting coordinate information, velocity, and bounding box size information of an object included in image analysis information prepared in advance, and training by outputting control information for controlling the corresponding edge camera Intelligent Collaboration Server. 3. The method of claim 2,
Wherein the processor analyzes the at least one of the first image analysis information and the second image analysis information according to the rule-based analysis technique to determine that the rule matches at least one rule included in the rule stored in advance, And generates control information. 6. The method of claim 5,
The processor may further include control information for zooming in or zooming out the edge camera according to the rule when the bounding box for detecting the object included in the first and second image analysis information is less than or equal to a predetermined threshold size, Wherein the intelligent collaboration server is configured to generate the intelligent collaboration server. 3. The method of claim 2,
Wherein the processor analyzes the first and second image analysis information according to a result of analyzing the first and second image analysis information having the highest reliability according to the event-based analysis technique, the learning-based analysis technique, and the rule- And generates control information. The method according to claim 1,
Wherein the control information is generated in the form of metadata including at least one of a camera ID, a camera PTZ setup information, and a frame reanalysis request, and is transmitted to at least one of the edge camera and the image analysis server. In an intelligent collaboration system,
At least one edge camera for capturing an image and generating first image analysis information for analyzing the captured image,
An integrated control server for receiving the image, the first image analysis information, and the camera detail information from the edge camera,
An image analysis server for receiving the image from the integrated control server and generating second image analysis information obtained by analyzing the image,
An intelligent collaboration server that analyzes at least one of the first and second image analysis information and generates control information for controlling the at least one edge camera through the camera detail information based on the analysis result, system. In a collaboration-based analysis method in an intelligent collaboration server,
Receiving at least one of first image analysis information analyzed by one or more edge cameras and second image analysis information analyzed by an image analysis server;
Analyzing at least one of the first and second image analysis information based on an event-based analysis technique, a learning-based analysis technique, and a rule-based analysis technique;
And generating control information for controlling the at least one edge camera through the camera detail information based on the analysis result,
Wherein the edge camera transmits the captured image together with the first image analysis information and its own camera detailed information to the integrated control server, and the image analysis server receives the captured image from the integrated control server, A method of analyzing a collaboration based on generating image analysis information.

Images