KR20220000492A - Method for transmitting cctv video data using artificial intelligence - Google Patents

Abstract

An image data relay method, device and computer program using artificial intelligence are provided. According to one embodiment of the present invention, the image data relay method using artificial intelligence is a method performed by a computing device. The method includes the steps of: collecting a plurality types of image data; processing the plurality types of image data according to a preset image data processing rule; and transmitting the processed image data and analysis result information generated by analyzing the processed image data.

Description

CCTV video data relay method using artificial intelligence {METHOD FOR TRANSMITTING CCTV VIDEO DATA USING ARTIFICIAL INTELLIGENCE}

Various embodiments of the present invention relate to a method, apparatus, and computer program for relaying image data using artificial intelligence.

With the development of information technology and the introduction of ubiquitous computing, modern society delivers public services, contents, and new additional services beyond the infrastructure based on public service networks (electricity, gas, water and sewage, etc.) connected by various pipes or physical lines. Therefore, attention is paid to cutting-edge information and communication networks that can create high added value.

In addition, as society develops, crimes are becoming more intelligent and faster, and violent and unmotivated crimes are also becoming more frequent. Due to the nature of these crimes, it is not easy to find witnesses or evidence, and there are many difficulties in securing statements.

CCTV (Closed-Circuit Television) cameras were introduced to monitor crime in these situations and to improve the safety of society.

CCTV cameras can be used as evidence of crime by taking action when an incident occurs while monitoring and storing images in real time by using a monitor agent for a wide area or by searching the stored images after an incident occurs.

However, in the case of a conventional CCTV camera, it is composed of a full-set according to a special purpose (eg, facial recognition, intrusion detection, fire and accident monitoring, traffic signal violation, etc.) There is a problem that the initial introduction cost or cost problem occurs during expansion due to incompatibility with .

In addition, since the conventional surveillance system using a CCTV camera has a closed system and application structure, it is difficult to change the intended use or apply the application environment, and there is a problem in that it has to depend only on the customization of the provider.

In addition, conventionally, a built-in image analysis system is applied, but for this purpose, there is a burden of cost to purchase a full-set system (particularly, the higher the real-time support, the higher the cost), and the burden on maintenance and upgrade is increased. there is a problem.

Korean Patent Laid-Open Patent No. 10-2011-0060342 (2011.06.08).

The problem to be solved by the present invention is to provide an image data relay method, apparatus and computer program using artificial intelligence that can link and utilize existing CCTV infrastructure by securing compatibility between devices.

Another object to be solved by the present invention is to provide a method, apparatus and computer program for relaying image data using artificial intelligence that can freely change use or apply application environment as the purpose of use and environment change.

The problem to be solved by the present invention is to provide an image data relay method, an apparatus, and a computer program that can be maintained and upgraded without the need to purchase a full-set system by utilizing a cloud service.

The problems to be solved by the present invention are not limited to the problems mentioned above, and other problems not mentioned will be clearly understood by those skilled in the art from the following description.

An image data relay method using artificial intelligence according to an embodiment of the present invention for solving the above problems is a method performed by a computing device, the method comprising: collecting a plurality of image data; preset image data processing rules processing the plurality of image data according to the method, and transmitting the processed image data and analysis result information generated by analyzing the processed image data.

In various embodiments, the processing may include analyzing the plurality of image data to extract one or more image data in which an object is recognized from among the plurality of image data, and the object included in the extracted one or more image data. and generating metadata including analysis result information indicating information on the analyzed properties of the object by analyzing the properties of the object, wherein the delivering includes: It may include the step of delivering.

In various embodiments, the processing may include analyzing the plurality of image data to extract one or more image data in which an object is recognized from among the plurality of image data, and dividing each of the extracted one or more image data into a plurality generating unit image data of , and selecting one or more unit image data corresponding to the region including the object from among the plurality of unit image data, wherein the transmitting includes: The method may include transmitting only the selected one or more unit image data.

In various embodiments, the processing may include analyzing the plurality of image data and extracting one or more image data in which an object is recognized from among the plurality of image data, dividing each of the extracted one or more image data into a plurality of generating unit image data of , selecting one or more unit image data corresponding to a region not including the object from among the plurality of unit image data, and correcting the selected one or more unit image data to select the one or more units The method may include downgrading the resolution of the image data, and the transmitting may include transmitting the one or more image data of which the resolution of the selected one or more unit image data is lowered.

In various embodiments, the processing may include analyzing the plurality of image data to extract one or more image data for which an object is recognized from among the plurality of image data, and analyzing the extracted one or more image data to determine the and determining whether an event has occurred, wherein the delivering includes generating a clip for each event in response to the occurrence of the event of the object, the generated event clip, the extracted one or more image data, and the analysis result information may include the step of synchronizing and transmitting the .

In various embodiments, the delivering may include matching the processed image data with the analysis result information and delivering it to a server that analyzes and determines whether an event has occurred for an object included in the image data, but through the server Based on the number of image data to be analyzed and determined whether the event has occurred, the number of image data to be analyzed and determined whether the event has occurred among the processed image data is determined, and the processed image is equal to the determined number of image data It may include the step of determining whether an event has occurred for the data and transmitting it to the server.

In various embodiments, the transmitting may include analyzing whether the event has occurred and a property of the generated event generated by analyzing whether an event has occurred for an object included in the processed image data, the processed image data and the analysis Image data for analyzing whether or not the event occurs so that the data is transmitted to the server in synchronization with the result information, and the speed of data transmission to the server is less than the preset speed, so that the data transmission speed to the server is greater than or equal to the preset speed may include determining the number of

In various embodiments, the transmitting may include analyzing whether the event has occurred and a property of the generated event generated by analyzing whether an event has occurred for an object included in the processed image data, the processed image data and the analysis The result information is synchronized and transmitted to the server, but based on the cost data according to the pre-stored amount of image data transmission and server usage, the cost according to the number of data transmitted to the server and the number of data processed by the server is minimized. The method may include determining the number of image data to be analyzed whether or not an event has occurred.

An image data relay apparatus using artificial intelligence according to another embodiment of the present invention for solving the above-described problems includes a memory for storing one or more instructions and a processor for executing the one or more instructions stored in the memory, The processor may execute the image data relay method using artificial intelligence according to an embodiment of the present invention by executing the one or more instructions.

A computer program according to another embodiment of the present invention for solving the above-described problems is combined with a computer that is hardware, so that the computer can perform the image data relay method using artificial intelligence according to an embodiment of the present invention. It may be stored in a readable recording medium.

Other specific details of the invention are included in the detailed description and drawings.

According to various embodiments of the present invention, the existing CCTV infrastructure can be linked and utilized by ensuring compatibility between devices.

In addition, as the purpose of use and the environment are changed, there is an advantage that a change of use or application of an application environment is possible freely.

In addition, by utilizing the cloud service, there is an advantage that the system can be maintained and upgraded without the need to purchase a full-set of the system.

Effects of the present invention are not limited to the effects mentioned above, and other effects not mentioned will be clearly understood by those skilled in the art from the following description.

1 is a diagram illustrating an image data relay system using artificial intelligence according to an embodiment of the present invention.
2 is a diagram illustrating an image data relay system using artificial intelligence according to another embodiment of the present invention.
3 is a hardware configuration diagram of an image data relay apparatus using artificial intelligence according to another embodiment of the present invention.
4 is a flowchart of an image data relay method using artificial intelligence according to another embodiment of the present invention.
5 is a diagram exemplarily illustrating image data processed by an image data relay apparatus using artificial intelligence, according to various embodiments.
6 is a diagram illustrating an image data relay system using artificial intelligence to which a plurality of image data relay devices are applied, according to various embodiments.
7 is a diagram exemplarily illustrating a configuration for providing an alarm when an event occurs, according to various embodiments.

Advantages and features of the present invention and methods of achieving them will become apparent with reference to the embodiments described below in detail in conjunction with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below, but may be implemented in various different forms, and only the present embodiments allow the disclosure of the present invention to be complete, and those of ordinary skill in the art to which the present invention pertains. It is provided to fully understand the scope of the present invention to those skilled in the art, and the present invention is only defined by the scope of the claims.

The terminology used herein is for the purpose of describing the embodiments and is not intended to limit the present invention. In this specification, the singular also includes the plural unless specifically stated otherwise in the phrase. As used herein, “comprises” and/or “comprising” does not exclude the presence or addition of one or more other components in addition to the stated components. Like reference numerals refer to like elements throughout, and "and/or" includes each and every combination of one or more of the recited elements. Although "first", "second", etc. are used to describe various elements, these elements are not limited by these terms, of course. These terms are only used to distinguish one component from another. Accordingly, it goes without saying that the first component mentioned below may be the second component within the spirit of the present invention.

Unless otherwise defined, all terms (including technical and scientific terms) used herein will have the meaning commonly understood by those of ordinary skill in the art to which this invention belongs. In addition, terms defined in a commonly used dictionary are not to be interpreted ideally or excessively unless specifically defined explicitly.

As used herein, the term “unit” or “module” refers to a hardware component such as software, FPGA, or ASIC, and “unit” or “module” performs certain roles. However, “part” or “module” is not meant to be limited to software or hardware. A “unit” or “module” may be configured to reside on an addressable storage medium or to reproduce one or more processors. Thus, as an example, “part” or “module” refers to components such as software components, object-oriented software components, class components and task components, processes, functions, properties, Includes procedures, subroutines, segments of program code, drivers, firmware, microcode, circuitry, data, databases, data structures, tables, arrays, and variables. Components and functionality provided within “parts” or “modules” may be combined into a smaller number of components and “parts” or “modules” or as additional components and “parts” or “modules”. can be further separated.

Spatially relative terms "below", "beneath", "lower", "above", "upper", etc. It can be used to easily describe the correlation between a component and other components. Spatially relative terms should be understood as terms including different directions of components during use or operation in addition to the directions shown in the drawings. For example, when a component shown in the drawing is turned over, a component described as “beneath” or “beneath” of another component may be placed “above” of the other component. can Accordingly, the exemplary term “below” may include both directions below and above. Components may also be oriented in other orientations, and thus spatially relative terms may be interpreted according to orientation.

In this specification, a computer refers to all types of hardware devices including at least one processor, and may be understood as encompassing software configurations operating in the corresponding hardware device according to embodiments. For example, a computer may be understood to include, but is not limited to, smart phones, tablet PCs, desktops, notebooks, and user clients and applications running on each device.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

Each step described in this specification is described as being performed by a computer, but the subject of each step is not limited thereto, and at least a portion of each step may be performed in different devices according to embodiments.

1 is a diagram illustrating an image data relay system using artificial intelligence according to an embodiment of the present invention.

Referring to FIG. 1 , an image data relay system using artificial intelligence according to an embodiment of the present invention may include an image data relay device 100 , a plurality of camera modules 200 , and a server 300 .

Here, the image data relay system using artificial intelligence shown in FIG. 1 is according to an embodiment, and its components are not limited to the embodiment shown in FIG. 1, and may be added, changed, or deleted as necessary. have.

In one embodiment, the image data relay apparatus 100 collects a plurality of image data generated by each of the plurality of camera modules 200 (eg, CCTV cameras provided in a predetermined area or area) performing a photographing operation. Also, it is possible to provide an image data relay service provided to the server 300 by processing the plurality of collected image data.

At this time, the image data relay device 100 performs primary analysis on real-time streaming image data collected in real time from the plurality of camera modules 200 to detect and classify objects (eg, people, animals, vehicles, and other objects). may be performed, and metadata including information related to the extracted object may be generated according to the execution result. For example, the image data relay apparatus 100 may extract information related to an object by analyzing image data using the first artificial intelligence model. However, the present invention is not limited thereto.

Here, the first artificial intelligence model may be an artificial intelligence model pre-learned using various data such as object-related information and images as learning data, but is not limited thereto.

In various embodiments, the image data relay apparatus 100 transmits a plurality of image data collected from a plurality of camera modules 200 to the server 300, By filtering the image data, only one or more image data (eg, image data in which an object is recognized) among a plurality of image data may be selectively transmitted to the server 300 .

In various embodiments, the image data relay device 100 performs secondary analysis of one or more image data in which an object is recognized among a plurality of image data collected from a plurality of camera modules 200 to add to the one or more image data in which an object is recognized. It is possible to detect an event and a motion for the event, and transmit one or more image data including information related to the event (eg, whether an event has occurred, properties of the generated event, etc.) to the server 300 . For example, the image data relay apparatus 100 may detect an event and motion for one or more image data in which an object is recognized by analyzing image data using the second artificial intelligence model, and may generate a clip for each event. It can be transmitted to the server 300 in synchronization with the image data and metadata.

Here, the second artificial intelligence model may be a model in which information related to an object and information related to an event generated from the object have been previously learned as learning data, but is not limited thereto.

In an embodiment, the server 300 may receive, store and manage a plurality of image data (or image data processed by the image data relay device 100 ) from the image data relay device 100 . For example, the server 300 may index and store clips and metadata for each event provided from the image data relay apparatus 100 . However, the present invention is not limited thereto.

In various embodiments, the server 300 may perform a deduplication function, and image data that overlaps with a plurality of image data provided from the image data relay device 100 among a plurality of pre-stored image data through the deduplication function is displayed. It can be determined whether there is, and the overlapping image data can be removed according to the determination result.

In various embodiments, the server 300 may receive one or more image data in which an object is recognized from the image data relay device 100, but by performing an object recognition function on one or more image data in which an object is not recognized, The image data relay apparatus 100 may perform a function of identifying a false positive for detecting the missing image data.

In various embodiments, the server 300 may detect events and motions for the plurality of image data by secondary analysis of the plurality of image data provided from the image data relay apparatus 100 . For example, the server 300 may detect an event and motion for the plurality of image data by analyzing the plurality of image data using the second artificial intelligence model.

At this time, when the server 300 collects image data for which the event occurrence is determined from the image data relay device 100 (eg, when collecting secondary analyzed image data from the image data relay device 200), A separate secondary analysis operation may not be performed. However, the present invention is not limited thereto, and even if the secondary analyzed image data is collected from the image data relay device 200 according to the setting value of the monitoring user, such as the monitoring monitoring grade and the safety grade, the secondary analysis is repeatedly performed to determine whether an event occurs can be judged

2 is a diagram illustrating an image data relay system using artificial intelligence according to another embodiment of the present invention.

Referring to FIG. 2 , the image data relay apparatus 100 in the image data relay system using artificial intelligence according to another embodiment of the present invention operates a network switch 21 through a plurality of existing CCTV control infrastructures. A plurality of image data may be provided from the camera module 200, and a plurality of image data may be processed (eg, image processing and primary analysis of a real-time streaming image) according to a preset image data processing rule, and a plurality of Meta data including the extracted analysis result information can be generated by analyzing each of the image data of , and this can be transmitted to the server 300, which is a higher-level cloud system. However, the present invention is not limited thereto.

The server 300 may analyze and determine the DD image data and metadata transmitted from the image data relay device 100 . Here, the server 300 is based on the artificial intelligence of MS or AWS, and may be designed to change the rules in order to continuously improve the accuracy of data analysis and determination.

In addition, the server 300 provides a plurality of image data, object-related information, and event-related information to the worker terminal 24 of a worker who monitors and monitors a plurality of images in real time through an interface platform 25 . A user interface (UI) that provides In this case, the server 300 may classify and edit image data (eg, search for specific conditions and actions, store and structure event data) to match the conditions requested by the user through the UI.

Here, the interface platform 25 may be a platform area composed of a user application and an API that connects to the server 300 by relaying a user input obtained through it.

In addition, the server 300 determines whether an event has occurred for a plurality of image data, and, when it is determined that an event occurs, guides the worker terminal 24 for monitoring the image data or the monitoring device 23 of the control room. can provide For example, as shown in FIG. 7 , the server 300 analyzes a plurality of image data and when an event occurrence is detected, the worker terminal 24 or the monitoring device 23 of the control room through the interface platform 25 . to provide notification information, including information related to the event (e.g. object information, event information (e.g. trespassing, roaming, abandoned, fighting, arson, or signs that such an event is expected to occur, etc.), route of movement, etc. can

Thereafter, the server 300 may provide a link (eg, a “shortcut” link in FIG. 7 ) between the image data determined that the event has occurred and the notification information, and obtain a user input for selecting the link from the worker. In this case, image data determined that the event has occurred and various information related to the event may be output through the UI.

In this case, the server 300 may determine whether an event detected according to a notification condition set by the worker satisfies the notification condition, and may provide notification information only for an event determined to meet the notification condition.

The DVR/NVR 22 can temporarily store and manage a plurality of video data generated from the existing CCTV control infrastructure in the DVR/NVR 22, and store the plurality of video data in a video data monitoring control room (eg, CCTV control room). A plurality of image data sources may be output to the monitoring device 23 (eg, dashboard) provided in the CCTV control room by providing it to the server. Hereinafter, a hardware configuration of the image data relay apparatus 100 using artificial intelligence according to another embodiment of the present invention will be described with reference to FIG. 3 .

3 is a hardware configuration diagram of an image data relay apparatus using artificial intelligence according to another embodiment of the present invention.

Referring to FIG. 3 , an image data relay apparatus 100 (hereinafter, referred to as “computing apparatus 100”) according to another embodiment of the present invention may include a processor 110 and a memory 120 . In various embodiments, the computing device 100 may further include a network interface (or communication interface) (not shown), storage (not shown), and a bus (not shown).

In an embodiment, the processor 110 may control the overall operation of each component of the computing device 100 . The processor 110 may include a central processing unit (CPU), a micro processor unit (MPU), a micro controller unit (MCU), or any type of processor well known in the art.

In various embodiments, the processor 110 may perform an operation on at least one application or program for executing the method according to the embodiments of the present invention. In various embodiments, the processor 110 includes one or more cores (not shown) and a graphic processing unit (not shown) and/or a connection path (eg, a bus, etc.) for transmitting and receiving signals to and from other components. can do.

In various embodiments, the processor 110 temporarily and/or permanently stores a signal (or data) processed inside the processor 110 , a random access memory (RAM) and a read access memory (ROM). -Only Memory, not shown) may be further included. In addition, the processor 110 may be implemented in the form of a system on chip (SoC) including at least one of a graphic processing unit, a RAM, and a ROM.

In an embodiment, the processor 110 executes one or more instructions stored in the memory 120 to perform a method (eg, an image data relay method using artificial intelligence) to be described in relation to FIG. 4 . have.

For example, the processor 110 executes one or more instructions stored in the memory 120 to collect a plurality of image data, process the plurality of image data according to a preset image data processing rule, and An image data relay method using artificial intelligence may be performed, including transmitting analysis result information generated by analyzing the image data and the processed image data. However, the present invention is not limited thereto.

In one embodiment, the memory 120 may store various data, commands, and/or information. The memory 120 may store programs (one or more instructions) for processing and controlling the processor 110 . Programs stored in the memory 120 may be divided into a plurality of modules according to functions.

In various embodiments, steps of a method or algorithm described in connection with an embodiment of the present invention may be implemented directly in hardware, as a software module executed by hardware, or by a combination thereof. A software module may include random access memory (RAM), read only memory (ROM), erasable programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), flash memory, hard disk, removable disk, CD-ROM, or It may reside in any type of computer-readable recording medium well known in the art to which the present invention pertains.

The components of the present invention may be implemented as a program (or application) to be executed in combination with a computer, which is hardware, and stored in a medium. Components of the present invention may be implemented as software programming or software components, and similarly, embodiments may include various algorithms implemented as data structures, processes, routines, or combinations of other programming constructs, including C, C++ , may be implemented in a programming or scripting language such as Java, assembler, or the like. Functional aspects may be implemented in an algorithm running on one or more processors. Hereinafter, an image data relay method using artificial intelligence performed by the computing device 100 will be described with reference to FIG. 4 .

4 is a flowchart of an image data relay method using artificial intelligence according to another embodiment of the present invention.

Referring to FIG. 4 , in step S110 , the computing device 100 may collect a plurality of image data from a plurality of camera modules 200 provided in a predetermined area.

In various embodiments, the computing device 100 collects a plurality of image data from a plurality of camera modules 200 in response to receiving an image data collection request from a user, a plurality of camera modules ( 200), a plurality of image data may be collected according to any one of the methods of collecting the plurality of image data. However, the present invention is not limited thereto.

Here, the plurality of camera modules 200 may be CCTV cameras already installed in the existing CCTV infrastructure, but is not limited thereto.

Here, the computing device 100 is described as collecting a plurality of image data from a plurality of camera modules 200 provided in a predetermined area, but this is an example, and a plurality of pieces of image data are collected from a sound module provided in a predetermined area. may collect voice data of , or a plurality of sensor data from an IoT sensor module provided in a predetermined area.

In step S120 , the computing device 100 may process the plurality of image data collected in step S110 . For example, the computing device 100 may process a plurality of image data according to a preset image data processing rule.

Here, the image data processing may be a process of filtering unnecessary image data among a plurality of image data and extracting meaningful information (eg, object recognition information and event information, etc.) by analyzing necessary image data, but is not limited thereto.

In various embodiments, the computing device 100 may extract one or more image data in which an object is recognized from among the plurality of image data by first analyzing the plurality of image data. For example, the computing device 100 analyzes each of a plurality of image data using a first artificial intelligence model (eg, an object recognition model) that has been previously learned from various data such as information and images related to an object as learning data, Image data in which an object is recognized may be selected from among the plurality of image data.

Thereafter, the computing device 100 may analyze the properties of the object included in the one or more extracted image data to generate metadata including analysis result information indicating information on the analyzed properties of the object.

For example, as shown in FIG. 5 , the computing device 100 analyzes the image data of the original from which the object is recognized, and attributes such as the motion or shape of the object included in the image data (eg, the type, number, characteristics of an object, movement, etc.) can be detected. Also, the computing device 100 may generate metadata indicating properties of the detected object. However, the present invention is not limited thereto.

In various embodiments, the computing device 100 may process a plurality of image data according to a result of a primary analysis of the plurality of image data (eg, an object recognition result).

For example, the computing device 100 generates a plurality of unit image data by dividing each of one or more image data in which an object is recognized among a plurality of image data, and one corresponding to a region including the object among the plurality of unit image data. More than one unit image data can be selected.

Thereafter, the computing device 100 may generate object image data (a form in which unit image data corresponding to a region not including an object is deleted) by cutting only one or more unit image data corresponding to the region including the object.

As another example, the computing device 100 generates a plurality of unit image data by dividing each of one or more image data in which an object is recognized among a plurality of image data, and corresponds to a region that does not include an object among the plurality of unit image data. One or more unit image data may be selected.

Thereafter, the computing device 100 corrects one or more unit image data corresponding to a region not including an object with respect to each of the one or more image data in a direction in which the capacity of the image data decreases (eg, lowering the resolution and changing the screen to an achromatic color). configuration, number of frames, etc.)

For example, the computing device 100 may adjust the number of frames according to the importance of image data. For example, in the image of the part where the object is recognized, the number of frames may not be reduced in order to determine the motion of the object, or a relatively large number of frames may be transmitted to the server. On the other hand, the image of the part in which the object is confirmed not to be recognized can be transmitted to the server by reducing the number of frames relatively significantly.

In an embodiment, the server may perform object recognition on the image transmitted from the computing device 100 again. In the process, an object that the computing device 100 does not recognize or an artificial intelligence model stored in the computing device 100 does not recognize another type of object (eg, a model that recognizes a person in the computing device 100) However, when a model for recognizing an animal or a specific object may not be mounted) is recognized, an image of a predetermined time range before and after the frame in which the object is recognized is retransmitted to the computing device 100 (eg, more frames or higher resolution, etc.).

In an embodiment, the computing device 100 may adjust the number of frames transmitted to the server 300 according to the movement of the recognized object. For example, the computing device 100 may transmit a larger number of frames to the server 300 as the change between frames increases, but is not limited thereto.

In an embodiment, the computing device 100 may transmit predetermined information to the server 300 based on an interval between transmitted frames. For example, code information based on the number of skipped frames may be transmitted to the server 300 , thereby reducing the amount of transmitted data. As a specific example, when frames are sampled at intervals of 5 frames in the image, there is no recognized object, when frames are sampled at intervals of 3 frames, objects of relatively low importance are recognized, and when frames are sampled at intervals of 1 frame Assuming that an object of relatively high importance is recognized, the server may determine the type of the object included in the corresponding frames only by the interval between the transmitted frame numbers, but is not limited thereto. In addition, more information may be transmitted based on the interval of several frames. For example, when an image sampled at intervals of one frame in succession three times is received, the server 300 may determine it as a header, and may obtain information based on the frame interval of an image transmitted thereafter. For example, when frames sampled at 3 frame intervals, 2 frame intervals, and 5 frame intervals are received after the header, it may be determined that the object of the code corresponding to 325 is included in the corresponding image. A code table for this may be shared between the server and the computing device in advance.

That is, the computing device 100 processes and transmits a plurality of image data in a form to be transmitted to the server 300, but extracts only a portion in which an object is recognized or corrects the resolution of a portion in which an object is not recognized to be transmitted. By reducing the amount of image data as much as possible, it is possible to alleviate problems such as the amount of data processing in the server 300 and the amount of data provided to the server 300 and cost.

In operation S130 , the computing device 100 may determine whether an event occurs with respect to one or more image data for which an object is recognized among a plurality of image data. For example, the computing device 100 performs secondary analysis of each of one or more image data (eg, one or more image data in which an object is recognized among a plurality of image data) extracted through step S120 to determine the object included in the image data. It is possible to determine whether an event has occurred.

In various embodiments, the computing device 100 uses a second artificial intelligence model (eg, an event analysis model) previously learned from data about the object and data related to the occurrence of an event of the object as learning data from one or more image data. It is possible to determine whether an event has occurred and to extract various types of information related to the generated event. However, the present invention is not limited thereto.

In various embodiments, when receiving a control command indicating to perform an event occurrence determination operation from the server 300 , the computing device 100 performs an event occurrence determination operation in response to the control command, thereby performing a plurality of images It may be determined whether an event occurs with respect to one or more image data in which an object is recognized among data.

In various embodiments, the computing device 100 determines whether an event occurs with respect to one or more image data for which an object is recognized among a plurality of image data according to a preset condition, or image data to be determined whether an event occurs can determine the number of

First, the computing device 100 analyzes whether an event occurs and determines whether an event has occurred through the server 300 (eg, a server that detects and determines whether an event has occurred) based on a plurality of images based on the number of image data to be determined. The number of image data to be analyzed and determined whether or not an event has occurred among the data may be determined, and whether an event has occurred in the image data may be determined by the determined number of image data.

For example, when the amount of image data provided to the server 300 is less than the reference data amount, that is, the amount of image data provided to the server 300 is small, the computing device 100 determines whether an event has occurred in the server 300 . If it is determined that it is sufficient to determine whether an event has occurred on the image data only The number of image data to be analyzed and determined is determined as 0), and only an operation of recognizing an object among a plurality of image data may be performed and provided to the server 300 .

On the other hand, when the amount of image data provided to the server 300 exceeds the reference data amount, the computing device 100 determines whether an event occurs with respect to some image data among one or more image data for which an object is recognized. can do.

At this time, the computing device 100 determines the difference between the number of image data provided to the server 300 and the reference value, that is, the number of transmitted image data exceeding the reference data amount as the number of image data to analyze and determine whether an event has occurred. can decide However, the present invention is not limited thereto.

Next, the computing device 100 analyzes whether an event occurs according to a difference between a plurality of image data or a speed at which one or more image data for which an object is recognized among the plurality of image data is transmitted to the server 300 and a preset speed, and The number of image data to be determined may be determined.

For example, when the speed at which data is transmitted to the server 300 is greater than or equal to a preset speed, the computing device 100 determines whether an event occurs with respect to a plurality of image data or one or more image data for which an object is recognized among the plurality of image data. can be analyzed and judged.

At this time, the computing device 100 may determine (maximum value) the number of image data to analyze and determine whether an event has occurred so that the speed at which the data is transmitted to the server 300 is included within a range that is greater than or equal to a preset speed, , it is possible to determine whether an event has occurred with respect to the image data by the determined number of image data.

On the other hand, when the speed at which data is transmitted to the server 300 is less than a preset speed, the computing device 100 analyzes whether an event occurs with respect to a plurality of image data or one or more image data in which an object is recognized among the plurality of image data. and the number of image data to be analyzed whether an event occurs or not may be determined so that the determining operation is not performed or the speed of data transmission to the server 300 is greater than or equal to the preset speed, and the number of image data equal to the determined number of image data It is possible to determine whether an event has occurred.

Next, the computing device 100 may analyze whether an event has occurred and determine the number of image data to be determined based on cost data according to the pre-stored amount of image data transmission and server usage.

In general, communication costs and server usage fees may increase or decrease depending on data transmission and server usage. In particular, when using a cloud server, since the cost of using the cloud is significant, there is a need to adjust the amount of data transmitted to and processed by the server in order to reduce costs.

That is, the computing device 100 has a minimum cost according to the number of data transmitted to the server 300 and the number of data processed by the server 300 based on cost data according to the pre-stored image data transmission amount and server usage amount. It is possible to determine the number of image data to be analyzed to determine whether an event has occurred.

In various embodiments, the computing device 100 provides a preset data throughput (eg, the capability (computing power) of the computing device 100 , initial introduction cost, installation and maintenance cost, etc.) according to the performance of the computing device 100 . It is possible to determine the number of image data to be analyzed whether or not an event has occurred based on the initially adjusted value). However, the present invention is not limited thereto, and any method of determining whether the computing device 100 performs an event occurrence operation for a plurality of image data or determining the number of image data for determining whether an event occurs may be applied.

In various embodiments of the present disclosure, the computing device 100 and the server 300 analyze a plurality of image data to determine whether an event has occurred, but analyze a relatively simple event through the computing device 100 (eg, a person This falling event) may be set to be performed, and analysis of a relatively complex event may be performed through the server 300 .

Also, with respect to a plurality of different types of events, the computing device 100 and the server 300 may be set to detect different types of events. To this end, the computing device 100 and the server 300 perform the same function of event detection, but may use different artificial intelligence models that have been pre-learned using information on different types of events as learning data.

Also, the computing device 100 and the server 300 may be set to repeatedly analyze an event for the same object.

In addition, the computing device 100 performs event analysis on any one or more image data among a plurality of image data, and the server 300 analyzes events on the remaining image data that has not been event-analyzed through the computing device 100 . can be set to perform.

In addition, the computing device 100 performs event analysis on one or more image data for which an object is recognized among a plurality of image data, and the server 300 repeatedly performs event analysis on the event-analyzed image data through the computing device 100 . It may be set to analyze and verify the event analysis result. However, the present invention is not limited thereto, and various methods for performing the function of the computing device 100 and the server 300 determining whether an event has occurred may be applied.

That is, in various embodiments of the present invention, data processing speed, cost, etc. may be optimized by balancing the data throughput of the computing device 100 and the server 300 .

In step S140 , the computing device 100 may transmit the image data processed through steps S120 and S130 and analysis result information generated by analyzing the processed image data to the server 300 .

In various embodiments, the computing device 100 may transmit one or more image data in which an object is recognized among a plurality of image data and metadata indicating information related to an object extracted from the image data together to the server 300 .

In addition, when each of the plurality of image data is processed through step S120 (eg, only unit image data in which an object is recognized or the resolution of unit image data in which an object is not recognized is adjusted), the computing device 100 , the processed image data may be transmitted to the server 300 .

In various embodiments, when determining whether an event has occurred with respect to the entire image data or a part of the image data according to a request by the server 300 or a preset condition, the computing device 100 responds to the occurrence of an event of the object A clip for each event may be generated, and the generated event clip, image data, and analysis result information (meta data) may be synchronized (eg, time stamped) according to time and transmitted to the server 300 .

In various embodiments, the computing device 100 may store a plurality of image data, one or more image data in which an object is recognized, metadata, and event related data in a temporary storage device (eg, DVR) provided therein, In response to receiving a request for providing image data from the server 300 , pre-stored image data and various data may be transmitted.

The image data relay method using the aforementioned artificial intelligence has been described with reference to the flowchart shown in the drawings. For simple explanation, the image data relay method using artificial intelligence has been described by showing a series of blocks, but the present invention is not limited to the order of the blocks, and some blocks are performed in an order different from that shown and operated in this specification or may be performed simultaneously. In addition, new blocks not described in the present specification and drawings may be added, or some blocks may be deleted or changed.

In the image data relay method using artificial intelligence according to an embodiment of the present invention, it is based on collecting image data from a plurality of camera modules 200 in the existing CCTV control infrastructure and processing it according to a preset image data processing rule. However, the present invention is not limited thereto, and the voice data collected from a plurality of speaker (sound) modules is processed according to a preset voice data processing rule or sensor data sensed from a plurality of IoT-based sensors is collected. It can be processed according to the set sensor data processing rules. Hereinafter, a form to which the plurality of image data relay devices 100 are applied will be described with reference to FIG. 6 .

6 is a diagram illustrating an image data relay system using artificial intelligence to which a plurality of image data relay devices are applied, according to various embodiments.

Referring to FIG. 6 , an image data relay system according to various embodiments of the present disclosure may include a plurality of image data relay devices 100 - 1 to 100 -N.

The video data relay device 100 may perform a function of controlling the CCTV system of a certain scale (eg, one factory and one building unit), and divides a predetermined area into an area of a certain size, and divides each A plurality of image data relay devices 100-1 to 100-N may be included in order to control the CCTV system for the area of .

Each of the plurality of image data relay devices 100-1 to 100-N includes a first AI model (eg, a Darknet-based object recognition model) and a second AI model (eg, an event detection model such as motion detection) Various artificial intelligence models including

Through this, each of the plurality of image data relay devices 100-1 to 100-N performs primary analysis (eg, object recognition) or secondary analysis (eg, event detection) on the plurality of image data by itself. , and through this, the extracted result data may be purified and selected and transmitted to the server 300 .

At this time, each of the plurality of image data relay devices 100-1 to 100-N performs different functions so as to perform optimal functions in consideration of various factors such as the installation environment, requests of users and managers, and safety ratings. One or more AI models among a plurality of AI models may be selectively loaded.

In addition, each of the plurality of image data relay devices 100 - 1 to 100 -N may mount a plurality of the same artificial intelligence model based on the amount of data to be processed.

That is, the plurality of image data relay devices 100-1 to 100-N may have different roles and functions depending on the installation environment or situation, and different AI models may be mounted according to the functions to be performed. can

For example, the first image data relay device 100-1 among the plurality of image data relay devices 100-1 to 100-N is equipped with only the first artificial intelligence model to perform only the object recognition function for image data. In addition, the second image data relay apparatus 100 - 2 may be equipped with both the first artificial intelligence model and the second artificial intelligence model to perform both an object recognition function for image data and a function to determine whether an event has occurred. However, the present invention is not limited thereto, and a plurality of artificial intelligence models performing different functions may be mounted on each of the plurality of image data relay devices 100-1 to 100-N, and may be used depending on the installation environment or situation. Various methods can be applied, such as a method of selectively using only an artificial intelligence model.

In various embodiments, each of the plurality of image data relay devices 100-1 to 100-N is based on an embedded Linux operating system and GPU programming (eg, CUDA, OpenCL), which may be linked to an IP camera (Real-RTSP). Time Streaming Protocol) server.

The server 300 may be an image data streaming server and a cloud server, and may index and store image data clips and metadata.

In addition, the server 300 may provide a monitoring service for monitoring a plurality of image data, and may manage accounts and privileges for users and devices receiving the monitoring service.

In addition, the server 300 may collect big data for image data analysis using an artificial intelligence model.

The server 300 may provide an application 32 that provides a web-based control solution. For example, the server 300 may provide a WEB/APP service to the building control center server 33 through the application 32 to provide a monitoring, notification and control solution through the terminal.

In the above, embodiments of the present invention have been described with reference to the accompanying drawings, but those of ordinary skill in the art to which the present invention pertains can realize that the present invention can be embodied in other specific forms without changing the technical spirit or essential features thereof. you will be able to understand Therefore, it should be understood that the embodiments described above are illustrative in all respects and not restrictive.

100: Image data relay device (or computing device) using artificial intelligence
200: camera module
300 : server

Claims (10)

A method performed by a computing device, comprising:
collecting a plurality of image data;
processing the plurality of image data according to a preset image data processing rule; and
Transmitting the processed image data and the analysis result information generated by analyzing the processed image data,
Video data relay method using artificial intelligence. According to claim 1,
The processing step is
analyzing the plurality of image data and extracting one or more image data from which an object is recognized from among the plurality of image data; and
and analyzing the properties of the object included in the one or more extracted image data to generate metadata including analysis result information indicating information on the properties of the analyzed object,
The delivering step is
Transmitting the extracted one or more image data and the metadata,
Video data relay method using artificial intelligence. According to claim 1,
The processing step is
analyzing the plurality of image data and extracting one or more image data from which an object is recognized from among the plurality of image data; and
generating a plurality of unit image data by dividing each of the one or more extracted image data, and selecting one or more unit image data corresponding to an area including the object from among the plurality of unit image data;
The delivering step is
Transmitting only the selected one or more unit image data from among the processed image data,
Video data relay method using artificial intelligence. According to claim 1,
The processing step is
analyzing the plurality of image data and extracting one or more image data from which an object is recognized from among the plurality of image data;
generating a plurality of unit image data by dividing each of the one or more extracted image data, and selecting one or more unit image data corresponding to a region not including the object from among the plurality of unit image data; and
Compensating the selected one or more unit image data to lower the resolution of the selected one or more unit image data;
The delivering step is
Transmitting the one or more image data of which the resolution of the selected one or more unit image data is downgraded,
Video data relay method using artificial intelligence. According to claim 1,
The processing step is
analyzing the plurality of image data and extracting one or more image data from which an object is recognized from among the plurality of image data; and
and analyzing the one or more extracted image data to determine whether an event of the object has occurred,
The delivering step is
generating a clip for each event in response to the occurrence of an event of the object, and synchronizing and transmitting the generated event clip with the one or more extracted image data and the analysis result information;
Video data relay method using artificial intelligence. According to claim 1,
The delivering step is
Matching the processed image data with the analysis result information and delivering it to a server that analyzes and determines whether an event has occurred for an object included in the image data,
Based on the number of image data to be analyzed and determined whether the event has occurred through the server, the number of image data to be analyzed and determined whether the event has occurred among the processed image data is determined, and the determined number of image data Comprising the step of determining whether an event has occurred for the processed image data as much as possible and delivering it to the server,
Video data relay method using artificial intelligence. According to claim 1,
The delivering step is
The event occurrence and the properties of the generated event generated by analyzing whether an event has occurred for an object included in the processed image data are synchronized with the processed image data and the analysis result information and transmitted to the server,
When the data transmission speed to the server is less than a preset speed, determining the number of image data to be analyzed whether the event occurs so that the data delivery speed to the server is greater than or equal to the preset speed,
Video data relay method using artificial intelligence. According to claim 1,
The delivering step is
The event occurrence and the properties of the generated event generated by analyzing whether an event has occurred for an object included in the processed image data are synchronized with the processed image data and the analysis result information and transmitted to the server,
Based on the pre-stored image data transmission amount and cost data according to the server usage amount, the image data to be analyzed whether the event has occurred so that the cost according to the number of data transmitted to the server and the number of data processed by the server is minimized. determining the number;
Video data relay method using artificial intelligence. a memory storing one or more instructions; and
a processor executing the one or more instructions stored in the memory;
The processor by executing the one or more instructions,
An apparatus for performing the method of claim 1 . A computer program stored in a computer-readable recording medium in combination with a computer, which is hardware, to perform the method of claim 1.

Images