KR102641534B1 - Cloud-edge integrated platform system - Google Patents

Abstract

The present invention relates to a cloud-edge integrated platform system applicable to the fields of video analysis and monitoring, cloud computing, artificial intelligence, and data security based on the interaction between edge devices and cloud servers, and more specifically, to at least one an edge device that acquires image data through a camera, generates metadata from the acquired image data, and transmits at least one of the image data and the metadata; and a cloud server that receives the metadata from the edge device and, upon determining that an event has occurred based on the received metadata, additionally transmits the video data from the edge device. will be.

Description

Cloud-Edge Integration Platform System{CLOUD-EDGE INTEGRATED PLATFORM SYSTEM}

The present invention relates to a cloud-edge integrated platform system, particularly applicable to the fields of video analysis and monitoring, cloud computing, artificial intelligence, and data security based on the interaction between edge devices and cloud servers. It's about.

Most existing video analysis and monitoring systems are based on a centralized architecture and adopt a method of transmitting and processing all video data to a central server. This requires large-scale servers and expensive graphics processing units (GPUs) to collect, transmit, and process large amounts of video data, resulting in high operating costs and energy consumption.

Additionally, issues related to personal information protection are emerging as an important issue, and video data may contain sensitive personal information. Therefore, interest in the need for personal information protection and de-identification technology is increasing.

(0001) Domestic Patent Publication No. 10-2022-0041722

The technical problem that the present invention seeks to solve is to provide a cloud-edge integrated platform system that can improve the efficiency of video analysis and monitoring systems and solve personal information protection issues through cooperation between edge devices and cloud servers.

Another technical problem that the present invention aims to solve is a cloud-edge integrated platform that implements video analysis and de-identification technology in edge devices, and the cloud server can support real-time monitoring and event detection by performing various analyzes based on structured data. The goal is to provide a system.

Another technical challenge that the present invention aims to solve is a cloud-edge integrated platform that provides innovative solutions to technical challenges in the fields of image analysis, cloud computing, artificial intelligence, and data security, and can overcome the limitations of existing systems. The goal is to provide a system.

The technical problems to be achieved in the present invention are not limited to the technical problems mentioned above, and other technical problems not mentioned will be clearly understood by those skilled in the art from the description below. You will be able to.

The cloud-edge integrated platform system of the present invention for achieving the above technical problem includes a first processing unit that acquires image data through at least one camera and generates metadata from the acquired image data, and uses the camera. an edge device that transmits at least one of image data acquired through and metadata generated through the first processing unit; and a cloud server that receives the metadata from the edge device and transmits the video data from the edge device only when it is determined that an event has occurred through the received metadata, wherein the edge device receives the video data and It is characterized in that it further includes at least one memory for storing the metadata.

The above aspects of the present invention are only some of the preferred embodiments of the present invention, and various embodiments reflecting the technical features of the present invention will be described in detail below by those skilled in the art. It can be derived and understood based on.

The present invention described above has the following effects.

First, the present invention implements video analysis and de-identification technology in edge devices, enabling much more efficient video processing and monitoring compared to centralized architectures, and transmits only events and meta information occurring in edge devices to the cloud server, reducing network load. can be reduced and transmission bandwidth can be saved.

In addition, the present invention enhances personal information protection by applying personal information de-identification technology in edge devices, that is, video data containing personal information is de-identified before being transmitted to the cloud, and the cloud server stores the de-identified data. The problem of personal information infringement can be solved by only processing the information.

In addition, the cloud server implements a real-time dashboard based on transmitted metadata and structured data and supports high-speed search, allowing you to quickly search and monitor events such as fire, appearance of specific objects, and abnormal behavior.

Additionally, since video analysis is performed on edge devices, the problem of purchasing expensive GPUs and power consumption in central servers can be solved.

Additionally, since the cloud server stores only event-related videos, storage costs are reduced and cloud construction and operation costs can be reduced.

The effects that can be obtained from the present invention are not limited to the effects mentioned above, and other effects not mentioned will be clearly understood by those skilled in the art from the description below.

1 is a conceptual diagram according to an embodiment of the cloud-edge integrated platform system of the present invention;
Figure 2 is an example of the cloud-edge integrated platform system of the present invention;
Figure 3 is an example of edge device analysis and data to be transmitted to a cloud server according to the present invention;
Figure 4 is an example of a real-time dashboard implementation according to the present invention.

Hereinafter, some embodiments of the present invention will be described in detail through illustrative drawings. When adding reference numerals to components in each drawing, it should be noted that identical components are given the same reference numerals as much as possible even if they are shown in different drawings. Additionally, when describing embodiments of the present invention, if detailed descriptions of related known configurations or functions are judged to impede understanding of the embodiments of the present invention, the detailed descriptions will be omitted.

Additionally, when describing the components of an embodiment of the present invention, terms such as first, second, A, B, (a), and (b) may be used. These terms are only used to distinguish the component from other components, and the nature, sequence, or order of the component is not limited by the term. When a component is described as being "connected," "coupled," or "connected" to another component, that component may be directly connected or connected to that other component, but there is no "interconnection" between each component. It should be understood that they may be “connected,” “coupled,” or “connected.”

Figure 1 is a configuration diagram according to an embodiment of the cloud-edge integrated platform system of the present invention, Figure 2 is an exemplary diagram of the cloud-edge integrated platform system of the present invention, and Figure 3 is an edge device analysis and cloud server according to the present invention. This is an example of data to be transmitted, and Figure 4 is an example of a real-time dashboard implementation according to the present invention.

With reference to FIGS. 1 to 4, the cloud-edge integrated platform system of the present invention is described as follows.

The cloud-edge integrated platform system of the present invention may be configured to include at least one camera 1, an edge device 10, and a cloud server 20.

The camera 1 can be any known camera and can be composed of at least one camera. The positions of the cameras 1 may consist of at least one camera 1 in one location, or may be configured of at least one camera 1 in each location. In the present invention, it can be composed of at least one camera (1), and by using multiple cameras (1), it can cover a variety of fields of view and collect image data from various angles and positions.

The camera 1 may be equipped with an interface for data communication with the edge device 10. In general, data can be transmitted using methods such as Ethernet, USB, and wireless communication.

The camera 1 uses an image sensor to detect light in the surrounding environment and converts it into an image, and the converted image can be saved in digital format.

An image processor can process the acquired images and perform necessary operations. This process may include brightness correction, color correction, noise removal, etc., and AI-based image analysis may also occur at this stage.

The camera 1 can transmit processed image data through data communication with the edge device 10. The transmitted data can be used for further processing and analysis at the edge device.

The camera 1 can be configured in various forms and functions, and through this, it can acquire image data in cooperation with the edge device 10.

A camera-based edge device can be used as the edge device 10, and the camera-based edge device can acquire and analyze video data. Camera types and resolutions can vary, and detailed image data can be obtained using high-definition cameras.

In addition to video, edge devices using various sensors can also be applied. For example, various data from the environment can be collected and analyzed using thermal imaging cameras, ultrasonic sensors, radar, and motion detection sensors.

The edge device 10 can collect and analyze not only video data but also data using various communication technologies.

Additionally, by applying an embedded system-based edge device, that is, the edge device 10 may include a small computing system, through which data processing and analysis may be performed locally. For example, you can utilize an embedded system such as Raspberry Pi.

Unmanned aerial vehicles (UAV) or drone-based edge devices can also be applied. An unmanned aerial vehicle or drone can collect image data in the air and transmit it to the edge device 10. These devices allow larger areas to be monitored and data collected.

Edge devices including AI chips and accelerators can also be applied. Some edge devices 10 may include AI accelerators, GPUs, or specialized hardware to perform high-performance data processing. This allows real-time video analysis and complex algorithms to be run.

The edge device 10 may generate metadata from the acquired image data and transmit at least one of the image data and the metadata to the cloud server 20.

The edge device 10 acquires image data through the camera 1, and the image data may be collected using a lens system and an image sensor.

The edge device 10 may include a memory 11, and the memory 11 may store the image data and metadata.

The first processing unit 12 may process and analyze the acquired image data. Referring to FIG. 3, the first processing unit 12 may analyze acquired image data and perform object recognition, face recognition, abnormal behavior analysis, and other tasks. At this time, AI algorithms can be used to process image data and extract related information. For example, in the case of a person, only key points (skeletons) can be extracted, and only the extracted coordinate information can be displayed.

The edge device 10 can recognize objects such as fire, weapons, people, and vehicles by analyzing acquired image data. This allows identification and detection of important objects in the captured environment.

The edge device 10 can detect and recognize faces in image data. Facial recognition technology can be used to identify people and extract relevant information. If necessary, personal information can be protected by applying de-identification technology.

The edge device 10 can extract vehicle information and enhance personal information protection by recognizing vehicle license plates and applying de-identification technology if necessary.

The edge device 10 can track the movement of objects and analyze the movement paths of people, vehicles, etc. This information can be used to detect dangerous situations and analyze route change patterns.

The edge device 10 can detect and analyze abnormal behavior in the environment. For example, abnormal behavior such as falling, assault, or smoking can be detected and necessary measures can be taken.

The edge device 10 can record the location information and time stamp of an object as metadata, through which the time and location of an event in video data can be identified and used for analysis.

Keypoint analysis refers to a technology that identifies and tracks important points in an image, and keypoints can be used to perform facial recognition, object tracking, and other analysis tasks.

This image data analysis technology can be executed in the first processing unit 12 within the edge device 10. Using this technology, various events and objects occurring in the environment captured through the camera (1) are identified and related information is extracted, stored as metadata, and transmitted to the cloud server (20) to support various analysis tasks. You can.

The first processing unit 12 may store processed image data and metadata in the memory 11 . At this time, the memory 11 can efficiently manage data and compress and store it if necessary.

If necessary, the edge device 10 can transmit metadata, necessary image data, and analysis result data to the cloud server 20. For example, only de-identified metadata can be transmitted in normal times, and original video data captured to the cloud server 20 can be transmitted only when an event occurs. The occurrence of an event includes the occurrence of a dangerous situation, for example, when two people are walking to a certain place and only one person is identified through metadata transmitted from the edge device 10, or the movement path of the two people is unusual, or This could be a case where a person on the roof of a building suddenly disappears. In this case, the cloud server 20 detects an event situation through metadata transmitted from the edge device 10, and when it is determined that an event has occurred, it requests and receives the original video data from the edge device 10, and receives the received video data. You can check the exact situation through video data.

As shown in FIG. 2, event generation may be performed in the edge device 10. The edge device 10 may check the occurrence of an event through a stored AI algorithm, and if an event occurs, it may transmit original image data rather than metadata. Of course, metadata and original video data can also be transmitted together. In addition, even when it is determined that an event has occurred, the edge device 10 can de-identify personal information-related matters and transmit only de-identified metadata, for example, transmitting an event occurrence video or an event capture screen. Additionally, video data can be transmitted from the camera 1 to the cloud server 20 in real time.

Additionally, metadata transmitted from the cloud server 20 can be checked and the original video data can be configured to be transmitted only when the original video data is requested from the edge device 10.

Basically, the cloud server 20 is configured to receive only metadata normally from the edge device 10 and to receive the video data from the edge device 10 only when it is determined that an event has occurred through the transmitted metadata. You can.

The cloud server 20 may receive metadata and necessary image data transmitted from the edge device 10. The received data may be filtered and compressed in the edge device 10.

The cloud server 20 connected to the edge device 10 may be equipped with a large capacity memory 21 and may store and manage image data and metadata transmitted from the edge device 10. The memory 21 can be used to process image data and metadata in real time.

The second processing unit 22 analyzes and processes the received structured data, and in this process, detects dangerous situations, vehicle collision situations, object recognition, face recognition, abnormal behavior analysis, vehicle license plate recognition, and path by detecting abnormalities in human movement patterns. Various analysis tasks, such as change pattern analysis, can be performed.

The second processing unit 22 may perform analysis, processing, and management of image data. In this process, various analysis algorithms and technologies can be used to process data and generate results.

The second processing unit 22 may receive image data and metadata collected from the edge device 10. The second processing unit 22 can analyze the image data, that is, perform tasks such as object recognition, face recognition, license plate recognition, object tracking, and abnormal behavior analysis using various image processing and computer vision algorithms.

The second processing unit 22 may utilize predefined algorithms and models for this analysis. For example, deep learning neural networks can be used to perform face recognition or object recognition, and computer vision technology can be used to extract object properties and location information.

Analyzed data can be processed through several steps. Result data can be generated by extracting and organizing only the necessary information. For example, face recognition results can be converted into facial features and identification information.

Analyzed and processed data and related information may be stored in memory 21. These storage spaces support long-term storage of data and can be managed so that they can be retrieved when needed.

The second processing unit 22 can support high-speed search by utilizing meta data and result data. This allows you to quickly search for specific events or objects, and this information can be used to implement dashboards.

Analyzed data can be used for statistical and analysis work. For example, various information and statistical data can be generated through analysis of path change patterns between people, analysis of vehicle size and movement patterns, and key point analysis.

Through this process, the second processing unit 22 can analyze and process the data received from the cloud server 20 to generate important information and results, and the generated information can be monitored or searched in real time by the user to identify important events and results. It can be used to check data. For example, the cloud server 20 can detect abnormal behavior such as face recognition, falling, smoking, etc. and provide statistical information through analysis of path change patterns between people, vehicle size and movement pattern analysis, key point analysis, etc.

The cloud server 20 can support high-speed search by utilizing meta information and the properties and location information of objects in the image. Through this, the location of a specific event or object can be quickly searched, and using this information, a dashboard can be implemented, as shown in FIG. 4. That is, the edge device 10 can analyze image data captured through the camera 1, generate metadata, and transmit the generated metadata to the cloud server 20. The transmitted metadata can be output to the dashboard of the cloud server 20. At this time, as shown in FIG. 4, on the dashboard, person and vehicle information may be displayed as data containing only features and location coordinates rather than actual images. Additionally, if it is determined that an event has occurred, the edge device 10 may transmit the initially captured original image data to the cloud server 20 and output the original image data on the dashboard of the cloud server 20. Whether or not an event occurs can be configured to automatically analyze the algorithm through the second processing unit 22 and/or be determined by a separate manager.

In this way, the cloud server 20 can process and manage data received from the edge device 10 to provide advanced analysis, statistical operations, high-speed search, and dashboard implementation, which allows users to perform real-time monitoring and important It can be implemented to allow quick access to information.

In the case of statistical data, for example, it may be time series analysis data such as the volume of pedestrian and vehicle traffic by day, the number of visitors for each section of the trail, or analysis data on the direction in which people move a lot. Based on this, various data can be used, such as selecting a place to install a billboard. It can be used in related industries.

The present invention described above provides innovative technologies in the fields of video analysis, cloud computing, personal information protection, real-time monitoring, and high-speed search, and can be utilized in various industries and application fields.

In the above, even though all the components constituting the embodiment of the present invention have been described as being combined or operated in combination, the present invention is not necessarily limited to this embodiment. That is, as long as it is within the scope of the purpose of the present invention, all of the components may be operated by selectively combining one or more of them. In addition, terms such as “include,” “comprise,” or “have” described above mean that the corresponding component may be present, unless specifically stated to the contrary, and thus do not exclude other components. Rather, it should be interpreted as being able to include other components. All terms, including technical or scientific terms, unless otherwise defined, have the same meaning as generally understood by a person of ordinary skill in the technical field to which the present invention pertains. Commonly used terms, such as terms defined in a dictionary, should be interpreted as consistent with the contextual meaning of the related technology, and should not be interpreted in an idealized or overly formal sense unless explicitly defined in the present invention.

The above description is merely an illustrative explanation of the technical idea of the present invention, and various modifications and variations will be possible to those skilled in the art without departing from the essential characteristics of the present invention. Accordingly, the embodiments posted in the present invention are not intended to limit the technical idea of the present invention, but are for illustrative purposes, and the scope of the technical idea of the present invention is not limited by these examples. The scope of protection of the present invention should be interpreted in accordance with the claims below, and all technical ideas within the equivalent scope should be construed as being included in the scope of rights of the present invention.

1: Camera
10: Edge device 11: Memory
12: Processing unit 20: Cloud server
21: memory 22: second processing unit

Claims (1)

A first processing unit that acquires image data through at least one camera and generates metadata from the acquired image data, and selects one of the image data acquired through the camera and the metadata generated by the first processing unit. An edge device that transmits at least one; and
A cloud server that receives the metadata from the edge device and receives the video data from the edge device only when it is determined that an event has occurred based on the received metadata,
The edge device is,
A cloud-edge integrated platform system further comprising at least one memory for storing the image data and the metadata.