KR20240090079A - high definition camera image compression and depression support system and method thereof - Google Patents

Abstract

The present invention relates to a high-definition real-time camera image compression and restoration support system and a compression and restoration method. The system generates compressed data from compressed images received from at least one camera and transmits them through a communication network. It is equipped with an image collector that decompresses and restores compressed data received from the image acquisition device through a communication network, and the image acquisition device extracts an AI area of interest using a deep learning technique for the image received from the camera and determines the AI interest area. Image data corresponding to the area is processed in accordance with a maintenance pattern set to minimize the compression rate, thereby generating a compressed image. According to this high-definition real-time camera video compression and restoration support system and its compression and restoration method, regions of high importance within the video are extracted using deep learning techniques, and regions of low importance are processed to have a high compression rate, thereby improving compression efficiency. It provides the advantage of increasing both the transmission and restoration efficiency of overcompressed video.

Description

High-definition real-time camera image compression and restoration support system and its compression and restoration method {high definition camera image compression and depression support system and method there}

The present invention relates to a high-definition real-time camera image compression and restoration support system and a compression and restoration method. Specifically, the present invention relates to a system for supporting high-definition real-time camera image compression and restoration, and in detail, analyzes images acquired by a camera using deep learning techniques to generate a region of interest map and input images based on this. It relates to a high-definition real-time camera video compression and restoration support system that supports compression and restoration at a high compression rate, and its compression and restoration methods.

The scope of use of CCTV devices is expanding beyond the traditional level of monitoring abnormal situations in factory equipment or large buildings to include black boxes for electric kickboards and home security systems. In addition, with the development of camera and network technology, the performance of CCTV has improved, and various functions have been added, such as installing 4K high-definition cameras, receiving CCTV images in real time from remote locations, and sending voice warnings through speakers installed in the CCTV. .

CCTV equipped with high-definition cameras can monitor a wider area than low-definition cameras because the number of pixels per degree (PPD) is high. However, due to the nature of CCTV, which requires monitoring the surrounding environment 24 hours a day, the higher the image quality of the camera, the more storage space is required, which shortens the video storage period. Also, due to bandwidth limitations when transmitting real-time video, the video quality is lowered to HD level or lower. There are still problems that need to be addressed.

As one of the methods to solve the above problem, a technology has been introduced to monitor a large area without increasing the storage space required for image storage by adding a mechanical rotation driver to the camera. However, there is a problem that the possibility that the occurrence of an abnormal situation cannot be detected cannot be ruled out because the camera is shooting in a different direction when an abnormal situation occurs.

Another method to solve the above problem is to use image processing techniques. A method has been introduced to separate the background in the image from the main object and store the background in low quality and the object in high quality. This method has a high image compression rate when the size of the object is small and small compared to the background, but there is a problem in that the compression rate is significantly lower when there are a large number of objects in the image or the distance between the object and the camera is close.

Therefore, there is a need to develop a new method that has high compression efficiency and enables real-time high-definition video transmission to create a CCTV device that can fully utilize the performance of high-definition cameras to observe a wide range of areas for a long time.

Republic of Korea Patent No. 10-1614607 (Pan-tilt device and CCTV camera system, registered on April 22, 2016) Republic of Korea Patent No. 10-1951232 (High-definition CCTV video system using separate storage of object areas and adaptive background video, registered on February 18, 2019)

The present invention was created to solve the above requirements. It automatically extracts the region of interest using deep learning techniques and adjusts the compression rate of the image captured by the camera based on the extracted region of interest information according to the importance of the region. The purpose is to provide a high-definition real-time camera video compression and restoration support system and a compression and restoration method that can increase both compression efficiency and transmission and restoration efficiency of compressed video by processing differently.

In order to achieve the above object, a high-definition real-time camera image compression and restoration support system according to the present invention includes an image acquisition device that generates compressed data from compressed images received from at least one camera and transmits them through a communication network; an image collector that decompresses and restores compressed data received from the image acquirer through the communication network, wherein the image acquirer extracts an AI region of interest using a deep learning technique for the image received from the camera; , Image data corresponding to the AI area of interest is processed in accordance with a maintenance pattern set to minimize the compression rate to generate the compressed image.

According to one aspect of the present invention, the image acquirer is constructed to recognize an object using the YOLO (You Only Look Once) algorithm and set the boundary area of the recognized object as the AI area of interest.

Preferably, at least one of the image acquirer and the image collector supports a user to specify an area of interest, and the image acquirer has a compression rate for image data corresponding equally to the AI area of interest for the designated user area of interest. The compressed image is generated by processing corresponding to the maintenance pattern set to be minimized.

In addition, the image collector restores the original image from the received compressed data using a deep learning algorithm based on a generative adversarial network, and outputs the restored original image to the collection display unit.

In addition, in order to achieve the above object, the compression and restoration method of the high-definition real-time camera image compression and restoration support system according to the present invention generates compressed data from compressed video that compresses the video received from at least one camera and transmits the compressed data through a communication network. In the image compression and restoration method of a high-definition real-time camera image compression and restoration support system including an image acquisition unit for transmitting and an image collector for decompressing and restoring compressed data received from the image acquisition unit through the communication network, . Extracting an AI area of interest from the image received from the camera using a deep learning technique, processing the image data corresponding to the AI area of interest in accordance with a maintenance pattern set to minimize the compression rate to generate the compressed image, transmitting compressed data corresponding to the generated compressed video to the video collector through a communication network; me. Restoring the original image from the received compressed data using a deep learning algorithm based on a generative adversarial network for the compressed data collected through the communication network, and outputting the restored original image to the collection display unit; Includes.

According to the high-definition real-time camera image compression and restoration support system and its compression and restoration method according to the present invention, regions of high importance within the image are extracted using deep learning techniques, and regions of low importance are processed to have a high compression rate. This provides the advantage of increasing both compression efficiency and transmission and restoration efficiency of compressed video.

1 is a diagram showing a high-definition real-time camera image compression and restoration support system according to the present invention;
Figure 2 is a flow diagram showing the image compression and transmission process of the image acquisition unit of Figure 1;
Figure 3 is a flow chart showing the restoration process of compressed data received by the image collection unit of Figure 1.

Hereinafter, a high-definition real-time camera image compression and restoration support system and its compression and restoration method according to a preferred embodiment of the present invention will be described in more detail with reference to the attached drawings.

Figure 1 is a diagram showing a high-definition real-time camera image compression and restoration support system according to the present invention.

Referring to FIG. 1, the high-definition real-time camera image compression and restoration support system 100 according to the present invention includes an image acquirer 110 and an image collector 150.

The image acquirer 110 generates compressed data from a compressed image received from at least one camera 122 and transmits it to the image collector 150 through the communication network 50. Here, the communication network 50 may be a wireless and/or wired communication network.

The image acquirer 110 extracts an AI area of interest using a deep learning technique for the image received from the camera 122, and processes the image data corresponding to the AI area of interest in accordance with a maintenance pattern set to minimize the compression rate. This creates a compressed video.

In addition, the image acquirer 110 supports the user to designate an area of interest within the image area, and responds to a maintenance pattern set to minimize the compression rate for image data that corresponds equally to the AI area of interest for the designated user area of interest. It is desirable to build it so that it can be processed to produce compressed images.

Here, for example, in the case of the camera 122 that monitors a crosswalk, the user interest area may be designated as the user interest area, and the user interest area may be designated at the entrance of a major facility in the factory to monitor intruders. In addition, a specific area in the image can be designated as a user interest area for various purposes.

The image acquisition unit 110 includes an imaging unit 120, a local manipulation unit 132, a local display unit 134, a local communication unit 136, and a local image processing unit 140.

The imaging unit 120 is equipped with at least one CCTV camera 122 and provides images captured by the applied camera 122 to the local image processing unit 140.

The camera 122 applied to the imaging unit 120 may be a camera with a wide viewing angle to monitor a wide area, or a camera equipped with a lens module or mirror module capable of omnidirectional image shooting. Additionally, when multiple cameras 122 are used in the imaging unit 120, a camera that combines images taken in multiple directions at one point or simply combines images taken at different points may be used.

The local manipulation unit 132 is capable of setting supported functions with the support of the local image processing unit 140. The local manipulation unit 132 may be configured to designate a user area of interest within the imaging area received through the imaging unit.

The local display unit 134 is controlled by the local image processing unit 140 to display display information.

The local communication unit 136 is controlled by the local image processing unit 140 and communicates with the local collection unit 150 through the communication network 50.

The local image processing unit 140 extracts an AI area of interest using a deep learning technique for the image received from the camera 122, generates a map of the AI area of interest, which is location information for the AI areas of interest, and creates an AI area of interest. The image data corresponding to is processed in accordance with the maintenance pattern set to minimize the compression rate to generate a compressed image.

In addition, when a user-designated user interest area is set, the local image processing unit 140 extracts the user interest area from the image received from the camera 122 and provides a map of the user interest area, which is location information about the user interest areas. The image data corresponding to the user's area of interest is processed according to a maintenance pattern set to minimize the compression rate, thereby generating a compressed image.

Here, the image data corresponding to the AI area of interest and the user area of interest may be set in such a way that the original image is maintained as is without compression. In other words, image data corresponding to the AI area of interest and the user area of interest can be constructed so that the compression rate is 0 (zero)%.

The local image processing unit 140 has a built-in memory unit that temporarily stores received image data.

The local image processing unit 140 applies a mask (Mask(x,y)) to the input data (Input(x, y)) for each pixel in each image frame received from the imaging unit 120 and performs the math below. It is constructed to generate compressed images according to Equation 1.

<Equation 1>

Compressed(x, y) = Input(x, y) x Mask(x, y)

Here, Mask(x, y) is

It is determined by Equation 2 below.

<Equation 2>

Mask(x, y) = 1 only if Random(x, y, u, n) + UserRoi(x, y) + AiRoi(x, y) > 0 else 0

Here, Compressed(x,y) is the compressed image, Input(x,y) is the input image, UserRoi is a map of the user area of interest, AiRoi is a map of the AI area of interest, and Random is a random noise image. The pixel value follows a normal distribution, uniform distribution, or similar probability distribution with an average of u, and Random (x, y, u, n) is produced in advance and used by the local image processing unit 140 and This refers to the pixel at the (x, y) position of the nth image among N random noise images recorded in the image restoration unit 160, which will be described later.

Here, the random noise image is an image shared equally by the image acquirer 110 and the image collector 150, and the number of pixels with a value of 1 in the mask image can be adjusted by adjusting the size of the average u, which is This means that compression rate and image quality can be controlled by average u. Additionally, in the user area of interest map and the AI area of interest map, the values of all non-zero pixels may be set to be the same or may be set differently depending on the type of object or area. In addition, the center and periphery of the area of interest can be adjusted to have different values so that the image quality of the more important parts is higher.

Compressed data generated by the local image processing unit 140 includes the number of the random noise image used to create the mask image and information on the AI area of interest required to generate a map of the AI area of interest.

The map information of the user's area of interest is set to be shared by the local image processing unit 140 and the image restoration unit 160 during the initial setup process, and is constructed so that the map information of the user's area of interest is omitted when transmitting actual compressed data. desirable.

The local image processing unit 140 may be configured to recognize objects in the image using the YOLO (You Only Look Once) algorithm and set the boundary area of the recognized object as the AI area of interest.

The local communication unit 136 is controlled by the local control unit 136 and transmits the compressed data to be transmitted to the image collector 150 through the communication network 50.

Unlike this, the local manipulation unit 132 and the local display unit 134 are omitted, and support designation of a user area of interest from the image collector 150, which will be described later, and setting information about the designated user area of interest is provided to the local image processing unit 140. Of course, it can be configured to receive and register through the local communication unit 136.

The image collector 150 decompresses the compressed data received from the image acquirer 110 through the communication network 50 and restores it to the original image.

The image collector 150 restores the original image from the received compressed data using a deep learning algorithm based on a generative adversarial network, and outputs the restored original image to the collection display unit 154.

The image collector 150 includes a collection and communication unit 156, a collection operation unit 152, a collection display unit 154, a database (current) 170, and an image restoration unit 160.

The collection and communication unit 156 is controlled by the image restoration unit 160 and performs communication, and receives compressed data transmitted from the image acquisition unit 110 through the communication network 50 and provides it to the image restoration unit 160.

The collection operation unit 152 is configured to set supported functions, and the collection display unit 154 is controlled by the image restoration unit 160 to display the restored image.

The database (DB) 170 is controlled by the image restoration unit 160 and records the collected images.

The image restoration unit 160 stores the compressed data received through the collection and communication unit 156 in a database (DB) using a deep learning algorithm based on an adversarial generative network, and extracts the original image from the compressed data. It is restored, and the restored original image is output to be displayed on the collection display unit 154.

Below, the image compression and restoration process will first be described with reference to FIGS. 2 and 4 of the local image processing unit 140.

First, determine whether a user-specified region of interest exists (step 201).

If it is determined in step 201 that a user-specified region of interest exists, a user-specified region of interest map 402 is generated by demarcating boundary areas for the user-specified regions of interest within the image frame (step 202).

Next, it is determined whether the camera 122 is driven (step 203), and if it is determined that the camera 122 is driven, image acquisition is attempted to obtain the input image 401 (step 204). After step 204, it is determined whether image acquisition was successful (step 205), and if it is determined that image acquisition was successful, the AI area of interest is automatically extracted using deep learning techniques (step 206). As an example, the YOLO (You Only Look Once) algorithm can be used to recognize objects such as people or cars and designate the bounding box of the recognized location as the AI area of interest. Alternatively, you can extract the important area using a saliency extraction algorithm and designate the oval or square area containing the area as the AI area of interest. In addition, other deep learning techniques can be used to detect important areas or objects in the video and designate them as AI areas of interest.

Afterwards, it is determined whether the extraction of the AI area of interest was successful (step 207), and if it is determined that the extraction of the AI area of interest was successful, an AI area of interest map 403 is generated (step 208). For reference, Figure 4 shows that people and cars are extracted and designated as AI areas of interest, and an AI area of interest map is created as a circle.

Afterwards, it is determined whether a user-specified region of interest exists in the input image (step 209), and if it is determined that a user-specified region of interest exists, a composite region of interest map (404) including both the user-specified region of interest and the AI region of interest is created. Create (step 210).

After step 210, a mask image 405 is generated (step 211) as previously explained through Equations 1 and 2, and a compressed image in which some pixels of the input image 401 are removed using the generated mask image. (406) is generated (step 212), and compressed data generated through a data compression process for the generated compressed video is transmitted through a communication network (step 214).

Here, the compressed data 407 is generated through a data compression process that serializes the number of the random noise image used to create the mask image 405, the AI region of interest information required to generate the AI region of interest map, and each pixel of the compressed image. do. The data compression step may include information on a user-specified region of interest 402 when the input image 401 is the first image acquired from the camera 212 or when the image is in a specific order repeated at regular intervals. For example, information on the user-specified region of interest 402 may be transmitted from the image acquirer 110 to the image collector 150 every 1 second or every 1 minute. Compressed data compressed in the image acquisition device 110 through the above process is transmitted to the image collector 150 through a wireless or wired network.

Next, the process of restoring compressed data received from the image collector 150 will be described with reference to FIGS. 3 and 5.

First, the image collector 150 continuously performs a process of receiving compressed data 501 (step 301) and determines whether reception of the compressed data was successful (step 302).

If it is determined that the compressed data has been successfully received in step 302, the compressed data 501 is stored in the database 170 (step 303).

Next, a data decompression process is performed to decompress the stored compressed data (step 310). Here, the data decompression process reverses the data compression process. Afterwards, the number of the random noise image, user-specified region of interest (402) information, and AI region of interest information (403) are extracted from the compressed data (501) to restore the composite region of interest map (502), and then using Equation 2 The mask image 503 used for data compression is restored (step 311), and the compressed image 504 is restored by sequentially assigning pixel data serialized to the non-zero pixels in the mask image 503 (step 312). . Afterwards, the original image 401 is restored from the compressed image 504 using a deep learning algorithm based on a generative adversarial network (GAN), and the restored image 505 is output on the screen (step 314). .

Next, the system determines whether it is a termination condition (step 315). If it is a termination condition, restoration by decompression is terminated. If it is not a termination condition, the system returns to step 301.

In this way, this system can minimize image quality damage in important surveillance areas by combining deep learning-based region-of-interest map generation and adversarial generative neural network-based image compression and restoration techniques.

According to the high-definition real-time camera video compression and restoration support system and its compression and restoration method described above, regions of high importance within the video are extracted using deep learning techniques, and regions of low importance are processed to have a high compression rate. This provides the advantage of increasing both compression efficiency and transmission and restoration efficiency of compressed video.

110: Image acquisition device
150: Video collector

Claims (10)

An image acquisition device for generating compressed data from compressed images received from at least one camera and transmitting the compressed data through a communication network;
an image collector that decompresses and restores compressed data received from the image acquirer through the communication network,
The image acquisition unit extracts an AI region of interest using a deep learning technique for the image received from the camera, and processes the image data corresponding to the AI region of interest in accordance with a maintenance pattern set to minimize the compression rate to obtain the compressed image. A high-definition real-time camera video compression and restoration support system characterized by generating a. The method of claim 1, wherein the image acquirer recognizes the object using the YOLO (You Only Look Once) algorithm and sets the boundary area of the recognized object as the AI area of interest. Support system. The method of claim 1, wherein at least one of the image acquirer and the image collector supports a user to designate an area of interest, and the image acquirer provides image data equally corresponding to the AI area of interest for the designated user area of interest. A high-definition real-time camera image compression and restoration support system characterized in that the compressed image is generated by processing corresponding to a maintenance pattern set to minimize the compression rate. The method of claim 3, wherein the image acquirer applies a mask (Mask(x,y)) to the input data (Input(x, y)) for each pixel in the image frame to generate a compressed image according to the following relational equation. do,
Compressed(x, y) = Input(x, y) x Mask(x, y)
The mask is
Mask(x, y) = 1 only if Random(x, y, u, n) + UserRoi(x, y) + AiRoi(x, y) > 0 else 0,
The Compressed(x,y) is a compressed image, the Input(x,y) is an input image, the UserRoi is a map of the user area of interest, the AiRoi is a map of the AI area of interest, and the Random is a random In a noisy image, the pixel value follows a normal distribution, uniform distribution, or similar probability distribution with an average of u, and Random(x, y, u, n) is a pre-made and recorded N A high-definition real-time camera image compression and restoration support system characterized by representing the pixel at the (x, y) position of the nth image among random noise images. The method of claim 4, wherein the compressed data includes the number of the random noise image used to produce the mask image and information on the AI area of interest required to generate a map of the AI area of interest. Restoration support system. The method of claim 5, wherein the image collector restores the original image from the received compressed data using a deep learning algorithm based on a generative adversarial network, and outputs the restored original image to the collection display unit. A high-definition real-time camera video compression and restoration support system. An image acquirer that generates compressed data from compressed images received from at least one camera and transmits them through a communication network, and an image collector that decompresses and restores the compressed data received from the image acquirer through the communication network. In the video compression and restoration method of the high-definition real-time camera video compression and restoration support system,
go. Extracting an AI area of interest from the image received from the camera using a deep learning technique, processing the image data corresponding to the AI area of interest in accordance with a maintenance pattern set to minimize the compression rate to generate the compressed image, transmitting compressed data corresponding to the generated compressed video to the video collector through a communication network;
me. Restoring the original image from the received compressed data using a deep learning algorithm based on a generative adversarial network for the compressed data collected through the communication network, and outputting the restored original image to the collection display unit; An image compression and restoration method for a high-definition real-time camera video compression and restoration support system, comprising: The method of claim 7, wherein the provisional step recognizes an object using the YOLO (You Only Look Once) algorithm and sets the boundary area of the recognized object as the AI area of interest. Support for compressing and restoring high-definition real-time camera images. Video compression and restoration methods in the system. The high-definition real-time camera according to claim 8, wherein the provisional step generates the compressed image by processing the image data corresponding to the user-designated area of interest in accordance with the maintenance pattern set to minimize the compression rate. Video compression and restoration method in video compression and restoration support system. The method of claim 9, wherein the first step applies a mask (Mask(x,y)) to the input data (Input(x, y)) for each pixel in the image frame to generate a compressed image according to the following relational equation. do,
Compressed(x, y) = Input(x, y) x Mask(x, y)
The mask is
Mask(x, y) = 1 only if Random(x, y, u, n) + UserRoi(x, y) + AiRoi(x, y) > 0 else 0,
The Compressed(x,y) is a compressed image, the Input(x,y) is an input image, the UserRoi is a map of the user area of interest, the AiRoi is a map of the AI area of interest, and the Random is a random In a noisy image, the pixel value follows a normal distribution, uniform distribution, or similar probability distribution with an average of u, and Random(x, y, u, n) is a pre-made and recorded N An image compression and restoration method for a high-definition real-time camera image compression and restoration support system, characterized in that it refers to the pixel at the (x, y) position of the nth image among random noise images.

Images