KR20200055297A - PTZ camera control system and mothod based on reinforcement learning - Google Patents

Abstract

Disclosed in the present invention are a PTZ camera control system based on reinforcement learning and a method thereof, which can increase precision of an image analysis service such as object detection, tracking, and scene recognition by improving the quality of an image through the physical movement of a camera instead of through an image processing software. To this end, the reinforcement learning-based PTZ camera control system of the present invention comprises: a means of receiving image analysis metadata generated by an image analysis; a means of extracting metadata related to an object or an event by analyzing the received image analysis metadata; a means of generating camera movement prediction information by predicting the movement of a PTZ camera suitable to the current state of the object or the event by means of a reinforcement learning model from the extracted metadata related to the object or the event; a means of configuring the PTZ camera movement prediction information as movement prediction information metadata; and a means of transmitting the configured PTZ camera movement prediction information metadata to a VMS server.

Description

PTZ camera control system and mothod based on reinforcement learning}

The present invention relates to a technology for controlling a PTZ camera in an intelligent video analysis service environment.

The present invention is an information communication and broadcasting research and development project of the Ministry of Science and ICT and the Information and Communication Technology Promotion Center in 2017. It was carried out as a result of research on the development of core technologies for intelligent defense surveillance systems.

Recently, in the security / safety, business, and personal service fields, intelligent video analysis services such as facility control using CCTV and judging risk situations are expanding, and the video analytics service industry is expected to grow with the development of deep learning technology. Due to the nature of the image analysis service, the accuracy of the image analysis algorithm depends on the quality of the received image. That is, since the reliability of the image analysis service changes according to the quality of the input image, the input image must be acquired to facilitate image analysis.

In the previous method, to improve the quality of the image, the quality was improved by using a software-based image processing method such as image enhancement, but for objects with very small size or occlusion of the detection object. In the case of an image analysis, there is a limit, and for this, it is necessary to physically move the camera.

The present invention is a system and method for controlling a PTZ camera based on reinforcement learning to improve the accuracy of an image analysis service such as object detection, tracking, and scene understanding by improving the quality of an image through physical camera movement rather than image processing software. To suggest.

In order to solve the above problems, the present invention provides a PTZ camera control system and control method interworking with a VMS server.

The PTZ camera control system based on reinforcement learning of the present invention includes means for receiving image analysis metadata generated by image analysis, means for analyzing the received image analysis metadata to extract metadata related to an object or event, and the extracted A means for predicting movement of a PTZ camera suitable for a current object or event state from an object or event-related metadata with an enhanced learning model, and generating camera movement prediction information, and configuring the PTZ camera movement prediction information as movement prediction information metadata And means for transmitting the configured PTZ camera movement prediction information metadata to a VMS server.

Here, the object or event-related metadata extracted from the means for extracting the object or event-related metadata may include position and size data of the object or event.

In addition, the means for extracting the object or event-related metadata may further include means for correcting the resolution of the object or event in the image to match the resolution of the reinforcement learning model.

In addition, the camera movement prediction information generated by the camera movement prediction information generating means may include a movement direction and a movement value of the camera.

In addition, the configured camera movement prediction information metadata may include control information including each camera control system ID and camera ID, camera pan, camera tilt, camera zoom, and camera function on / off information for each system ID. have.

On the other hand, in the reinforcement learning based PTZ camera control method of the present invention, generating image analysis metadata through image analysis, and analyzing the image analysis metadata to relate the object or event including the location and size data of the object or event Extracting metadata, predicting movement information including the movement direction and movement value of a PTZ camera suitable for the current object or event state from the extracted object or event-related metadata with a reinforcement learning model to generate camera movement prediction information And configuring the PTZ camera movement prediction information as movement prediction information metadata.

The configuration and operation of the present invention introduced above will be further clarified through specific embodiments described with reference to the drawings.

According to the present invention, by improving the image quality by controlling the PTZ camera based on reinforcement learning, it is possible to secure the accuracy and reliability of an image analysis service such as anomaly detection, and use a software-based image processing method. It is possible to effectively perform image analysis in the case of obscured objects or very small objects.

1 is a flowchart of a method for controlling a PTZ camera based on reinforcement learning according to an embodiment of the present invention
Figure 2 is an artificial neural network structure diagram used in the reinforcement learning model for performing reinforcement learning
3 is a block diagram of PTZ camera movement prediction information metadata

Advantages and features of the present invention and methods for achieving them will be made clear by referring to embodiments described in detail with reference to the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below, but may be implemented in various different forms, and only the present embodiments allow the disclosure of the present invention to be complete so that ordinary knowledge in the technical field to which the present invention pertains is provided. It is provided to fully inform the possessor of the scope of the invention. The technical scope of the invention is defined by the claims.

On the other hand, the terms used in this specification are for describing the embodiments and are not intended to limit the present invention. In the present specification, the singular form also includes the plural form unless otherwise specified in the phrase. As used herein, "comprises" or "comprising" means the presence of one or more other components, steps, operations and / or elements other than the components, steps, operations and / or elements mentioned, or Addition is not excluded.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. In adding reference numerals to the components of each drawing, the same components are assigned the same reference numerals as possible, even though they are shown on different drawings, and in describing the present invention, detailed descriptions of related well-known components or functions When the subject matter of the present invention can be obscured, the detailed description thereof will be omitted.

1 is a flowchart of a method for controlling a PTZ camera based on reinforcement learning according to an embodiment of the present invention.

A method for controlling a PTZ camera based on reinforcement learning in an intelligent image analysis service environment will be described with reference to FIG. 1. It will be apparent to those skilled in the art that the method algorithm description of the present invention described below can replace the description of the device configuration of the PTZ camera control system based on reinforcement learning according to the present invention. The PTZ camera control system based on reinforcement learning can be implemented as part of a video management (management) system (VMS) server together with an image analysis system. REST API can be used as a means of interaction between the camera control system and the image analysis system and the VMS server.

As described above, the spirit of the present invention can be implemented in terms of a device or a method. In particular, a function or process of each component of the present invention is a digital signal processor (DSP), processor, controller, or application of ASIC. -specific IC), programmable logic devices (FPGA, etc.), at least one of other electronic devices, and combinations thereof can be implemented as hardware elements. It can also be implemented as software in combination with hardware elements or independently, which can be stored on a record carrier.

110: Receiving metadata in the form of JSON generated by image analysis.

JavaScript Object Notation (JSON) form or format is an open standard format that uses human-readable text to pass data objects consisting of attribute-value pairs or key-value pairs, especially when sending and receiving data over the Internet. It is a way to express the material. There is no big limitation on the type of data, and it is particularly suitable for expressing variable values of computer programs.

The metadata in JSON format generated by image analysis can be received by the REST API in the image analysis system. For example, a REST API based image receiving unit may be configured in the image analysis system to receive metadata.

120: After performing step 110, analyzing the received image analysis metadata to extract only metadata related to object or event information in the image.

Here, the location and size data of the object or event in the image are extracted and corrected to a 640 × 480 resolution range, for example, in accordance with the reinforcement learning environment.

130: After performing step 120, predicting the movement of the PTZ camera using reinforcement learning.

This is a step of predicting the movement direction and value of the PTZ camera suitable for the current object or event state from the location and size metadata of the object or event extracted in the previous step 120 using reinforcement learning.

Reinforcement learning (RL) is a new methodology for machine learning that solves probabilistic decision-making problems. Unlike supervised learning, reward or reward functions are given to maximize the average of reward values to be obtained in the future. It is a machine learning technique to find the policy function of. That is, unlike supervised learning, the target value is a reward and the predicted value is a policy or action.

2 shows an artificial neural network structure used in a reinforcement learning model for performing reinforcement learning. The artificial neural network shown in FIG. 2 has four hidden layers 30-1, 30-2, 30-3, and 30-4 between the input layer 10 and the output layer 20. ). The following equation was used to create a reinforcement learning model using this artificial neural network.

The above formula is a formula for setting a learning goal in reinforcement learning, and the definition of each variable is as follows.

: 반복 횟수,

: 총 반복 횟수

: Number of repetitions,

: Total number of iterations

: 현재 단계의 상태,

: 다음 단계의 상태. 좀 더 구체적으로,

는 객체의 현재 좌표 및 크기이고,

은 120과 130 단계에서 추정된 액션값과 방향으로 PTZ 카메라를 이동시켰을 때의 객체의 좌표 및 크기이다(객체의 좌표 및 크기

임).

: Status of the current stage,

: Status of the next step. More specifically,

Is the current coordinate and size of the object,

Is the coordinates and size of the object when the PTZ camera is moved in the direction and action values estimated in steps 120 and 130 (the coordinates and size of the object)

being).

: 현재 단계의 액션값,

: 다음 단계의 액션값 (본 발명에서 액션값은 (0.02, 0.06, 0.1, 0.14, 0.18, 0.22, 0.26, 0.3) 중 선택된 값)

: Action value of the current step,

: Action value of the next step (in the present invention, the action value is a value selected from (0.02, 0.06, 0.1, 0.14, 0.18, 0.22, 0.26, 0.3))

: 무작위성을 조절하는 가중치(Weight)(E-greedy 방법 사용)

: Weight to control randomness (using E-greedy method)

: 현재 단계의 Reward

: Reward at the current stage

: 인공신경망의 Weight,

: 복사된 인공신경망의 Weight

: Weight of artificial neural network,

: Weight of the copied artificial neural network

: Deep Q-learning 함수 (Q값을 산출한다)

: Deep Q-learning function (calculates Q value)

를 이용해 현재 상태

와 현재 액션값

로 Q값(딥 Q-러닝 함수값)을 계산한다(현재 학습 목표 설정). ②번 항은 ①번 항에서 선택한

값으로 실제 액션을 수행한 후 변경된 다음 단계의 상태

와 새롭게 선택된 액션값

를 이용하여 복사된 인공신경망

을 통해 최대가 되는 Q값을 계산한다. 산출된 Q값은 가중치

로 조절하며, ①번 항 수행 후에 습득한 Reward를 결합한다(다음 단계 학습 목표 설정). ① 항과 ② 항의 에러를 줄일 수 있는

를 학습함으로써 강화학습이 수행된다.In the above equation, item ① is an artificial neural network.

Use Presence

And current action value

Calculate the Q value (deep Q-learning function value) with (Current learning goal setting). Item ② is selected in the item ①

The status of the next step that changed after performing the actual action with the value

And the newly selected action value

Artificial neural network copied using

Calculate the maximum Q value. The calculated Q value is the weight

Adjust to, and combine the Rewards acquired after performing ① (setting the next step learning goal). The errors in terms ① and ② can be reduced.

Reinforcement learning is performed by learning.

140: After performing step 130, constructing the predicted camera movement information into metadata in the form of JSON.

In one embodiment of the present invention, JSON-type PTZ camera movement prediction information metadata is configured as shown in FIG. 3. The system ID was configured with the code name 'collabo_id', and the control information was configured with the code name 'collabo_ctrl_info'. Camera ID, camera pan, camera tilt, camera zoom, and camera function on / off information were assigned to each control information.

Step 150: If the motion prediction information metadata of the PTZ camera is configured by performing step 140, transmitting the PTZ camera motion prediction information metadata to a video management system (VMS) server using a REST API.

160: checking the position and size of the object or event in the video from the received PTZ camera movement prediction information metadata.

Since the physical movement of the PTZ camera is predicted based on reinforcement learning, the quality of an image is improved, and accuracy of an image analysis service such as detection of objects or events, tracking, and scene understanding can be greatly improved.

In the above, the configuration of the present invention has been described in detail through a preferred embodiment of the present invention, but those skilled in the art to which the present invention pertains are disclosed herein without changing the technical spirit or essential features of the present invention. It will be understood that it may be implemented in a specific form different from the content. It should be understood that the embodiments described above are illustrative in all respects and not restrictive. The protection scope of the present invention is defined by the claims below, rather than the above detailed description, and all changes or modifications derived from the claims and equivalent concepts should be interpreted to be included in the technical scope of the present invention. .

Artificial neural network input layer 10, output layer 20, hidden layer (30-1, 30-2, 30-3, 30-4)

Claims (8)

As a PTZ camera control system that works with a VMS server,
Means for receiving image analysis metadata generated by image analysis,
Means for analyzing the received image analysis metadata and extracting only metadata related to an object or event,
Means for predicting the movement of the PTZ camera suitable for the current object or event state from the extracted object or event-related metadata with a reinforcement learning model and generating camera movement prediction information,
Means for configuring the PTZ camera movement prediction information as movement prediction information metadata,
Reinforcement learning-based PTZ camera control system comprising a means for transmitting the configured PTZ camera movement prediction information metadata to a VMS server. The system of claim 1, wherein the object or event-related metadata extracted from the means for extracting the object or event-related metadata includes position and size data of the object or event. In claim 1, The means for extracting the object or event-related metadata
Reinforcement learning-based PTZ camera control system further comprises a means for correcting the resolution of the object or event in the image to match the resolution of the reinforcement learning model. The PTZ camera control system of claim 1, wherein the camera movement prediction information generated by the camera movement prediction information generating means includes a movement direction and a movement value of the camera. The method of claim 1, wherein the configured camera movement prediction information metadata includes control information including each camera control system ID and camera ID, camera pan, camera tilt, camera zoom, and camera function on / off information for each system ID. Including reinforcement learning based PTZ camera control system. Generating image analysis metadata through image analysis,
Analyzing the image analysis metadata and extracting object or event-related metadata including object and event location and size data,
Generating camera movement prediction information by predicting movement information including a movement direction and a movement value of a PTZ camera suitable for a current object or event state from the extracted object or event-related metadata with a reinforcement learning model,
And constructing the PTZ camera movement prediction information as movement prediction information metadata. The method of claim 6, further comprising transmitting the configured PTZ camera movement prediction information metadata to a video control system (VMS) server. The method of claim 6, wherein the configured camera movement prediction information metadata includes control information including each camera control system ID and camera ID, camera pan, camera tilt, camera zoom, and camera function on / off information for each system ID. Including reinforcement learning based PTZ camera control method.

Images