RU2636745C1 - Method and system of monitoring territory using controlled video camera - Google Patents

Abstract

FIELD: physics.

SUBSTANCE: method for monitoring a territory is provided, comprising the steps of: receiving frames of an observable area through a primary passage of a predetermined route controlled by the video camera, at least, one frame being received at each point of the route; with the help of a detection mechanism, the presence of target observation objects is determined on the frames; in the case of detecting observation targets, determining pixel coordinates on, at least, one frame of, at least, one target observation object; calculating a camera viewing direction providing an image of, at least, one observation target based on the pixel coordinates of the target observation object defined in the previous step, the internal calibration and the direction of view of the controlled video camera; driving the controllable camera, at least, once in the viewing direction of, at least, one target observation object, obtaining, at least, one frame containing the target observation object and processing the frame with a detection mechanism; on the basis of the data on the detection of the target observation objects by, at least, one repeated pass, a decision is made about the presence of the target observation object in the frame obtained at the primary pass.

EFFECT: increasing the reliability of detection of the target observation object.

9 cl, 5 dwg

Description

FIELD OF TECHNOLOGY

[0001] The invention relates, in General, to computer systems and methods, and in particular to systems and methods for monitoring the territory using a managed video camera.

BACKGROUND

[0002] In the prior art, there are monitoring systems and methods in which monitoring is carried out using PTZ cameras, and if only PTZ cameras are used, the camera must follow a route for continuous monitoring, sequentially changing the viewing direction, moving from one point to another.

[0003] Patent application US 20150116488 Al Method for controlling a monitoring system and a system for its implementation ”, patent holder: Ivan Sergeevich Shishalov, Andrei Viktorovich Filimonov, Oleg Andreevich Gromazin, Sergei Vyacheslavovich Brunov, published: 04/30/2015 .

[0004] This invention relates generally to the field of video surveillance, and more particularly to a method for controlling a forest monitoring system. The technical result consists in increasing the reliability of detection (probability of detection), reducing the likelihood of a false alarm, or false detection of an object, reducing the time required to detect, examine and analyze information about the territory. The technical result is achieved due to the method, which includes the following steps: first collect current information about the object of observation; create a route for inspecting the territory with at least one means of observation, consisting of many points with fixed values of the orientation of the means of observation, which are chosen in such a way as to optimally inspect all possible territory according to the technical characteristics of the means of observation, terrain and height of the structure, and which a plurality of observation sites is determined, wherein the observation means looks at each site motionlessly with a predetermined magnification value.

[0005] This method assumes that at each moment of time, only part of the territory or object is observed. Moreover, the full inspection cycle is characterized by a certain time T, during which the camera passes the route, and returns to the starting point, starting the route from the beginning.

[0006] Obviously, under certain conditions, this time can be significant: when the territory is large and the detail of the site is high (for example, when forest fires are detected, the camera should be approached as close as possible, and the inspection area is tens of square kilometers).

[0007] In this case, the time of detection of the object from the moment of its occurrence (in case the object is stationary) will be associated with the time of inspection of the route T, and the longer the inspection time, the longer the time from the moment of occurrence to the detection of the object. This characteristic, by analogy with radar methods, can be called time resolution.

[0008] Moreover, for automated systems (in the general case, and for direct observation by the operator), the more information is accumulated about the object, the less likely it is to miss the desired object. In particular, the longer each specific point is taken, the less likely it is to miss the desired object. Similarly, we can give an example about the camera zoom (ZOOM): the larger the camera’s zoom, the less likely it is to miss the desired object, because the object will be larger in the image, but the viewing angle will decrease, which means that the full cycle of exploring the territory will take longer, because the number of points in the inspection route will increase.

[0009] As you can see, these two parameters (detection reliability and detection efficiency) conflict with each other: with an increase in the inspection time, the detection reliability increases, but, obviously, the detection efficiency decreases, and vice versa.

[00010] The situation for automated systems is further complicated by the presence of false positives, the likelihood of which should be reduced. The probability of a false response depends on the time of observation and the sensitivity of the detection mechanism, in particular on the method of operation and the settings of the detection mechanism that performs the detection.

[00011] In general, the main approach to solving this problem is to optimize the routes for inspecting the territory for the task at hand. If such systems are used to detect forest fires, the influence factor may be the probability of forest fires (for example, it is known that there is a greater probability of fires in some direction, it is advisable to conduct more detailed monitoring in this direction), or the detection conditions (for example, it is known , which means poor visibility in this direction due to fog, which means that this direction can be excluded from inspection by spending time on inspecting directions with better visibility).

[00012] All such approaches require the availability of sufficiently detailed information about the target object of observation, which is often difficult to obtain. In addition, part of the influencing parameters may change during the monitoring process, which also complicates the implementation of such approaches, for example, as in the patent US 20150116488 described above.

SUMMARY OF THE INVENTION

[00013] The invention addresses the disadvantages inherent in solutions known in the art.

[00014] The technical result is to increase the reliability of detection of the target object of observation.

[00015] With an increase in the reliability of detection of the desired object, the likelihood of false alarms is reduced and there is no need to obtain information about the object or the territory of observation in advance (or during the observation process).

[00016] The indicated technical result is achieved due to the method of monitoring the territory with the help of a controlled video camera, in which a sequence of frames of the observed area is obtained through the initial passage of the given route controlled by the video camera, at least one frame is received at each point of the route; determining on the obtained sequence of frames the presence of target objects of observation using a detecting mechanism; in the case of detection of target objects of observation, determine the pixel coordinates on at least one frame of the sequence of at least one target object of observation; calculating a camera viewing direction providing an image of at least one monitoring target based on: pixel coordinates of the monitoring target defined in the previous step, internal calibration, and the viewing direction of the controlled video camera; point the controlled camera at least once in the direction of view of at least one target to be monitored, obtain at least one frame containing the target to be monitored and process the frame with a detecting mechanism based on the data on the detection of the target to be monitored at least in one pass , decide on the presence of the target object of observation on the frame obtained during the initial pass.

[00017] In some embodiments of the technical solution, upon repeated passage, the number of frames is greater than the number of frames on the first passage.

[00018] In some embodiments of the technical solution, more than one frame is obtained at each shooting point, and with repeated passage, the shooting time of each shooting position is increased.

[00019] In some embodiments of the technical solution, the re-passage increases the proximity of the camera.

[00020] In some embodiments of the technical solution, the detection ability is changed by configuring the detection mechanism.

[00021] Also, the indicated technical result is achieved thanks to the territory monitoring system using a controlled video camera, which includes a controlled video camera; one or more data processing devices; one or more storage devices; one or more programs, where one or more programs are stored on one or more data storage devices and executed on one or more data processing devices, and one or more programs includes the following instructions, which when executed on at least one processing device data, allow: to obtain a sequence of frames of the observed territory through the initial passage of a given route controlled by a video camera, and at least one frame is received at each point of the route; to determine on the obtained sequence of frames the presence of target objects of observation using the detecting mechanism; in case of detection of target objects of observation, to determine the pixel coordinates on at least one frame of the sequence of at least one target object of observation; calculate the viewing direction of the video camera, providing an image of at least one target of the observation based on the pixel coordinates of the target object of observation defined in the previous step, internal calibration and the viewing direction of the managed video camera; to guide the managed video camera at least once in the direction of view of at least one target to be monitored, and to obtain at least one frame containing the target to be monitored, and to process the said frame with a detecting mechanism; on the basis of data on the detection of target objects of observation of at least one repeated pass, make a decision on the presence of the target object of observation on the frame obtained during the initial pass.

BRIEF DESCRIPTION OF THE DRAWINGS

[00022] The features and advantages of this technical solution will become apparent from the following detailed description and the accompanying drawings, in which:

[00023] In FIG. 1 shows an example implementation of a method for monitoring a territory using a controlled video camera.

[00024] FIG. 2 shows a sequence of frames that was obtained by first walking around the camera.

[00025] In FIG. 3 shows an embodiment in which two monitoring targets are detected by a detection mechanism.

[00026] In FIG. 4 shows an embodiment in which the target is not detected by the detection mechanism.

[00027] In FIG. 5 shows an embodiment in which a target of observation is detected by a detecting mechanism.

DETAILED DESCRIPTION OF THE INVENTION

[00028] Below will be described the concepts and definitions necessary for the detailed disclosure of the ongoing technical solution.

[00029] The technical solution may be implemented as a distributed computer system.

[00030] In this solution, a system means a computer system, a computer (electronic computer), CNC (numerical control), PLC (programmable logic controller), computerized control systems, and any other devices that can perform a given, well-defined sequence of operations (actions, instructions).

[00031] An instruction processing device is understood to mean an electronic unit or an integrated circuit (microprocessor) executing machine instructions (programs).

[00032] An instruction processing device reads and executes machine instructions (programs) from one or more data storage devices. Storage devices may include, but are not limited to, hard disks (HDD), flash memory, ROM (read only memory), solid state drives (SSD), optical media (CD, DVD, etc.).

[00033] A program is a sequence of instructions for execution by a computer control device or an instruction processing device.

[00034] A detecting mechanism is a set of measures aimed at identifying the target object in the obtained sequence of frames, moreover, a computer vision algorithm or a hardware-software complex can be used as a detecting mechanism.

[00035] Detection ability is a complex characteristic of a system or a detecting mechanism, which is determined by the fact that an increase in detection ability is one of the following options, or a combination thereof

a) reduction in the probability of missing a target while maintaining the probability of false alarms,

b) reducing the likelihood of false alarm while maintaining the probability of missing the target,

c) at the same time reducing the probability of missing the target object of observation and the probability of false alarm,

d) a change in the probability of false alarm and omission of the target object of observation, increasing the efficiency of solving a specific problem.

[00036] Frame - data received from the matrix, a set of points (pixels), each of which is assigned a characteristic (brightness) or several characteristics (brightness, color components).

[00037] Camera calibration (calibration characteristics of the camera) - a mathematical model and its parameters that allow you to relate the image pixel and the direction of arrival of the optical beam (direction of observation) for a given pixel.

[00038] A controlled video camera is a camera that supports remote control of the direction (often the viewing angle of the vertical and horizontal) and zoom.

[00039] A guided camera bypass route is a sequence of points characterizing the viewing direction (vertical, horizontal) and proximity, at each of which points receive photo or video data of the territory or object of interest.

[00040] The target of observation is the object for detection of which this method and this system are designed, for example, in the case of forest fires, this object may be smoke.

[00041] Resolution is a value that determines the number of dots (raster image elements) per unit area (or unit length).

[00042] FIG. 1 is a flowchart showing a method for monitoring a territory using a managed video camera, which comprises the following steps:

[00043] Step 101: a sequence of frames of the observed area is obtained through an initial passage of a given route controlled by a video camera, at least one frame being received at each point of the route.

[00044] In this step, at least one frame is received at least at one point. Detection ability depends on the initial passage of a given route controlled by a video camera, i.e. if you get a large number of frames, then you can reduce the likelihood of a false alarm and reduce the likelihood of missing a target.

[00045] In an exemplary embodiment, the precontrolled video camera traverses a high speed bypass route with the minimum acceptable characteristics of the detecting mechanism (for example, with low thresholds). In FIG. Figure 2 shows the sequence of frames that was obtained by first walking around the camera at high speed with the minimum acceptable thresholds for the detection mechanism to operate, which passes the observation route with a sufficiently small predetermined approximation (wide viewing angle). Due to the high speed of the route bypass, a controlled video camera can go the route for a small number of route points (for example, to determine the optimal number of route points, it is necessary to divide 360 degrees by the camera viewing width).

[00046] Step 102: on the obtained sequence of frames, the presence of target objects of observation is determined using the detecting mechanism.

[00047] After obtaining a sequence of frames of the observed area, a detection mechanism is applied to said frames. For example, in one of the frames that was obtained in the previous step, the azimuth is 173.045 degrees, the elevation angle is 2.348 degrees, with an overview of the managed video camera 52.7 × 31.5 degrees (image and video resolution 1920 × 1080) with a detecting mechanism with a minimum Allowed thresholds of operation was detected target monitoring. As a detecting mechanism, various methods can be used that are obvious to a person skilled in the art, for example, segmentation of flame and smoke and / or detection of moving areas and / or analysis of spatial changes in brightness, not limited to. Surveillance targets are shown in FIG. 3 and highlighted by quadrangles.

[00048] Each detecting mechanism that provides the detection and identification of any objects has many configurable parameters that affect the quality of detection by the mechanism of target objects. These parameters can change during operation and depend on the conditions of application of the mechanism. Thus, the detecting mechanism is customizable. Detection parameters depend on such settings in a non-trivial way.

[00049] As the detecting mechanism, object detection algorithms described in the information source [1] can be used. As an example, but not limited to, methods for detecting and extracting objects based on temporal filtering, which additionally use the information obtained in previous frames, can be used. The effectiveness of such methods is certainly higher than that of methods that use only a priori information about the spatial and statistical properties of the background and object when detecting. When bypassing the controlled camera along the route at high speed and processing the received frames with a detecting mechanism with minimum acceptable thresholds of sensitivity, a rather high probability of detecting an object arises, but the likelihood of false alarms also increases.

[00050] Territory inspection options may vary in various ways. For example, you can vary the sequence of settings of the detecting mechanism and the approximations of the camera. You can vary the observation times of a particular waypoint. It is also possible that the settings of the detecting mechanism, the camera’s proximity, and the time of observation of a specific route point (or the number of frames received at a specific point of the route) vary simultaneously.

[00051] Step 103: in the case of detection of target objects of observation, determine the pixel coordinates on at least one frame of the sequence of at least one target object of observation.

[00052] In the indicated exemplary embodiment, the detected objects have the following coordinates: the first object x = 393, y = 470 (pixel coordinates from the upper left corner), the second object x = 1461, y = 491.

[00053] Step 104: calculating a viewing direction of the camera, providing an image of at least one target to be observed based on the pixel coordinates of the target to be monitored as determined in the previous step, internal calibration and orientation of the managed video camera.

[00054] After passing the bypass route of the managed video camera, determine the direction of the camera view and the pixel coordinates on the frame along which you want to direct the camera based on the internal calibration of the video camera and the orientation of the managed video camera on which objects were detected, and send the managed video camera to execute a frame on these zoomed coordinates. Moreover, the degree of increase in approximation can be set both initially (for example, with a given magnification factor), and based on the limitations and capabilities of the managed video camera.

[00055] For the indicated exemplary embodiment, the calibration characteristic can be set by the resolution of the picture and the viewing angles horizontally and vertically 52.7 × 31.5 degrees (image resolution 1920 × 1080). Based on the calibration characteristics and the pixel coordinates of the object in the image, the angles of the object relative to the optical center of the camera can be calculated (Fig. 3), which will be equal, for the first object -15 degrees (to the left of the center), and for the second object 2 degrees (above the center )

[00056] To determine the angles of the camera’s viewing direction, you can add the obtained angles to the camera’s viewing direction in which the image was obtained (azimuth of 173.045 degrees, elevation - 2.348 degrees), or select such directions so that the resulting object is in the frame, namely the azimuth = 156.29, elevation = -0.016 (camera view 2.91 × 1.64 degrees) for the first point, azimuth of 168.16 and elevation angle of -0.337, respectively, for the second.

[00057] Step 105: point the controlled video camera at least once in the direction of view of at least one target to be monitored, obtain at least one frame containing the target to be monitored, and process the frame with a detecting mechanism.

[00058] In an embodiment, the guided video camera is first guided by the coordinates and direction of the first quadrangle (Fig. 3) with the maximum possible approximation: azimuth = 156.29, elevation = -0.016 (camera view 2.91 × 1.64 degrees), after which they receive a frame at a given approximation.

[00059] In the embodiment, due to the repeated operation of the detecting mechanism, it turns out that there is no detection target in the frame at the given coordinates. Then the mechanism can be configured with lower thresholds of sensitivity (minimum acceptable characteristics of the detecting mechanism). If even now it turns out that there is no detection target at the given coordinates at the given coordinates, then it is decided that the target is missing (see Fig. 4).

[00060] Next, in an embodiment, the guided video camera is guided by the coordinates and direction of the second quadrangle with the maximum possible approximation, azimuth angle of 168.16 and elevation angle of -0.337, respectively. Smoke was detected in this quadrangle by a detecting mechanism.

[00061] In addition, the reliability of detection of objects can be changed in the following ways:

[00062] - by increasing the approximation - in this case, the detected target becomes larger, its movements and movements are better distinguishable, that is, the probability of detection increases;

[00063] - by increasing the observation time at a specific point in the observation space — in this case, it is possible to use detection mechanisms using accumulation, then the probability of detecting an object increases.

[00064] Optimum configurations of the mechanism can be found to ensure that the detection reliability is maintained, without increasing the likelihood of false alarms, with a minimum inspection time.

[00065] Step 106: based on the detection data of the observation targets of at least one repeated pass, a decision is made about the presence of the target to be observed on the frame obtained in the primary pass.

[00066] The identified surveillance target is shown in FIG. 5.

[00067] In some embodiments, the claimed method is performed on a territory monitoring system using a managed video camera.

[00068] In one embodiment, this technical solution can be made in the form of a territory monitoring system using a managed video camera, including: a managed video camera, one or more data processing devices, one or more data storage devices, one or more programs, where one or more programs are stored on one or more data storage devices and executed on one or more data processing devices, and one or more programs includes the following instructions: receive a sequence of firewood of the observed territory by means of an initial passage of a given route controlled by a video camera, and at least one frame is received at each point of the route; determining on the obtained sequence of frames the presence of target objects of observation using a detecting mechanism; in the case of detection of target objects of observation, determine the pixel coordinates on at least one frame of the sequence of at least one target object of observation; calculating a camera viewing direction providing an image of at least one monitoring target based on the pixel coordinates of the monitoring target determined in the previous step, internal calibration, and the viewing direction of the controlled video camera; point the controlled camera at least once in the direction of view of at least one target to be observed, get at least one frame containing the target to be observed, and process the frame with a detecting mechanism; on the basis of data on the detection of target objects of observation of at least one repeated pass, a decision is made on the presence of the target object of observation on the frame obtained during the primary pass.

[00069] In some embodiments, the method is implemented physically on a camera.

[00070] The system can be implemented using a mobile phone, personal or portable computer, or using other computing means suitable for performing the necessary program-logic functions.

[00071] A system may include one or more of the following components: a processing component, a camera, memory, a power component, a multimedia component, an input / output (I / O) interface, a touch component, a data transmission component.

[00072] In some embodiments, the processing component mainly controls all system operations, for example, display, data transfer, camera operation, and recording operation. The processing component may include one or more processors that implement instructions to complete all or part of the steps of the above methods. In addition, the processing component may include one or more modules for a convenient interaction process between the processing component and other components. For example, the processing component may include a multimedia module for conveniently facilitating interaction between the multimedia component and the processing component.

[00073] The memory is configured to store various types of data to support system operation. Examples of such data include instructions from any application or method, images, videos, etc. The memory may be implemented in the form of any type of non-volatile memory, non-volatile memory or a combination thereof, for example, Static Random Access Memory (RAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read-Only Memory (EPROM), Programmable Read-Only Memory (EPROM) Storage Device (ROM), Permanent Storage Device (ROM), magnetic memory, flash memory, magnetic and whether an optical disk.

[00074] In some embodiments, the power component provides electricity to various components of the system. A power component may include a power management system, one or more power supplies, and other nodes for generating, controlling, and distributing power to the system.

[00075] In some embodiments, the multimedia component includes a screen providing an output interface between the system and the user. In some embodiments, the implementation of the screen may be a liquid crystal display (LCD) or touch panel (SP). If the screen includes a touch panel, the screen may be implemented as a touch screen for receiving an input signal from a user. The touch panel includes one or more touch sensors in the sense of gesturing, touching and sliding the touch panel. The touch sensor can not only feel the touch or the gesture of turning over, but also determine the duration of time and pressure associated with the operation mode of touch and sliding.

[00076] An I / O interface provides an interface between a processing component and a peripheral interface module.

[00077] The sensor component comprises one or more sensors and is configured to provide various aspects of evaluating a system state. For example, a sensor component may detect on / off states. systems, the relative arrangement of components, for example, the display and keypad of the system, the change in position of the system or one component of the system, the presence or absence of contact between the user and the system, and the orientation or acceleration / deceleration of the system. The sensor component comprises a proximity sensor configured to detect the presence of an object in the vicinity when there is no physical contact. The sensor component contains an optical sensor (for example, CMOS or CCD image sensor), made with the possibility of use in the visualization of the application. In some embodiments, the sensor component comprises an acceleration sensor, a gyroscope, a magnetic sensor, a pressure sensor, or a temperature sensor.

[00078] The communication component (data transmission component) is configured to facilitate wired or wireless communication between the system and other devices. The system can access a wireless network based on communication standards, but not limited to, such as WiFi, 2G or 3G, or combinations thereof. In one exemplary embodiment, the data transmission component receives a broadcast signal or broadcast, associated information from an external broadcast control system via a broadcast channel. In one embodiment, the data transmission component comprises a near field communication module (NFC) to facilitate short-range communication, or a radio frequency identification (RFID) module, an infrared data transmission module (IrDA), or ultra-wideband (UWB) technology, or a Bluetooth module (BT) or other means suitable for wireless data transmission.

[00079] In an exemplary embodiment, the memory includes instructions that are executed by the processor of the system to implement the methods described above to increase the reliability of detection of the target surveillance object. For example, a non-volatile computer-readable medium may be a ROM, random access memory (RAM), CD, magnetic tape, floppy disks, optical storage devices and the like.

The source of information

Forsyth D.A., Pons D. Computer Vision. Modern Approach = Computer Vision: A Modern Approach // M.: William. - 2004.

Claims (26)

1. A method of monitoring the territory using a managed video camera, comprising stages in which:

receive a sequence of frames of the observed territory through the initial passage of a given route controlled by a video camera, and at least one frame is obtained at each point of the route;

using the detecting mechanism on the obtained sequence of frames determine the presence of target objects of observation;

in the case of detection of target objects of observation, determine the pixel coordinates on at least one frame of the sequence of at least one target object of observation;

calculating the viewing direction of the video camera, providing an image of at least one target of the observation based on the pixel coordinates of the target object of observation defined in the previous step, internal calibration and the viewing direction of the managed video camera;

point the controlled video camera at least once in the viewing direction of the at least one target to be monitored in the previous step, obtain at least one frame containing the target to be monitored, and process the frame with a detecting mechanism;

on the basis of data on the detection of target objects of observation of at least one repeated pass of the route of the primary passage, a decision is made on the presence of the target object of observation on the frame obtained during the primary passage. 2. The method according to claim 1, characterized in that the detecting mechanism is a computer vision algorithm. 3. The method according to claim 1, characterized in that the detecting mechanism is a hardware-software complex. 4. The method according to claim 1, characterized in that upon repeated passage, the number of frames is greater than during the first passage. 5. The method according to claim 1, characterized in that more than one frame is obtained at each shooting point, and with repeated passage, the time interval between frames is increased. 6. The method according to claim 1, characterized in that upon repeated passage increase the approximation of the camera. 7. The method according to claim 1, characterized in that the detecting ability is changed by configuring the parameters of the detecting mechanism. 8. The method according to claim 1, characterized in that the parameter of the detecting mechanism is a sensitivity threshold. 9. The territory monitoring system using a controlled video camera includes:

managed video camera;

one or more data processing devices;

one or more storage devices;

one or more programs where one or more programs are stored on one or more data storage devices and executed on one or more data processing devices, and one or more programs includes instructions that, when executed on at least one data processing device, allow:

to obtain a sequence of frames of the observed territory through the initial passage of a given route controlled by a video camera, and at least one frame is received at each point of the route;

to determine on the obtained sequence of frames the presence of target objects of observation using the detecting mechanism;

in case of detection of target objects of observation, to determine the pixel coordinates on at least one frame of the sequence of at least one target object of observation;

calculate the viewing direction of the video camera, providing an image of at least one target of the observation based on the pixel coordinates of the target object of observation defined in the previous step, internal calibration and the viewing direction of the managed video camera;

to guide the managed video camera at least once in the direction of view of at least one target to be observed at a certain point in the previous step and to obtain at least one frame containing the target to be observed and to process the said frame with a detecting mechanism; on the basis of data on the detection of target objects of observation of at least one repeated pass of the route of the primary passage, make a decision on the presence of the target object of observation on the frame obtained during the primary passage.

Images