KR102579511B1 - Video Surveillance Device and Driving Method Thereof, Video Surveillance System and Computer Readable Recording Medium - Google Patents

Abstract

The present invention relates to a video surveillance device and a method of driving the device, a video surveillance system, and a computer-readable recording medium. The video surveillance device according to an embodiment of the present invention includes a fixed camera installed in a certain area, A communication interface unit that receives captured images of an area, analyzes the received captured images, detects objects in the images, and tracks the detected objects using at least one of the pan, tilt, and zoom control functions of the PTZ camera. It may include a control unit that acquires information on the tracking object according to tracking.

Description

Video surveillance device and driving method of the device, video surveillance system and computer readable recording medium {Video Surveillance Device and Driving Method Thereof, Video Surveillance System and Computer Readable Recording Medium}

The present invention relates to a video surveillance device and a method of driving the device, a video surveillance system, and a computer-readable recording medium, and more specifically, to a video surveillance device that operates as a PTZ (Pan, Tilt, Zoom) camera or operates inside or outside the camera. It is a video surveillance device that receives and analyzes images from fixed cameras, tracks detected objects, and transmits the detailed information and location information of objects obtained by analyzing images captured by PTZ cameras to external devices such as control servers. and a method of driving the device, a video surveillance system, and a computer-readable recording medium.

In addition to providing surveillance video and pan/tilt/zoom control functions, which are the original functions, recent PTZ cameras perform video analysis based on deep learning and provide the results. However, since the PTZ camera only provides the device's original video and video analysis results, it must rely on a third external device to link with other external cameras and provide detailed and comprehensive information using the video analysis results.

For this reason, when analyzing and linking images from multiple cameras, the complexity of building a surveillance system increases due to dependence on external devices. Additionally, since the pan/tilt/zoom control of the PTZ camera is determined by an external device, there is a problem of poor responsiveness in tracking objects detected by video analysis.

Korean Patent Publication No. 10-0888935 (2009.03.10) Korean Patent Publication No. 10-0872403 (2008.12.01) Korea Patent Publication No. 10-2018-0060335 (2018.06.07) Korea Patent Publication No. 10-2019-0056650 (2019.05.27)

An embodiment of the present invention is a device that operates as a PTZ camera or operates inside or outside the camera, and tracks a detected object by receiving and analyzing images from a fixed camera, and detects objects obtained by analyzing images captured by the PTZ camera. The purpose is to provide a video surveillance device that transmits detailed information and location information to an external device such as a control server, a method of operating the device, a video surveillance system, and a computer-readable recording medium.

A video surveillance device according to an embodiment of the present invention includes a communication interface unit that receives captured images of a fixed area in a random area from a fixed camera installed in a random area, and an object in the image that analyzes the received captured images. Detects and tracks the detected object using at least one control function among the pan, tilt, and zoom of a PTZ (Pan, Tilt, Zoom) camera to track the tracked object according to the tracking. It includes a control unit that acquires information.

The control unit analyzes the captured image of the fixed camera to obtain overall information including events of the detected objects, and as information on the tracking object, detailed information including properties of the object and indicating the location of the object. Location information can be obtained.

The control unit may acquire information on gender or clothing as the detailed information when the tracking object is a human object, and may acquire information on the vehicle model or vehicle number as the detailed information when the tracking object is a vehicle object.

The control unit may use a feature point on the ground preset using an image captured by the PTZ camera and a position description information specified in relation to the feature point as the location information of the tracking object.

The control unit may continuously track the tracking object by analyzing images captured by the PTZ camera when the tracking object deviates from the field of view of the fixed camera.

The control unit may compress a full-screen image including the tracking object and an object image of the tracking object, and transmit the compressed data, event information, and detailed information of the tracking object to an external device of the manager. .

In addition, a method of driving a video surveillance device according to an embodiment of the present invention includes a communication interface unit receiving a captured image of a fixed area in a random area from a fixed camera installed in a random area, and a control unit receiving the received image. Detects objects in the video by analyzing the captured video, and tracks the detected object using at least one of the pan, tilt, and zoom control functions of the PTZ (Pan, Tilt, Zoom) camera. It includes the step of acquiring information on the tracking object according to the tracking.

The acquiring step is to analyze the captured image of the fixed camera to obtain overall information including events of the detected objects, detailed information including the properties of the object as information on the tracking object, and the location of the object. You can obtain location information indicating .

In the acquiring step, if the tracking object is a human object, information on gender or clothing may be acquired as the detailed information, and if the tracking object is a vehicle object, information on the vehicle type or vehicle number may be acquired as the detailed information. .

The acquisition step may use a preset feature point on the ground using an image captured by the PTZ camera as the location information of the tracking object and location description information specified in relation to the feature point.

The acquiring step may include continuously tracking the tracking object by analyzing a captured image of the PTZ camera when the tracking object leaves the field of view of the fixed camera.

In the driving method, the control unit compresses the full screen image including the tracking object and the object image of the tracking object, and compresses the compressed data, event information, and detailed information of the tracking object through the compression to an external device of the manager. It may further include the step of transmitting to .

A video surveillance system according to an embodiment of the present invention includes a fixed camera installed in a random area to capture a fixed area, analyzes captured images from the fixed camera, detects objects in the video, and stores the detected objects in the PTZ camera. It includes a video surveillance device that tracks using at least one control function among pan, tilt, and zoom and acquires information on the tracked object according to the tracking.

In addition, it is a computer-readable recording medium containing a program for executing a method of driving a video surveillance device according to an embodiment of the present invention, wherein the method of driving a video surveillance device includes a method of operating a video surveillance device from a fixed camera installed in a certain area to Receiving a captured image of a fixed area, analyzing the received captured image to detect an object in the image, and panning, tilting, and zooming the detected object by a PTZ camera. A step of tracking using at least one control function and acquiring information about the tracked object according to the tracking is executed.

According to an embodiment of the present invention, objects are detected by directly receiving and analyzing images from an (external) fixed camera without relying on a third external device, so images from multiple cameras can be analyzed and linked without an additional device.

In addition, according to an embodiment of the present invention, objects detected by directly analyzing images from an external fixed camera are directly tracked by directly controlling the pan/tilt/zoom functions of the PTZ camera without relying on a third external device, thereby quickly tracking the object. can be tracked.

According to an embodiment of the present invention, not only the image analysis results based on the wide view image of the fixed camera but also the image analysis results based on the image enlarged using the zoom function of the PTZ camera are obtained, so that information such as events generated by the object being tracked is obtained. In addition, overall information including background information to which the object belongs, detailed information such as car model, license plate number, gender, and clothing of the object being tracked, as well as an enlarged image of the object, can be provided.

According to an embodiment of the present invention, when the object being tracked deviates from the angle of view of the external fixed camera, the object can be continuously tracked by analyzing the original image of the PTZ camera device.

According to an embodiment of the present invention, location information of the object being tracked can be provided based on location information according to pre-entered pan/tilt/zoom.

1 is a diagram showing a video surveillance system according to an embodiment of the present invention;
Figure 2 is a block diagram illustrating the detailed structure of the PTZ camera of Figure 1;
Figure 3 is a flowchart showing the operating process of a PTZ camera according to the first embodiment of the present invention;
Figure 4 is a flowchart showing the operating process of a PTZ camera according to a second embodiment of the present invention;
Figure 5 is a flowchart showing the operating process of a PTZ camera according to a third embodiment of the present invention;
Figure 6 is a flowchart showing the operating process of a PTZ camera according to a fourth embodiment of the present invention;
Figure 7 is a block diagram illustrating the detailed structure of a video surveillance device according to an embodiment of the present invention, and
Figure 8 is a flowchart showing the driving process of the video surveillance device of Figure 7.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

1 is a diagram showing a video surveillance system according to an embodiment of the present invention.

As shown in FIG. 1, the video surveillance system 90 according to an embodiment of the present invention may be named variously, such as a camera interlocking system or a camera collaboration system, and may include a fixed camera (or first type camera) 101, 102, 103, 104), PTZ camera (or second type camera) 105, and part or all of the communication network 106, and a control server that controls incidents and accident events through camera footage, etc. It may further include external devices.

Here, “including part or all” means that the video surveillance system 90 is configured by omitting some components such as the control server, or the fixed cameras 101, 102, 103, and 104 are configured by omitting the communication network 106. and the PTZ camera 105 performs direct (e.g. P2P, etc.) communication, or some or all of the components that make up the PTZ camera 105 are network devices (e.g., wireless exchange devices, etc.) that make up the communication network 106. ), etc., and is explained as including everything in order to facilitate a sufficient understanding of the invention.

The fixed cameras (101, 102, 103, 104) capture images input through the lens through an imaging device such as a CCD (Charge Coupled Device) or CMOS (Complementary Metal-Oxide Semiconductor) and transmit the images to the PTZ camera (105). do. Fixed cameras (101, 102, 103, 104) are installed in various locations and can capture preset fixed areas according to the angle of view and transmit the captured images to the PTZ camera 105. Wired and wireless communication or wireless communication is used to transmit the captured images. Captured images can be provided by communicating with the PTZ camera 105 in various forms, such as near or far. Of course, each fixed camera (101, 102, 103, 104) communicates with the PTZ camera (105) based on identification information such as device ID, and each fixed camera (101, 102, 103, 104) has an internal memory. Communication can be performed based on device identification information, such as the device ID of the PTZ camera 105, which is previously stored in the back. The fixed cameras 101, 102, 103, and 104 in FIG. 1 may be referred to as a first fixed camera, a second fixed camera, etc. for convenience of explanation, and may operate in the form of photographing a preset fixed area (or region). there is. In other words, the manager who installs and manages the fixed cameras 101, 102, 103, and 104 can photograph the fixed area according to the angle specified when the camera is first installed. For example, the fixed area may be an intersection or a specific area in a crime-prone area.

The PTZ camera 105 refers to a second type of camera that has different functions and operates in a different way than the fixed cameras 101, 102, 103, and 104, for example, pan, tilt, and zoom. It may refer to a camera capable of operations such as (zoom). Typically, the PTZ camera 105 can perform PTZ operation by remote control from a control server, etc., and thus, it is possible to film a wide area according to the PTZ operation based on the place where the PTZ camera 105 is installed. Of course, it is also possible to enlarge and photograph a specific area through a zoom operation.

The PTZ camera 105 according to an embodiment of the present invention detects objects by analyzing (filmed) images transmitted from fixed cameras 101, 102, 103, and 104, and detects events generated by the objects based on event areas and rules. can be detected. In other words, the PTZ camera 105 according to an embodiment of the present invention is a device such as a video analysis module (or video surveillance module) for analyzing (filmed) video transmitted from the fixed cameras 101, 102, 103, and 104. It may include, and it may also be possible to analyze captured images taken by the PTZ camera 105 through the corresponding device.

The PTZ camera 105 captures and analyzes images input through a lens using an imaging device such as CCD or CMOS to detect objects. The PTZ camera 105 is a video camera equipped with an electric zoom lens and is attached to a pan and tilt driving device. The camera can be rotated and the lens can be zoomed in and out through control signals inside the PTZ camera 105. there is.

The PTZ camera 105 receives images from fixed cameras 101, 102, 103, and 104 or analyzes the images from the PTZ camera 105 in real time to detect, track, and classify moving objects, etc., and based on this, the user can It detects a designated event and automatically controls the pan, tilt, and zoom of the PTZ camera 105 to perform PTZ tracking of a moving object. In addition, the PTZ camera 105 analyzes the images input from the fixed cameras 101, 102, 103, and 104 to obtain object information and event detection results, images when an event occurs, or images from the PTZ camera 105. , Detailed information on the tracked object and an enlarged image of the tracked object obtained by analyzing the image of the PTZ camera 105 are transmitted to external devices such as a control server through the wired and wireless communication network 106. Here, in the case of a human object, the detailed information may be gender or clothing, and in the case of a vehicle object, it may be the vehicle type or license plate number.

More specifically, the PTZ camera 105 according to an embodiment of the present invention can perform various operations in conjunction with at least one fixed camera 101, 102, 103, and 104. First, the PTZ camera 105 receives and analyzes images from the fixed cameras 101, 102, 103, and 104, tracks the detected object, and analyzes the images from the fixed cameras 101, 102, 103, and 104 to detect any more If the object cannot be tracked, an operation of continuously tracking the object can be performed by analyzing the video of the PTZ camera 105, that is, the video taken by the camera itself. In addition, the PTZ camera 105 analyzes the images of the fixed cameras 101, 102, 103, and 104 to obtain overall information about the detected object, and enlarges the image including the object to obtain detailed information about the object (control You can perform a transmission operation (to an external device such as a server). Furthermore, the PTZ camera 105 according to an embodiment of the present invention can perform operations to input location information according to pan, tilt, and zoom, and can also perform operations to obtain and transmit location information of objects. there is. Various detailed operations of the PTZ camera 105 will be discussed in more detail later through drawings.

The communication network 106 includes wired and wireless communication networks, such as a local area network (LAN), a wide area network (WAN), or a value added network (VAN), or a mobile radio network. Communication Network), satellite communication network, Bluetooth, Wibro (Wireless Broadband Internet), HSDPA (High Speed Downlink Packet Access), etc. can be implemented with all types of wireless networks.

More specifically, the communication network 106 may be configured in various forms. The communication network 106 may include both wired and wireless communication networks. For example, a wired or wireless Internet network may be used or linked as the communication network 106. Here, the wired network includes Internet networks such as cable networks and public switched telephone networks (PSTN), and the wireless communication network includes CDMA, WCDMA, GSM, EPC (Evolved Packet Core), LTE (Long Term Evolution), and Wibro networks. It means including. Of course, the communication network 106 according to an embodiment of the present invention is not limited to this, and can be used as an access network for a next-generation mobile communication system to be implemented in the future, for example, a cloud computing network in a cloud computing environment, a 5G network, etc. For example, if the communication network 106 is a wired communication network, the access point within the communication network can connect to the telephone company's exchange office, etc., but in the case of a wireless communication network, data is processed by connecting to the SGSN or GGSN (Gateway GPRS Support Node) operated by the communication company, or Data can be processed by connecting to various repeaters such as BTS (Base Transceiver Station), NodeB, and e-NodeB.

Communication network 106 may also include access points. Access points may include small base stations, such as femto or pico base stations, which are often installed in buildings. Here, femto or pico base stations can be classified according to the maximum number of fixed cameras (101, 102, 103, 104) or PTZ cameras (105) that can be connected, according to the classification of small base stations. Of course, it may include a short-distance communication module for performing short-range communication such as ZigBee and Wi-Fi with a fixed camera (101, 102, 103, 104) or a PTZ camera (105). Access points can use TCP/IP or RTSP (Real-Time Streaming Protocol) for wireless communication. Here, in addition to Wi-Fi, short-range communication can be performed using various standards such as Bluetooth, Zigbee, infrared (IrDA), RF (Radio Frequency) such as UHF (Ultra High Frequency) and VHF (Very High Frequency), and ultra-wideband communication (UWB). You can. Accordingly, the access point extracts the location of the data packet, specifies the best communication path for the extracted location, and forwards the data packet to the next device, such as the PTZ camera 105 or the control server, along the designated communication path. . Access points can share multiple lines in a typical network environment and may include, for example, routers, repeaters, and repeaters.

In addition to the above, the fixed cameras 101, 102, 103, 104 and the PTZ camera 105 of FIG. 1 according to the embodiment of the present invention can perform various operations, and other details can be discussed later. So, I would like to replace it with those contents.

FIG. 2 is a block diagram illustrating the detailed structure of the PTZ camera of FIG. 1.

As shown in FIG. 2, the PTZ camera 105 of FIG. 1 according to an embodiment of the present invention includes an external video receiver 201, a decoder 202, an image analysis unit 203, an encoder 204, and an image generation unit. It may include part or all of the unit 205, the PTZ control unit 206, and the PTZ driver 207.

Here, “including some or all” means that the PTZ camera 105 is configured by omitting some components such as the decoder 202 or encoder 204, or that some components such as the image generator 205 are configured. This means that it can be configured by being integrated with other components such as the image analysis unit 203, and is explained as being all-inclusive to facilitate a sufficient understanding of the invention.

The external image receiver 201 receives images from the fixed cameras 101 to 104. The external image receiver 201 can communicate with the fixed cameras 101 to 104 by wire or wirelessly, and can receive captured images through communication with the fixed cameras 101 to 104.

The decoder 202 may perform an operation to decompress the images received from the fixed cameras 101 to 104. Video compression, that is, encoding, and decoding are already well known to those skilled in the art, so further explanation will be omitted.

The image analysis unit 203 may analyze images received from the fixed cameras 101 to 104 and images from the PTZ camera 105 to perform object detection and event detection and determine object tracking. That is, the image analysis unit 203 analyzes images from the fixed cameras 101 to 104 in addition to the images from the PTZ camera 105, extracts a comprehensive analysis result, and enables object tracking based on this.

More specifically, the image analysis unit 203 can analyze captured images and more accurately detect people or vehicle objects within the video frame through pixel analysis. Depending on the pixel analysis of the video frame, people or vehicle objects can be recognized and detected based on attributes (e.g. shape, color, etc.), feature point analysis results, etc.). Then, the recognized object can be extracted and the movement of the object can be tracked by tracking the corresponding object in the input video frame according to time changes. In this process, the image analysis unit 203 analyzes captured images, that is, video frames, captured by the PTZ camera 105 when the tracked object leaves the field of view of the fixed cameras 101 to 104 to determine whether an object identical to the tracked object is detected. This can be determined, and continuous tracking of the tracking object can be performed through this process. In addition, events can be detected by comprehensively considering the situation between the tracked object and its surrounding objects, and such event detection can be determined by comparison with previously stored data for event detection. For example, if a human object and a vehicle object are in close contact, it can be judged as an accident. In addition, the image analysis unit 203 can perform various operations, and other details will be discussed later.

The encoder 204 may compress the entire image including the object being tracked and the enlarged object image into a JPEG image based on the image results analyzed by the image analysis unit 203. For example, if an event occurs, it can be seen that the object image related to the event occurrence is compressed.

Additionally, the image generator 205 may correspond to a part that captures an image through an imaging device such as a CCD or CMOS of the PTZ camera 105. In addition to an imaging device such as a lens, the image generator 205 may further include various devices for mechanically realizing the PTZ operation of the lens.

The PTZ control unit 206 can control pan, tilt, and zoom driving units to track objects. The PTZ control unit 206 may include a processor such as a CPU or MPU, and may further include a memory that carries a program for driving the driver. The PTZ control unit 206 secures an image of the event when an event occurs as a result of tracking an object in the video, for example, according to a preset method of the program, and secures the image of the event and secures the encoder 204 so that the image is compressed and transmitted to an external device such as a control server. ) can also be controlled.

The PTZ driver 207 may include various driving means for driving the image generator 205. Typically, the PTZ driver 207 may include a motor, and can drive the motor to pan, tilt, or zoom under the control of the PTZ controller 206. In other words, at least one of pan, tilt, and zoom operations can be performed.

The PTZ camera 105 according to an embodiment of the present invention is configured with an external image receiver 201 and a decoder 202, which do not exist in general PTZ cameras, to receive and process images from the fixed cameras 101 to 104, and is configured to receive and process images from the fixed cameras 101 to 104. It is characterized by including an image analysis unit 203 that analyzes images from the fixed cameras 101 to 104 in addition to the images from the camera 105, extracts comprehensive analysis results, and enables object tracking based on them. can see.

Figure 3 is a flowchart showing the operating process of the PTZ camera according to the first embodiment of the present invention. More precisely, images from the fixed cameras 101 to 104 are received and analyzed, and detected objects are tracked, and the fixed cameras 101 This is a flow chart to explain how to continuously track an object by analyzing the video of the PTZ camera 105 when the object can no longer be tracked by analyzing the video of ~ 104).

Referring to FIG. 3 together with FIG. 1 for convenience of explanation, the PTZ camera 105 of FIG. 1 receives images transmitted from the fixed cameras 101 to 104 (S301). Then, the PTZ camera 105 decodes the received compressed video (S302).

Next, the PTZ camera 105 analyzes the decoded video and detects objects such as people and vehicles present in the video (S303).

Additionally, the PTZ camera 105 detects events generated by objects based on the event area and rules (S304). As mentioned earlier, for example, if there is a vehicle object around a human object, the distance between the objects can be measured, and if the measured distances are close, it can be judged to be an accident. At this time, the severity of the accident can be determined by further determining whether the human object is lying on the floor or bleeding. Data for such judgment refers to event areas and rules, etc., and can be pre-stored in memory, and the event can be considered to be judged by comparison with the corresponding data.

Furthermore, the PTZ camera 105 tracks the object for which an event has been detected. Object tracking is performed on the fixed camera 101 based on information that maps the feature point coordinates of the fixed cameras 101 to 104 and the pan/tilt coordinates of the PTZ camera 105. ~ 104) Tracking can be performed by moving to the location of the PTZ camera 105 mapped to the object location in the image (S305). A method of mapping the feature point coordinates of the fixed cameras (101 to 104) and the coordinates of the PTZ camera (105) and the method of moving to the location of the mapped PTZ camera (105) are described in Korean Patent Publication No. 10-0888935 (2009.03.10). ) As in “Linking method between two cameras in an intelligent video surveillance system,” video received from fixed cameras (101 to 104) can be used instead of still images when setting coordinates.

In addition, based on the results of analyzing the images received from the fixed cameras 101 to 104, the PTZ camera 105 determines whether the object being tracked is beyond the image range, that is, the angle of view, of the fixed cameras 101 to 104 and can no longer be tracked. Check whether . If the object being tracked is within the image range (angle of view) of the fixed cameras 101 to 104 and continuous tracking is possible, it is checked whether the object tracking time limit set by the user has been reached. If the object tracking time limit has not yet been reached, the object is tracked by continuously analyzing the images received from the fixed cameras (101 to 104). If object tracking is no longer possible or the object tracking time limit has been reached, step S309 can be performed.

When the PTZ camera 105 in FIG. 1 is tracking an object in step S305, it analyzes the image of the PTZ camera 105 and detects the object (S307).

Additionally, the PTZ camera 105 finds the object being tracked in step S305 among the objects detected in step S307 and manages the information separately (S308). The method of finding the object being tracked in step S305 among the objects detected in step S307 is to select an object that matches the object type (person, vehicle, etc.) among the objects located at the center point of the PTZ camera 105 image as the object being tracked. Considered. When the PTZ camera 105 is tracking an object, as will be explained in step S404 of FIG. 4, the object is zoomed in and tracked, so the object being tracked does not deviate significantly from the center point of the PTZ camera 105 image.

Subsequently, unlike the previous step in which the PTZ camera 105 tracked the object by analyzing the images received from the fixed cameras 101 to 104, the PTZ camera 105 analyzes the image of the PTZ camera 105 and tracks the object (S309). In step S308, separately managed objects are continuously tracked based on the results of analyzing the video from the PTZ camera 105.

Additionally, the PTZ camera 105 checks whether the object being tracked is beyond the image range (angle of view) of the PTZ camera 105 and can no longer be tracked (S310). If tracking is no longer possible, object tracking ends. If continuous tracking is possible, check whether the object tracking time limit set by the user has been reached. If the object tracking time limit is reached, object tracking is terminated. Otherwise, object tracking continues in step S309.

In addition to the above, the PTZ camera 105 of FIG. 1 or a video surveillance device mounted inside the PTZ camera 105 or operating externally connected can perform various operations, and other details have been sufficiently explained previously. I would like to replace it with contents.

Figure 4 is a flowchart showing the operating process of the PTZ camera according to the second embodiment of the present invention. More precisely, the images of the fixed cameras 101 to 104 are analyzed to obtain overall information about the detected object and This is a flow chart to explain how to obtain and transmit detailed information of an object by enlarging an image.

For convenience of explanation, referring to FIG. 4 together with FIG. 1, the PTZ camera 105 of FIG. 1 analyzes images from the fixed cameras 101 to 104 and detects an object to be tracked (S401). Here, the tracking object may be a person or a vehicle object.

The PTZ camera 105 tracks the object detected in step S401 by controlling the pan/tilt/zoom of the PTZ camera 105 (S402).

In addition, the PTZ camera 105 can detect a tracked object separately managed in step S308 of FIG. 3 by analyzing the video captured by the PTZ camera 105 (S403).

Then, based on the results of analyzing the video of the PTZ camera 105, the longer of the horizontal and vertical lengths of the bounding box of the object being tracked is 30% of the video of the PTZ camera 105. The image is enlarged by controlling the zoom function of the PTZ camera 105 (S404). Of course, embodiments of the present invention will not be particularly limited to such magnification. If the horizontal length is long, it is enlarged to be 30% of the horizontal length of the PTZ camera 105 image, and if the vertical length is long, it is enlarged to be 30% of the vertical length of the PTZ camera 105 image. When enlarging an image, if the maximum zoom ratio of the PTZ camera 105 is reached, the image is no longer enlarged.

The PTZ camera 105 analyzes the enlarged image of the PTZ camera 105 and detects detailed information (e.g., vehicle model, license plate number, gender, clothing, etc.) of the object being tracked (S405).

Additionally, the PTZ camera 105 compresses the image in the object bounding box range into a JPEG image based on the results of analyzing the video of the PTZ camera 105 (S406).

The PTZ camera 105 can acquire bounding box information of the detected object, event information generated by the detected object, etc. based on the results of analyzing the images of the fixed cameras 101 to 104 (S407).

Additionally, the PTZ camera 105 displays the object's bounding box by overlaying it on the entire screen of the video frames of the fixed cameras 101 to 104 and then compresses it into a JPEG image (S408).

Subsequently, the PTZ camera 105 may transmit the information acquired in step S407, the image acquired in step S408, the information acquired in step S405, and the image acquired in step S406 (for example, by mixing them) to an external device such as a control server. (S409).

In addition to the above, the PTZ camera 105 of FIG. 1 or a video surveillance device mounted inside the PTZ camera 105 or operating externally connected can perform various operations, and other details have been sufficiently explained previously. I would like to replace it with contents.

Figure 5 is a flowchart showing the operating process of a PTZ camera according to the third embodiment of the present invention, and more precisely, a flowchart for explaining a method of inputting location information according to pan/tilt/zoom.

For convenience of explanation, referring to FIG. 5 together with FIG. 1, the PTZ camera 105 displays a cross (+) sign by overlaying it at the exact center of the screen of the PTZ camera 105 (S501).

In addition, the PTZ camera 105 adjusts the pan/tilt of the PTZ camera 105 according to the user's interface, etc. to select a specific location on the ground as a feature point, and the image of the PTZ camera 105 so that the feature point matches the cross mark on the screen. Move (S502). A feature point is a point used to describe the location of the object being tracked, and is not intended to provide location coordinates such as accurate direction and distance. It is information used to describe the location of the object as “near xxx,” etc. In the case of an intersection, feature points can be selected by designating one for each intersection and each road branching from the intersection, and if the road shown in the image is long, feature points can be selected by adding feature points at intervals.

Next, the PTZ camera 105 can input and save a name that describes the feature point specified by the user interface (S503).

In addition, the PTZ camera 105 analyzes the image of the PTZ camera 105 through an interface with the user, repeats the above steps until a sufficient number of feature points are designated to represent the location information of the detected object, and then collects the information. Save (S504).

In addition to the above, the PTZ camera 105 of FIG. 1 or a video surveillance device mounted inside the PTZ camera 105 or operating externally connected can perform various operations, and other details have been sufficiently explained previously. I would like to replace it with contents.

Figure 6 is a flowchart showing the operating process of a PTZ camera according to the fourth embodiment of the present invention, and more precisely, it is a flowchart explaining a method of obtaining and transmitting location information of an object.

For convenience of explanation, referring to FIG. 6 together with FIG. 1, the PTZ camera 105 of FIG. 1 according to an embodiment of the present invention is an object detected by analyzing the images of the fixed cameras 101 to 104 or the PTZ camera 105. is tracked using the pan/tilt/zoom function of the PTZ camera 105 (S601).

In addition, the PTZ camera 105 searches for the feature point closest to the location of the part where the object being tracked is in contact with the ground based on the results of analyzing the image of the PTZ camera 105 among the stored feature points as described in FIG. 5 (S602). .

Next, the PTZ camera 105 checks whether the feature point closest to the tracking object was first discovered (S603). Unless it is discovered for the first time, it is checked whether the feature point closest to the tracking object is the same as the feature point discovered previously. If the feature point is discovered for the first time or is different from a previously discovered feature point, step S604 is performed.

The PTZ camera 105 can determine the searched feature point as the location of the object and transmit that information to an external device (S604).

Next, the PTZ camera 105 checks whether object tracking has ended and, if object tracking has ended, may terminate (the operation of the system or camera) (S605).

In addition to the above, the PTZ camera 105 of FIG. 1 or a video surveillance device mounted inside or externally connected to the PTZ camera 105 can perform various operations, and other details have been sufficiently explained previously. I would like to replace it with .

Figure 7 is a block diagram illustrating the detailed structure of a video surveillance device according to an embodiment of the present invention.

As shown in FIG. 7, the video surveillance device 105' according to an embodiment of the present invention is configured to be mounted inside the PTZ camera 105 of FIG. 1, or mounted outside the PTZ camera 105. It is a device that can operate in conjunction (e.g., edge device, etc.) and includes part or all of a communication interface unit 700, a control unit 710, an image analysis unit 720, and a storage unit 730.

Here, "including part or all" means that the video surveillance device 105' is configured by omitting some components, such as the storage unit 730, or that some components, such as the video analysis unit 720, are configured by the control unit ( 710), it means that it can be configured by being integrated with other components such as

The communication interface unit 700 can communicate with external devices such as the fixed cameras 101 to 104 of FIG. 1 and a control server, respectively. For example, when the video surveillance device 105' is mounted inside the PTZ camera 105 of FIG. 1, the communication interface unit 700 may be used by improving the communication module previously used for remote control. will be. The communication interface unit 700 also provides compressed data in the case of the communication network 106 of FIG. 1 in order to encode the image in which an event occurs, that is, transmit compressed data to an external device such as a control server, under the control of the control unit 710. You will be able to.

The control unit 710 may perform overall control operations of the communication interface unit 700, image analysis unit 720, and storage unit 730 of FIG. 7. Furthermore, it may be responsible for the PTZ control function of the PTZ camera 105. For example, in order to analyze the captured images provided from the fixed cameras 101 to 104, the control unit 710 temporarily stores the captured images in the storage unit 730, retrieves them, and provides them to the image analysis unit 720. You can. Additionally, the control unit 710 may control the PTZ control operation of the PTZ camera 105 at the request of the video analysis unit 720. Typically, the control unit 710 controls the lens (or the focus of the lens) of the PTZ camera 105 when the tracked object deviates from the angle of view of the fixed cameras 101 to 104 as a result of analysis of the captured image, that is, the camera is captured under PTZ control. It can operate to analyze the captured video and determine whether the tracking object is detected in the captured video. In this process, the control unit 710 can receive images captured by the PTZ camera 105 and provide them to the image analysis unit 720 for image analysis. Of course, in addition to this, the control unit 710 can perform various operations in conjunction with the image analysis unit 720, and since the details have been sufficiently explained above, those contents will be used instead.

The image analysis unit 720 may perform the operation of the PTZ camera 105 previously described in FIGS. 3 to 6 . In other words, the image analysis unit 720 can analyze images captured by the fixed cameras 101 to 104 as well as images captured by the PTZ camera 105. Of course, the PTZ camera 105 can be viewed as operating in conjunction with the fixed cameras 101 to 104. For example, when the tracked object deviates from the field of view of the fixed cameras 101 to 104, the PTZ camera 105 may operate, and the image captured by the PTZ camera 105 may be analyzed by the image analysis unit 720. By analyzing in , it is possible to have continuity in tracking the tracking object.

In addition, the image analysis unit 720 may perform an operation to generate detailed information or location information of the tracking object by analyzing the captured image captured by PTZ operation in the PTZ camera 105, etc. If overall information to determine events occurring between detected or tracked objects is obtained through video analysis of the fixed cameras 101 to 104, detailed information such as gender, clothing, vehicle model, and license plate number of the tracked object may be obtained. It can be information. Additionally, the location information may be the location information of the object being tracked, which is acquired based on location information according to preset pan, tilt, and zoom. Location information is information used to describe the location of an object, such as “around xxx.” Location information can be acquired through the PTZ image of the PTZ camera 105 based on feature points of a specific location on the ground that are programmatically set in advance by the user interface.

Furthermore, the image analysis unit 720 may perform an operation to compress the image of the tracking object when an event occurs in the tracking object. To this end, the image analysis unit 720 may be configured to include an encoder (or encoder, compressor). The compressed data of the captured image in which an event occurred may be requested to the control unit 710 to be transmitted to an external device.

The storage unit 730 may temporarily store various types of data processed under the control of the control unit 710. The storage unit 730 not only stores image data of captured images provided by fixed cameras 101 to 104, but also temporarily stores image data of captured images captured by the PTZ camera 105 and then retrieves them for image analysis. It can be provided to unit 720.

In addition to the above, the communication interface unit 700, control unit 710, image analysis unit 720, and storage unit 730 of FIG. 7 can perform various operations, and other details have been sufficiently explained previously. I want to replace it with fields.

The communication interface unit 700, control unit 710, image analysis unit 720, and storage unit 730 of FIG. 7 according to an embodiment of the present invention are composed of hardware modules that are physically separated from each other, but each module has an internal It will be possible to store software to perform the above operations and execute it. However, the software is a set of software modules, and each module can be formed of hardware, so there will be no particular limitation on the configuration of software or hardware. For example, the storage unit 730 may be hardware, such as storage or memory. However, since it is possible to store information through software (repository), the above content will not be specifically limited.

Meanwhile, as another embodiment of the present invention, the control unit 710 may include a CPU and memory, and may be formed as a single chip. The CPU includes a control circuit, an arithmetic unit (ALU), an instruction interpretation unit, and a registry, and the memory may include RAM. The control circuit performs control operations, the operation unit performs operations on binary bit information, and the command interpretation unit includes an interpreter or compiler, which can convert high-level language into machine language and machine language into high-level language. , the registry may be involved in software data storage. According to the above configuration, for example, at the beginning of the operation of the video surveillance device 105', the data operation processing speed is increased by copying the program stored in the video analysis unit 720, loading it into memory, that is, RAM, and then executing it. It can be increased quickly. In the case of deep learning models, they can be loaded into GPU memory rather than RAM and executed by accelerating the execution speed using GPU.

Figure 8 is a flowchart showing the driving process of the video surveillance device of Figure 7.

For convenience of explanation, referring to FIG. 8 together with FIG. 7, the video surveillance device 105' according to an embodiment of the present invention is a device configured inside or outside the PTZ camera 105 of FIG. 1, and can be installed in any area. A captured image of a fixed area in a random area is received from the fixed cameras 101 to 104 (S800). Of course, the video surveillance device 105' can also receive video captured by the PTZ camera 105.

In addition, the video surveillance device 105' analyzes the received images captured by the fixed cameras 101 to 104, detects objects in the images, and controls at least one of the pan, tilt, and zoom of the PTZ camera 105 for the detected objects. Tracking is performed using the function to generate information on the tracking object according to the tracking (S810). Here, information on the tracking object may include detailed information such as the object's gender, clothing, vehicle, and license plate number, and location information indicating the location of the tracking object.

In addition, the video surveillance device 105' of FIG. 7 can perform various operations, and other details have been sufficiently explained with reference to FIGS. 3 to 6, so those will be replaced.

Even though all the components constituting the embodiment of the present invention are described as being combined or operating in combination, the present invention is not necessarily limited to this embodiment. That is, as long as it is within the scope of the purpose of the present invention, all of the components may be operated by selectively combining one or more of them. In addition, although all of the components may be implemented as a single independent hardware, a program module in which some or all of the components are selectively combined to perform some or all of the combined functions in one or more pieces of hardware. It may also be implemented as a computer program having. The codes and code segments that make up the computer program can be easily deduced by a person skilled in the art of the present invention. Such computer programs can be stored in non-transitory computer readable media and read and executed by a computer, thereby implementing embodiments of the present invention.

Here, a non-transitory readable recording medium refers to a medium that stores data semi-permanently and can be read by a device, rather than a medium that stores data for a short period of time, such as a register, cache, or memory. . Specifically, the above-described programs may be stored and provided on non-transitory readable recording media such as CD, DVD, hard disk, Blu-ray disk, USB, memory card, ROM, etc.

Although embodiments of the present invention have been described above with reference to the attached drawings, those skilled in the art will understand that the present invention can be implemented in other specific forms without changing its technical idea or essential features. You will be able to understand it. Therefore, the embodiments described above should be understood in all respects as illustrative and not restrictive.

101, 102, 103, 104: Fixed camera 105: PTZ camera
106: Communication network 201: External video receiver
202: decoder 203, 720: video analysis unit
204: Encoder 205: Image generation unit
206: PTZ control unit 207: PTZ driving unit
700: Communication interface unit 710: Control unit
730: storage unit

Claims (14)

a communication interface unit that receives captured images of a fixed area in a random area from a fixed camera installed in the random area; and
Analyzing the received captured video, detecting an object in the video, and controlling the detected object using at least one of the pan, tilt, and zoom functions of a PTZ (Pan, Tilt, Zoom) camera A control unit that tracks using and acquires information on the tracked object according to the tracking;
The control unit uses, as location information of the tracking object, a preset feature point on the ground using an image captured by the PTZ camera and location description information designated to describe the location of the object in relation to the feature point,
The control unit overlays and displays a cross (+) mark in the exact center of the screen of the PTZ camera, selects a designated location on the ground as a feature point, and moves the image of the PTZ camera so that the selected feature point matches the cross mark on the screen. sets a feature point, and uses a name describing the feature point input through an interface with the user in relation to the set feature point as the location description information,
In the case of an intersection, the control unit selects a feature point by designating one feature point at the intersection and each road branching from the intersection, and selects the feature point by adding feature points at intervals when the length is longer than the standard visible in the image. , video surveillance device. According to paragraph 1,
The control unit analyzes the captured image of the fixed camera to obtain overall information including events of the detected objects, and as information on the tracking object, detailed information including properties of the object and indicating the location of the object. A video surveillance device that acquires location information. According to paragraph 2,
The control unit acquires information on gender or clothing as the detailed information when the tracking object is a human object, and acquires information on the vehicle type or license plate number as the detailed information when the tracking object is a vehicle object. . delete According to paragraph 1,
The control unit is a video surveillance device that continuously tracks the tracking object by analyzing images captured by the PTZ camera when the tracking object leaves the field of view of the fixed camera. According to paragraph 1,
The control unit compresses a full-screen image containing the tracking object and an object image of the tracking object, and transmits the compressed data, event information, and detailed information of the tracking object to an external device of the manager. Video surveillance device. A communication interface unit receiving a captured image of a fixed area in a random area from a fixed camera installed in the random area; and
The control unit analyzes the received captured image, detects an object in the image, and applies the detected object to at least one of pan, tilt, and zoom of a PTZ (Pan, Tilt, Zoom) camera. Tracking using the control function of and acquiring information on the tracked object according to the tracking;
Using a preset feature point on the ground using an image captured by the PTZ camera as location information of the tracked object and location description information designated to describe the location of the object in relation to the feature point;
After overlaying and displaying a cross (+) mark in the exact center of the screen of the PTZ camera, a designated location on the ground is selected as a feature point, and the feature point is set by moving the image of the PTZ camera so that the selected feature point matches the cross mark on the screen. and using a name describing the feature point input through an interface with a user in relation to the set feature point as the location description information; and
In the case of an intersection, a feature point is selected by designating one feature point at the intersection and each road branching from the intersection, and when the feature point is longer than the standard length visible in the image, a feature point is selected by adding feature points at intervals;
A method of driving a video surveillance device, including further. In clause 7,
The acquisition step is,
By analyzing the captured image of the fixed camera, overall information including events of the detected objects is obtained, and detailed information including the properties of the object and location information indicating the location of the object are acquired as information of the tracked object. How to drive a video surveillance device. According to clause 8,
The acquisition step is,
A method of driving a video surveillance device, wherein when the tracking object is a human object, information on gender or clothing is acquired as the detailed information, and when the tracking object is a vehicle object, information on a car model or vehicle number is acquired as the detailed information. delete In clause 7,
The acquisition step is,
Continuously tracking the tracking object by analyzing images captured by the PTZ camera when the tracking object deviates from the field of view of the fixed camera. In clause 7,
The control unit compresses a full-screen image including the tracking object and an object image of the tracking object, and transmits the compressed data, event information, and detailed information of the tracking object to an external device of the manager. A method of driving a video surveillance device, further comprising ;. A fixed camera installed in a random area to photograph a fixed area; and
Detects an object in the video by analyzing the video captured by the fixed camera, and tracks the detected object using at least one control function among the pan, tilt, and zoom of the PTZ camera. Includes a video surveillance device that acquires information on tracked objects according to,
The video surveillance device,
As the location information of the tracked object, a preset feature point on the ground using the captured image of the PTZ camera and a location description information designated to describe the location of the object in relation to the feature point are used,
The video surveillance device,
After overlaying and displaying a cross (+) mark in the exact center of the screen of the PTZ camera, a designated location on the ground is selected as a feature point, and the feature point is set by moving the image of the PTZ camera so that the selected feature point matches the cross mark on the screen. In relation to the set feature point, a name describing the feature point entered through an interface with the user is used as the location description information,
The video surveillance device,
In the case of an intersection, a feature point is selected by designating one feature at the intersection and each road branching from the intersection, and when the length visible in the video is longer than the standard, feature points are selected by adding feature points at intervals. . A computer-readable recording medium that stores a program for executing a method of driving a video surveillance device,
The driving method of the video surveillance device is,
Receiving a captured image of a fixed area in a random area from a fixed camera installed in the random area; and
The received captured video is analyzed to detect an object in the video, and the detected object is tracked using at least one control function among the pan, tilt, and zoom of the PTZ camera. Executing a step of acquiring information on the tracking object according to the
Using a preset feature point on the ground using an image captured by the PTZ camera as location information of the tracked object and location description information designated to describe the location of the object in relation to the feature point;
After overlaying and displaying a cross (+) mark in the exact center of the screen of the PTZ camera, a designated location on the ground is selected as a feature point, and the feature point is set by moving the image of the PTZ camera so that the selected feature point matches the cross mark on the screen. and using a name describing the feature point input through an interface with a user in relation to the set feature point as the location description information; and
In the case of an intersection, a feature point is selected by designating one feature point at the intersection and each road branching from the intersection, and when the feature point is longer than the standard length visible in the image, a feature point is selected by adding feature points at intervals;
Further executable, computer-readable recording medium.

Images