KR102631391B1 - Crowd monitoring apparatus and method for monitoring crowd - Google Patents

Abstract

A crowd monitoring device and method are provided. The crowd monitoring device is connected through a communication network with camera devices placed within the target area. The crowd monitoring device includes a data collection unit that collects image data from the camera devices, identifies a target object from the image data of each of the camera devices based on a machine learning object detection method, and provides information about the identified target object. a target object detection unit that generates object data including, a detection unit that generates crowd detection data including information on whether a crowd occurs in the target area based on the object data, and crowd alarm data based on the crowd detection data. It includes a crowd alarm unit that outputs.

Description

Crowd monitoring apparatus and method {CROWD MONITORING APPARATUS AND METHOD FOR MONITORING CROWD}

The present invention relates to crowd monitoring devices and methods.

Recently, as gatherings such as rallies and festivals are held freely, crowds are becoming more frequent within limited areas. If a crowd occurs in a small area, there is a problem of increased security vulnerability and an increased possibility of accidents due to bottlenecks.

Accordingly, the demand for real-time monitoring of crowds is increasing.

Meanwhile, a control system using closed circuit television (CCTV) is widely used to prevent crimes and secure post-mortem evidence.

However, video data acquired from multiple CCTV cameras can generally be displayed in real time on the display devices of traffic control personnel, or stored on a server and used as evidence after the fact when necessary.

Accordingly, attempts are continuing to expand the areas of use of image data acquired from CCTV cameras.

The problem to be solved by the present invention is to provide a device and method for monitoring the presence of crowds in a target area based on image data acquired from a plurality of CCTV cameras installed in the target area.

The problems of the present invention are not limited to the problems mentioned above, and other technical problems not mentioned will be clearly understood by those skilled in the art from the description below.

Crowd monitoring devices according to embodiments for solving the above problems are connected to camera devices placed in the target area through a communication network. The crowd monitoring device includes a data collection unit that collects image data from the camera devices; Based on a machine learning object detection method, a target object detection unit that identifies a target object from image data of each of the camera devices and generates object data including information about the identified target object; a detection unit that generates crowd detection data including information on whether a crowd occurs in the target area, based on the object data; and a crowd alarm unit that outputs crowd alarm data based on the crowd detection data.

The detection unit may count the number of target objects identified from the image data based on the object data, and generate the crowd detection data if the number of target objects is greater than or equal to the crowd threshold number.

The detection unit detects the distance between target objects identified from the image data based on the object data, and when the distance between the target objects is less than a threshold distance is more than a threshold number, the density of the crowd is determined. Dense detection data containing information can be generated. The crowd alarm unit may output the crowd alarm data based on at least one of the crowd detection data and the crowding detection data.

The detection unit may generate movement data including information on the movement speed and direction of movement of each target object identified from the image data, based on the object data.

In addition, the crowd monitoring device may further include a crowd prediction unit that generates crowd prediction data including information about the possibility of crowd occurrence in a bottleneck management area among the target areas, based on the movement line data. The crowd alarm unit may output the crowd alarm data based on at least one of the crowd detection data and the crowd prediction data.

The crowd monitoring method according to embodiments for solving the above problem includes collecting image data from each camera device deployed in the target area; Based on a machine learning object detection method, identifying a target object from image data of each of the camera devices and generating object data including information about the identified target object; Based on the object data, generating crowd detection data including information on whether a crowd occurs in the target area; and outputting crowd alarm data based on the crowd detection data.

Generating the crowd sensing data may include counting the number of target objects identified from the image data based on the object data; and generating the crowd detection data if the number of target objects is greater than or equal to the crowd threshold number.

The crowd monitoring method includes detecting distances between target objects identified from the image data, based on the object data; And if the distance between the target objects is less than the threshold distance or more than the threshold number, generating crowding detection data including information about the density of the crowd may be further included. The outputting the crowd alarm data may include outputting the crowd alarm data based on the crowd detection data and the crowding detection data.

The crowd monitoring method includes generating movement data including information on the movement speed and direction of movement of each target object identified from the image data, based on the object data; And based on the movement data, it may further include generating crowd prediction data including information about the possibility of crowds occurring in a bottleneck management area among the target areas. The outputting the crowd alarm data may include outputting the crowd alarm data based on the crowd detection data and the crowd prediction data.

Specific details of other embodiments are included in the detailed description and drawings.

A crowd monitoring device according to embodiments includes a target object detection unit that identifies a target object from image data of each camera device and generates object data based on a machine learning object detection method, and whether a crowd occurs in the target area based on the object data. It includes a detection unit that generates crowd detection data including information about, and a crowd alarm unit that outputs crowd alarm data based on the crowd detection data.

In this way, according to embodiments, target objects among image data are identified using a machine learning object detection method, and crowd detection data is generated based on object data including information about the identified target object, thereby determining whether a crowd occurs. The reliability and accuracy of sensing monitoring devices can be improved.

Effects according to the embodiments are not limited to the contents exemplified above, and further various effects are included in the present specification.

1 is a block diagram showing a crowd monitoring system according to embodiments.
Figure 2 is a block diagram showing the crowd monitoring device of Figure 1.
3 is a flowchart showing a crowd monitoring method according to embodiments.
FIG. 4 is image data showing an example of the step of detecting object data in FIG. 3.
Figure 5 is a flowchart showing a crowd monitoring method according to one embodiment.
Figure 6 is a flowchart showing a crowd monitoring method according to another embodiment.

Since the present invention can make various changes and have various embodiments, specific embodiments will be illustrated in the drawings and described in detail in the detailed description. This is not intended to limit the present invention to specific embodiments, but may be understood to include all changes, equivalents, and substitutes included in the spirit and technical scope of the present invention.

In describing the present invention, terms such as first and second may be used to describe various components, but the components may not be limited by the terms. The above terms may be used only for the purpose of distinguishing one component from another. For example, a first component may be named a second component, and similarly, the second component may also be named a first component without departing from the scope of the present invention.

The term “and/or” may include any of a plurality of related stated items or a combination of a plurality of related stated items.

When a component is said to be "connected" or "connected" to another component, it means that it may be directly connected to or connected to that other component, but that other components may also exist in between. It can be understood. On the other hand, when a component is referred to as being “directly connected” or “directly connected” to another component, it can be understood that there are no other components in between.

The terms used in this document are only used to describe specific embodiments and are not intended to limit the invention. Singular expressions may include plural expressions, unless the context clearly indicates otherwise.

In this document, terms such as “comprise” or “have” are intended to designate the presence of features, numbers, steps, operations, components, parts, or combinations thereof described in the specification, and one or more other features. It can be understood that it does not exclude in advance the possibility of the existence or addition of elements, numbers, steps, operations, components, parts, or combinations thereof.

Unless otherwise defined, all terms used herein, including technical or scientific terms, may have the same meaning as commonly understood by a person of ordinary skill in the technical field to which the present invention pertains. Terms defined in commonly used dictionaries can be interpreted as having meanings consistent with the meanings they have in the context of related technologies, and unless clearly defined in this application, are interpreted as having an ideal or excessively formal meaning. It may not work.

In addition, the embodiments disclosed in this document are provided to provide a more complete explanation to those with average knowledge in the art, and the shapes and sizes of elements in the drawings may be exaggerated for clearer explanation.

In describing the present invention in this document, if it is determined that a detailed description of a related known function or configuration may unnecessarily obscure the gist of the present invention, the detailed description may be omitted.

Various embodiments described in this document may be implemented in a recording medium readable by a computer or similar device using software, hardware, or a combination thereof.

According to hardware implementation, embodiments of the present invention include ASICs (application specific integrated circuits), DSPs (digital signal processors), DSPDs (digital signal processing devices), PLDs (programmable logic devices), and FPGAs (field programmable gate arrays). It may be implemented using at least one of processors, controllers, micro-controllers, microprocessors, and electrical units for performing functions.

Meanwhile, according to software implementation, embodiments such as procedures or functions in the present invention may be implemented with a separate software module that performs at least one function or operation.

Hereinafter, specific embodiments will be described with reference to the attached drawings.

1 is a block diagram showing a crowd monitoring system according to embodiments.

Referring to FIG. 1, the crowd monitoring system according to embodiments includes a crowd monitoring device 100, camera devices 200 installed in the target area (IMD1 to IMDn, n is a natural number of 2 or more), and a safety alarm device 300. ) may include.

The crowd monitoring device 100 may be linked to the camera devices 200 and the safety alarm device 300 through a wired or wireless communication network 400.

The network 400 represents a connection structure that allows information exchange between nodes, such as a plurality of terminals and servers. The network may include a local area network (LAN), a wide area network (WAN), the World Wide Web (WWW), a wired or wireless data communication network, a telephone network, or a wired or wireless television network. Wireless data communication networks include 3G, 4G, 5G, 3GPP (3rd Generation Partnership Project), 5GPP (5th Generation Partnership Project), LTE (Long Term Evolution), WIMAX (World Interoperability for Microwave Access), Wi-Fi, and Internet. (Internet), LAN (Local Area Network), Wireless LAN (Wireless Local Area Network), WAN (Wide Area Network), PAN (Personal Area Network), RF (Radio Frequency), Bluetooth (Bluetooth) network, NFC (Near- Field Communication) network, satellite broadcasting network, analog broadcasting network, DMB (Digital Multimedia Broadcasting) network, etc.

The crowd monitoring device 100 collects video data from each of the camera devices 200 installed in the target area, monitors whether a crowd occurs in the target area based on the collected video data, and provides information about whether a crowd occurs. Crowd sensing data can be generated including:

The crowd monitoring device 100 may output crowd alarm data based on crowd detection data.

The camera devices 200 may be closed-circuit television (CCTV) cameras that acquire image data in real time and transmit the acquired image data only to at least one specific server.

Each of the camera devices 200 is distributed within the target area and can generate and transmit image data captured in the target area in real time.

Image data from the camera devices 200 includes image information, time information at the time each camera device (IMD1 to IMDn) acquired the image, location information at the point where the camera devices (IMD1 to IMDn) are placed, and camera device information. (IMD1 to IMDn) may further include at least one of the location information of the area being photographed.

According to embodiments, the crowd monitoring device 100 may be directly linked to the camera devices 200.

Alternatively, according to embodiments, the crowd monitoring device 100 may be associated with a specific server associated with the camera devices 200.

When the safety alarm device 300 receives crowd alarm data from the crowd monitoring device 100, it can send an alarm notifying the danger caused by the crowd to the control system in charge of security and safety. Safety alarm device 300 may be part of a control system.

Figure 2 is a block diagram showing the crowd monitoring device of Figure 1.

Referring to FIG. 2, the crowd monitoring device 100 according to embodiments includes a data collection unit 110 that collects image data from camera devices 200 placed in the target area, based on a machine learning object detection method. A target object detection unit 120 identifies a target object from the image data of each of the camera devices 200 and generates object data including information about the identified target object, and determines whether a crowd occurs in the target area based on the object data. It may include a detection unit 130 that generates crowd detection data including information about, and a crowd alarm unit 140 that outputs crowd alarm data based on the crowd detection data.

The data collection unit 110 may be directly connected to the camera devices 200 through a wired or wireless network 400.

Alternatively, the data collection unit 110 may be connected to a specific server associated with the camera devices 200 through a wired or wireless network.

The data collection unit 110 may classify the collected image data based on the location information of the collected image data. As an example, based on the location information of the area captured by the camera devices (IMD1 to IMDn), image data acquired by some of the camera devices 200 adjacent to the bottleneck management area may be classified for monitoring of the bottleneck management area. You can.

The bottleneck management area may include intersection areas where three or more roads converge, open space areas where rallies are held, and road areas connected to public transportation hubs such as subway stations.

The target object detection unit 120 may identify the target object from image data based on a machine learning object detection method.

The machine learning object detection method is a box segmentation method that detects objects by segmented block area, a method of detecting objects by area or class using a class-specific probability mapping method, and a method of detecting objects by area or class by setting a region or area of interest to detect blocks and objects within the area of interest. Detection method, a method of segmenting and detecting objects of the same color in the image through shape detection for each major color such as red, blue, yellow, and green, and a method of detecting objects of the same color in the image by analyzing the segmentation block and the edge shape for each segmentation block. It may include a method of detecting an object with an edge shape.

The target object detector 120 may identify objects included in the image data based on a machine learning object detection method and then identify the target object among the objects.

Objects may include human objects such as faces, heads, hands, arms, and legs, transportation objects such as vehicles and bicycles, belongings objects such as hats, shoes, and bags, and background objects such as signs, buildings, and signs. there is. However, this is only an example, and the types of objects may be set in various ways depending on the area where the camera devices 200 are installed.

According to embodiments, the target object may be a human object.

The target object detector 120 may generate object data including information about the identified target object.

Information about the target object may include object identification information such as the identification ID of the target object, the location of the target object, and the size of the target object.

Alternatively, the information about the target object may further include object location information, such as the location of the target object.

Alternatively, the information about the target object may further include object movement information such as the movement speed and direction of movement of the target object.

In this way, as object data is generated by identifying a target object from video data using a machine learning object detection method, the detection accuracy of the target object can be gradually improved, and thus the accuracy and reliability of crowd monitoring can be improved.

The detection unit 130 may generate crowd detection data including information on whether a crowd occurs in the target area, based on the object data.

According to embodiments, the detector 130 may count the number of target objects identified from image data based on object identification information among object data.

Additionally, the detection unit 130 may generate crowd detection data if the number of target objects is greater than or equal to the crowd threshold number.

According to one embodiment, the detector 130 may detect the distance between target objects identified from image data based on object identification information and object location information among object data.

In addition, if the distance between target objects is less than the critical distance, the detection unit 130 may generate crowding detection data including information about the density of the crowd.

According to another embodiment, the crowd monitoring device 100 may further include a crowd prediction unit 150.

According to another embodiment, the detection unit 130 provides information on the movement speed and direction of movement of each target object identified from the image data based on object identification information, object location information, and object movement information among the object data. It is possible to generate movement data including:

The crowd prediction unit 150 may generate crowd prediction data including information about the possibility of crowd occurrence in a bottleneck management area among the target areas, based on the movement data.

The crowd prediction unit 150 may generate crowd prediction data by predicting when more than a critical number of target objects will reach the bottleneck management area based on movement line data.

The crowd prediction data may include prediction information about the time when a crowd occurs in the bottleneck management area, the time when the crowd in the bottleneck management area grows beyond a critical value, and the size of the crowd in the bottleneck management area.

Meanwhile, the detection unit 130 may correct object data based on the moving line data.

That is, the detector 130 may identify a stationary object that does not move during a critical period among target objects identified from image data, based on object data and movement data. The detection unit 130 may perform correction to remove the identified stationary object from the object data.

In this way, when generating crowd detection data, crowd detection data, and crowd prediction data for crowd monitoring, the influence of stationary objects that are not elements that change the crowd can be removed. As a result, the accuracy and reliability of crowd monitoring can be improved.

The crowd alarm unit 140 may output crowd alarm data based on at least one of crowd detection data and crowding detection data generated by the detection unit 130.

Alternatively, the crowd alarm unit 140 may output crowd alarm data based on at least one of crowd detection data and crowd prediction data provided by the detection unit 130.

Crowd alarm data from the crowd alarm unit 140 may be transmitted to the safety alarm device 300.

3 is a flowchart showing a crowd monitoring method according to embodiments. FIG. 4 is image data showing an example of the step of detecting object data in FIG. 3.

Referring to FIG. 3, the crowd monitoring method according to embodiments includes collecting image data from each of the camera devices 200 deployed in the target area (S10), based on a machine learning object detection method, the camera device Identifying a target object from each of the image data and generating object data including information about the identified target object (S20), based on the object data, detecting a crowd including information about whether a crowd occurs in the target area It may include generating data (S31, S32) and outputting crowd alarm data based on crowd detection data (S40).

According to embodiments, the step of generating crowd detection data includes counting the number of target objects identified from the image data based on the object data (S31), and if the number of target objects is more than the crowd threshold number, the crowd It may include generating sensing data (S32).

In the step of collecting video data (S10), the data collection unit 110 of the crowd monitoring device 100 may receive video data from camera devices 200 placed in the target area through the wired and wireless network 400. there is.

The data collection unit 110 may receive image data from the camera devices 200 at a predetermined period.

Alternatively, the data collection unit 110 may receive image data from the camera devices 200 in real time.

In the step of generating object data (S20), the target object detection unit 120 of the crowd monitoring device 100 may identify the target object from the image data based on a machine learning object detection method.

Referring to FIG. 4, the target object detector 120 may identify objects (person, backpack, handbag, car, traffic light) included in the image data and then detect the target object (person) among the objects. .

According to embodiments, the target object may be a person.

Object data may include information about the identified target object.

Information about the target object may include object identification information such as the identification ID of the target object, the location of the target object, and the size of the target object.

In the step of generating crowd detection data (S30), the detection unit 130 of the crowd monitoring device 100 counts the number of target objects identified from the image data based on object identification information among the object data (S31), If the number of counted target objects is greater than the crowd threshold number, crowd detection data can be generated. (S32)

The crowd threshold number may be set based on the width of the area displayed by the image data of each of the camera devices 300 and the number and width of roads arranged in the area.

The crowd threshold number may be set further based on the moving speed of the target object. For example, the faster the average moving speed of target objects, the higher the crowd threshold number may be.

In step S40 of outputting crowd alarm data, the crowd alarm unit 140 of the crowd monitoring device 100 may output crowd alarm data based on crowd detection data.

Figure 5 is a flowchart showing a crowd monitoring method according to one embodiment.

Referring to FIG. 5, the crowd monitoring method according to one embodiment includes the steps of generating crowding detection data (S33, S34) after the crowding detection data generating steps (S31, S32), Since it is substantially the same as the crowd monitoring method of FIG. 3, redundant description will be omitted below.

According to one embodiment, generating crowding detection data includes detecting distances between target objects identified from image data based on object data (S33), and when the distance between target objects is less than a threshold distance. If the threshold number or more is derived, a step (S34) of generating crowding detection data including information about the density of the crowd may be included.

In step S33 of detecting the distance between target objects, the detection unit 130 may detect the distance between target objects identified from the image data based on object identification information and object location information among the object data.

Then, in step S34 of generating crowding detection data, the detection unit 130 counts cases where the distance between target objects is less than the threshold distance, and when the number of cases where the distance between target objects is less than the threshold distance is greater than or equal to the threshold number. , crowding detection data containing information about crowd density can be generated.

Thereafter, in step S40 of outputting crowd alarm data, the crowd alarm unit 140 may output crowd alarm data based on at least one of crowd detection data and crowding detection data.

Figure 6 is a flowchart showing a crowd monitoring method according to another embodiment.

Referring to FIG. 6, the crowd monitoring method according to another embodiment is similar to that of FIGS. 3 and 3, except that it further includes a step of generating movement data (S35) and a step of generating crowd prediction data (S50). Since it is virtually the same as the crowd monitoring method in 4, redundant explanations are omitted below.

According to another embodiment, the crowd monitoring device 100 may further include a crowd prediction unit 150.

In step S35 of generating movement data, the detection unit 130 determines the movement speed and movement direction of each target object identified from the image data based on object identification information, object location information, and object movement information among the object data. Movement data including information about can be generated.

In the step of generating crowd prediction data (S50), the crowd prediction unit 150 may generate crowd prediction data including information about the possibility of crowd occurrence in a bottleneck management area among the target areas, based on the movement line data.

Thereafter, in step S40 of outputting crowd alarm data, the crowd alarm unit 140 may output crowd alarm data based on at least one of crowd detection data and crowd prediction data.

In addition, after the step of generating movement data (S35), the detection unit 130 may correct the object data based on the movement data.

That is, the detector 130 may identify a stationary object that does not move during a critical period among target objects identified from image data, based on object data and movement data. The detection unit 130 may perform correction to remove the identified stationary object from the object data.

In this way, when generating crowd detection data, crowd detection data, and crowd prediction data for crowd monitoring, the influence of stationary objects that are not elements that change the crowd can be removed. As a result, the accuracy and reliability of crowd monitoring can be improved.

Although embodiments of the present invention have been described above with reference to the attached drawings, those skilled in the art will understand that the present invention can be implemented in other specific forms without changing its technical idea or essential features. You will be able to understand it. Therefore, the embodiments described above should be understood in all respects as illustrative and not restrictive.

100: crowd monitoring device 200: camera devices
300: Safety alarm device

Claims (9)

In the crowd monitoring device connected through a communication network with camera devices placed within the target area,
a data collection unit that collects image data from the camera devices;
Based on a machine learning object detection method, a target object detection unit that identifies target objects from image data of each of the camera devices and generates object data including information on the identified target objects;
a detection unit that generates crowd detection data including information on whether a crowd occurs in the target area, based on the object data; and
A crowd alarm unit that outputs crowd alarm data based on the crowd detection data,
The sensing unit,
Based on the object data, further generate movement data including information on the movement speed and direction of movement of each target object identified from the image data,
Based on the object data and the movement line data, identify a stationary object that does not move during a critical period among the target objects,
Performing correction to remove the stationary object on the object data,
Generating the crowd detection data based on the corrected object data
Crowd monitoring device. According to claim 1,
The sensing unit,
Based on the object data, count the number of target objects identified from the image data,
A crowd monitoring device that generates the crowd detection data when the number of target objects is greater than or equal to the crowd threshold number. According to claim 1,
The sensing unit,
Based on the object data, detecting distances between target objects identified from the image data,
If the distance between the target objects is less than the threshold, the number of cases where the distance between the target objects is less than the threshold is greater than or equal to the threshold, generating crowding detection data including information about the density of the crowd,
The crowd monitoring device outputs the crowd alarm data based on at least one of the crowd detection data and the crowd detection data. delete According to claim 1,
Based on the movement data, it further includes a crowd prediction unit that generates crowd prediction data including information about the possibility of crowd occurrence in a bottleneck management area among the target areas,
The crowd monitoring device outputs the crowd alarm data based on at least one of the crowd detection data and the crowd prediction data. In a method in which a crowd monitoring device connected to camera devices placed in a target area and a communication network monitors whether a crowd occurs in the target area,
collecting image data from each of the camera devices placed within the target area;
Based on a machine learning object detection method, identifying target objects from image data of each of the camera devices and generating object data including information about the identified target objects;
Based on the object data, generating crowd detection data including information on whether a crowd occurs in the target area; and
Outputting crowd alarm data based on the crowd detection data,
The step of generating the crowd sensing data is,
Based on the object data, generating movement data including information on the movement speed and movement direction of each target object identified from the image data;
Identifying a stationary object that does not move during a critical period among the target objects based on the object data and the movement line data;
performing correction to remove the stationary object on the object data; and
Generating the crowd sensing data based on the corrected object data.
Crowd monitoring methods. According to clause 6,
The step of generating the crowd sensing data is,
Counting the number of target objects identified from the image data based on the object data; and
A crowd monitoring method comprising generating the crowd detection data when the number of target objects is greater than or equal to the crowd threshold number. According to clause 6,
Based on the object data, detecting distances between target objects identified from the image data; and
If the distance between the target objects is less than the threshold distance or more than the threshold number, generating crowding detection data including information on the density of the crowd,
The step of outputting the crowd alarm data includes outputting the crowd alarm data based on the crowd detection data and the crowding detection data. According to clause 6,
Based on the movement data, it further includes generating crowd prediction data including information about the possibility of crowd occurrence in a bottleneck management area among the target areas,
The step of outputting the crowd alarm data includes outputting the crowd alarm data based on the crowd detection data and the crowd prediction data.