KR102663282B1 - Crowd density automatic measurement system - Google Patents

Abstract

The present invention relates to an automatic crowd density measurement system that can calculate, predict, and guide crowd density in real time using captured images. More specifically, it relates to an automatic crowd density measurement system that calculates the crowd density in real time by using captured images. At least one photography means for photographing a set area by calculating the crowd density; And identifying people from images captured through the above-mentioned photographing means, counting the total number of people in the captured images by applying different counting methods depending on the distance at which each identified person is located, and counting the total number of people in the captured images based on this. It relates to an automatic crowd density measurement system that includes a processing unit that calculates local crowd density.

Description

Crowd density automatic measurement system {CROWD DENSITY AUTOMATIC MEASUREMENT SYSTEM}

The present invention relates to an automatic crowd density measurement system, and particularly to an automatic crowd density measurement system that can calculate and predict crowd density in real time using captured images.

Recently, there has been increasing interest in crowding, crowd control, and safety issues in cities or areas where large-scale events occur. Accordingly, quickly identifying and managing crowd density is very important to prevent safety accidents.

However, methods to date are usually based on manual surveys or empirical estimates and have limitations in terms of accuracy and efficiency.

Accordingly, we would like to use video data obtained through filming methods to calculate and predict crowd density in real time and present a method to quickly provide guidance on dangerous situations.

(0001) Domestic Patent Publication No. 10-1176947

The technical problem to be solved by the present invention is to provide an automatic crowd density measurement system that can calculate and predict crowd density in real time using image data obtained through a photography means.

Another technical problem that the present invention aims to solve is to provide an automatic crowd density measurement system that can accurately calculate the total number of people by suggesting different counting methods depending on the distance when counting the number of people in an image.

Another technical problem that the present invention aims to solve is to provide an automatic crowd density measurement system that can predict crowd density and detect dangerous situations in real time through captured image data and crowd flow analysis.

The technical problems to be achieved in the present invention are not limited to the technical problems mentioned above, and other technical problems not mentioned will be clearly understood by those skilled in the art from the description below. You will be able to.

The automatic crowd density measurement system of the present invention for achieving the above technical problem calculates the crowd density in real time by counting the number of people from images captured through a photographing means, and includes at least one photographing means for photographing a set area; And identifying people from images captured through the above-mentioned photographing means, counting the total number of people in the captured images by applying different counting methods depending on the distance at which each identified person is located, and counting the total number of people in the captured images based on this. It is characterized by including a processing unit that calculates the crowd density of the area.

In addition, the photographing means includes a wide-angle camera that photographs the entire set area; and at least one pan/tilt/zoom camera that precisely photographs the specific location when the crowd density at a specific location is determined to be greater than a set value in the image captured through the wide-angle camera. do.

In addition, the method for counting the number of people is that, if the distance at which the person is located is greater than the set first separation distance, it is judged to be a long distance and only the head of the person is counted, and if the distance at which the person is located is less than the first set separation distance, it is considered a close distance. It is characterized by counting the entire human body.

In addition, the processing unit is characterized in that it predicts the crowd density by identifying the crowd density in the video together with the crowd flow chart.

In addition, the processing unit is characterized in that it predicts an area where crowd density will increase based on the crowd density and the crowd flow chart.

In addition, the processing unit is characterized in that it identifies roads on which people travel from images captured through the photographing means, and predicts areas where crowd density will increase by reflecting the width of the identified roads.

In addition, when a dangerous situation occurs where the crowd density exceeds a set value, it further includes a display unit that detects and provides guidance.

In addition, the display unit is characterized in that it is placed at least one of a control center that monitors the image through the photographing means or a place that can be recognized by people located in the set area.

The above aspects of the present invention are only some of the preferred embodiments of the present invention, and various embodiments reflecting the technical features of the present invention will be described in detail below by those skilled in the art. It can be derived and understood based on.

The present invention described above has the following effects.

First, the captured video data can be processed to calculate crowd density in real time, allowing the crowd situation to be monitored in real time and immediate response measures taken.

In addition, by identifying people at various distances and presenting a method for accurately counting the number of people according to the distance to them, accurate analysis of crowds occurring near and far is possible.

In addition, it is possible to predict crowd density and detect dangerous situations through video data and crowd flow analysis, so dangerous areas can be predicted in advance and various measures can be taken.

Additionally, the display unit allows the control center or general users to monitor crowd density in real time and take immediate action, enabling faster safety measures.

Additionally, the aforementioned effects can provide great help to field operators and managers in the areas of crowd management and safety management.

The effects that can be obtained from the present invention are not limited to the effects mentioned above, and other effects not mentioned will be clearly understood by those skilled in the art from the description below.

1 is a conceptual diagram of the configuration of an automatic crowd density measurement system according to an embodiment of the present invention.

Hereinafter, some embodiments of the present invention will be described in detail through illustrative drawings. When adding reference numerals to components in each drawing, it should be noted that identical components are given the same reference numerals as much as possible even if they are shown in different drawings. Additionally, when describing embodiments of the present invention, if detailed descriptions of related known configurations or functions are judged to impede understanding of the embodiments of the present invention, the detailed descriptions will be omitted.

Additionally, when describing the components of embodiments of the present invention, terms such as first, second, A, B, (a), and (b) may be used. These terms are only used to distinguish the component from other components, and the nature, sequence, or order of the component is not limited by the term. When a component is described as being "connected," "coupled," or "connected" to another component, that component may be directly connected or connected to that other component, but there is no "interconnection" between each component. It should be understood that they may be “connected,” “coupled,” or “connected.”

Figure 1 is a conceptual diagram according to an embodiment of the automatic crowd density measurement system of the present invention.

First, the automatic crowd density measurement system according to the present invention will be described with reference to FIG. 1 as follows.

The automatic crowd density measurement system of the present invention calculates the crowd density in real time by counting the number of people from images captured through a photographing means, and may be configured to include a photographing means 10 and a processing unit 20.

The photographing means 10 may be composed of a wide-angle camera 11 and a pan/tilt/zoom camera 12. The wide-angle camera 11 can capture images of the entire set area, and the captured images can be transmitted to the processing unit 20 and used to estimate initial crowd density.

The image captured through the wide-angle camera 11 may be transmitted to the processing unit 20.

The processing unit 20 can identify each person by analyzing the received video. Technologies such as facial recognition and body recognition can be used to identify individual people. For this purpose, computer vision technology and image processing algorithms can be used.

Computer vision technology used in the processing unit 20 may include various image processing algorithms and technologies. These technologies can be used to analyze and understand video data.

Computer vision technologies may include object recognition, object tracking, segmentation, optical character recognition, and clustering technologies.

Object recognition technology is a technology that identifies specific objects in images. To this end, learned algorithms and deep learning technology can be used to extract object characteristics and recognize objects by comparing them with a database. For example, facial recognition technology can be used to identify a person's face.

Object tracking technology is a technology that tracks the movement of a specific object in an image in real time. For this purpose, the difference between the previous frame and the current frame can be analyzed and the object's movement path can be predicted. Object tracking can be used to calculate crowd density or analyze the movement of a specific object.

Segmentation technology is a technology that accurately extracts the boundaries of objects from images. Through this, it is possible to distinguish between objects and backgrounds and accurately extract the areas of individual objects. Segmentation can be used to identify individual people, analyze crowd density, etc.

Optical Character Recognition (OCR) technology is a technology that recognizes and extracts text from images. By recognizing text in a specific area from images captured with a wide-angle camera, road names, traffic lights, information signs, etc. can be analyzed and processed.

Clustering technology is a technology that automatically groups data with similar characteristics. It can be used to cluster people in video to identify individuals and calculate crowd density.

The above-mentioned computer vision technologies can be combined or used individually depending on various situations, and the processing unit 20 can utilize these technologies to analyze image data and use them to calculate crowd density.

Meanwhile, the present invention can apply image preprocessing technology. For example, super resolution, low-light enhancement, and defoggy technologies can be applied.

As described above, high-resolution conversion technology is a technology that converts a low-resolution image into a high-resolution image. In general, low-resolution video can occur when the image or video lacks detail or is of low quality. High-resolution conversion technology provides a way to reconstruct these low-resolution images into clearer, higher-resolution images. To achieve this, various algorithms and techniques can be used to estimate high-resolution information in an image or video and generate additional details for low-resolution input.

Low-light improvement technology is a technology that improves images captured in low-light conditions. Videos shot in low-light environments often lack brightness and details are blurred or lost.

Low-light improvement technology can use various technologies to improve the quality of low-light images. For example, the visual quality of low-light images can be improved by adjusting the brightness of the image, applying noise reduction techniques, or correcting details.

Fog removal technology is a technology that improves blurry or hazy images due to weather conditions such as fog or smog. Fog or smog can cause haze in images, reducing visual clarity and contrast. Fog removal technology uses various algorithms and techniques to remove this haze phenomenon and improve the clarity, contrast, and color of the image, such as analyzing and correcting pixel intensity and color differences in the image to remove haze, or using optical absorption models. You can use methods such as applying to remove fog.

These image preprocessing technologies can be useful in image analysis and computer vision tasks by improving the visual quality of images and restoring or correcting detailed information.

After identifying each person, the processing unit 20 can calculate the location and distance of the people.

The following procedure can be performed to calculate the positions and distances of people from images captured through the wide-angle camera 11.

First, the image captured with the wide-angle camera 11 is preprocessed to extract necessary information. This process may include image distortion correction, background removal, noise removal, etc.

The processing unit 20 can recognize and track individuals using the computer vision technology described above. For this purpose, object recognition algorithms and tracking algorithms can be used. Object recognition refers to the process of extracting and identifying people's characteristics, and tracking refers to the process of tracking people's movements in real time.

Individual locations of people can be determined through object recognition and tracking. To do this, you can mark the locations of people in the video or save them as coordinates. Coordinates can usually be expressed in pixel units within an image, creating coordinate data indicating the location of people.

The distance can be calculated using the coordinate data, that is, the location coordinates of people. Distance calculations can generally be performed using the Euclidean distance. Euclidean distance is a method of calculating the straight-line distance between two points, and can use people's location coordinates to calculate the distance between individual people.

Meanwhile, the distance and area to the camera can be calculated using only the captured image.

For example, depth estimation technology is a technology that estimates the distance of an object in an image, through which the distance to the camera can be estimated. Depth estimation technology can be implemented in a variety of ways, and technologies such as real-time depth map generation, stereo vision, and time difference stereo vision can be used. Through this, the depth information of the captured scene can be estimated and the distance to the camera can be calculated.

Additionally, the distance to the camera can be estimated by analyzing the size and ratio of the object in the captured image. For this purpose, comparisons and calculations can be made based on the size of a standard object measured in advance or the average height of a person. This is a method of estimating distance using the relationship between the size and distance of an object.

Additionally, the distance to the camera can be estimated using image processing technology. For this purpose, a method of calculating the distance can be used by analyzing the object size, pixel density, perspective, etc. in the image. This method creates a statistical model for distance based on the size and pixel density of the object and can estimate the distance by analyzing the captured image.

Through the above procedure, the location and distance of each person can be calculated from the image captured through the wide-angle camera 11, and based on this, the number of people can be counted and the crowd density can be calculated.

Crowd density can be calculated based on location information of people extracted from images captured through the wide-angle camera 11. Crowd density can be calculated through the following procedure.

First, the individual location coordinates of people extracted from the wide-angle camera 11 can be utilized, and as described above, can be expressed as pixel coordinates in the image or real space coordinates.

You can define a set area that you want to analyze to calculate crowd density. The set area may be defined differently depending on a given problem or application, and for example, it may be an area set for roads, squares, event locations, etc.

In this way, crowd density can be estimated by calculating the number of people within the set area. Crowd density calculation methods can be applied in various ways and may vary depending on the situation and needs. For example, a method of calculating the number of pixels within a set area from the locations of people or directly counting the number of people located within a set area may be used.

The crowd density calculation result is generally expressed as the number of people or the number of people per unit area, that is, the crowd density can be expressed as the number of people within a set area, or the number of people per unit area.

Through this procedure, the crowd density in a set area can be calculated using the location information of people extracted from images captured through a wide-angle camera, thereby predicting areas where crowd density will increase and providing more accurate crowd management and safety measures based on this. can be established.

By utilizing the wide viewing angle of the wide-angle camera 11, the locations of people within a set area can be identified, and crowd density counting methods can be applied differently depending on the location and distance of the people. For example, people located more than the set first separation distance may be judged as far away and only the head of the person may be counted, while people located less than the first separation distance may be judged as close and the person's entire body may be counted. In other words, in the captured video, people located close by appear relatively large and are easy to identify through image processing, but for people located far away, it is not easy to grasp the overall shape individually, so only the number of heads is relatively easy to identify. It can be identified and counted.

Image processing software can include a variety of technologies and algorithms that can be used for near and far distances. In general, near-field image processing refers to targeting a relatively small area, and long-distance image processing may refers to targeting a large area.

Since detailed object detection and tracking are important at close range, object detection algorithms must be used to identify people or other objects. For example, you can use YOLO (You Only Look Once) or Faster R-CNN (Region-based Convolutional Neural Networks), which are deep learning-based object detection algorithms. For object tracking, visual tracking algorithms can be used to track the movement path of individual objects.

Additionally, image correction and improvement are important in near-field image processing. The quality of the image can be improved by performing distortion correction, color correction, and noise removal, and filtering techniques or image restoration algorithms can be used.

Additionally, computer vision technologies such as object recognition, segmentation, and motion estimation can be used in short-range image processing. Deep learning-based algorithms can be used to identify objects, analyze motion, and extract features.

Additionally, near-field image processing requires the ability to store, analyze, and visually display the analysis results. To achieve this, you can use database systems or graph and chart creation libraries to manage and visualize data.

On the other hand, it may be difficult to accurately detect and track small-sized people or objects at long distances, so different approaches and techniques may be used.

In distant images, the size of objects may be relatively small or the resolution may be low, so to compensate for this, it is necessary to adjust the size of objects in the image and normalize them as necessary. Here, resizing means enlarging or reducing the image, and normalization means standardizing the image and converting it to a consistent size.

It can be difficult to accurately detect small-sized people or objects from a distance. Therefore, in long-distance image processing, objects can be detected using size invariant technology or feature extraction algorithms. For example, feature extraction algorithms such as Scale-Invariant Feature Transform (SIFT) or Speeded-Up Robust Features (SURF) can be used.

Additionally, in long-distance images, objects must be analyzed in consideration of their surroundings, and it is important to understand the size and shape of the object and its relationship with the surrounding background or other objects. For this purpose, context modeling algorithms can be used to detect and classify objects. For example, you can use Conditional Random Field (CRF), which utilizes information from surrounding pixels, for context modeling.

Additionally, in distant images, the resolution is low and detailed information is often lacking. Super Resolution technology is a technique that restores low-resolution images to high-resolution images, and can be used to improve object detection and tracking at a distance. For example, deep learning-based super resolution technology can be used to improve the resolution of distant images.

In this way, the total crowd density can be calculated by calculating the total number of people using different methods for counting the number of people depending on the near and far distances, and through this, the initial crowd density can be estimated.

The initial crowd density estimate using the wide-angle camera 11 can be used for subsequent analysis and prediction.

The pan/tilt/zoom camera 12 can be used to precisely photograph specific points where crowd density increases beyond a set value. If the crowd density detected by the wide-angle camera 11 exceeds the set threshold, the pan/tilt/zoom camera 12 operates to automatically photograph the point and can provide detailed images of the point. .

Meanwhile, when a point occurs where the crowd density value through the wide-angle camera 11 increases beyond a set value, the specific point may be photographed through at least one pan/tilt/zoom camera 12. For example, the specific point can be photographed through two pan/tilt/zoom cameras 12, and if the specific point is photographed from the front through one pan/tilt/zoom camera 12, the other pan/tilt/zoom camera 12 can be photographed from the front. The specific point can be photographed from an angle different from the front through the /tilt/zoom camera 12. To this end, at least a plurality of pan/tilt/zoom cameras 12 may be configured, and each configuration position may be varied. This arrangement can solve the problem of not counting people accurately when they overlap.

Image analysis through the pan/tilt/zoom camera 12 can also apply the image analysis technology for the wide-angle camera 11 described above.

The processing unit 20 can predict the crowd density by analyzing the crowd density and the crowd flow chart together. Through this, the processing unit 20 can predict areas where crowd density will increase, and the display unit 30 can provide guidance and warning when a dangerous situation occurs.

Additionally, the processing unit 20 can identify roads on which people travel from images captured through the photographing means 10. In addition, the width of the identified road can be calculated and the calculated results can be reflected to predict areas where crowd density will increase.

To achieve this, first, objects related to the road are detected in the image using image processing technology. For this purpose, object detection technology, one of the computer vision technologies, can be used. Object detection is a technology that identifies specific objects, such as roads, in an image and displays them as bounding boxes.

Road areas can be extracted based on object detection results. In this step, the bounding box corresponding to the road is identified among the object detection results, and the corresponding area is recognized as a road.

The width of the road can be calculated from the extracted road area. For this purpose, various geometric calculations and pattern analysis techniques can be used to estimate the width of the road. For example, the pixel distance in the horizontal or vertical direction can be measured in the extracted road area, or the road width can be estimated through comparison with the environment around the road. There may be a certain relationship between road width and population density, and the risk can be calculated by combining the estimated road width and population density data. For this purpose, risk values can be assigned according to road width and population density using predefined risk models or rules.

Using this method, roads can be automatically detected in images acquired through the imaging means 10, road widths can be calculated, and reflected in the risk level according to population density.

The display unit 30 can be used to detect a dangerous situation where the crowd density exceeds a set value and provide related information.

The display unit 30 may be placed in at least one of a control center that monitors images captured through the photographing means 10 or a location that can be recognized by people located in a set area.

The display unit 30 may include various types of display devices. For example, an LCD monitor, LED panel, electronic signboard, projector, etc. can be used. These display devices can display captured images, crowd density information, guidance messages about dangerous situations, route guidance, etc.

The display unit 30 may include audio functions such as voice guidance or alarm sounds. Through this, it is possible to provide information visually and at the same time provide auditory guidance.

When the display unit 30 detects a dangerous situation in which the crowd density increases beyond a set value, it can generate an appropriate warning signal. For example, people can be guided through alarm sounds, flashing emergency lights, warning messages, etc.

Additionally, the display unit 30 may transmit a notification of a dangerous situation to a control center or a manager in a set area. This allows you to take timely response actions and, if necessary, contact emergency services or relevant authorities for assistance.

The display unit may be a mobile device carried by people located in an area being photographed through a photographing means.

The display function can be implemented through a specific application installed on a portable mobile device. This application can display captured video, crowd density information, and dangerous situation notifications to the user, and deliver warning messages when necessary.

Additionally, it is possible to provide warnings about dangerous situations by sending notification messages to mobile devices carried by people located in a set area. For this purpose, information can be delivered to the user using push notifications or SMS messages.

In addition, the user's current location can be determined by using the GPS (Global Positioning System) of the mobile device carried or other location-based technology, and crowd density information for that location can be provided. This allows users to check the crowd conditions in their area in real time.

Additionally, the captured video data can be shared and displayed to the user on the screen of a portable mobile device. For this purpose, video can be delivered and displayed in real time using video streaming technology or screen sharing applications.

According to the present invention described above, it is possible to monitor crowd situations in real time and take immediate response measures, enable accurate analysis of crowds occurring near and far, and predict risk areas in advance and take various measures. In addition to enabling faster safety measures, the aforementioned effects can provide great help to field operators and managers in the areas of crowd management and safety management.

In the above, even though all the components constituting the embodiment of the present invention have been described as being combined or operated in combination, the present invention is not necessarily limited to this embodiment. That is, as long as it is within the scope of the purpose of the present invention, all of the components may be operated by selectively combining one or more of them. In addition, terms such as “include,” “comprise,” or “have” described above mean that the corresponding component may be present, unless specifically stated to the contrary, and therefore do not exclude other components. Rather, it should be interpreted as being able to include other components. All terms, including technical or scientific terms, unless otherwise defined, have the same meaning as generally understood by those skilled in the art to which the present invention pertains. Commonly used terms, such as terms defined in a dictionary, should be interpreted as consistent with the contextual meaning of the related technology and, unless explicitly defined in the present invention, should not be interpreted in an idealized or overly formal sense.

The above description is merely an illustrative explanation of the technical idea of the present invention, and various modifications and variations will be possible to those skilled in the art without departing from the essential characteristics of the present invention. Accordingly, the embodiments posted in the present invention are not intended to limit the technical idea of the present invention, but are for illustrative purposes, and the scope of the technical idea of the present invention is not limited by these examples. The scope of protection of the present invention should be interpreted in accordance with the claims below, and all technical ideas within the equivalent scope should be construed as being included in the scope of rights of the present invention.

10: Photography means 11: Wide-angle camera
12: Pan/Tilt/Zoom camera 20: Processing unit
30: display unit

Claims (8)

This method calculates crowd density in real time by counting the number of people from images captured through filming means.
At least one photographing means for photographing a set area; and
People are identified from images captured through the above-mentioned photography means, and the total number of people in the captured images is counted by applying different people counting methods depending on the distance at which each identified person is located, and based on this, the set area is determined. It includes a processing unit that calculates the crowd density of,
The processing unit,
Computer vision technologies, including object recognition, object tracking, segmentation, optical character recognition, and clustering techniques; and
Apply image processing algorithms including high-resolution conversion, low-light improvement, and fog removal technology.
The processing unit,
Predict crowd density by identifying the crowd density in the video and the crowd flow chart.
Predict areas where crowd density will increase based on the crowd density and the crowd flow chart,
Identify roads on which people travel from images captured through the above filming means, and predict areas where crowd density will increase by reflecting the width of the identified roads.
When a dangerous situation occurs where the crowd density exceeds a set value, it further includes a display unit that detects and provides guidance,
The display unit,
Videos of crowded conditions, crowd density information, guidance messages about dangerous situations, and route guidance are displayed visually and audibly through LCD monitors, LED panels, electronic signs, and projectors placed in crowded areas.
It is placed in a place that can be seen by people located in the set area,
The photography means is,
Wide-angle camera that covers the entire set area; and
When the crowd density in a specific place is determined to be greater than a set value in the image captured through the wide-angle camera, it includes at least one pan/tilt/zoom camera that precisely photographs the specific place,
The pan/tilt/zoom camera consists of a plurality of pan/tilt/zoom cameras that photograph the specified location from different angles,
The display unit includes a mobile device carried by people located in an area taking pictures through the photography means,
Crowd density information and dangerous situation notifications are displayed through a specific application installed on the portable mobile device, and images captured through the above-described filming means are also displayed,
The method for counting the number of people is,
If the distance at which the person is located is greater than the set first separation distance, it is judged to be far away and only the person's head is counted,
An automatic crowd density measurement system that counts the entire body of a person by determining that the distance at which the person is located is less than the set first separation distance. delete delete delete delete delete delete delete