KR102633938B1 - Method for crowd density estimation using cctv video and appartaus therefor - Google Patents

Abstract

The present invention relates to a video analysis method using CCTV images, and more specifically, to a new method of calculating crowd density using CCTV images and a device to which such a method is applied. The method for calculating crowd density by video analysis according to an embodiment of the present invention includes a first length related to at least one of the width, length, and height of a person's head and a second length related to the distance between people. Setting steps, obtaining surveillance images of the surveillance area from a photographing device, analyzing the surveillance images to identify at least one human head as an object, a step indicating the size at which the object is displayed in the surveillance images. Obtaining one measurement value, obtaining a second measurement value indicating the distance between the object and at least one adjacent object based on the size displayed in the surveillance image, the ratio between the first measurement value and the first length Calculating the adjacent distance of the object based on the second measurement value and the ratio, classifying the object as a dense object when the adjacent distance of the object is less than or equal to the second length, and It may include calculating the crowd density of the surveillance area based on the number of dense objects identified from the surveillance video.

Description

Crowd density calculation method and device using CCTV video {METHOD FOR CROWD DENSITY ESTIMATION USING CCTV VIDEO AND APPARTAUS THEREFOR}

The present invention relates to a video analysis method using CCTV images, and more specifically, to a new method of calculating crowd density using CCTV images and a device to which such a method is applied.

CCTV is a surveillance system aimed at maintaining security and public order, and is installed and operated in various private and public areas. In particular, many public CCTV systems have been introduced recently for the purposes of crime prevention, traffic safety, and disaster response. To achieve this goal, various video analysis techniques based on surveillance video acquired using CCTV have been proposed. These video analysis techniques are used for the purpose of automatically identifying risk factors for the site being monitored from the surveillance video and taking appropriate safety measures.

These risk factors may include the possibility of unexpected accidents due to overcrowding. From many unfortunate accident cases at home and abroad, it can be seen that failure to respond early when a situation where too much crowding of people occurs in a limited space can lead to an accident situation that poses a serious risk to the lives and safety of citizens.

Conventionally, in order to determine such crowd overcrowding using CCTV, actual measurements of the surveillance target area captured in the surveillance video were required. For example, there was a method in which the area of a specific space captured in surveillance video was adjusted in advance to be a horizontal unit area, and then if more than a certain number of people were detected in the space, it was considered overcrowded. This prior art may be partially useful in detecting crowds in units that are easy for humans to recognize, such as the so-called “number of people per square meter,” but each camera requires pre-calibration and responds when changes occur in the shooting angle or background environment. There is a difficult problem.

Another conventional technology was to use image analysis using artificial intelligence to identify individual people and calculate the person's three-dimensional location information to ultimately determine whether the crowd was overcrowded. This method, unlike the prior art described above, may require somewhat less prior calibration of the camera. However, on the other hand, when using this technology, the process of recalculating a person's location requires significant computational resources, so advanced artificial intelligence and information processing equipment to support it are required. It will require significant time and resources to integrate such a sophisticated system into an existing, widely installed CCTV system.

The present invention is intended to overcome the limitations of the prior art described above, and its object is to provide a method that can calculate crowd density without prior camera correction using a relatively simple person recognition method.

A method of calculating crowd density by video analysis according to an embodiment of the present invention for solving the above-described technical problem includes a first length related to at least one of the width, length, and height of a person's head, and a distance between people. Setting a second length related to the interval of, Obtaining a surveillance image of the surveillance area from a photographing device, Analyzing the surveillance image to identify at least one human head as an object, Wherein the object is the surveillance image Obtaining a first measurement value representing the size displayed in the image, obtaining a second measurement value representing the distance between the object and at least one adjacent object based on the size displayed in the surveillance image, the first measurement value representing the distance between the object and at least one adjacent object based on the size displayed in the surveillance image Calculating a ratio between a measurement value and the first length, and calculating an adjacent spacing distance of the object based on the second measurement value and the ratio, crowding the object when the adjacent spacing distance of the object is less than or equal to the second length. It may include classifying objects as objects, and calculating crowd density in the surveillance area based on the number of dense objects identified from the surveillance video.

The method includes setting a first reference number of people related to the density of the crowd, if the number of dense objects is greater than or equal to the first reference number of people, determining that the crowd density is greater than or equal to the first standard density, and determining a first standard number of people for the surveillance area. 1. The method may further include generating warning information, and transmitting the first warning information using at least one of display, voice, vibration, light, text, and communication.

The method is related to the density of the crowd and sets a second standard number of people that is greater than the first standard number of people. When the number of dense objects is greater than the second standard number of people, the crowd density is determined to be greater than or equal to the second standard density. Determining and generating second warning information for the surveillance area, and transmitting the second warning information in at least one of display, voice, vibration, light, text, and communication, The first warning information and the second warning information may be different.

The step of transmitting the first warning information may include transmitting the first warning information to at least one person located in the surveillance area.

The step of transmitting the first warning information may include transmitting the first warning information to at least one of a manager of the surveillance area, a manager of the imaging device, and a safety manager of the surveillance area.

The step of transmitting the warning information includes transmitting the first warning information to at least one of a private or public management organization, police agency, fire agency, military agency, central administrative agency, and local administrative agency related to the surveillance area. May include steps.

The method further includes the step of setting a third standard number of people related to the density of the crowd, and the step of calculating the crowd density is performed when the number of objects identified in the surveillance video is less than or equal to the third standard number of people. It may include determining that the crowd density is less than or equal to the third standard density regardless of the number of objects.

The surveillance image is an image expressed by pixel information, the first measurement value is obtained based on the number of pixels that the object occupies in the display image, and the second measurement value is between the object and the adjacent object. It can be obtained based on the number of pixels present in .

The method further includes the step of obtaining a third measurement value indicating the size at which the adjacent object is displayed in the surveillance image, and the step of calculating the adjacent interval distance is based on the ratio of the first measurement value and the third measurement value. It may include correcting the second measurement value based on the second measurement value.

The method further includes the step of setting a fourth standard number of people indicating the number of adjacent objects to be included in the calculation of the adjacent spacing distance, and the step of calculating the adjacent spacing distance includes the fourth standard adjacent to the object. Specifying a number of adjacent objects, calculating an individual adjacent distance for each adjacent object from the object, and calculating the adjacent distance based on an average of the individual adjacent distances, and calculating the average of the adjacent objects. Can be derived by any one of the average methods belonging to the arithmetic mean, geometric mean, harmonic mean, weighted mean, median, mode, and power mean.

The step of calculating the adjacent spacing distance includes obtaining individual second measurement values from the object with respect to other objects whose horizontal positions overlap with the object in the surveillance image, and determining the second measurement value with the smallest second measurement value with respect to the object. Specifying a fourth standard number of adjacent objects, calculating an individual adjacent spacing distance for each adjacent object from the individual second measurement value, and calculating the adjacent spacing distance based on the average of the individual adjacent spacing distances. may include.

The step of calculating the adjacent interval distance includes obtaining an individual second measurement value from the object to at least one other object identified in the surveillance video, and obtaining an individual adjacent interval for each other object from the individual second measurement value. Calculate the distance, designate the fourth standard number of adjacent objects with the smallest individual adjacent distance to the object, and determine the adjacent object based on the average of the individual adjacent distances for each adjacent object. It may include a calculating step.

A device for calculating crowd density by video analysis according to an embodiment of the present invention for solving the above-described technical problem includes a memory, at least one processor connected to the memory, and a shooting device for acquiring surveillance images of a surveillance area. and a unit, wherein the memory stores a first length related to at least one of the width, length, and height of the person's head, and a second length related to the distance between the people, and the processor stores the image. An image collection unit that acquires surveillance images from a device, an object extraction unit that analyzes the surveillance images to identify at least one human head as an object, and obtains a first measurement value indicating the size at which the object is displayed in the surveillance images. and obtain a second measurement value representing the distance between the object and at least one adjacent object based on the size displayed in the surveillance image, calculate the ratio between the first measurement value and the first length, and calculate the ratio between the first measurement value and the first length. 2 Calculate the adjacent distance of the object based on the measurement value and the ratio, classify the object as a dense object if the adjacent distance of the object is less than the second length, and classify the object as a dense object identified from the surveillance image. It may include a crowd density calculation unit that calculates the crowd density of the surveillance area based on the number.

The memory further stores a first standard number of people related to crowd density, and the processor determines that the crowd density is greater than or equal to the first standard density when the number of dense objects is greater than or equal to the first standard number of people, and the monitoring It may further include a warning determination unit that generates first warning information for the area, and an information providing unit that transmits the first warning information by at least one method among display, voice, vibration, light, text, and communication.

The memory is related to the density of the crowd and further stores a second standard number of people that is greater than the first standard number of people, and the warning judgment unit is configured to determine if the crowd density is greater than the second standard number of people. 2, and is configured to determine that the density is higher than the standard density and generate second warning information for the surveillance area, wherein the information provider displays the second warning information using at least one method of voice, vibration, light, text, and communication. It is configured to transmit, and the first warning information and the second warning information may be different.

The memory further stores a third standard number of people related to the density of the crowd, and the crowd density calculation unit is configured to, when the number of objects identified in the surveillance video is less than the third standard number of people, regardless of the number of the crowd objects. It may be configured to determine that the crowd density is below the third standard density.

The surveillance image is an image expressed by pixel information, the crowd density calculation unit obtains the first measurement value based on the number of pixels that the object occupies in the display image, and the second measurement value is the It may be configured to obtain based on the number of pixels existing between an object and the adjacent object.

The crowd density calculation unit obtains a third measurement value indicating the size of the adjacent object displayed in the surveillance video, corrects the second measurement value based on the ratio of the first measurement value and the third measurement value, and It may be configured to calculate the adjacent spacing distance based on the second measurement value.

The memory further stores a fourth standard number of people indicating the number of adjacent objects to be included in the calculation of the adjacent spacing distance, and the crowd density calculation unit specifies an adjacent object of the fourth standard number of people adjacent to the object, configured to calculate an individual adjacent distance from the object to each adjacent object, and calculate the adjacent distance based on an average of the individual adjacent distances, wherein the average is an arithmetic mean, geometric mean, harmonic mean, It can be derived by any one of the weighted average, median, mode, and power average methods.

The processor further includes a basic information unit storing basic information for operating the device in the memory, and the basic information unit includes the first length, the second length, a first reference number of people related to the density of the crowd, and the crowd. At least one of a second standard number of people related to the density of the crowd and greater than the first standard number of people, a third standard number of people related to the density of the crowd, and a fourth standard number indicating the number of adjacent objects to be included in the calculation of the adjacent spacing distance. It may be configured to receive input and store it in the memory.

According to the method and/or device according to the present invention, it is possible to make a relatively accurate judgment about the density of the crowd even by analyzing the image using a relatively simple algorithm. Therefore, it can be easily added to an existing CCTV system and used to monitor crowded situations and provide a function to warn in advance.

1 is a conceptual diagram of CCTV surveillance video and estimation of relative length based on the surveillance video according to an embodiment of the present invention;
Figure 2 is a conceptual diagram of a ratio relationship showing the derivation of the adjacent distance between people according to an embodiment of the present invention;
Figure 3 is a conceptual diagram showing the pixel representation of the adjacent distance between people according to an embodiment of the present invention;
4 is a conceptual diagram of calculating the adjacent distance between adjacent objects according to an embodiment of the present invention;
Figure 5 is a conceptual diagram of calculating the adjacent distance between adjacent objects by correction according to an embodiment of the present invention;
6 is a conceptual diagram of a method for calculating crowd density in a surveillance area based on the number of dense objects identified from surveillance images according to an embodiment of the present invention;
Figure 7 is an example of an environment settings screen for utilizing the crowd density calculation method according to an embodiment of the present invention;
8 is a conceptual diagram of configuring a device for calculating crowd density by image analysis or a system including such a device according to an embodiment of the present invention;
Figure 9 is a conceptual diagram of an embodiment of a crowd density calculation and information provision system according to a preferred embodiment of the present invention, and
Figure 10 is a flowchart summarizing the crowd density calculation method according to an embodiment of the present invention.

Since the present invention can make various changes and have various embodiments, specific embodiments will be illustrated in the drawings and described in detail. However, this is not intended to limit the present invention to specific embodiments, and should be understood to include all changes, equivalents, and substitutes included in the spirit and technical scope of the present invention.

Terms such as first, second, etc. may be used to describe various components, but the components should not be limited by the terms. The above terms are used only for the purpose of distinguishing one component from another. For example, a first component may be referred to as a second component, and similarly, the second component may be referred to as a first component without departing from the scope of the present invention. The term “and/or” includes any combination of a plurality of related stated items or any of a plurality of related stated items and, unless otherwise indicated, is non-exclusive. When items are listed in the present application, it is merely an exemplary description to easily explain the spirit and possible implementation methods of the invention of the present application, and therefore, it is not intended to limit the scope of the embodiments of the present invention.

As used herein, “A or B” may mean “only A,” “only B,” or “both A and B.” In other words, as used herein, “A or B” may be interpreted as “A and/or B.” For example, as used herein, “A, B or C” refers to “only A,” “only B,” “only C,” or “any and all combinations of A, B, and C ( It can mean “any combination of A, B and C)”.

The slash (/) or comma used in this specification may mean “and/or.” For example, “A/B” can mean “A and/or B.” Accordingly, “A/B” can mean “only A,” “only B,” or “both A and B.” For example, “A, B, C” can mean “A, B, or C.”

As used herein, “at least one of A and B” may mean “only A,” “only B,” or “both A and B.” In addition, in this specification, the expression "at least one of A or B" or "at least one of A and/or B" means "at least one It can be interpreted the same as "at least one of A and B".

Additionally, as used herein, “at least one of A, B and C” means “only A”, “only B”, “only C”, or “A, B and C”. It can mean “any combination of A, B and C.” Also, “at least one of A, B or C” or “at least one of A, B and/or C” means It may mean “at least one of A, B and C.”

When a component is said to be "connected" or "connected" to another component, it is understood that it may be directly connected to or connected to the other component, but that other components may exist in between. It should be. On the other hand, when it is mentioned that a component is “directly connected” or “directly connected” to another component, it should be understood that there are no other components in between.

The terms used in this application are only used to describe specific embodiments and are not intended to limit the invention. Singular expressions include plural expressions unless the context clearly dictates otherwise. In this application, terms such as “comprise” or “have” are intended to designate the presence of features, numbers, steps, operations, components, parts, or combinations thereof described in the specification, but are not intended to indicate the presence of one or more other features. It should be understood that this does not exclude in advance the possibility of the existence or addition of elements, numbers, steps, operations, components, parts, or combinations thereof.

Unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as generally understood by a person of ordinary skill in the technical field to which the present invention pertains. Terms defined in commonly used dictionaries should be interpreted as having a meaning consistent with the meaning in the context of the related technology, and should not be interpreted in an ideal or excessively formal sense unless explicitly defined in the present application. No.

In describing the invention in this application, embodiments may be described or illustrated in terms of unit blocks that perform the described function or functions. In the present application, the blocks may be expressed as one or more devices, units, modules, units, etc. The blocks may be implemented in hardware using one or more logic gates, integrated circuits, processors, controllers, memories, electronic components, or information processing hardware implementation methods that are not limited thereto. Alternatively, the blocks may be implemented in software by an implementation method of application software, operating system software, firmware, or information processing software that is not limited thereto. One block may be implemented separately into a plurality of blocks that perform the same function, or conversely, one block may be implemented to simultaneously perform the functions of a plurality of blocks. The blocks may also be implemented physically separated or combined by any criteria. The blocks may be implemented to operate in an environment where their physical locations are not specified and are spaced apart from each other by a communication network, the Internet, a cloud service, or a communication method that is not limited thereto. All of the above implementation methods are within the scope of various embodiments that can be taken by a person skilled in the art of information and communication technology to implement the same technical idea, so any detailed implementation methods are all included in the technical idea scope of the invention in this application. It should be interpreted as

Hereinafter, preferred embodiments of the present invention will be described in more detail with reference to the attached drawings. In order to facilitate overall understanding when describing the present invention, the same reference numerals are used for the same components in the drawings, and duplicate descriptions for the same components are omitted. Additionally, it is assumed that the plurality of embodiments are not mutually exclusive, and that some embodiments may be combined with one or more other embodiments to form new embodiments.

The method for calculating crowd density by video analysis according to an embodiment of the present invention includes a first length related to at least one of the width, length, and height of a person's head and a second length related to the distance between people. It may be a method of measuring the distance between people by comparing the first length and the second length in a relative manner compared to the size of the person's head, and then calculating the degree of crowding of people according to the relative measurement value. there is. In the above method, since the human head has a constant average size within a relatively small error range depending on the person, the display of the object identified as the human head is performed regardless of where and at what distance the object is recorded in the image. It can be based on the principle that the actual length can be estimated from the size.

Figure 1 is a conceptual diagram of CCTV surveillance video and estimation of relative length based on the surveillance video according to an embodiment of the present invention. The surveillance image 100 of FIG. 1 is obtained from a photographing device configured to monitor a predetermined surveillance area. For example, it may be an on-site image obtained from a CCTV camera overlooking a surveillance area where crowds may be concentrated. However, it is not limited to this.

In the surveillance video 100, various people are photographed in various sizes at various distances. When trying to measure the degree of crowding of people from such images, the position and angle of view of the photographing device must be set in advance to provide information on the measurement distance in advance, or the spatial location of the person can be measured regardless of the distance. Means that could be determined had to be taken into consideration. However, according to one embodiment of the present invention, in the analysis of the surveillance video 100, only an image processing means capable of recognizing and identifying the human head as an object and determining the size at which the object is displayed in the surveillance video is used. It may be used.

As illustrated in FIG. 1, the head size of the first person 110 in the surveillance video 100 is displayed as the first display length 120, and the head size of the second person 112 is displayed as the second display length. It is displayed as a length (122), and the head size of the third person (114) can be considered to be displayed as a first displayed length (124).

When the surveillance video 100 is acquired as a bitmap video frame consisting of a plurality of pixels, the first, second, and third people 110, 112, and 114 use the video analysis technique. Identified by , and especially when identifying the head of the person, the first, second, and third display lengths 120, 122, and 124 may be calculated as a predetermined number of pixels.

At this time, if the first length, which represents the average head size of a person, is set to an arbitrary constant length, the size of the head of the person identified in the surveillance video 100 is considered to be generally the same, that is, the first person If the head sizes of (110), the second person (112), and the third person (114) are also considered to be the same, the first, second, and third display lengths (120, 122, and 124) are different from each other. Even if it is calculated by the number of pixels, it can be considered to be the first length. Accordingly, a predetermined ratio relationship can be calculated and derived between the number of pixels of the first, second, and third display lengths 120, 122, and 124 and the first length.

By calculating the ratio relationship as above, the distance between people appearing in the surveillance video can also be calculated. For example, when trying to find the distance between the first person 110 and the adjacent third person 114, the pixel where the head object of the first person 110 is located and the head of the third person 114 It is possible to measure how many other pixels unrelated to the objects exist between pixels where an object is located. When the number of irrelevant pixels is referred to as the distance display length 134, the actual distance between the people, that is, the adjacent distance between people, can be calculated from the distance display length 134 according to the ratio relationship.

Figure 2 is a conceptual diagram of a ratio relationship showing the derivation of the adjacent distance between people according to an embodiment of the present invention.

For an ideal explanation, assume that there are three people located on the same horizon in a surveillance video. These are tentatively named the first person 210, the second person 212, and the third person 214, respectively. The first, second, and third people (210, 212, 214) include the first head object, the second head object, and the third head object (210-1, 212-1, and 214-1), respectively. It can be identified through video analysis from the surveillance video in this way.

In the surveillance video, the first head object 210-1 appears at a first display length 220, the second head object 212-1 appears at a second display length 222, and the third head object 210-1 appears at a first display length 220. Assume that the head object 214-1 appears at the first display length 224.

In addition, in the surveillance video, the adjacent interval between the first head object 210-1 and the second head object 212-1 is a first distance display length 232, and the first head object 210-1 It is assumed that the adjacent distance between 1) and the third head object 214-1 is represented by the second distance display length 234.

At this time, the first length, which represents the average head size of a person, is set to an arbitrary constant length, for example, 180 mm, and the size of the head of the person identified in the surveillance video can be considered to be generally the same. In this case, the first head object, the second head object, and the third head object (210-1, 212-1, 214-1) are all considered to be objects of 180 mm, and therefore are also considered to be the same size. , the first, second, and third display lengths 220, 222, and 224 may be regarded as the first length displayed in the surveillance video.

Accordingly, a predetermined ratio relationship can be calculated and derived between the number of pixels of the first, second, and third display lengths 220, 222, and 224 and the first length. And from the ratio relationship, it becomes possible to invert the first distance display length 232 and the second distance display length 234 into the actual adjacent spacing distance.

For example, the first length is L _r1 , the first display length 220 is L _d1 , the second display length 222 is L _d2 , and the first distance display length 134 is D _d1 . At this time, if we want to find the actual adjacent distance of the first distance display length 232 and call it D _r1 , the following proportional equation is established.

As described above, among the numbers appearing in the proportional equation of Equation 1, the only unknown number is D _r1 , which is the actual adjacent spacing distance. Therefore, it can be seen that this can be easily derived by inverting from the arbitrarily designated first length. For example, when the first distance display length 232 is twice the first display length 220, the actual adjacent spacing distance D _r1 is (180:D _r1 :180)=(1:2:1 ), and therefore it may be possible to estimate it as 360mm.

Figure 3 is a conceptual diagram showing the pixel representation of the adjacent distance between people according to an embodiment of the present invention.

When the surveillance video is acquired as a bitmap image frame consisting of a plurality of pixels, the first, second, and third display lengths 220, 222, and 224 shown in FIG. 2 Each can be calculated as a predetermined number of pixels. That is, the number of pixels occupied by the first head object 310, the second head object 312, and the third head object 314 belonging to the first, second, and third people in the surveillance video is as described above. It can be identified through image analysis.

At this time, the standard representing the number of pixels, for example the width, the number of first display pixels 320 for the first head object 310, and the second display pixel for the second head object 312 It can be seen as the number 322, and the third display pixel number 324 for the third head object 314.

In addition, the distance filled with irrelevant pixels existing between the first head object 310 and the second head object 312 from the surveillance image, that is, the adjacent interval distance, is the first distance pixel number 332, and the The adjacent distance between the first head object 310 and the third head object 314 can be viewed as the second distance pixel number 334.

At this time, by determining the first length to be a constant length as described above, a proportional expression can be established between the actual number of pixels appearing in the surveillance image and the actual length, as shown in Equation 1 above. Therefore, if the first, second, and third display pixel numbers 320, 322, and 324 are considered to be the first length, the first and second distance pixel numbers 332, 334 are also considered to be the actual adjacent numbers. It becomes possible to invert the gap distance.

Based on the example shown in FIG. 3, the first, second, and third display pixel numbers 320, 322, and 324 in the surveillance image each correspond to 10 pixels, and the first distance pixel When the number 332 is 14 pixels, the second distance pixel number 334 is 34 pixels, and the first length is designated as 180 mm, the first distance pixel number 332 is actually 252 mm, and the second distance pixel number 332 is 252 mm. The number of distance pixels (334) can be considered to actually represent a distance of 612 mm. In other words, the actual distance between the people, that is, the adjacent distance between people, can be calculated based on the pixels appearing in the surveillance video.

In other words, if the crowd density calculation method according to the present invention as described above is explained more generally, it is as follows. First, a first length related to at least one of the width, length, and height of a person's head can be set. Next, surveillance images of the surveillance area can be obtained from the imaging device. Next, the surveillance video can be analyzed to identify at least one human head as an object. Next, a first measurement value indicating the size of the object displayed in the surveillance video can be obtained. At this time, the surveillance image may be an image expressed by pixel information. Next, the first measurement value may be obtained based on the number of pixels that the object occupies in the display image, and the second measurement value may be obtained based on the number of pixels existing between the object and the adjacent object. . Next, the ratio between the first measurement and the first length may be calculated, and the adjacent distance between the objects may be calculated based on the second measurement and the ratio.

As described above, when calculating the adjacent distance, the calculation may be easier if the number of display pixels of the object that is the target of distance comparison is the same or close. For example, referring to Equation 1 above, if the first display length L _d1 and the second display length L _d2 are the same, one of the two can be omitted from the proportional expression, and therefore the first display length It can be seen that the actual adjacent spacing distance D _r1 can be obtained from the first distance display length D _d1 simply by replacing L _d1 with the first length L _r1 , making the calculation simpler. To facilitate such calculation, according to an embodiment of the present invention, the calculation of the adjacent distance may be limited to other objects whose horizontal positions overlap with the object in the surveillance image.

Figure 4 is a conceptual diagram of calculating the adjacent distance between adjacent objects according to an embodiment of the present invention. Figure 4 shows a first object 410, a second object 412, a third object 414, and a fourth object 416, each representing a human head. Based on the first object 410, the distance to each other adjacent object is the first distance display length 432, the second distance display length 434, and the third distance display length 436. It may appear. In the example shown in FIG. 4, the remaining objects 412, 414, and 416 share a predetermined horizontal position section 440 based on the first object 410, and in the present invention, the horizontal positions overlap. It can be interpreted to include these cases. For example, if the surveillance image is a bitmap image composed of a number of pixels defined by the horizontal coordinate axis x and the vertical coordinate axis y, and all objects include at least one pixel that shares the same y value, In the context of the invention, these objects can be described as having overlapping horizontal positions.

When acquiring surveillance images using a photographing device for a given surveillance area, objects with overlapping horizontal positions can be considered to be located at the same distance from the photographing device according to general optical projection laws. Therefore, even if the first to fourth objects 410, 412, 414, and 416 appear to have slightly different sizes in the imaging device, they can be considered to be substantially all objects of the same size corresponding to the first length. . In this case, any one of the first to fourth objects (410, 412, 414, 416), preferably the first length, is determined by a proportional expression such as Equation 1 based on the display size of the first object 410. By substituting , the first to third distance display lengths 432, 434, 436 can be converted into actual adjacent spacing distances.

However, according to another embodiment of the present invention, when calculating the adjacent distance, the distance may be calculated even between objects located at arbitrary positions. For example, in Equation 1, when the first display length L _d1 and the second display length L _d2 are significantly different, both are replaced by the same first length L _r1 to establish a proportional equation. You can see that there are difficulties. Nevertheless, if the premise that both the first display length L _d1 and the second display length L _d2 are regarded as the same first length L _r1 is established, the proportional expression such as Equation 1 is replaced with the following equation A modified proportional equation can be assumed as shown in Equation 2.

That is, assuming that the first display length L _d1 and the second display length L _d2 are displayed differently at a ratio of 1:a, the proportional equation for calculating the first distance display length will be modified to additionally reflect this ratio. In addition, the actual adjacent distance D _r1 can be calculated by calculating additional correction values b and c to reflect the three-dimensional perspective relationship in the proportional equation.

In a simple exemplary method that does not limit the implementation method of the present invention, assuming that the reflection of perspective is simplified and applied by the average method, b and c are each set to 1/2 (ab + c) can be determined in the simple form of (1/2)a+(1/2)=((1+a)/2). In this example calculation method, when L _d1 is 10 pixels, D _d1 is 20 pixels, and L _d2 is 15 pixels, L _d1 and L _d2 have a ratio of 1:1.5, (10:20:15)=( Assuming that L _d1 :1.25D _d1 :1.5L _d2 ), the actual adjacent spacing distance D _r1 is (L _d1 :D _d1 :L _d2 )=(10:16:10) by the corrected proportional relationship. It can be seen as having a relationship of 10:16:10)=(180:D _r1 :180), and therefore it can be estimated as 288mm.

In other words, to generalize, a first measurement value indicating the size of the first object displayed in the surveillance image is obtained, and the distance between the first object and the second object is a first measurement value representing the size displayed in the surveillance image. Obtaining a second measurement value, obtaining a third measurement value indicating a size at which the second object is displayed in the surveillance video, correcting the second measurement value based on the ratio of the first measurement value and the third measurement value, and The ratio between one measurement value and the first length may be calculated, and the adjacent distance between the objects may be calculated based on the second measurement value and the ratio.

Figure 5 is a conceptual diagram for calculating the adjacent distance between adjacent objects by correction according to an embodiment of the present invention. Figure 5 shows a first object 510, a second object 512, and a third object 514, each representing a human head. Although each of the objects is equally regarded as the first length, in the surveillance video according to an embodiment of the present invention, each of the objects has a first display length 520, a second display length 522, and It may appear as the third display length (524). Based on the first object 510, the distance to each other adjacent object may be measured only in the horizontal direction, such as the first horizontal distance display length 532 or the second horizontal distance display length 534. Or, by calculation reflecting the vertical positional relationship that each object has in the surveillance video (for example, by the Pythagorean law using a right triangle formed by the distance in the x-direction and the distance in the y-direction) ) It may be the obtained first actual distance display length (542) or the second actual distance display length (544). Each of the display lengths 532, 534, 542, and 544 can be measured in pixels, and depending on the calculation method, they may be treated as abstract pixel distance values in decimal units rather than integers. When using the corrected calculation method using Equation 2 as an example using the above measurement values, the distance indicators in the image, such as each display length (532, 534, 542, 544), are compared to the actual adjacent ones. You can see that it can be converted to an interval distance.

As described above, according to the present invention, the adjacent distance from each object identified in the surveillance video to another object can be estimated by simple calculation. Based on the adjacent distance, according to an embodiment of the present invention, when the adjacent distance between an object and surrounding adjacent objects is calculated to be within a predetermined length, for example, a second length, this means that people are crowded together and interacting with each other. It is judged to be in a close contact state, and therefore can be classified as a dense object.

According to one embodiment of the present invention, the crowd density of the surveillance area can be calculated based on the number of dense objects identified from the surveillance video. Figure 6 is a conceptual diagram of a method for calculating crowd density in a surveillance area based on the number of dense objects identified from surveillance images according to an embodiment of the present invention.

Referring to the surveillance image of the first example 610, according to the above-described calculation method, the adjacent distance from an arbitrary person belonging to the first example, that is, the first object 615, to surrounding adjacent objects is calculated. You can. At this time, according to an embodiment of the present invention, one or more adjacent objects are designated, individual adjacent distances for the designated adjacent objects are obtained, and then the adjacent distances are calculated by merging the individual adjacent distances. You may also derive it.

According to one embodiment of the present invention, when referring to the individual adjacent spacing distance for one or more adjacent objects as described above, a standard number of people can be determined for how many adjacent objects are to be designated. Each example in FIG. 6 is based on an example of calculating individual adjacent distances for four adjacent objects adjacent to one object, with the standard number of people set to 4. However, the standard number of people may be determined as a number other than 4, or may be determined variably, and if it is determined variably, the range shall be 1 or more and any upper or lower limit may deviate from the spirit of the present invention. It is obvious that it is not done. Of course, the purpose of the present invention is not impaired even if one or more adjacent objects are selected based on criteria other than the standard number of people.

In a preferred embodiment of the present invention, the adjacent distance specifies adjacent objects of the reference number adjacent to the object, calculates an individual adjacent distance for each adjacent object from the object, and calculates the individual adjacent distance. It can be calculated by calculating the adjacent interval distance based on the average of the distance. At this time, the average may be derived by any one of the average methods belonging to the arithmetic mean, geometric mean, harmonic mean, weighted mean, median, mode, and power mean, and any of the mean methods can be used by those practicing the present invention. Can be applied by choice.

In one embodiment of the present invention, the calculation of the adjacent interval distance is performed by obtaining individual distance display lengths from the object for other objects whose horizontal positions overlap with the object in the surveillance video, and determining the distance between the object and the object. Designate the adjacent objects of the standard number of people in the order of smallest distance display length, calculate the individual adjacent interval distance for each adjacent object from the individual distance display length, and calculate the individual adjacent interval distance based on the average of the individual adjacent interval distances. This can be done by calculating the adjacent spacing distance.

In another embodiment of the present invention, the calculation of the adjacent interval distance is performed by obtaining individual distance display lengths from the object to at least one different object identified in the surveillance video (preferably, adjacent to the object). (may be obtained for all objects that are likely to do so), calculate the individual adjacent spacing distance for each other object from the individual distance display length, and determine the standard number of people with the smallest individual adjacent spacing distance to the object. It may include designating adjacent objects and calculating the adjacent distance based on the average of the individual adjacent distances for each adjacent object.

In any embodiment of the present invention, from the method of designating the adjacent object, the standard number of people and the method for determining the same when the adjacent object is designated by the standard number of people, the method of measuring the individual distance display length, and the individual distance display length The method of calculating each individual adjacent distance, and the method of calculating the at least one individual adjacent distance in an integrated manner to finally derive the adjacent distance of the object, can be applied to the above-mentioned or various other applied methods. A person skilled in the art will easily understand that the present invention can be implemented without departing from the technical spirit of the present invention.

Referring again to FIG. 6, the surveillance video of the first example 610 illustrates a case where crowd density is low. According to one embodiment of the present invention, in order to calculate the crowd density for the first example 610, the crowded objects identified within the first example 610 may be counted. For the coefficient, neighboring adjacent objects can be designated from at least one object including the first object 615 and the adjacent distance to the adjacent objects can be obtained using various methods as described above. At this time, the example in FIG. 6 specifies four adjacent objects based on the standard number of people.

If the adjacent distance of the first object 615 is calculated to be within the second length as described above, the first object 615 may be classified as a dense object, and the first example 610 This could be any other object that appears in . However, in the first example 610, if the adjacent distance between the first object 615 and the four adjacent objects is sufficiently large, the first object 615 may not be classified as a dense object, and the This may be any other object appearing in the first example 610. For example, if the result of the coefficient is less than the first crowd density, it is determined that the crowd density of the first example 610 is at least lower than the first reference density that can be estimated based on the first crowd density. You can.

The surveillance video of the second example 620 illustrates a case where the crowd density is at a normal level. In order to calculate the crowd density for the second example 610 by various methods as described above, the dense objects identified within the second example 620 can be counted, and for this purpose Adjacent objects surrounding the at least one object including the second object 625 can be designated, and the adjacent distance to the adjacent objects can be obtained. If the result of the coefficient is, for example, greater than the first crowd density, the crowd density of the second example 620 can be determined to be at least higher than the first reference density that can be estimated based on the first crowd density. there is.

According to an embodiment of the present invention, in the case of the second example 620, it is determined that a situation exceeding the first standard density is observed within the surveillance area indicated by the corresponding surveillance video, and warning information is provided regarding this. can be created. By using this warning information, the purpose of the present invention to respond early when a situation where too many people are crowded in a limited space occurs can be realized.

If the entity that generates the warning information (i.e., the entity that calculates the above-described crowd density) is a computing device such as a computer, such warning information may be expressed as being “sent,” but in the present invention, warning information is transmitted. In addition to the dictionary meaning, when warning information is generated by the above crowd density calculation method, it should be understood to include any means for reaching the final target of the warning. Methods for transmitting the warning information may include, for example, display, voice, vibration, light, text, communication, etc., but are not limited thereto.

As a simple example, when the warning information is generated from a computer device, the warning information is displayed on the screen, displayed by turning on an alarm light, guided by a warning voice, guided by an alarm sound, or touched by vibration of the device. It may be recognized by the person doing it. In addition, it is sent from the computer device in the form of a communication message according to any standard through a wired or wireless circuit or communication network, and is transmitted to any other type of device or system including a computer device. It may also be transmitted from any other device or system using the above example method. For example, warning information is generated from a computer device and transmitted to any portable communication terminal device via a mobile communication network, and warning information may be provided from the portable communication terminal device through display, voice, vibration, light, text, etc. .

The intended warning target may include at least one person, but depending on the implementation method, it may be a device or system other than a person.

When the warning target is a person, for example, the warning information may be transmitted to at least one person located in the surveillance area. In this case, a warning broadcast may be played through speakers installed in the surveillance area, or a message may be sent in the form of a disaster text message to mobile communication terminals owned by people within the surveillance area. In this case, people located in the surveillance area will be able to recognize the situation where crowd density is increasing and take self-help measures such as leaving the place or dispersing the crowd.

As another example, the warning information may be transmitted to at least one of a manager of the surveillance area, a manager of the imaging device, and a safety manager of the surveillance area. The manager may mean an authorized person in charge of the filming device or the surveillance area monitored by it, for example, in the case of a facility, the safety manager of the facility, in the case of an event site, the safety manager of the event, or the street ( In the case of a street, the person may be in charge of maintaining order on the street. In particular, the manager of the surveillance area may include the information protection manager of the facility, region, or local government with jurisdiction over the surveillance area, and the manager of the filming device may include the CCTV system installed in the surveillance area. may include personnel in charge of facilities, regions, or local governments involved in the operation of the surveillance area, and when referred to as a safety manager of the surveillance area, he/she has jurisdiction over the surveillance area, or is capable of accessing or deploying to the surveillance area, or has already accessed the surveillance area. Alternatively, it may include one or more assigned police officers, security guards, or other personnel responsible for control. Of course, it is obvious that it can mean a manager other than the above-mentioned examples and is not limited thereto. In this case, the manager may take appropriate response measures such as controlling the number of people to reduce crowd density, restricting access to the surveillance area, or dispatching and deploying related support personnel.

If the warning target is a device or system rather than a person, for example, the warning information may be distributed to any of the private or public management organizations, police agencies, fire agencies, military agencies, central administrative agencies, and local administrative agencies related to the surveillance area. It may be transmitted to at least one destination, such as a related device, system, or communication network.

The private or public management organization may include an organization such as an operator of a facility including the surveillance area or an event organizer. The above police agencies may include national police, autonomous police, investigative agencies, or other autonomous crime prevention organizations. The firefighting agency may include a public firefighting agency, a private disaster response team, or an autonomous firefighting organization, and may broadly include public or private health care facilities. The military agencies may include military units at all levels, military agencies, and other appropriate military organizations that can take appropriate measures in the surveillance area. The above-mentioned central administrative agencies are administrative agencies responsible for controlling crowd density, and include, for example, agencies such as the Office of the President, the Office of the Prime Minister, the Ministry of Public Administration and Security, the National Police Agency and its local offices, and the National Fire Agency and its local offices at the time of filing the application for the present invention. can do. The local administrative agency refers to a local administrative agency with jurisdiction over the monitoring area, for example, a metropolitan or basic government under the local autonomy system, a department and/or agency affiliated therewith, or other attached local administrative agency, that is, It may be a special city hall, metropolitan city hall, special self-governing city administration, special self-governing provincial office, provincial office, etc., or it may be a city hall, district office, or county office, or it may be an administrative office with jurisdiction over the town, township, dong, or ri belonging to it, or each level of the above. It may be a department belonging to an administrative district office. In addition, the device, system, or communication network to which the above warning information is transmitted refers to those installed in or operated by the above-mentioned targets or other similar targets, for example, the server of the CCTV integrated control center, or the disaster communication network and It could be the same thing. It is also obvious that the examples listed above correspond to some of the specific objects that can be specifically listed for the preferred implementation of the present invention, and are merely examples that do not limit the purpose of implementing the present invention.

Referring again to FIG. 6, the surveillance video of the third example 630 illustrates a case where the crowd density is at a significant level. In order to calculate the crowd density for the third example 630 by various methods as described above, the dense objects identified within the third example 630 can be counted, and for this purpose Adjacent objects surrounding the at least one object including the third object 635 can be designated, and the adjacent distance to the adjacent objects can be obtained. If the result of the coefficient is, for example, greater than the second crowd density, the crowd density of the third example 630 can be determined to be at least higher than the second standard density that can be estimated based on the second crowd density. there is. At this time, the number of people in the second crowd may be greater than the number of people in the first crowd as exemplified above.

According to an embodiment of the present invention, in the case of the third example 630, it is determined that a situation exceeding the second standard density is observed within the surveillance area indicated by the corresponding surveillance video, and warning information is provided regarding this. can be created. In addition, if it is determined that it is a crowded situation with a higher density than the first standard density, and the warning information in the embodiment related to the second example 620 is called the first warning information, the content and/or transmission format are Different second warning information may be generated and also sent.

According to another embodiment of the present invention, different levels of crowd density are calculated as above according to the difference in crowd density between the second example 620 and the third example 630, and different types of warning information can be generated. Similarly, the standard density is defined by at least two different stages, and step-by-step warning information corresponding to each standard density may be generated.

Meanwhile, according to another embodiment of the present invention, a situation may be defined in which calculation to identify dense objects is essentially unnecessary, as shown in FIG. 6 above. For example, in the case of the first example 610, if the absolute number of objects, that is, people, detected in the surveillance video is below a certain level, there is a risk of exceeding the predetermined standard density at which risk is expected in any case. It may be judged that there is no such thing. In this case, if the number of objects identified in the first example 610 is, for example, less than the third number of crowded objects, performing a series of calculations for detecting the dense objects can be omitted, and the first example 610 The crowd density of the example 610 may be determined to be lower than the third standard density that can be estimated based on the third crowd density, regardless of the number of crowd objects. The present invention has the effect of simplifying the density calculation method, but by following the above-described embodiments, such simplified calculation can also achieve the effect of further saving.

Figure 7 is an example of an environment setting screen for utilizing the crowd density calculation method according to an embodiment of the present invention. Figure 7 shows an example of the form of a user interface for setting up such a device when all of the above-described embodiments are combined and implemented, and when such a method is implemented in the form of a computer device.

According to one embodiment of the present invention, an environment setting screen 700 as shown in FIG. 7 may be provided. The environment description screen 700 may be displayed divided into a calculation condition setting unit 710 and an emergency level standard setting unit 720.

For example, the calculation condition setting unit 710 specifies a case where the number of people on the screen shown by the surveillance video is more than a certain number of people (712), or sets a standard size (714) for the head of a person captured in the surveillance video. Alternatively, it may include a screen for setting the number of people nearby (716) for calculating the adjacent distance to calculate density. Compared to the previously disclosed embodiment, the certain number of people 712 may correspond to the third crowded number of people, the reference size 714 may mean the first length, and the number of surrounding people 716 ) may mean the standard number of people for designating the adjacent object, but is not necessarily limited to this.

For example, the emergency level standard setting unit 720 determines the degree of risk by setting emergency levels by step according to the number of people whose adjacent distances are within a predetermined length 728 among the people in the screen shown by the surveillance video. It may be configured to input content. For example, the 1st stage standard number of people (722), the 2nd level standard number of people (724), and the 3rd level number of people (726) are each set, so that if the number of people exceeds each standard number, the emergency level of the corresponding level is designated. there is. If necessary, the predetermined length 728 may also be defined differently for each step. Compared to the previously disclosed embodiment, the predetermined length 728 may mean the second length that is a standard for classifying objects recognized in the surveillance video as dense objects, and the first, second, and third steps The stage standard number of people (722, 724, 726) may refer to the number of people at various stages related to the judgment that the density increases step by step, such as the number of people in the first crowd and the number of people in the second crowd, and the emergency stage is the number of people in the first crowd. This may refer to a situation in which several stages of warning information are provided, such as warning information and second warning information, in which the content and/or transmission method of the warning is configured differently in each stage, but is not necessarily limited to this.

According to a preferred embodiment of the present invention, the emergency stage may mean generating warning information that notifies the CCTV integrated control center in charge of the occurrence of such a situation to the subject of event monitoring when risk level 1 is determined, and the risk If level 2 is determined, this may mean notifying the personal computer or mobile phone of the person in charge of risk management of the occurrence of such a situation and generating warning information to turn on warning lights in the integrated control center, and if level 3 risk is determined. Voice announcements are sent to the surveillance area, disaster text messages are sent to the surveillance area, and the situation is reported through the police (112), fire department (119), the Presidential Office (Presidential Security Service, etc.), and related disaster communication networks. This may mean generating warning information for automatic reporting. In addition to these examples, these emergency stages and the corresponding generation types of warning information may be configured in various stages and forms by the operator. In addition, depending on the implementation method and the method of generating/transmitting warning information, the present invention may be implemented to determine and respond in a continuous method that changes without steps by increasing or decreasing the number, rather than in stages divided by whether or not the number is more than a certain number. It does not go against the purpose.

Each standard information set in FIG. 7 may be selected to depend on a preset value to be applied as a default value if some of it is ignored, not used, or not user-defined. For example, according to an embodiment of the present invention, a function may be provided to select a specific type of setting value and set it to a used or unused state using a means such as a check box 730.

Figure 8 is a conceptual diagram of configuring a device for calculating crowd density through image analysis or a system including such a device according to an embodiment of the present invention. In one embodiment of the present invention, the present invention includes a crowd density calculation device 800 that calculates the crowd density by a method including the above-described embodiments, and a photographing unit 810 attached thereto that acquires surveillance images of the surveillance area. ), and a transmitting means 880 for transmitting warning information generated by the crowd density calculation device 800. The photographing unit 810 and the transmitting means 880 may be separate devices from the crowd density calculation device 800, and thus a system consisting of a plurality of devices may be configured.

According to one embodiment of the present invention, the crowd density calculation device 800 includes a memory 801, at least one processor 802, an image collection unit 820 that acquires surveillance images from the photographing device, and the surveillance. An object extraction unit 830 that analyzes the video to identify at least one human head as an object, obtains a first measurement value indicating the size of the object displayed in the surveillance video, and detects at least one adjacent object adjacent to the object. Obtain a second measurement value representing the distance between the two based on the size displayed in the surveillance image, calculate the ratio between the first measurement value and a predetermined first length representing the size of the person's head, and calculate the second measurement value and Calculate the adjacent spacing distance of the object based on the ratio, classify the object as a dense object if the adjacent spacing distance of the object is less than a predetermined second length, and calculate the adjacent spacing distance of the object based on the ratio. A crowd density calculation unit 840 that calculates the crowd density of the surveillance area based on the crowd density calculation unit 840, which determines that the crowd density is more than the standard density when the number of crowd objects is greater than the standard number of people, and generates warning information for the surveillance area. A warning determination unit 850, an information providing unit 860 that transmits the warning information by at least one method of display, voice, vibration, light, text, or communication, and stores basic information for operating the device in the memory. It may be configured to include a basic information unit 870.

The memory 801 may be configured as a computer-readable recording medium. The computer-readable recording medium may include a temporary or volatile recording medium that is maintained only while the processor 802 operates, such as a processor cache, RAM, or flash memory, or a hard disk or floppy disk. Magnetic media such as disks and magnetic tapes, optical media such as CD-ROMs and DVDs, magneto-optical media such as floptical disks, or It may include a recording medium that is relatively non-volatile or capable of long-term recording, such as solid state memory, or it may include a read-only recording medium such as ROM placed on hardware. , Furthermore, the hardware itself is configured to perform operations equivalent to a series of program commands through a hard-wired structure using circuit wiring, and is also a type of recording medium on which program commands implementing embodiments of the present invention are recorded. It is obvious to those skilled in the art that this may be included.

The processor 802 may be a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable array (FPA), a programmable logic unit (PLU), a microprocessor, or an instruction ( It can refer to one or more general-purpose or special-purpose computers, such as any other device capable of executing and responding to instructions.

The processor 802 may be configured to execute an operating system (OS) and one or more software running on the operating system. Additionally, a processing device may access, store, manipulate, process, and generate data in response to the execution of software. For convenience of understanding, even if the processor 802 is expressed in the singular, those skilled in the art will understand that the processor 802 includes a plurality of processing elements and/or a plurality of types. It can be seen that it may include processing elements. For example, a processing device may include a plurality of processors or one processor and one controller. Additionally, it can be implemented by various processing configurations, such as a parallel processor or multi-core processor.

The software may include a computer program, code, instructions, or a combination of one or more of these, and may configure a processing unit to operate as desired, independently or collectively. ) can command the processing unit. The software may be used on any type of machine, component, physical device, virtual equipment, or computer storage to be interpreted by the processor 802 or to provide instructions or data to the processor 802. It may be permanently or temporarily embodied in a medium or device, or in a transmitted signal wave. The software may be distributed over networked computer systems and stored or executed in a distributed manner.

The software may also be implemented in the form of program instructions that can be executed through various computer means including the processor 802 and recorded or stored in the memory 801. Program commands, data files, data structures, etc. may be recorded singly or in combination in the memory 801. The program instructions stored in the memory 801 are based on a command system specifically designed and configured for embodiments of the present invention, or a command system known and available to those skilled in the art of computer software, such as assembly language, C, You can also follow command systems exemplified by C++, Java, Python languages, etc. The above command system and its program instructions should be understood to include not only machine language code such as that generated by a compiler, but also high-level language code that can be executed by a computer using an interpreter, etc.

The above-described embodiments of the memory 801 and the processor 802 are not mutually exclusive, and may be selected or combined as needed. For example, one hardware device may be configured to operate as one or more software modules to perform the operations of embodiments of the invention, and vice versa.

According to one embodiment of the present invention, in the crowd density calculation device 800, the image collection unit 820, the object extraction unit 830, the crowd density calculation unit 840, and the warning judgment unit ( 850), the information providing unit 860, and the basic determination unit 870 can be interpreted as functional units included in the processor 802.

At this time, in the disclosure of this patented invention, “included” in the processor 802 means that the operation of each functional unit includes software implementation or hardware implementation including the various embodiments described above in relation to the processor 802. It may mean that it can be implemented by implementation or a combination of such implementations, or that each of the functional units is configured to operate by independently or jointly providing and/or utilizing the processor 802. It refers to any act of operating a functional part that performs a specific function in conjunction with the processor 802 in software or hardware form, and means that it can be clearly understood by a person skilled in the art. . In addition, similar to the processor 802, each of the functional units is a software implementation recorded in the memory 801, or operates by providing and/or utilizing the memory 801 independently or in common. , It is also obvious that it can be configured to operate in conjunction with the memory 801 to perform a specific function.

In a preferred embodiment of the present invention, the photographing unit 810 is a device capable of acquiring the surveillance image as a moving image by photographing image information in continuous frames in the visible light region, and is preferably a CCTV camera. It can be. However, in another embodiment of the present invention, the photographing unit 810 is a device configured to photograph a ray region other than visible light (which may include infrared rays, ultraviolet rays, and X-rays), and the surveillance image includes such Information captured in the ray region may also be included. In another embodiment, the photographing unit 810 is a device configured to obtain three-dimensional information of the surveillance area 135 through a binocular image (stereograph) method and/or a depth recognition method, The surveillance video may include such 3D information in the form of 3D information expressions including voxels, meshes, and point clouds. The photographing unit 810 is a camera array that photographs the same, overlapping, or different portions of the display area where the screen 920 can be viewed simultaneously by two or more independent camera devices. It may be implemented in any form, and the surveillance video may refer to a set of two or more independent image information obtained from the camera array. It is obvious that various embodiments of the photographing unit 810 can be implemented simultaneously or in combination with each other, and furthermore, they contribute to achieving the purpose of the present invention by being substantially the same or continuous with the technical idea intended by the embodiments. It is reasonable to view all of the configurations as modified embodiments of the present invention.

The image collection unit 820 may be a functional unit that receives surveillance images captured by the photographing unit 810. The photographing unit 810 may mean, for example, one of at least one CCTV camera that views a surveillance area where there is a risk of crowds, but is not limited thereto. The received surveillance video may be in the form of an image expressed by pixel information, but any method of implementation can establish the operation of the crowd density calculation device 800 according to the present invention even if it is other means. It is okay to do so. For example, the image may be composed of only differential values and/or motion vectors that indicate only changes from the existing image. In addition, the image may be a digital image encoded as a type of vector image, an analog signal image formed of continuous signals, or a three-dimensional display such as a voxel, mesh, or point cloud. It may be any form of visual information that can be acquired for the purpose of surveillance video. In addition, the image collection unit may be configured to further perform an operation of converting the image provided in a first form from the photographing unit 810 into a second form suitable for operation of the density calculation device 800 and receiving and processing the image.

The object extraction unit 830 may be configured to obtain surveillance images of the surveillance area from the image collection unit 820 and analyze the surveillance images to identify at least one human head as an object. The object extraction unit 830 may be configured to include a pattern recognition or object recognition algorithm for identifying the object. To identify the head object, the object extraction unit 830 may include an image analysis artificial intelligence that operates based on at least one processor 802 and at least one memory 801. For example, according to one embodiment of the present invention, the object extraction unit 830 may be configured to include artificial intelligence based on a deep neural network capable of machine learning. In order to realize the identification operation that the object extraction unit 830 must perform, the artificial intelligence may include an image analysis artificial intelligence model previously trained through supervised learning or unsupervised learning. According to an embodiment of the present invention, the artificial intelligence model may be implemented as a convolutional neural network (CNN).

The crowd density calculation unit 840 calculates the crowd density according to the present invention by using the same method as described with reference to FIGS. 1 to 6 or any other method that produces substantially the same results. Can be configured to run.

For example, according to one embodiment of the present invention, the crowd density calculation unit 840 reads a first length related to at least one of the width, length, and height of a person's head from the memory 801, and , the first measurement value is obtained based on the number of pixels that each object identified by the object extraction unit 830 occupies in the display image, and based on the number of pixels existing between the object and the adjacent object. Obtain the second measurement value, calculate the ratio between the first measurement value and the first length, calculate the adjacent spacing distance of the object based on the second measurement value and the ratio, and determine the proximity of the object to the surrounding area. When the adjacent distance between objects is calculated to be within a predetermined length, for example, a second length, this is judged to be a state in which people are crowded and closely attached to each other, and is classified as the dense object, and the dense object identified from the surveillance video is judged to be in close contact with each other. It may be configured to calculate crowd density in the surveillance area based on the number of objects.

In addition, according to an embodiment of the present invention, at least a first object and a second object are identified from the surveillance image, a first measurement value indicating a size at which the first object is displayed in the surveillance image is obtained, and the first object is identified. Obtaining a second measurement value indicating the size at which the distance between the first object and the second object is displayed on the surveillance image, obtaining a third measurement value representing the size at which the second object is displayed on the surveillance image, and obtaining the first measurement value representing the size at which the second object is displayed on the surveillance image. It may be configured to further perform an operation of correcting the second measurement value based on the ratio of the measurement value and the third measurement value.

The warning determination unit 850 generates warning information according to the present invention by using the same method as described with reference to FIGS. 6 to 7 or other methods that produce substantially the same results for calculating the crowd density. It can be configured.

For example, according to one embodiment of the present invention, the warning determination unit 850 reads information for determining the first reference density from the memory 801, and the crowd density calculation unit 840 If it is determined that the calculated crowd density exceeds the first standard density, it may be configured to generate warning information. In addition, the warning determination unit 850 may be configured to read information from the memory 801 for determining the second standard density, which is higher than the first standard density, and may be configured to read information for determining the second standard density, which is higher than the first standard density. It may be configured to read more information to determine the density of the level. In addition, when it is determined that the crowd density calculated by the crowd density calculation unit 840 exceeds the density for various stages including the second standard density, a first warning whose content and/or transmission format is at least partially different from It may be configured to generate information, secondary warning information, and other various levels of warning information. That is, the standard density is defined by at least two different stages, and stage-level warning information corresponding to each standard density may be generated.

Furthermore, according to an embodiment of the present invention, the warning determination unit 850 does not perform a stage in which the crowd density is divided by whether or not the crowd density is above a certain standard density, but a continuous stage that changes without a stage due to an increase or decrease in the density judgment standard. Even if the method is configured to determine the level or content of the warning, it does not violate the spirit of the present invention.

The information providing unit 860 may be a functional unit for performing an operation of transmitting the warning information generated by the warning determination unit 850 through the transmitting means 880. In this case, “transmitting” is as described above. Likewise, in addition to the dictionary meaning, when warning information is generated by the above crowd density calculation method, it should be understood to include the generation of any information to ultimately reach the intended warning target. The method or content of generating the information may be the same as that described with reference to FIG. 7 or the like in relation to the embodiment of the warning information, or may be implemented by any other method that produces substantially the same results.

The basic information unit 870 may be a functional unit that stores in the memory 801 basic information necessary for each functional unit to operate in order to achieve the operation purpose of the density calculation device 800. According to an embodiment of the present invention, the basic information unit 870 includes the first length related to at least one of the width, length, and height of a person's head, the second length related to the distance between people, A first standard number of people related to the density of the crowd, a second standard number of people related to the density of the crowd and greater than the first standard number of people, a third standard number of people related to the density of the crowd, and adjacent objects to be included in the calculation of the adjacent spacing distance. It may be a function unit configured to receive at least one of the fourth standard number of people representing the number of and store it in the memory. Therefore, in one embodiment of the present invention, the basic information unit 870 may be a functional unit that operates in relation to the display of and input to the environment setting screen shown in FIG. 7.

The transmitting means 880 should be understood to include any transmitting means for ultimately reaching the intended warning target when information for transmission in relation to warning information is generated from the information providing unit 860. something to do. Methods for transmitting the warning information may include, for example, display, voice, vibration, light, text, communication, etc., but are not limited thereto. As in the above-described example, the transmitting means may be a screen that displays warning information, a warning light that turns on, a speaker that plays a voice or warning sound, or a device such as a vibrating buzzer. Alternatively, the information provider 860, that is, the density calculation device 800, refers to a line including a wired or wireless circuit or communication network in the form of a communication message according to any standard, or the line may refer to any other type of device or system including a computer device connected to the density calculation device 800 and configured to transmit the warning information.

Figure 9 is a conceptual diagram of an embodiment of a crowd density calculation and information provision system according to a preferred embodiment of the present invention. The system 900 of FIG. 9 can be understood as a kind of preferred embodiment embodying the method described with reference to FIGS. 1 to 7 and the device and/or system as shown in FIG. 8, but is not the only, particularly recommended, or particularly preferred embodiment. It does not imply a more advantageous embodiment.

In the system 900 of FIG. 9, a plurality of CCTVs 910 may be configured as a photographing unit. The plurality of CCTVs 910 provide video information in the form of streaming video or recorded video to the CCTV video collection unit 940, and the streaming video information is collected by the CCTV streaming video collection function 942. Can be collected by the CCTV recorded video file collection function (944). The CCTV recorded video file collection function 944 may be configured to re-collect video information collected in real time by the CCTV streaming video collection function 942 in the form of a file. The plurality of CCTVs 910 are connected to the CCTV image collection unit 940 via a standard RTSP (Real-time Streaming Protocol) server 920, a VMS (Video management systems) server 922, or , It can be placed on a network via a network video recorder (NVR) server 924 server.

The CCTV surveillance video provided by the CCTV image collection unit 940 may be provided to the crowd density calculation unit 950. The crowd density calculation unit 950 extracts a head object from the surveillance video (952) and determines the occurrence of a step-by-step event according to the density according to the number of pixels of the head object and the number of pixels between the head objects (954). It may be configured to include. The crowd density calculation unit 950 can generate warning information in situations where the crowd density exceeds a predetermined density, and the warning information is provided to the information provision unit 960, and the information provision unit 960 generates and provides information providing API ( 962) can be configured to be transmitted.

Additionally, for the operation of the system 900 as described above, various system operation information may be set by the basic information unit 930. For example, a function for setting (932) information for determining the occurrence of a staged event according to the density according to the number of pixels of the head object and the number of pixels between the head objects, or a function for managing all APIs connected to the system It may be configured to include a function 934 or a function 936 for setting information for information management and synchronization for each of the plurality of CCTVs 910.

Figure 10 is a flowchart summarizing the crowd density calculation method according to an embodiment of the present invention. Figure 10 is a general summary of the crowd density calculation method according to the present invention, and the system 900 of Figure 9 is the method described through Figures 1 to 7 and the device and/or system as shown in Figures 8 to 9. It can be understood as a kind of preferred embodiment of the operating method, but does not mean that it is the only, particularly recommended, or particularly more advantageous embodiment.

In the method according to an embodiment of the present invention, surveillance images can first be acquired from a capture unit such as CCTV (S1010). Next, the acquired surveillance video can be analyzed to identify at least one human head as an object, and the display size of each object displayed in the surveillance video can be obtained (S1020). If the number of objects is less than the first standard number (S1025), subsequent calculations related to adjacent objects can be skipped and it can be determined that there is no problem (S1095).

If not, each object may be sequentially selected (S1030), and adjacent objects with a standard number of people adjacent to the object may be designated to create a group of adjacent objects (S1040). Next, the individual adjacent interval distance for each adjacent object belonging to the adjacent object group can be calculated from the object (S1050). Next, the individual adjacent spacing distance can be corrected (S1060) based on the difference in display size of each object. Next, the average adjacent distance of the object may be calculated based on the average of the individual adjacent distances (S1070). The above calculation procedure can be repeated for each object in the screen (S1080).

When the calculation of the adjacent distance is completed for all objects, the number of dense objects whose adjacent distance, that is, the average distance between objects, is less than or equal to the standard distance is counted, and it is determined whether the number of dense objects is greater than the second standard number (S1085 )can do. If the number of dense objects is less than the second standard number, it can be determined (S1095) that there is no problem. Conversely, if the number is greater than the second standard number, it can be determined (S1090) that a risk factor has been discovered.

After the acquired surveillance image is judged to be dangerous (S1090) or not dangerous (S1095), a new surveillance image may be acquired (S1010) and the above procedure may be repeated.

As described above, the flow chart of FIG. 10 generalizes and shows the sequential implementation method of the main implementation method of the present invention disclosed in the present invention, and some non-essential steps shown in the flow chart of FIG. 10 may be omitted, On the other hand, some steps may be executed repeatedly, may be configured to be judged differently in several steps, and may also change before and after the processing order.

Although the present invention has been described above with reference to the drawings and examples, as already described above, this does not mean that the scope of protection of the present invention is limited by the drawings or examples presented above, and those skilled in the art will It will be understood that various modifications and changes can be made to the present invention without departing from the spirit and scope of the present invention as set forth in the claims.

Claims (20)

In the method of calculating crowd density by video analysis,
setting a first length related to at least one of the width, length, and height of a person's head and a second length related to the distance between people;
Setting a third standard number of people related to crowd density;
Obtaining surveillance images of the surveillance area from a photographing device;
Analyzing the surveillance video to identify at least one human head as an object;
If the number of objects identified in the surveillance video is less than or equal to the third standard number of people, omitting subsequent steps and calculating that the crowd density is less than or equal to the third standard density regardless of the number of crowded objects;
If the number of objects identified in the surveillance video exceeds the third standard number of people,
Obtaining a first measurement value indicating a size at which the object is displayed in the surveillance video;
Obtaining a second measurement value representing the distance between the object and at least one adjacent object based on a size displayed in the surveillance image;
calculating a ratio between the first measurement and the first length, and calculating an adjacent spacing distance of the object based on the second measurement and the ratio;
classifying the object as the dense object on an object-by-object basis when the adjacent distance between the objects is less than or equal to the second length; and
A method comprising: calculating the crowd density of the surveillance area by counting the number of dense objects identified from the surveillance video.
According to claim 1,
Setting a first standard number of people related to the density of the crowd;
When the number of crowded objects is greater than or equal to the first standard number of people, determining that the crowd density is greater than or equal to the first standard density and generating first warning information for the surveillance area; and
Transmitting the first warning information in at least one of display, voice, vibration, light, text, and communication methods. The method further comprising a.
According to claim 2,
setting a second standard number of people related to the density of the crowd and greater than the first standard number of people;
When the number of crowded objects is greater than or equal to the second standard number of people, determining that the crowd density is greater than or equal to the second standard density and generating second warning information for the surveillance area; and
Transmitting the second warning information by at least one method among display, voice, vibration, light, text, and communication,
The first warning information and the second warning information are different.
According to claim 2,
The method of transmitting the first warning information includes transmitting the first warning information to at least one person located in the surveillance area.
According to claim 2,
The method of transmitting the first warning information includes transmitting the first warning information to at least one of a manager of the surveillance area, a manager of the imaging device, and a safety manager of the surveillance area.
According to claim 2,
The step of transmitting the warning information includes transmitting the first warning information to at least one of a private or public management organization, police agency, fire agency, military agency, central administrative agency, and local administrative agency related to the surveillance area. A method comprising steps.
delete According to claim 1,
The surveillance video is an image expressed by pixel information,
The first measurement value is obtained based on the number of pixels that the object occupies in the surveillance image,
Wherein the second measurement is obtained based on the number of pixels existing between the object and the adjacent object.
According to claim 1,
Further comprising obtaining a third measurement value representing the size at which the adjacent object is displayed in the surveillance image,
The method of claim 1 , wherein calculating the adjacent spacing distance includes correcting the second measurement based on a ratio of the first measurement and the third measurement.
According to claim 1,
It further includes setting a fourth standard number of people representing the number of adjacent objects to be included in the calculation of the adjacent spacing distance,
The step of calculating the adjacent distance includes specifying the fourth standard number of adjacent objects adjacent to the object, calculating individual adjacent distances for each adjacent object from the object, and determining the individual adjacent distances. Comprising the step of calculating the adjacent interval distance based on the average,
The method wherein the average is derived by any one of the average methods belonging to arithmetic mean, geometric mean, harmonic mean, weighted mean, median, mode, and power mean.
According to claim 10,
The step of calculating the adjacent spacing distance is,
Obtain individual second measurements from the object for other objects whose horizontal positions overlap with the object in the surveillance video, and designate the adjacent object of the fourth reference number of people with the smallest second measurement value with the object, , calculating an individual adjacent spacing distance for each adjacent object from the individual second measurement value, and calculating the adjacent spacing distance based on an average of the individual adjacent spacing distances.
According to claim 10,
The step of calculating the adjacent spacing distance is,
Obtaining an individual second measurement value from the object to at least one other object identified in the surveillance video, calculating an individual adjacent interval distance for each other object from the individual second measurement value, and calculating an individual second measurement value from the object to the object. A method comprising designating adjacent objects of the fourth standard number with the smallest adjacent distance, and calculating the adjacent distance based on the average of the individual adjacent distances for each adjacent object.
In a device for calculating crowd density by video analysis,
Memory;
at least one processor connected to the memory; and
It includes a filming unit that acquires surveillance images of the surveillance area,
The memory stores a first length related to at least one of the width, length, and height of a person's head, a second length related to the distance between people, and a third reference number of people related to the density of the crowd,
The processor,
An image collection unit that acquires surveillance images from the photographing unit;
an object extraction unit that analyzes the surveillance video and identifies at least one human head as an object; and
If the number of objects identified in the surveillance video is less than the third standard number of people, subsequent operations are skipped and the crowd density is calculated to be less than the third standard density regardless of the number of dense objects, and the object identified in the surveillance video is calculated as If the number exceeds the third reference number of people, a first measurement value indicating the size of the object displayed in the surveillance video is obtained, and the distance between the object and at least one adjacent adjacent object is displayed in the surveillance video. Obtaining a second measurement representing a size, calculating a ratio between the first measurement and the first length, calculating an adjacent spacing distance of the object based on the second measurement and the ratio, and If the adjacent distance is less than or equal to the second length, the object is classified as the dense object on an object basis, and the crowd density of the surveillance area is calculated by counting the number of the dense objects identified from the surveillance video. A device comprising a density calculation unit.
According to claim 13,
The memory further stores a first reference number of people related to the density of the crowd,
The processor,
When the number of crowded objects is greater than or equal to the first standard number of people, a warning determination unit determines that the crowd density is greater than or equal to the first standard density and generates first warning information for the surveillance area; and
An information provider that transmits the first warning information in at least one of display, voice, vibration, light, text, and communication methods. The device further includes a.
According to claim 14,
The memory further stores a second reference number of people, which is related to the density of the crowd and is greater than the first reference number of people,
The warning determination unit is configured to determine that the crowd density is greater than the second standard density when the number of crowded objects is greater than or equal to the second standard number of people, and to generate second warning information for the surveillance area,
The information provider is configured to transmit the second warning information by at least one method among display, voice, vibration, light, text, and communication,
The first warning information and the second warning information are different from each other.
delete According to claim 13,
The surveillance video is an image expressed by pixel information,
The crowd density calculation unit obtains the first measurement based on the number of pixels that the object occupies in the surveillance image, and the second measurement is obtained based on the number of pixels existing between the object and the adjacent object. A device configured to:
According to claim 13,
The crowd density calculation unit obtains a third measurement value indicating the size of the adjacent object displayed in the surveillance video, corrects the second measurement value based on the ratio of the first measurement value and the third measurement value, and The device is configured to calculate the adjacent spacing distance based on the second measurement.
According to claim 13,
The memory further stores a fourth reference number indicating the number of adjacent objects to be included in the calculation of the adjacent spacing distance,
The crowd density calculation unit specifies adjacent objects of the fourth standard number of people adjacent to the object, calculates individual adjacent distances for each adjacent object from the object, and calculates the individual adjacent distances based on the average of the individual adjacent distances. Configured to calculate the adjacent spacing distance,
The apparatus, wherein the average is derived by any one of averaging methods belonging to arithmetic mean, geometric mean, harmonic mean, weighted mean, median, mode, and power mean.
According to claim 13,
The processor further includes a basic information unit that stores basic information for operating the device in the memory,
The basic information unit includes the first length, the second length, a first standard number of people related to the density of the crowd, a second standard number of people related to the density of the crowd and greater than the first standard number of people, and a third information related to the density of the crowd. A device configured to receive at least one of the standard number of people and a fourth standard number of people indicating the number of adjacent objects to be included in the calculation of the adjacent spacing distance and store them in the memory.

Images