JP2020173775A - Object detector and congestion state management device - Google Patents

Abstract

To optimize a hyper parameter relating to the accuracy of object detection.SOLUTION: An object detector pertaining to the present invention comprises: storage means in which a first threshold and a second threshold larger than the first threshold are stored; acquisition means for acquiring an image; detection means for detecting a prescribed object existing in a space on the basis of the image; calculation means for calculating, for each of the prescribed objects detected, a value that indicates the likelihood of the prescribed object being existent; determination means for determining a congestion area, out of the areas in the image, in which a greater number of the prescribed objects than a prescribed count are detected; first detection means for detecting an object, among the prescribed objects detected in the congestion area, the value of which is larger than the first threshold; and second detection means for detecting an object, among the prescribed objects detected in a non-congestion area, except the congestion area, out of the areas in the image, the value of which is larger than the second threshold.SELECTED DRAWING: Figure 9

Description

The present invention relates to an object detection device and a congestion status management device.

Conventionally, vacant seat management systems for commercial facilities and the like have been known. For example, Patent Document 1 describes a digital signage control device that displays a congestion status of each store in a digital signage arranged near an entrance plaza of a commercial complex (shopping mall). The monitoring device 30 installed in each monitored store is connected to each vacant seat detection device such as a sensor 30a for detecting human feeling and a camera 30b for image analysis of the presence of a person, and is on the cloud via the network 40. The vacant seat information of the monitored store is transmitted to the control server 10.

Further, Patent Document 2 describes a congestion information providing system that can determine the congestion status with one camera and can secure more real-time performance. The congestion information providing system includes a fixed-point camera 10 for observing a congestion situation, a congestion information calculation device 20 connected to the fixed-point camera 10, and an information display device 40 for displaying congestion information. The congestion information calculation device 20 measures the number of people in the image based on the person detecting means for detecting the person area included in the image captured by the fixed point camera 10 and the person area detected by the person detecting means. It includes a measuring means and a providing means for providing a user with congestion information based on the measurement result of the measuring means.

Japanese Patent No. 6153177 JP-A-2007-201556

When managing the congestion status of a food court in a complex commercial facility by using a vacant seat management system as described in Patent Document 1 or a congestion information providing system as described in Patent Document 2, the presence of a person is imaged. It is possible to take an image with the camera to be analyzed, measure the number of people in the image, and provide congestion information to the user based on the measurement result.

When detecting a person in an image, the accuracy of the person detection can generally be adjusted by a hyperparameter (threshold value) for controlling the behavior of a machine learning algorithm. Models used in machine learning have parameters that are not learned by machine learning and must be set by the analyst, which are called hyperparameters.

If the hyperparameters related to the person detection accuracy are adjusted low, it is easy to detect the person, but the accuracy that the detection result is certainly a person is low. That is, it becomes easier to detect objects other than humans as humans. On the other hand, if this hyperparameter is adjusted high, it is difficult to detect a person, but the accuracy that the detection result is certainly a person becomes high. That is, it becomes difficult for a person to detect an object other than a person.

In fact, the way to adjust hyperparameters is, for example, to create a model by brute force the values that look good, compare the results, and select the values that look better from the values that look good. It will be possible.

However, in a facility such as a food court, when there are stores that are crowded in a line and stores that are vacant, in the image, there are crowded areas where people are crowded and non-crowded areas where they are not. Exists. Therefore, in a congested area in an image, if only a part of the person appears in the image, for example, the people are densely overlapped with each other, the accuracy of the person detection is inevitably low. On the other hand, in a non-crowded area, most of the people are likely to appear in the image, and the accuracy of person detection is high.

Therefore, if there is a crowded area in the image where people are crowded and a non-crowded area where it is not, if the hyperparameters are adjusted high, it is easy to detect people even in the crowded area, but the detection result in the image is certain. On the other hand, if the hyperparameters are adjusted too low, it becomes difficult to detect people, especially in a congested area in the image.

The present invention has been proposed in view of the above points, and one aspect of the present invention is to optimize hyperparameters related to object detection accuracy.

In order to solve the above problems, the object detection device according to the present invention is an object detection device that detects a predetermined object existing in a predetermined space based on an image of the space, and has a first threshold value and the said. A storage means for storing a second threshold value larger than the first threshold value, an acquisition means for acquiring the image, and a detection means for detecting a predetermined object existing in the space based on the image were detected. For each of the predetermined objects, a calculation means for calculating a value indicating the certainty that the predetermined object exists, and a congested area in which a predetermined number or more of the predetermined objects are detected in the area in the image are determined. A determination means for detecting an object whose value is larger than the first threshold value among the predetermined objects detected in the congestion area, and the congestion area in the area in the image. It has a second detecting means for detecting an object whose value is larger than the second threshold among the predetermined objects detected in the non-congested area excluding the above.

According to the embodiment of the present invention, it is possible to optimize the hyperparameters related to the object detection accuracy.

It is a figure which shows the configuration example of the congestion situation management system which concerns on this embodiment. It is a figure which shows the hardware configuration example of the management server which concerns on this embodiment. It is a figure which shows the software configuration example of the management server which concerns on this embodiment. It is a figure which shows the data example of the congestion situation management DB which concerns on this embodiment. It is a figure which shows an example of the display screen of the digital signage which concerns on this embodiment. It is a figure which shows the sample image (the 1) which concerns on this embodiment. It is a figure which shows the sample image (the 2) which concerns on this embodiment. It is a flowchart which shows the person detection process which concerns on this embodiment. It is a flowchart which shows the person's knowledge which concerns on this embodiment. An example of the detection result based on the food court image according to this embodiment is shown. An example of the detection result based on the security checkpoint image according to this embodiment is shown.

Embodiments of the present invention will be described in detail with reference to the drawings.

<System configuration>
(Network configuration)
FIG. 1 is a diagram showing a configuration example of a congestion status management system according to the present embodiment. In the congestion status management system 100 of FIG. 1, a management server 10, a digital signage 20a, a mobile terminal 20b, a camera 31, and a GW (gateway) 32 are connected via a network 50.

The management server 10 is a device that determines the congestion status of the facility 30 based on the image captured by the camera 31 and distributes the congestion status information of the facility 30 to display devices such as the digital signage 20a and the mobile terminal 20b. The delivered congestion status information is displayed on the displays of the digital signage 20a and the mobile terminal 20b.

The digital signage 20a is an image display device such as a liquid crystal display installed on a facility entrance, a floor guide wall surface, or the like. The digital signage 20a is connected to the management server 10 on the cloud via the network 50, and displays the congestion status information of the facility 30.

The mobile terminal 20b is various information processing devices including a smartphone, a tablet terminal, a mobile phone, and the like possessed by the user. A general-purpose web browser or a dedicated application is installed in advance. The mobile terminal 20b is connected to the management server 10 on the cloud via the network 50 like the digital signage 20a, and displays the congestion status information of the facility 30.

The facility 30 is a predetermined space provided in a facility or the like to be monitored for the congestion status (hereinafter, simply referred to as a facility). In the present embodiment, for example, it is a food court facility, and the congestion status of the food court space can be monitored.

The camera 31 is an imaging device installed in the facility 30. The camera 31 captures images of the facility 30 (including still images and moving images) in real time, and the captured images are transmitted to the management server 10 one by one via the GW 32. In the present embodiment, one camera 31 covers the entire area of the facility 30 to be monitored for the congestion status within the imaging range. However, the number of cameras 31 installed is not limited to this, and the entire area of the facility 30 may be covered as an imaging range by a plurality of cameras 31 as needed.

The GW 32 is a communication relay device such as a router connected to the camera 31 and the network 50. The image captured by the camera 31 is transmitted to the management server 10 via the GW 32. The camera 31 and the GW 32 may be integrated with a communication function, and in this case, the GW 32 may not be required.

The network 50 is a communication network including wired and wireless. The network 50 includes, for example, the Internet, a public network, WiFi (registered trademark) and the like.

(Hardware configuration)
FIG. 2 is a diagram showing a hardware configuration example of the management server according to the present embodiment. The management server 10 includes a CPU (Central Processing Unit) 11, a ROM (Read Only Memory) 12, a RAM (Random Access Memory) 13, an HDD (Hard Disk Drive) 14, and a communication device 15.

The CPU 11 executes various programs and performs arithmetic processing. The ROM 12 stores programs and the like required at startup. The RAM 13 is a work area for temporarily storing the processing in the CPU 11 and storing the data. The HDD 14 stores various data and programs. The communication device 15 communicates with another device via the network 50.

(Software configuration)
FIG. 3 is a diagram showing a software configuration example of the management server according to the present embodiment. The management server 10 has image acquisition unit 101, person detection unit 102, accuracy calculation unit 103, area determination unit 104, congestion area detection unit 105, non-congestion area detection unit 106, output unit 107, and storage as main functional units. It has a part 108.

The image acquisition unit 101 acquires an image of the facility from the camera 31 installed in the facility 30.

The person detection unit 102 detects a person (person / human body) existing in the facility based on the image acquired by the image acquisition unit 101.

The accuracy calculation unit 103 calculates a value of the certainty degree indicating the certainty that the person exists for each person detected by the person detection unit 102.

The area determination unit 104 determines a congested area and a non-congested area among the areas in the image of the facility. Specifically, among the areas in the image of the facility, the area where a predetermined number or more of people are detected is determined as a congested area. In addition, among the areas in the image of the facility, the areas other than this congested area are determined to be non-congested areas.

The congestion area detection unit 105 detects a person whose certainty is higher than the first threshold value of the storage unit 108 among the persons detected in the congestion area.

The non-congested area detection unit 106 detects a person whose certainty is higher than the second threshold value of the storage unit 108 among the persons detected in the non-congested area.

The output unit 107 outputs the existing area of the person detected by the congested area detection unit 105 and the non-congested area detection unit 106 among the areas in the image of the facility.

The storage unit 108 stores a first threshold value (hyperparameter for a congested area) and a second threshold value (hyperparameter for a non-congested area) having a value larger than the first threshold value. It also stores the congestion status management DB 108a. The congestion status management DB 108a is a database for managing the congestion status information of the facility 30 in real time.

Each functional unit is realized by a computer program executed on hardware resources such as a CPU, ROM, and RAM of a computer constituting the management server 10. These functional parts may be read as "means", "module", "unit", or "circuit".

(Congestion status management DB)
FIG. 4 is a diagram showing a data example of the congestion status management DB according to the present embodiment. The congestion status management DB 108a is a database for managing the congestion status information of the facility 30 in real time. The congestion status management DB 108a has data items such as "facility ID", "facility name", and "equipment".

The "facility ID" indicates a unique identifier numbered for each facility in order to uniquely identify the facility. "Facility name" indicates the name of the facility.

“Equipment” refers to equipment to be monitored for congestion, and refers to specific equipment provided to users at the facility. Multiple facilities are set according to the target facility. For example, when the facility is a food court of a shopping center, the equipment to be monitored for the congestion status is each restaurant provided to the user. Further, for example, when the facility is an airport security checkpoint, the equipment to be monitored for the congestion status is each security checkpoint provided to the user.

Furthermore, congestion status information is managed and updated in real time for each "equipment". As an example, the information on the congestion status includes "empty", "slightly crowded", and "very crowded". However, the present invention is not limited to this, and management may be performed according to, for example, a specific number of people.

(Digital signage display screen)
FIG. 5 is a diagram showing an example of a display screen of the digital signage according to the present embodiment. The digital signage 20 of FIG. 5 is installed at the facility entrance, the floor guide wall surface, and the like, and displays the congestion status information of the facility 30 to the user.

The food court of the facility 30 contains restaurants such as "Cafe" 201, "Ramen" 202, "Takoyaki" 203, "Crepe" 204, and "Hamburger" 205. As shown in FIG. 5, information on the congestion status of each restaurant is displayed on the display screen of the digital signage 20a.

Specifically, "Cafe" 201 and "Crepe" 204 are "vacant", "Ramen" 202 and "Hamburger" 205 are "very crowded", and "Takoyaki" 203 is "slightly crowded". The congestion status information is displayed. By referring to the display screen of the digital signage 20a, the user can easily grasp each congestion status.

<Image processing>
FIG. 6 is a diagram showing a sample image (No. 1) according to the present embodiment. The image of FIG. 6 corresponds to the original image captured by the camera 31. In this original image, a place where people are crowded near the center of the image and a place where people are vacant near the bottom of the image are shown. Hereinafter, in the image, an image area in which people are crowded is referred to as a crowded area, and an image area in which people are not crowded is referred to as a non-crowded area.

FIG. 7 is a diagram showing a sample image (No. 2) according to the present embodiment. The image of FIG. 7 shows the processed image after the management server 10 executes the object detection process on the original image of FIG. In the processed image, for example, a plurality of objects such as "person" (person), "clock" (clock), and "handbag" (handbag) are detected, and a square is detected in the area where the detected object exists. It can be seen that the (rectangular) focus frame is displayed.

Of these, the image area a is a congested area because a predetermined number or more of people (persons) are detected in a rectangular area having a predetermined size. On the other hand, the image area b is a non-congested area because a predetermined number or more of people (persons) are not detected in the rectangular area of a predetermined size.

Further, when the management server 10 detects a person in the object detection process, the management server 10 also calculates a value (referred to as accuracy) indicating the certainty. That is, the accuracy corresponding to each detected object or each person (which can be said to be each focus frame) is calculated, and the accuracy is calculated so high that the detected object is probable as a person. On the contrary, the probability that the detected object is so low that it is not certain as a human being is calculated. And the lower the accuracy, the higher the possibility that the detected object is other than a person. For example, when most of a person (human body) is shown in an image, the certainty as a person is judged to be high, so that the certainty of the detection result is calculated to be high. On the other hand, when only a part of a person (human body) is shown in the image, the certainty as a person is judged to be low, and even if it is detected, the certainty of the detection result is calculated to be low.

Next, the management server 10 finally determines that the person is a person and detects the detection result whose accuracy is higher than the hyperparameter value among the detection results detected to be a person. Detection results lower than the hyperparameter value are excluded. Therefore, if the hyperparameters are adjusted low, human detection itself is likely to occur, but objects other than humans are also likely to be detected as humans. On the other hand, if this hyperparameter is adjusted high, human detection itself is difficult to detect, but objects other than humans are difficult to be detected as humans.

See FIG. 7 again. Since people are crowded in the image area a (congested area), it is often the case that only a part of the person (human body) is shown in the image because the people are densely overlapped with each other. Occasionally, only personal belongings such as hats, bags, shoes, or parts of them are shown. For example, if only the arm of a person hidden in the shadow of a person in the image is shown, the accuracy is low because it is detected as a person but the certainty as a person is judged to be low. It is calculated.

Therefore, in the image area a (congested area), if the hyperparameter is adjusted low, it becomes easier to detect a person whose image shows only a part of a person (human body), and there is an advantage that detection omission can be eliminated. is there. However, it becomes easier to detect objects other than humans as humans. On the other hand, if the hyperparameters are adjusted high, a person can be reliably detected, but a person or the like in which only a part of the person (human body) is shown in the image is difficult to be detected, and detection omission is likely to occur.

In the image area b (non-congested area), since people are not crowded in the image, most of the person (human body) is often shown in the image. Therefore, rather than adjusting the hyperparameters low to make it easier to detect people who only show a part of the person (human body) in the image, adjusting the hyperparameters high makes it possible to detect objects other than humans. There is a merit of those who eliminate the detection as a person and detect the person reliably.

In the past, the hyperparameters were often adjusted to moderate intermediate values without being too high or too low. However, it cannot be said that the adjustment is optimal for an image including a congested area and a non-congested area. That is, in the present embodiment, in an image including a congested area and a non-congested area, by applying a low hyperparameter to the congested area, a person or the like in which only a part of a person (human body) is shown in the image is detected. It will be easier to eliminate detection omissions. In addition, by applying high hyperparameters to non-congested areas, it is possible to eliminate the detection of objects other than humans as humans and ensure that humans are detected.

In particular, in the image area a (congested area), if the hyperparameter is adjusted low, it becomes easy to detect an object other than a person as a person. However, the adjustment of hyperparameters has a trade-off relationship between the risk of being determined to be absent and the risk of being determined to be absent. When comparing the two, it is more problematic that it is determined that there are no people in the congestion status management system or vacant seat management system that grasps the number of people in the crowded area.

<Information processing>
Next, the person detection process and the person detection process executed by the management server 10 will be described.

(People detection processing)
FIG. 8 is a flowchart showing a person detection process according to the present embodiment.

S1: The management server 10 determines whether or not the image captured by the camera 31 has been received. In addition, a "facility ID" for uniquely identifying the facility 30 is received together with the image. However, it is not always necessary to use the "facility ID", and it is sufficient that the management server 10 can identify which facility 30 the image belongs to, for example, by using the unique identifier of the GW 32. When the image is received, the process proceeds to S2.

S2: The management server 10 executes a person detection process on the received image (original image). The person detection process determines the presence or absence of a person in the image, and if there is a person, the "human existence area" where the person in the image will exist and the certainty that the person exists in that area. The "accuracy" indicating the above is calculated. The "human existence area" can be output, for example, in a square (rectangular) focus frame (FIG. 7).

S3: The management server 10 outputs the detection result. The detection result is a set of "human existence area" in which a person will exist in the image and "probability" indicating the certainty that a person exists in the area. The accuracy can be, for example, a value from 0 to 1. Also, the closer it is to 1, the higher the certainty that a person exists in that area.

In the human detection process, an image recognition algorithm using a machine learning method such as YOLO (You Only Look None) can be used.

(Supplement)
The above-mentioned person detection process will be supplemented below.
(1) For example, further, based on the physical size (size) of a predetermined object reflected in the image, and considering the size in the image physically assumed as a person, the term "human existence area" is used. By calculating the "accuracy", it is possible to improve the accuracy of detecting people and thus the accuracy of determining a congested area.

For example, referring to the images of FIGS. 6 and 7, an image object is installed in the center of the image, a seat is installed in the aisle, a lighting fixture is installed on the ceiling, and a glass window is installed on the side of the ceiling. Therefore, if at least the physical size of an object other than these people is known, the general size of a person (for example, 150 cm in height size) when calculating the "human existence area" and "accuracy" ~ 200 cm, face size is about 20-25 cm, face width is about 15-18 cm), which is extremely large or too small due to the comparison with the physical size of these objects. If it deviates from the above, that is, if the size of the person in the image is not appropriate, the "probability" can be corrected to be small even if the "human existence area" is not set or the "human existence area" is set.

In the case of an image, it goes without saying that size correction is performed because the front side is large and the depth side is small. In addition, since the sizes of image objects, lighting fixtures, glass windows, etc. often vary depending on the object, it is difficult to specify the physical size of these objects, but among them, which are the seats, etc. Even if it is a seat, its size can be specified by a substantially constant size (for example, the seat portion is about 50 to 60 cm). That is, it is desirable that the object used as a reference in the image is a standard product whose so-called size (known information) can be uniquely specified.

(2) Further, with reference to the physical position of a predetermined object (particularly a structural building, etc.) reflected in the image, and considering the position in the image that can be physically positioned as a person, the "human existence area" By calculating "" and "accuracy", it is possible to improve the accuracy of detecting people and thus the accuracy of determining a congested area.

For example, referring to the images of FIGS. 6 and 7, in the image, there is a passage from the front to the depth side, an image object in the center, a seat in the passage, a lighting fixture on the ceiling, and a glass window on the side of the ceiling. Etc. are installed. Therefore, it cannot be physically positioned as a person in the vicinity of the upper part of the image object, which is higher than the height of the person, in the position where it overlaps with the lighting fixture and the glass window, or in the vicinity thereof. In this way, if the position as a person in the image is not appropriate from the comparison relationship with the physical position of these objects, the "human existence area" is not set, or the "probability" is set as the "human existence area". It can be corrected to be small.

(3) In addition, the "human existence area" and "accuracy" are taken into consideration in consideration of the positional relationship (distance relationship) between the images of the person (human body) appearing in the image, especially only a part of the image such as the head and face. By calculating and, it is possible to improve the accuracy of detecting people and thus the accuracy of determining a congested area.

As described above, when most of the person (human body) is shown in the image, the certainty as a person is judged to be high, so the accuracy of the detection result is calculated to be high, but on the other hand, the person in the image. When only a part of (human body) is shown, the certainty as a person is judged to be low, so even if it is detected, the certainty of the detection result is calculated to be low. Further, as described above, the camera 31 captures an image in a form of looking down from relatively above in order to cover the entire area of the facility 30 as an imaging range. For this reason, in a crowded area, there are many images in which only a part of a person (human body), especially the head or face, is shown in the image.

Therefore, considering the positional relationship (distance relationship) between some images in the image, some images in which only a part such as the head and the face are shown exist at a position closer (closer distance) than a predetermined value to each other. If there are some images, or if some of the images overlap each other, the "accuracy" as the "human existence area" can be greatly corrected.
(4) Based on the physical position of a specific object (object registered in advance) that appears in the image, and considering the positional relationship (distance relationship) in the image as a person, the "human existence area" and "human existence area" By calculating "accuracy", it is possible to improve the determination accuracy of the congested area.

One of the characteristics is that people tend to concentrate around a given facility. Specifically, it is a typical example that they are lined up in front of the shops and security inspection lanes described later. Therefore, in consideration of the positional relationship (distance relationship) with a specific object (pre-registered shop, security inspection lane, etc.) in the image, a part of the image such as the head or face is specified. When it exists at a position (close distance) close to the object, its "accuracy" as a "human existence area" can be greatly corrected.

(People detection process)
FIG. 9 is a flowchart showing a person detection process according to the present embodiment.

S11: The management server 10 executes the congestion area determination process. In the congestion area determination process, when a predetermined number or more of people (people existing area) are detected in the predetermined rectangular area based on the detection result of S3 in the image, the rectangular area is determined as the congestion area. On the other hand, the other rectangular area, that is, the rectangular area in which a predetermined number or more of people (people existing area) are not detected is determined as a non-congested area.

S12: If it is determined that there is at least one congested area in the image, the process proceeds to S13 assuming that there is a congested area. If there is no congested area, proceed to S15.

S13: The management server 10 acquires the hyperparameters for the congested area. The hyperparameters for the congested area are stored in advance in the storage unit 108.

S14: The management server 10 detects a detection result (a set of a human existence area and an accuracy) detected from within the congestion area, whose accuracy is higher than that of the hyperparameter for the congestion area. The hyperparameter can take a value from 0 to 1, for example.

For example, suppose that the hyperparameter for a congested area is "0.01" (accuracy 1%). Further, as the detection results detected from within the congested area, for example, the accuracy of the detection result 1 is "0.5", the accuracy of the detection result 2 is "0.001", and the accuracy of the detection result 3 is "0.005". It is assumed that the accuracy of the detection result 4 is "0.015" and the accuracy of the detection result 5 is "0.5". As mentioned above, in a crowded area, people are crowded, so people are often crowded and overlapped, and only a part of the person (human body) is shown in the image, so the accuracy is low. Many detection results are output. In this case, the detection result 1, the detection result 4, and the detection result 5 having higher accuracy than the hyperparameter "0.01" for the congested area are detected.

S15: The management server 10 acquires hyperparameters for non-congested areas. The hyperparameters for non-congested areas are stored in advance in the storage unit 108.

S16: The management server 10 detects a detection result (a set of a human presence area and an accuracy) detected from within the non-congested area, whose accuracy is higher than that of the hyperparameter for the non-congested area.

Here, it is assumed that the hyperparameters for non-congested areas are higher than the hyperparameters for congested areas "0.01" described above, for example, "0.5". Further, as the detection results detected from within the non-congested area, for example, the accuracy of the detection result 6 is "0.95", the accuracy of the detection result 7 is "0.03", and the accuracy of the detection result 8 is "0.005". ". In a non-crowded area, since people are not crowded in the image, most of the person (human body) is often shown in the image, and when a person actually exists, the detection is fairly accurate. The result is output. In this case, the detection result 6 having a higher accuracy than the hyperparameter “0.5” for the non-congested area is detected.

S17: The management server 10 outputs the detection result. The detection result is at least a "human presence area" in which a person will be present in the image. Specifically, it is a "human existence area" as the detection result 1, a "human existence area" as the detection result 4, a "human existence area" as the detection result 5, and a "human existence area" as the detection result 6. ..

When the hyperparameters are uniformly "0.2" without determining the congested area and the non-congested area in the image as in the conventional case, the detection result is the "human existence area" as the detection result 1. It becomes the "human existence area" as the detection result 5 and the "human existence area" as the detection result 6. Compared with the detection result of this embodiment, the detection result 4 in the congested area is not detected. Since the accuracy of the detection result 4 is as low as "0.015", there is a possibility that the detection result 4 is actually a person or not a person. However, considering the characteristic that only a part of a person (human body) is often shown in the image in a crowded area, the risk of being determined that there is no person, etc., the detection result of the present embodiment is This is a desirable detection result for a congestion status management system or a vacant seat management system that grasps the number of people in a crowded area.

The management server 10 can finally display the congestion status information of the facility 30 by mapping the detection result with the position area corresponding to the restaurant.

<Facility application example>
FIG. 10 shows an example of the detection result based on the food court image according to the present embodiment. The camera 31 captures an image of the food court as the facility 30 in real time, and the captured image is transmitted to the management server 10 one by one via the GW 32. The image of the food court captured has a congested area a and a non-congested area b. The area around the restaurants of "Ramen", "Hamburger", and "Takoyaki" is judged to be a crowded area.

FIG. 11 shows an example of the detection result based on the security checkpoint image according to the present embodiment. The captured security checkpoint image has a congested area a and a non-congested area b. The security inspection lanes of "lane 1," "lane 2," and "lane 3" are determined to be crowded areas.

In the congested area shown in FIGS. 10 and 11, there are some people who are only partially shown in the image due to the fact that the people are densely overlapped with each other, so whether or not they are people in the person detection process. Areas where people exist with low accuracy are also detected. In the present embodiment, a human presence area having a relatively low accuracy in the congested area is also detected more positively than the non-congested area as an area in which a person is surely present as a detection result.

<Summary>
As described above, according to the congestion status management system 100 according to the present embodiment, when there is a crowded area in the image where people are crowded and a non-crowded area where it is not, the accuracy of person detection is related to the area. Hyperparameters can be optimized. Further, in a congestion status management system or a vacant seat management system that grasps the number of people in a crowded area, it is possible to preferably manage the congestion status, especially when there are a crowded area and a non-crowded area in the image.

In addition, although the present invention has been described by showing specific specific examples according to a preferred embodiment of the present invention, these specific examples do not deviate from the broad purpose and scope of the present invention defined in the claims. It is clear that various modifications and changes can be made to the example. That is, it should not be construed that the present invention is limited by the details of the specific examples and the accompanying drawings.

The facility may be any facility or facility that can be used by the user and is subject to monitoring of the congestion status. For example, it may be a queue at the entrance of a single restaurant / restaurant, an entrance gate to a public facility or a stadium, an attraction in an amusement park, or the like.

Further, the detection target is not limited to a person but may be a predetermined object. For example, the detection target can be a bicycle in a bicycle parking lot. That is, according to the congestion status management system 100 according to the present embodiment, it is possible to acquire an image in which bicycles are parked so as to overlap each other from the surveillance camera of the bicycle parking lot and manage the congestion status.

10 Management server 20a Digital signage 20b Mobile terminal 30 Facility 31 Camera 32 GW
50 Network 100 Congestion status management system 101 Image acquisition unit 102 People detection unit 103 Accuracy calculation unit 104 Area judgment unit 105 Congestion area detection unit 106 Non-congestion area detection unit 107 Output unit 108 Storage unit

Claims (8)

An object detection device that detects a person existing in a predetermined space based on an image of the space.
A storage means that stores a first threshold value and a second threshold value larger than the first threshold value, and
An acquisition means for acquiring the image and
A detection means that detects a person existing in the space based on the image, and
For each of the detected persons, a calculation means for calculating a value indicating the certainty that the person exists, and
A determination means for determining a congested area in which a predetermined number or more of the people are detected in the area in the image, and
Among the persons detected in the congested area, a first detection means for detecting a person whose value is larger than the first threshold value, and
A second detection means for detecting a person whose value is larger than the second threshold value among the persons detected in the non-congested area other than the congested area in the area in the image.
Have,
The calculation means reduces the value when the size of the person detected in the image of the space is not appropriate based on the comparison with the physical size of the predetermined object appearing in the image of the space. To correct,
An object detection device characterized by. The predetermined object is a standard product whose size is uniquely specified.
The object detection device according to claim 1. An object detection device that detects a person existing in a predetermined space based on an image of the space.
A storage means that stores a first threshold value and a second threshold value larger than the first threshold value, and
An acquisition means for acquiring the image and
A detection means that detects a person existing in the space based on the image, and
For each of the detected persons, a calculation means for calculating a value indicating the certainty that the person exists, and
A determination means for determining a congested area in which a predetermined number or more of the people are detected in the area in the image, and
Among the persons detected in the congested area, a first detection means for detecting a person whose value is larger than the first threshold value, and
A second detection means for detecting a person whose value is larger than the second threshold value among the persons detected in the non-congested area other than the congested area in the area in the image.
Have,
The calculation means corrects the value to a small value when the position of the person detected in the image of the space is not appropriate based on the comparison with the physical position of the predetermined object in the image of the space. To do,
An object detection device characterized by. The predetermined object is an object located higher than the height of a person.
The object detection device according to claim 3. An object detection device that detects a person existing in a predetermined space based on an image of the space.
A storage means that stores a first threshold value and a second threshold value larger than the first threshold value, and
An acquisition means for acquiring the image and
A detection means that detects a person existing in the space based on the image, and
For each of the detected persons, a calculation means for calculating a value indicating the certainty that the person exists, and
A determination means for determining a congested area in which a predetermined number or more of the people are detected in the area in the image, and
Among the persons detected in the congested area, a first detection means for detecting a person whose value is larger than the first threshold value, and
A second detection means for detecting a person whose value is larger than the second threshold value among the persons detected in the non-congested area other than the congested area in the area in the image.
Have,
The calculation means increases the value when a plurality of faces or heads of the person are detected in the image of the space, or when the positions of the faces or heads of the persons are closer than a predetermined value or overlap with each other. To correct,
An object detection device characterized by. An object detection device that detects a person existing in a predetermined space based on an image of the space.
A storage means that stores a first threshold value and a second threshold value larger than the first threshold value, and
An acquisition means for acquiring the image and
A detection means that detects a person existing in the space based on the image, and
For each of the detected persons, a calculation means for calculating a value indicating the certainty that the person exists, and
A determination means for determining a congested area in which a predetermined number or more of the people are detected in the area in the image, and
Among the persons detected in the congested area, a first detection means for detecting a person whose value is larger than the first threshold value, and
A second detection means for detecting a person whose value is larger than the second threshold value among the persons detected in the non-congested area other than the congested area in the area in the image.
Have,
When a plurality of faces or heads of the person are detected in the image of the space, the calculation means includes a predetermined object in which the position of the object in the image of the space is registered in advance, and the face or head of the person. If the position of the head is closer than the specified value, correct the above value significantly.
An object detection device characterized by. The predetermined object is equipment provided to the person.
The object detection device according to claim 6. A congestion status management device having the object detection device according to any one of claims 1 to 7.