CN113168694A

CN113168694A - Human detection device and human detection method

Info

Publication number: CN113168694A
Application number: CN201980079319.9A
Authority: CN
Inventors: 田中清明; 辻郁奈; 松永纯平
Original assignee: Omron Corp
Current assignee: Omron Corp
Priority date: 2018-12-26
Filing date: 2019-11-08
Publication date: 2021-07-23
Anticipated expiration: 2039-11-08
Also published as: JP7255173B2; DE112019006414T5; JP2020106970A; CN113168694B; US20220004749A1; WO2020137193A1

Abstract

A person detection device detects a person present in a detection target area by analyzing a fisheye image obtained by a fisheye camera provided above the detection target area, the person detection device including: a head detection unit that detects one or more head candidates from the fisheye image using an algorithm for detecting a human head; a human body detection unit that detects one or more human body candidates from the fisheye image using an algorithm for detecting a human body; and a determination unit configured to determine a pair satisfying a predetermined condition among the pairs of the head candidate and the human body candidate created by combining the detection result of the head detection unit and the detection result of the human body detection unit as a human.

Description

Human detection device and human detection method

Technical Field

The present invention relates to a technique for detecting a person using an image of a fisheye camera.

Background

In the fields of Building Automation (BA) and Factory Automation (FA), there is a need for applications in which the "number", "position", "operation route", and the like of people are automatically measured by an image sensor, and optimum control of equipment such as lighting and air conditioning is performed. In such applications, in order to obtain image information in as wide a range as possible, an ultra-wide-angle camera (referred to as a fisheye camera, an omnidirectional camera, a spherical (japanese: all celestial sphere) camera, or the like) equipped with a fisheye Lens (Fish Eye Lens) is often used.

The image taken by the fisheye camera is severely distorted. Therefore, when detecting a human body, a face, or the like from an image of a fisheye camera (hereinafter referred to as a "fisheye image"), it is a general method of correcting the fisheye image into an image with less distortion by performing plane development in advance and then performing detection processing (see patent document 1).

Documents of the prior art

Patent document

Patent document 1: japanese patent laid-open publication No. 2016-

Disclosure of Invention

Problems to be solved by the invention

However, the following problems exist in the prior art. One problem is that the overall processing cost increases because preprocessing such as planar expansion of the fisheye image occurs. This may make the real-time detection process difficult, resulting in a delay in the control of the apparatus, and is not preferable. A second problem is that an image of a person or an object existing at a position just below a boundary (a slit of an image) at the time of planar expansion, such as directly below a fisheye camera, is largely deformed or divided by the processing of planar expansion, and thus may not be detected accurately.

In order to avoid these problems, the present inventors have studied a method of directly (i.e., without performing plane expansion) detecting a fisheye image. However, in the case of a fisheye image, as compared with a normal camera image, the change in the appearance (inclination, distortion, size of a human body) of a person to be detected increases, and thus detection becomes difficult. In particular, when applications such as BA and FA are assumed, since there are many objects in the image that are likely to be mistaken for a human body or a head, such as a chair, a personal computer, a trash box, an electric fan, and a Circulator (Circulator), detection accuracy is likely to be degraded.

The present invention has been made in view of the above circumstances, and an object thereof is to provide a technique for detecting a person from a fish-eye image at high speed and with high accuracy.

Means for solving the problems

In order to achieve the above object, the present invention adopts the following configuration.

A first aspect of the present invention provides a human detection device that detects a human being present in a detection target area by analyzing a fisheye image obtained by a fisheye camera provided above the detection target area, the human detection device including: a head detection unit that detects one or more head candidates from the fisheye image using an algorithm for detecting a human head; a human body detection unit that detects one or more human body candidates from the fisheye image using an algorithm for detecting a human body; and a determination unit configured to determine a pair satisfying a predetermined condition among the pairs of the head candidate and the human body candidate created by combining the detection result of the head detection unit and the detection result of the human body detection unit as a human.

The "fisheye camera" is a camera equipped with a fisheye lens, and is a camera capable of shooting at a wider angle than a normal camera. An omnidirectional camera or a ball camera is also one of the fisheye cameras. The fisheye camera may be arranged to look down the detection target region from above the detection target region. Typically, the optical axis of the fisheye camera is arranged vertically downwards, but the optical axis of the fisheye camera may also be inclined with respect to the vertical. The "algorithm for detecting a human head" and the "algorithm for detecting a human body" are different algorithms in that the former has only a head as a detection target and the latter has a human body as a detection target. Here, the "human body" may be the whole body of a human, or may be the half body (for example, the upper body, the head, the body, and the like).

According to the present invention, since the fisheye image is not subjected to planar expansion, high-speed detection processing can be performed. In addition, since the head and the body are detected together from the image and determined as "human" only when they satisfy a predetermined condition, highly accurate detection can be realized.

The predetermined condition may include a condition related to a relative position of the head candidate and the human body candidate. In the fisheye image obtained by the fisheye camera, since the positional relationship between the region of the head and the region of the human body has a certain rule, the adequacy of the pair can be determined based on the relative position between the region of the head and the region of the human body (human certainty (からしさ)). Specifically, the predetermined condition may include a condition that the region of the head candidate overlaps with the region of the human body candidate. The predetermined condition may include a condition that the human body candidate exists at a coordinate closer to the center of the fisheye image than the head candidate.

The predetermined condition may include a condition related to a relative size of the head candidate and the human body candidate. Since the sizes of the head and the human body in the fisheye image obtained by the fixed camera can be assumed in advance, the adequacy of the pair (certainty of being a person) can be determined based on the relative sizes of the head candidate and the human body candidate. Specifically, the predetermined condition may include a condition that a size ratio of the head candidate to the human body candidate is within a predetermined range. Here, the determination unit may change the predetermined range according to coordinates of the head candidate or the human body candidate on the fisheye image.

The head detection unit may output the reliability of detection for each detected head candidate, the human body detection unit may output the reliability of detection for each detected human body candidate, and the predetermined condition may include a condition relating to the reliability of the head candidate and the reliability of the human body candidate. This can improve the reliability of the final detection result, that is, the detection accuracy.

For example, the determination unit may determine the total reliability from the reliability of the head candidate and the reliability of the human body candidate, and the predetermined condition may include a condition that the total reliability is greater than a threshold value. The total reliability may be any index as long as it is a function of the reliability of the head candidate and the reliability of the human body candidate. For example, the total, simple average, weighted average, or the like of the reliability of the head candidates and the reliability of the human body candidates may be used.

The determination unit may change the weight of the reliability of the head candidate and the reliability of the human body candidate when the total reliability is obtained, based on the coordinates of the head candidate or the human body candidate on the fisheye image. For example, a person positioned directly below the camera takes a shot of the head, but since the person takes only the left and right shoulders, the detection of the person becomes more difficult than the detection of the head. In this way, since it is changed depending on the coordinates on the image, which of the reliability of the head candidate and the reliability of the human body candidate is likely to be higher, the final determination accuracy can be improved by taking this characteristic into consideration when obtaining the total reliability.

In the case where either one of the reliability of the head candidate and the reliability of the human body candidate is sufficiently high, the determination unit may relax the condition on the reliability of the other. This is because if the reliability of either one is sufficiently high, it is considered that (even if the reliability of the detection of the other is slightly low) the certainty of being a human is high.

A second aspect of the present invention provides a human detection method for detecting a human being present in a detection target region by analyzing a fisheye image obtained by a fisheye camera provided above the detection target region, the human detection method including: a head detection step of detecting one or more head candidates from the fisheye image using an algorithm for detecting a human head; a human body detection step of detecting one or more human body candidates from the fisheye image by using an algorithm for detecting a human body; and a determination step of determining, as a human, a pair satisfying a predetermined condition among the pairs of the head candidate and the human body candidate created by combining the detection result of the head detection step and the detection result of the human body detection step.

The present invention can be understood as a person detection device having at least a part of the above-described means, a person recognition device that recognizes (recognizes) a detected person, a person tracking device that tracks a detected person, an image processing device, or a monitoring system. The present invention can also be understood as a person detection method, a person recognition method, a person tracking method, an image processing method, and a monitoring method including at least a part of the above-described processing. The present invention can also be understood as a program for implementing the method or a recording medium on which the program is recorded in a non-transitory manner. In addition, each of the above units and processes may be combined with each other as much as possible to constitute the present invention.

ADVANTAGEOUS EFFECTS OF INVENTION

According to the present invention, a person can be detected from a fisheye image at high speed and with high accuracy.

Drawings

Fig. 1 is a diagram showing an application example of the human detection device of the present invention.

Fig. 2 is a diagram showing a configuration of a monitoring system including a human detection device.

Fig. 3 is a diagram showing an example of a fisheye image.

Fig. 4 is a flowchart of the human detection process.

Fig. 5 is a diagram showing an example of the result of head detection.

Fig. 6 is a diagram showing an example in which the result of human body detection is superimposed on the result of head detection.

Fig. 7 is a diagram showing an example of the final determination result (human detection result).

Fig. 8 is a flowchart of the pairing process based on the relative position.

Fig. 9 is a flowchart of pairing processing based on relative sizes.

Fig. 10 is a diagram showing an example of the process of changing according to the position on the image.

Detailed Description

< application example >

An application example of the human detection device of the present invention is explained with reference to fig. 1. The person detection device 1 is a device that detects a person 13 present in a detection target area 11 by analyzing a fisheye image obtained by a fisheye camera 10 provided above the detection target area 11 (for example, on a ceiling 12 or the like). The person detection apparatus 1 detects, recognizes, and tracks a person 13 passing through the detection target area 11 in, for example, an office or a factory. The detection result of the human detection device 1 is output to an external device, and is used for counting the number of people, controlling various devices such as lighting and air conditioning, monitoring suspicious persons, and the like.

The human detection device 1 has one of the features that a fisheye image is directly used for human detection processing (i.e., without preprocessing such as plane expansion and distortion correction). This realizes high speed (real-time performance) of the detection process. Further, the human detection device 1 has one of the features of performing head detection and human body detection on a fisheye image, and performing final determination (determination of whether or not a human being is present) by combining the results of the head detection and the results of the human body detection. In this case, by taking the characteristics of the fisheye image into consideration, the matching between the head and the human body and the evaluation of the reliability can be performed, and highly accurate detection can be realized.

< monitoring System >

An embodiment of the present invention is explained with reference to fig. 2. Fig. 2 is a block diagram showing a configuration of a monitoring system to which a human detection device according to an embodiment of the present invention is applied. The monitoring system 2 is roughly provided with a fisheye camera 10 and a human detection device 1.

The fisheye camera 10 is an image pickup apparatus having an optical system including a fisheye lens and an image pickup device (an image sensor such as a CCD or a CMOS). The fisheye camera 10 may be installed on a ceiling 12 or the like of the detection target area 11 with its optical axis directed vertically downward, for example, as shown in fig. 1, and may capture an image of the detection target area 11 in all directions (360 degrees). The fisheye camera 10 is connected to the human detection device 1 by a wire (a USB cable, a LAN cable, or the like) or a wireless (WiFi or the like), and image data captured by the fisheye camera 10 is taken into the human detection device 1. The image data may be a monochrome image or a color image, and the resolution, frame rate, or format of the image data is arbitrary. In the present embodiment, it is assumed that a monochrome image taken at 10fps (10 sheets per second) is used.

Fig. 3 shows an example of a fisheye image captured from the fisheye camera 10. When the fisheye camera 10 is disposed so that the optical axis is directed vertically downward, an image viewed from the top of the head of a person present directly below the fisheye camera 10 appears at the center of the fisheye image. Further, as the angle of depression becomes smaller toward the end of the fisheye image, a human image appears as viewed obliquely from above. Further, the center distortion of the fisheye image is small, but the distortion of the image becomes large toward the end of the fisheye image. As described in the background section, conventionally, image processing such as detection or recognition is performed after a planar developed image in which distortion of a fisheye image is corrected is created, but in the monitoring system 2 of the present embodiment, the fisheye image shown in fig. 3 is used as it is (with distortion maintained) for the processing of detection or recognition. Thus, preprocessing such as distortion correction is omitted, and real-time monitoring is realized.

Returning to fig. 2, the human detection device 1 is explained. The human detection device 1 of the present embodiment includes an image input unit 20, a head detection unit 22, a human body detection unit 24, a determination unit 26, a storage unit 27, and an output unit 28. The head detection unit 22 and the human body detection unit 24 are also collectively referred to as "detection unit 21". The image input unit 20 has a function of acquiring image data from the fisheye camera 10. The acquired image data is stored in the storage unit 27. The head detection unit 22 has a function of detecting a head candidate from a fisheye image using an algorithm for detecting a human head. The head detection dictionary 23 is a dictionary in which image features of a head that may appear in a fisheye image are registered (registered) in advance. The human body detection unit 24 has a function of detecting a human body candidate from a fisheye image by using an algorithm for detecting a human body. The human body detection dictionary 25 is a dictionary in which image features of a human body that may appear in a fish-eye image are registered in advance. The determination unit 26 has a function of determining "a person" present in the fisheye image based on the detection results of the head detection unit 22 and the human body detection unit 24. The storage unit 27 has a function of storing a fisheye image, a detection result, a determination result, and the like. The output unit 28 has a function of outputting information such as a fisheye image, a detection result, and a determination result to an external device. For example, the output unit 28 may display information on a display as an external device, may transmit information to a computer as an external device, or may transmit information or a control signal to an illumination device, an air conditioner, or an FA device as an external device.

The human detection device 1 may be configured by a computer including, for example, a CPU (processor), a memory (memory), a storage (storage), and the like. In this case, the configuration shown in fig. 2 is realized by loading a program saved in the storage into the memory and executing the program by the CPU. The computer may be a general-purpose computer such as a personal computer, a server computer, a tablet terminal, and a smart phone, or may be an embedded computer such as an On-board computer. Alternatively, all or a part of the configuration shown in fig. 2 may be constituted by an ASIC, an FPGA, or the like. Alternatively, all or a portion of the structure shown in fig. 2 may be implemented by cloud computing or distributed computing.

< human detection processing >

Fig. 4 is a flowchart of the human detection process performed by the monitoring system 2. The overall flow of the human detection process will be described with reference to fig. 4. The flowchart of fig. 4 shows the processing for 1 frame of fisheye image. In the case where a fish-eye image is input at 10fps, the process of fig. 4 is performed 10 times in 1 second.

First, the image input unit 20 inputs a fisheye image of 1 frame from the fisheye camera 10 (step S40). Fig. 3 is an example of an input fisheye image. 5 persons were photographed in this image.

Next, the head detecting unit 22 detects the head of the person from the fish-eye image (step S41). In the case where a plurality of persons exist within the fisheye image, a plurality of heads are detected. In many cases, an object other than the head (for example, an object whose shape or color is easily confused with the head of a person, such as a ball, a PC, a circulator, or a round chair) may be erroneously detected. As described above, the detection result of the head detection unit 22 also includes an object other than the head, and is referred to as a "head candidate" at this stage. The detection result may include, for example, a circumscribed rectangle (also referred to as a "bounding box") of the detected head candidate. ) And the reliability of the detection (being the certainty of the head). Fig. 5 is an example of the result of head detection. Reference numeral 50 denotes a bounding box. In this example, objects 56 and 57 other than the head of the

person

51, 52, 53, 54 and 55 are also detected as head candidates. In addition, the head detection may use which algorithm. For example, a recognizer combining image features such as HoG or Haar-like with Boosting (Boosting) may be used, or head recognition based on deep learning (e.g., R-CNN, Fast R-CNN, YOLO, SSD, etc.) may be used.

Next, the human body detection unit 24 detects a human body from the fish-eye image (step S41). In the case where a plurality of persons exist in the fisheye image, a plurality of human bodies are detected. In many cases, an object that is not a human body (for example, an object whose shape or color is easily confused with a human body, such as an electric fan, a table, a chair, or a hat rack) is erroneously detected. In this way, since the detection result of the human body detection unit 24 also includes an object that is not a human body, the object is referred to as a "human body candidate" at this stage. The detection result may include, for example, a circumscribed rectangle of the detected human body candidate (also referred to as a "bounding box") and the reliability of the detection (the certainty of the human body (determination からしさ). Fig. 6 is an example in which the result of human body detection is superimposed on the result of head detection. Reference numeral 60 denotes a boundary frame of the human body candidate. In this example, in addition to the

human bodies

61, 62, 63, 64, 65, objects 66, 67 that are not human bodies are also detected as human body candidates. In addition, which algorithm is used for human body detection is all possible. For example, a recognizer combining HoG or Haar-like image features with enhancements may be used, as well as human recognition based on deep learning (e.g., R-CNN, Fast R-CNN, YOLO, SSD, etc.). In the present embodiment, the whole body of a human is detected as a human body, but the present invention is not limited thereto, and a part of the human body such as the upper body may be a detection target.

In addition, since the head detection and the human body detection are independent processes, the head detection and the human body detection may be performed in the order of human body detection → head detection, or may be processed in parallel.

Next, the determination unit 26 performs pairing of the head candidate and the human body candidate (step S43). The determination unit 26 selects only pairs satisfying a predetermined pairing condition from among 49 pairs of 7 head candidates 51 to 57 and 7 human body candidates 61 to 67 shown in fig. 6, for example. The pairing conditions and the pairing process will be described later in detail.

Next, the determination unit 26 obtains the reliability of each pair obtained in step S43 (step S44). The reliability of a pair is a measure indicating how high the probability that the pair (head candidate and human body candidate) represents the actual head and body of a human (certainty of being a human). The details of the reliability will be described later.

Next, the determination unit 26 extracts only pairs satisfying a predetermined reliability condition from the plurality of pairs obtained in step S43 (step S45). Then, the determination unit 26 finally determines the pair (combination of the head candidate and the human body candidate) extracted here as "human", and stores the determination result in the storage unit 27. The determination result may include information such as the position and size of a circumscribed rectangle (bounding box) including the head candidate and the human body candidate, and the reliability of the pair. Fig. 7 is an example of the final determination result (human detection result).

Finally, the output unit 28 outputs the determination result obtained in step S45 to the external device (step S46). The above process ends for the 1-frame fisheye image.

According to the person detection processing of the present embodiment, the fisheye image is directly analyzed, and person detection is directly performed from the fisheye image. Therefore, preprocessing such as plane expansion or distortion correction of the fisheye image can be omitted, and high-speed human detection processing can be performed. The method of using the fisheye image as it is for the detection processing has a problem of lowering the detection accuracy compared with the method of performing the detection processing after performing the plane development (distortion correction), but in the present embodiment, since logic of determining as "human" is employed when the head and the body are detected together from the fisheye image and they satisfy a predetermined condition, it is possible to realize very high-accuracy detection.

In the present embodiment, two conditions, namely, the pairing condition and the reliability condition, are used as the predetermined conditions, but only one of the conditions may be used as long as sufficient accuracy can be obtained. Alternatively, conditions other than the pairing condition and the reliability condition may be used.

< pairing >

A specific example of the pairing process and pairing conditions performed by the determination unit 26 will be described.

(1) Pairing based on relative position

Since the fisheye image is taken at an angle looking down on the person, the head region (bounding box) and the body region (bounding box) overlap as shown in fig. 6. In the case of a person present directly below the camera (i.e., on the optical axis), the head region and the center of the human body region substantially coincide with each other, but in the case of a person present at a position other than the head region, the human body region is captured closer to the center of the image than the head region (in other words, the center of the human body region and the center of the head region are arranged in this order as viewed from the center of the image). By taking into account the relative positions of the head region and the human body region using the characteristics of the fisheye image, the validity of the combination of the head candidate and the human body candidate can be evaluated.

Fig. 8 is a flowchart of the pairing process based on the relative position. First, the determination unit 26 generates a pair in which the head region and the human body region overlap each other, from all combinations of the head candidates and the human body candidates (step S80). In the case of the example of fig. 6, 6 pairs are generated. The reason why the number of persons is larger than the actual number of persons (5 persons) is that pairs of the human body candidates 62 and the head candidates 56 based on the erroneous detection are also generated.

Next, the determination unit 26 determines which of the head region and the human body region is close to the center of the image for each pair obtained in step S80, and extracts only a pair in which one of the human body regions is close to the center of the image (step S81). This determination may be performed by comparing the distance between the center of the head region and the center of the image with the distance between the center of the human body region and the center of the image, for example. By this processing, the pair of the human body candidate 62 and the head candidate 56 can be excluded. As a result, 5 pairs of head candidate 51 and human body candidate 61, head candidate 52 and human body candidate 62, head candidate 53 and human body candidate 63, head candidate 54 and human body candidate 64, head candidate 55 and human body candidate 65 are selected.

(2) Pairing based on relative size

When the position of the fisheye camera 10 is fixed with respect to the detection target region, the size of the head or the human body in the fisheye image can be approximately predicted. In addition, personal differences in the size of the body can be eliminated by calculating the relative sizes of the head and the body. By using the characteristics of the fisheye image, the validity of the combination of the head candidate and the human body candidate can be evaluated by considering the relative sizes of the head region and the human body region.

Fig. 9 is a flowchart of pairing processing based on relative sizes. First, the determination unit 26 generates a pair in which the head region and the human body region overlap each other, from all combinations of the head candidates and the human body candidates (step S90). The process is the same as step S80 of fig. 8. Next, the determination unit 26 calculates the size ratio of the head region and the human body region for each pair obtained in step S90 (step S91). For example, the area ratio of the bounding box may be determined as a size ratio, or the length ratio of the side or diagonal may be determined as a size ratio. Then, the determination unit 26 extracts only pairs whose size ratios have converged to a predetermined range (step S92). By such processing, it is possible to exclude the false detection object having a size significantly different from the actual head or human body from the counterpart.

However, the characteristic of the fisheye image is that the depression angle decreases toward the end of the image, and the size of the human body region increases relatively compared to the head region. That is, the size ratio of the head region and the human body region is not fixed in the entire image and varies depending on the position in the fisheye image. Therefore, the determination unit 26 may change the "predetermined range" used in step S102 according to the coordinates of the head candidate or the human body candidate on the image. For example, as shown in fig. 10, the fisheye image is divided into 25 regions L1 to L25, and a positive resolution range of the size ratio is set for each divided region. In the example of fig. 10, a forward solution range is set such that the size ratio (head region/body region) is smaller as the distance from the image center is longer. The determination unit 26 can perform appropriate determination corresponding to the position in the fisheye image by referring to the table shown in fig. 10 in the determination process of step S92. This can further improve the reliability of pairing.

< degree of reliability >

Several specific examples of the reliability determination by the determination unit 26 will be described.

(1) Individual decision

The determination unit 26 may determine that a pair is a human when the reliability of each of the head candidates and the human body candidates constituting the pair exceeds a predetermined threshold. That is, when the reliability of the head candidate is Ch, the reliability of the human body candidate is Cb, the threshold of the head candidate is Th, and the threshold of the human body candidate is Tb, it is determined that the head candidate is a human body candidate

Ch>Th and Cb>Tb

Human being

Ch is less than or equal to Th or Cb is less than or equal to Tb

The determination method that is not a person is a separate determination.

(2) Simple average

The determination unit 26 may determine the total reliability Cw from the reliability Ch of the head candidate and the reliability Cb of the human body candidate, and determine whether the pair is a human or not based on whether the total reliability Cw is greater than the threshold Tw or not. In the case of simple averaging, the total reliability Cw may be calculated by the following equation.

Cw＝(Ch+Cb)/2

(3) Weighted average

In the case of weighted averaging, the total reliability Cw can be calculated, for example, by the following equation.

Cw＝(w×Ch+(1－w)×Cb)/2

Where w is a weight. The weight w may be a fixed value or may be changed according to the coordinates of the head candidate or the human body candidate on the fisheye image. As shown in fig. 6, in the central portion of the image, the head is largely photographed, but hardly photographed into the human body. Further, the shooting rate of the human body increases as the image is directed toward the edge of the image. In consideration of the characteristics of the fisheye image, the weight w of the reliability Ch of the head candidate may be relatively increased in the central portion of the image, and the weight (1-w) of the reliability Cb of the human body candidate may be gradually increased toward the end portion of the image.

(4) Head priority

When the reliability Ch of the head candidate is extremely high, the determination unit 26 may perform the final determination of whether or not the head candidate is a human, without considering the reliability Cb of the human body candidate (or by making the weight of the reliability Cb of the human body candidate extremely small). Further, when the reliability Ch of the head candidate is extremely high, even when a human body candidate as a pair is not found, it can be determined as "human" (it is considered that the possibility that the body is hidden in the shadow is high). The threshold for determining whether or not the reliability Ch is extremely high may be set to a value greater than Th and Tw.

(5) Human body priority

When the reliability Cb of the human body candidate is extremely high, the determination unit 26 may perform the final determination of whether or not the head candidate is a human, without considering the reliability Ch of the head candidate (or minimizing the weight of the reliability Ch of the head candidate). Further, when the reliability Cb of the human body candidate is extremely high, even when the head candidate as a pair is not found, it can be determined as "human" (it is considered that the possibility that the head is blocked in the shadow is high). The threshold value for determining whether or not the reliability Cb is extremely high may be set to a value greater than Tb and Tw.

< others >

The above embodiments are merely illustrative of the configuration examples of the present invention. The present invention is not limited to the above-described specific embodiments, and various modifications can be made within the scope of the technical idea.

< appendix 1>

(1) A human detection device (1) that detects a human (13) present in a detection target region (11) by analyzing a fisheye image obtained by a fisheye camera (10) provided above the detection target region (11), the device comprising:

a head detection unit (22) that detects one or more head candidates from the fisheye image using an algorithm for detecting a human head;

a human body detection unit (24) for detecting one or more human body candidates from the fisheye image using an algorithm for detecting a human body; and

and a determination unit (26) that determines, as a human, a pair that satisfies a predetermined condition, from among the pairs of the head candidate and the human body candidate created by combining the detection result of the head detection unit (22) and the detection result of the human body detection unit (24).

(2) A person detection method for detecting a person (13) present in a detection target region (11) by analyzing a fisheye image obtained by a fisheye camera (10) provided above the detection target region (11), the person detection method comprising:

a head detection step (S41) for detecting one or more head candidates from the fisheye image by using an algorithm for detecting a human head;

a human body detection step (S42) for detecting one or more human body candidates from the fisheye image by using an algorithm for detecting a human body; and

and a determination step (S45) for determining, as a human, a pair that satisfies a predetermined condition among the pairs of the head candidate and the human body candidate created by combining the detection result of the head detection step and the detection result of the human body detection step.

Description of the reference symbols

1: human detection device

2 monitoring system

10 fisheye camera

11 detecting the object region

12: ceiling

13, human.

Claims

1. A human detection device that detects a human being present in a detection target region by analyzing a fisheye image obtained by a fisheye camera provided above the detection target region, the human detection device comprising:

a head detection unit that detects one or more head candidates from the fisheye image using an algorithm for detecting a human head;

a human body detection unit that detects one or more human body candidates from the fisheye image using an algorithm for detecting a human body; and

and a determination unit configured to determine a pair satisfying a predetermined condition among the pairs of the head candidate and the human body candidate created by combining the detection result of the head detection unit and the detection result of the human body detection unit as a human.

2. The human detection apparatus according to claim 1,

the predetermined condition includes a condition relating to relative positions of the head candidate and the human body candidate.

3. The human detection apparatus according to claim 1 or 2,

the prescribed conditions include the following conditions: the head candidate region and the human body candidate region overlap each other.

4. The human detection apparatus according to any one of claims 1 to 3,

the prescribed conditions include the following conditions: the human body candidate exists at a coordinate closer to the center of the fisheye image than the head candidate.

5. The human detection apparatus according to any one of claims 1 to 4,

the predetermined condition includes a condition relating to relative sizes of the head candidate and the human body candidate.

6. The human detection apparatus according to any one of claims 1 to 5,

the prescribed conditions include the following conditions: the size ratio of the head candidate to the human body candidate is within a predetermined range.

7. The human detection apparatus according to claim 6,

the determination unit changes the predetermined range according to the coordinates of the head candidate or the human body candidate on the fisheye image.

8. The human detection apparatus according to any one of claims 1 to 7,

the head detection unit outputs the reliability of detection for each detected head candidate,

the human body detection unit outputs the reliability of detection for each detected human body candidate,

the predetermined condition includes a condition relating to the reliability of the head candidate and the human body candidate.

9. The human detection apparatus according to claim 8,

the determination unit determines a total reliability based on the reliability of the head candidate and the reliability of the human body candidate,

the prescribed conditions include the following conditions: the total reliability is greater than a threshold.

10. The human detection apparatus according to claim 9,

the determination unit changes the weight of the reliability of the head candidate and the weight of the reliability of the human body candidate when the total reliability is obtained, based on the coordinates of the head candidate or the human body candidate on the fisheye image.

11. The human detection apparatus according to any one of claims 8 to 10,

when either one of the reliability of the head candidate and the reliability of the human body candidate is sufficiently high, the determination unit relaxes the condition for the reliability of the other.

12. A person detection method for detecting a person present in a detection target region by analyzing a fisheye image obtained by a fisheye camera provided above the detection target region, the person detection method comprising:

a head detection step of detecting one or more head candidates from the fisheye image using an algorithm for detecting a human head;

a human body detection step of detecting one or more human body candidates from the fisheye image by using an algorithm for detecting a human body; and

a determination step of determining, as a human, a pair satisfying a predetermined condition among the pairs of the head candidate and the human body candidate created by combining the detection result of the head detection step and the detection result of the human body detection step.

13. A program for causing a computer to execute the steps of the human detection method recited in claim 12.