JP7152048B2 - Human proximity detection device and human proximity detection method - Google Patents

Description

The present invention relates to a human proximity detection device and a human proximity detection method for detecting an approaching person.

For example, if the vehicle is carelessly moved while people are positioned around the vehicle, there is a risk of contact with the person. In order to prevent such a contact accident, a device for detecting an approaching person has been proposed (see Patent Documents 1 and 2, for example).

The surveillance system described in Patent Document 1 includes a control unit attached to a backhoe and connected to an imaging device that captures the surroundings. and calculate the approaching distance to the person or helmet recognized by the image, and if the approaching distance is less than the first distance and the person is in the warning area, a warning sound is generated, and the approaching distance is less than the second distance. When a person is in the stop area, the construction machine is stopped and a warning sound is generated. The approach distance is calculated by the determination unit of the control unit using the parallax of the stereo camera to calculate the distance to the subject (person or helmet) included in the target image.

The surroundings monitoring device described in Patent Document 2 detects the presence or absence of obstacles such as workers and objects around the work machine from the result of image processing using the scene captured by the camera and the distance measured by the distance sensor. If an obstacle exists, the positional relationship of the distance and direction from the machine body to the obstacle and the machine is calculated, and the contact risk level of the obstacle is calculated from the positional relationship and the movement of the work machine to change its posture such as excavation and turning. Then, the level is determined from the contact risk level, and the contents of the warning corresponding to the risk are output to an output device such as a monitor or voice.
In the surroundings monitoring device described in Patent Document 2, image processing is performed using a scene photographed by a camera. The distance data of the detected obstacles is integrated (fused).

JP 2019-157497 A JP 2010-198519 A

However, in order to measure the distance to a person approaching the vehicle, the surroundings monitoring device described in Patent Document 1 uses a stereo camera, and the surroundings monitoring device described in Patent Document 2 uses a millimeter wave radar. ing. The conventional human proximity detection devices described in Patent Documents 1 and 2 use special devices such as stereo cameras and millimeter wave radars, which complicates the device and increases the cost, hindering its widespread use. Resulting in.

SUMMARY OF THE INVENTION Accordingly, it is an object of the present invention to provide a human proximity detection device and a human proximity detection method that are capable of detecting the approach of a person with a simple configuration at a reduced cost.

A human approach detection apparatus of the present invention includes a camera that outputs a photographed image in a monitoring direction, a control unit that processes the photographed image from the camera to detect the approach of a person, and an approach notification from the control unit. a notifying unit, wherein the control unit includes: a detection unit that detects a human image from the captured image; an estimation unit that estimates a skeleton of the human image and associates the virtual skeleton with the human image; It is characterized by comprising a measurement unit that measures the length of the part, and a determination part that outputs the approach notification to the notification unit when the length of the part of the virtual skeleton exceeds a threshold value.

A human approach detection method of the present invention includes a camera that outputs a photographed image in a monitoring direction, a control unit that processes the photographed image from the camera and detects the approach of a person, and an approach notification from the control unit. and a notifying unit that notifies, the controller detects a human image from the photographed image, estimates the skeleton of the human image, and determines the human image. A length of a part of the associated virtual skeleton is measured, and the approach notification is output to the notification unit when the length of the part of the virtual skeleton exceeds a threshold value.

According to the present invention, a human image is detected from an image captured by a camera, a virtual skeleton is associated with the human image, and the length of a portion of the virtual skeleton is measured and compared with a threshold to detect a person approaching. It can be determined whether or not it is overdone. Therefore, the distance to a person can be measured without using a special camera or radar.

The measurement unit may measure shoulder width as the length of the part of the virtual skeleton. The measurement unit judges the person's approach position based on the length of the shoulder width as part of the virtual skeleton, so even if there is a difference in height, the difference in shoulder width is small, so it is possible to suppress the error in the position of the person due to the difference in physique. can.

The measurement unit measures the length from the shoulder to the waist when the determination unit determines that the shoulder width is equal to or less than the threshold value, and the determination unit measures the length from the shoulder to the waist when it exceeds the threshold value. An approach notification may be output at times. To detect the approach of a person facing obliquely sideways by judging the approaching position of the person from the length from the shoulder to the waist by a judging part even if the person looks obliquely sideways and the apparent shoulder width looks small. can be done.

As the length from the shoulder to the waist, when either one of the length from the right shoulder to the right waist or the length from the left shoulder to the left waist is equal to or less than a threshold, the determination unit compares the other with a threshold and approaches the length. It may determine the output of notifications. Even if it is impossible to measure either the length from the right shoulder to the right waist or the length from the left shoulder to the left waist when the person being monitored is facing sideways, by measuring and judging the other, it is possible to Even an approaching person can detect the approach.

The camera may be arranged at the rear end of the roof of the vehicle, facing obliquely downward to the rear. It is possible to reduce misidentification of the outline of a person due to the sunset or the rising sun being reflected in a position overlapping the person.

A setting unit that selectively instructs the threshold value may be provided. Since the setting unit selectively instructs the determination unit on the threshold value, the warning distance can be selected according to the circumstances around the vehicle.

The vehicle may further include a stop instruction section that outputs a stop notification to the vehicle when the determination section outputs the approach notification. The stop instruction unit outputs a stop notification to the braking unit of the vehicle, so that the vehicle can be stopped when a person approaches, so contact with the person can be avoided in advance.

INDUSTRIAL APPLICABILITY The present invention can measure the distance to a person without using a special camera or radar, so that the approach of a person can be detected with a simple configuration at a reduced cost. is possible.

BRIEF DESCRIPTION OF THE DRAWINGS It is a figure which shows the human approach detection apparatus which concerns on embodiment of this invention. 2 is a diagram for explaining the configuration of the human proximity detection device shown in FIG. 1; FIG. FIG. 4 is a diagram showing each point of person definition data stored in a storage unit; FIG. 2 is a flow chart for explaining a human approach detection method of the human approach detection device shown in FIG. 1; 2A is a diagram of an example of a photographed image from a camera of the human proximity detection device shown in FIG. 1, (A) is a diagram of an example of a photographed image including a human image, and (B) is a diagram in which a virtual skeleton is superimposed on the human image. An example diagram, (C) is an example diagram of a state in which a person approaches. FIG. 4 is a diagram for explaining the configuration of a virtual skeleton; FIG. It is a figure for demonstrating warning distance.

A human approach detection device according to an embodiment of the present invention will be described with reference to the drawings.
As shown in FIG. 1, the human approach detection device 10 is mounted on a forklift F, which is an example of a work vehicle (vehicle), and detects a person to be monitored who approaches the forklift F from behind.
As shown in FIG. 2, the human approach detection device 10 includes a camera 20 that outputs a photographed image in the monitoring direction (rear), a main body 100 that processes the photographed image from the camera 20 and detects the approach of a person, A notification unit 40 for notifying an approach notification from the main unit 100 is provided.

The camera 20 is a monocular camera that outputs captured images as moving images. The camera 20 is installed at the rear end of the head guard F2 (roof) located above the seat F1 and faces obliquely downward to the rear in order to monitor the rear of the forklift F, which is out of the field of view of the operator. The camera 20 can be, for example, an inexpensive and versatile USB camera. In this embodiment, the camera 20 has a wide-angle lens of 140° and can output a photographed image of approximately 2 million pixels (1920 pixels×1080 pixels).

The body part 100 is installed on the back surface of the head guard F2 by a magnet, but it may be located around the seat F1, around the handle F3, or at the rear end of the forklift F as long as it does not interfere with operation.
As shown in FIG. 2 , the body section 100 includes a control section 30 and a power supply section 50 .
The control unit 30 includes a detection unit 31 , an estimation unit 32 , a measurement unit 33 , a determination unit 34 , a setting unit 35 , a storage unit 36 and a stop instruction unit 37 .

The detection unit 31 detects a human image from the photographed image input from the camera 20 by referring to a database of human definition data stored in the storage unit 36 . The detection unit 31 is connected to the camera 20 by wire via a USB port formed in the control unit 30, but can be connected by wireless such as WIFI (registered trademark), for example. If the control unit 30 and the camera 20 are connected by USB, not only can signals be transmitted and received, but also power can be supplied from the control unit 30, so there is no need to wire a power line to the camera 20 separately.

The estimation unit 32 estimates a skeleton from the detected human image and associates the virtual skeleton with the human image. The virtual skeleton is superimposed on the captured image by associating the virtual skeleton with the estimation unit 32 .
A measuring unit 33 measures the length of a portion of the virtual skeleton. In this embodiment, the shoulder width and the length from the shoulder to the waist are measured as the length of a portion of the virtual skeleton.
Based on the threshold corresponding to the value read from the setting unit 35, the determination unit 34 issues an approach notification because the person is closer than the warning distance L1 (see FIG. 1) when the length of a part of the skeleton exceeds. Output to the notification unit 40 . Three types of thresholds are prepared for the warning distance L1 from the vehicle to the person: about 3 m, about 4 m, and about 5 m. Note that this distance can be appropriately determined as long as it is a distance at which the camera 20 can shoot.

The setting unit 35 outputs a setting value for selecting a threshold value used for determination by the determination unit 34 . The setting unit 35 is a three-contact slide switch that can select three states of about 3 m, about 4 m, and about 5 m as the set distance corresponding to the set value, but other switches may be used. The setting unit 35 can also be a flash memory to which data can be written from the outside.

In the storage unit 36, human definition data of various postures are stored as a database. As shown in FIG. 3, this person definition data includes right eye P11, left eye P12, right ear P21, left ear P22, nose P3, right shoulder P41, left shoulder P42, right elbow P51, left elbow P52, right hand P61, left hand P62. , right hip P71, left hip P72, right knee P81, left knee P82, right leg P91, and left leg P92, a total of 17 points are defined.

The stop instruction unit 37 outputs a stop notification to the forklift F when the determination unit 34 outputs an approach notification. This function can be enabled or disabled by setting the control unit 30, and can be selected on the forklift F side.

The notification unit 40 notifies the operator of the vehicle of the approach notification from the control unit 30 . The notification unit 40 can notify the surroundings by ringing a sound or lighting a light. It has a unit 41 and a second notification unit 42 built in the control unit 30 .

The first notification unit 41 is a post F4 that supports the head guard F2 of the forklift F, and can be a rotating light (warning light) attached to the front post F4. The rotating light functioning as the first notification unit 41 can be directly installed on the post F4 by the magnet provided on the bottom surface, but it may be installed indirectly by a stay. The second notification unit 42 can be a speaker that emits sound.

The power supply unit 50 is a DC-DC converter that steps down 12V from the battery, which is the power source of the forklift F, to 5V used by the control unit 30 .

The operation and usage state of the human approach detection device 10 according to the embodiment of the present invention configured as described above will be described with reference to the drawings.
First, the camera 20 outputs a photographed image of the rear of the forklift F, which is a working vehicle, to the control unit 30 (see step S10 in FIG. 4). For example, a captured image C1 as shown in FIG. 5A is output from the camera 20. FIG. This photographed image C1 includes a person as a person image M in the background.

In the control unit 30 shown in FIG. 2, which receives the captured image from the camera 20, the detection unit 31 refers to the database in the storage unit 36, and the human image that matches the pattern of the person definition data included in the database is detected as the captured image. determines whether it is included in
In this determination, when a key point is detected from a photographed image, a score of 0.0 to 1.0 is assigned as a reliability indicating accuracy. presume.

If it is estimated that the human image is included in the captured image from the camera 20, the detection unit 31 notifies the estimation unit 32 that the human image has been detected (see step S20).

Next, as shown in FIG. 5B, the estimation unit 32 associates the human image M of the captured image C2 with the virtual skeleton B as a virtual line simulating a human shape (see step S30).
Here, the shape of the virtual skeleton B will be described in detail. As shown in FIG. 6, the virtual skeleton B includes a head _BH , shoulders _{BS, right arm BAR, left arm B AL , right} upper body B _UR , left left body B _UL , It is formed from a _right leg _BLR and a left leg BLL.

The head _BH is a virtual line connecting the right ear P21, right eye P11, nose P3, left eye P12, and left ear P22. The shoulder BS is an imaginary line connecting the right shoulder _P41 and the left shoulder P42. The right arm _BAR is an imaginary line connecting the right shoulder P41 to the right elbow P51 and the right hand P61. The left arm B _AL is an imaginary line connecting the left shoulder P42, the left elbow P52, and the left hand P62. The right upper body _BUR is an imaginary line connecting the right shoulder P41 and the right hip P71. The left upper body _BUL is an imaginary line connecting the left shoulder P42 and the left waist P72. The _right leg BLR is an imaginary line connecting the right hip P71, the right knee P81, and the right foot P91. The left leg _BLL is an imaginary line connecting the left hip P72, the left knee P82, and the left leg P92.

Next, the measurement unit 33 shown in FIG. 2 measures the shoulder width (see step S40). The length of the shoulder width can be measured by counting the number of pixels of the shoulder width as the length of the shoulder BS in the virtual _skeleton B included in the image shown in FIG. The shoulder width measured by the measurement unit 33 is output to the determination unit 34 (see FIG. 2).

The determination unit 34 first reads a set value (set distance) from the setting unit 35 . The determination unit 34 determines a threshold value (first threshold value) for the length of the shoulder width based on the set value. For example, if the captured image is 1920 pixels×1080 pixels and the set distance of the setting unit 35 indicates 3 m, the first threshold is set to 40 pixels, and if the set distance is 4 m, the first threshold is set to 31 pixels. The first threshold may be 25 pixels if it indicates a distance of 5 m. In this embodiment, it is assumed that the distance is set to 4 m.

Next, the determination unit 34 compares the length of the shoulder width LS2 shown in FIG. 7 with the first threshold (see step S50). For example, it is assumed that a person in a low, hunched posture shown in the photographed image C3 shown in FIG. 5C is closer to the person than in the photographed image C2 shown in FIG. 5B.
At this time, when it is determined that the length of the shoulder width is larger than the first threshold, it indicates that the distance (detection distance) of the person positioned behind the forklift F is closer than the warning distance L1 of 4 m. ing. Therefore, the determination unit 34 outputs an approach notification to the first notification unit 41 and the second notification unit 42 .

The first notification unit 41 notifies the operator with light. Since the rotating light serving as the first notification unit 41 is installed on the front post F4, the first notification unit 41 is within the field of vision of the operator facing forward. It can make you notice the approach of people from.
Further, the second notification unit 42 notifies the operator by sound (warning sound). In addition, since the control unit 30 emits a warning sound, a person approaching from behind can also be made aware of the danger of approaching by the warning sound (see step S60).

In addition, the stop instructing unit 37 that has received the approach notification from the determination unit 34 adjusts the electrical level to a signal receivable by the forklift F, and outputs an amplified stop notification.
By doing so, in the forklift F, when the stop notification is received, the brake can be applied to stop the forklift, or the engine can be stopped.

For example, as shown in FIG. 7, when the shoulder width (the length of the shoulder BS (see FIG. 6)) is _LS1 , the determination unit 34 compares the shoulder width LS1 with the first threshold value, length LS1 is equal to or less than the first threshold (see determination in step S50).
In the example shown in FIG. 7, there is no problem because the person is at a distance L2 which is farther than the warning distance L1, but the person may be facing sideways and the apparent shoulder width may appear short.

In that case, the measurement unit 33 measures the length from the shoulder to the waist based on the virtual skeleton based on the notification from the determination unit 34 shown in FIG.
First, the measurement unit 33 measures the length from the right shoulder to the waist based on the length of the right upper body _BUR shown in FIG. 6 (see step S70). Then, the determination unit 34 determines a threshold value (second threshold value) for the length from the shoulder to the waist based on the set value. For example, if the setting unit 35 indicates a set distance of 3 m, the second threshold is 60 pixels, if the set distance is 4 m, the second threshold is 50 pixels, and if the set distance is 5 m, the second threshold is can be 40 pixels.

Next, the determination unit 34 compares the length from the right shoulder to the waist (right upper body) with the second threshold (see step S80).
If it is determined that the right upper body length is greater than the second threshold, it means that the person behind the forklift F is within the warning distance L1 (see FIGS. 1 and 7). Therefore, the determination unit 34 shown in FIG. 2 outputs an approach notification to the first notification unit 41 and the second notification unit 42 . In response to this approach notification, the notification unit 40 notifies in the same manner as when the approach is detected by the shoulder width (see step S60).

The determination unit 34 compares the length of the upper body on the right side with the second threshold value, and if it determines that the length of the upper body on the right side is equal to or less than the second threshold value, the person behind the forklift F gives a warning. This indicates that the distance is greater than or equal to L1.

Therefore, the measurement unit 33 next measures the length from the left shoulder to the waist based on the length of the left upper body _BUL shown in FIG. to the waist (upper body on the left side) is compared with the second threshold (see step S100).
In this way, after determining the length of the upper body on one side, the length of the upper body on the other side is determined by comparing with the second threshold to determine the output of the approach notification. Even if the person positioned behind faces sideways and the length of the upper body on one side cannot be measured, the approach of the person can be determined based on the length of the upper body on the other side. In the present embodiment, the right upper body is determined, and then the left upper body is determined. However, the right upper body may be determined after the left upper body is determined.

Thus, the control unit 30 completes one process and repeats the first step S10 to step S100.

As described above, in the human approach detection apparatus 10 according to the embodiment of the present invention, the detection unit 31 detects a human image from an image captured by the camera 20, associates the human image with a virtual skeleton, and determines the virtual skeleton. By measuring a part of the length and comparing it with thresholds (first threshold, second threshold), it is possible to determine whether the person is too close to the forklift F or not. Therefore, the distance to a person can be measured without using a special camera or radar. Therefore, the human approach detection device 10 can detect the approach of a person with a simple configuration, thereby improving safety and reducing costs.

In addition, since the determination unit 34 determines the approach position of the person based on the length of the shoulder width in the virtual skeleton, even if there is a difference in height, the difference in shoulder width is small. can be done.

Furthermore, even if the person faces sideways and the apparent shoulder width appears small, the determination unit 34 determines the approach position of the person based on the length of the upper body (the length from the shoulder to the waist). The approach of a person can also be detected. Also, by using the length of the upper body as the determination, it is possible to suppress the error in the position of the person due to the difference in physique rather than the length of the whole body (height). In addition, if it is the upper body, even if the person's height is lowered with the legs bent, the effect can be eliminated.

As shown in FIG. 1, the camera 20 is arranged at the rear end of the head guard F2 of the forklift F, facing obliquely downward to the rear. Misrecognition of contours can be reduced. In addition, since the forklift has a protruding counterweight at the rear of the vehicle body, the vicinity of the counterweight is a blind spot from the driver's seat, and there is a range that cannot be detected even if the camera 20 is installed at the rear end of the counterweight. There is a risk. By installing the camera 20 at the rear end of the head guard F2, it is possible to reduce blind spots due to the counterweight.

The setting unit 35 selectively instructs the determination unit 34 to set the threshold (the first threshold and the second threshold) to 3 m, 4 m, and 5 m, so that the warning distance is selected according to the situation of the workplace (around the forklift F). can do.

In the present embodiment, the notification unit 40 notifies when a person approaches from the warning distance L1, but the warning distance may be set stepwise. For example, the notification unit 40 gives stepwise notification according to the distance that the person approaches, such as 5m, 4m, and 3m. At this time, by changing the type of sound emitted by the notification unit 40, the color of the light, or the like, it is possible to easily grasp the approaching distance.

Also, in the present embodiment, a work vehicle, particularly a forklift, was described as an example of a vehicle, but the present invention may be applied to other work vehicles such as crane trucks, shovels, and trucks, as well as ordinary automobiles.

In addition, although the rearward direction, which tends to be out of the operator's field of view, is set as the monitoring direction, the camera may be installed as the monitoring direction including the front and the sides. In this case, multiple cameras are placed facing the surroundings, and in that case, the detection direction is identified by changing the type of sound emitted by the notification unit according to the direction in which the approach of a person is detected. You may make it possible.

In addition, the present invention is not limited to moving vehicles, and can be used to detect the approach of a person to a position where a camera is fixedly installed. For example, it can also be used for crime prevention such as unauthorized entry into the surrounding area of a building or a modeled object. When detecting the approach of a person and notifying an administrator located at a remote location, the notification unit can transmit a message through the Internet, which is an example of an electronic communication line, instead of using sound or light.

INDUSTRIAL APPLICABILITY The present invention can detect the approach of a person and can notify the approach of the person by the notification unit, so that it can be used in a place where it is necessary to avoid contact with the person or prevent the unnecessary intrusion of the person. can be set effectively.

10 human approach detection device 20 camera 30 control unit 31 detection unit 32 estimation unit 33 measurement unit 34 determination unit 35 setting unit 36 storage unit 37 stop instruction unit 40 notification unit 41 first notification unit 42 second notification unit 50 power supply unit 100 main unit Part F Forklift F1 Seat F2 Head guard F3 Handle F4 Post C1, C2, C3 Photographed image M Human image P11 Right eye P12 Left eye P21 Right ear P22 Left ear P3 Nose P41 Right shoulder P42 Left shoulder P51 Right elbow P52 Left elbow P61 Right hand P62 Left hand P71 Right waist P72 Left waist P81 Right knee P82 Left knee P91 Right leg P92 Left leg B Virtual skeleton B _H head B _S shoulder B _AR right arm B _AL left arm B _UR right upper body B _UL upper left body B _LR right leg B _LL left leg L1 warning distance L2 distance LS1, LS2 shoulder width

Claims (7)

A camera that outputs a photographed image in a monitoring direction, a control unit that processes the photographed image from the camera and detects the approach of a person, and a notification unit that notifies an approach notification from the control unit,
The control unit includes a detection unit that detects a human image from the captured image, an estimation unit that estimates a skeleton of the human image and associates the virtual skeleton with the human image, and a length of a part of the virtual skeleton. A human proximity detection device comprising: a measurement unit for measuring; and a determination unit for outputting the approach notification to the notification unit when the shoulder width measured as the length of the partial virtual skeleton exceeds a threshold value. The measurement unit measures the length from the shoulder to the waist when the determination unit determines that the shoulder width is equal to or less than the threshold,
2. The human approach detection device according to claim 1 , wherein the determination unit outputs an approach notification when the length from the shoulder to the waist exceeds a threshold. As the length from the shoulder to the waist, when either one of the length from the right shoulder to the right waist or the length from the left shoulder to the left waist is equal to or less than a threshold, the determination unit compares the other with a threshold and approaches the length. 3. The human approach detection device according to claim 2 , which determines output of notification. 4. The human approach detection device according to claim 1 , wherein the camera is arranged at the rear end of the roof of the vehicle so as to face obliquely downward to the rear. The human approach detection device according to any one of claims 1 to 4 , further comprising a setting unit that selectively instructs the threshold value. 6. The human approach detection device according to claim 1 , further comprising a stop instructing section that outputs a stop notification to the vehicle when the determination section outputs the approach notification. Human proximity detection comprising: a camera for outputting a photographed image in a monitoring direction; a controller for processing the photographed image from the camera to detect the approach of a person; and a notification unit for announcing an approach notification from the controller. A human approach detection method for a device, comprising:
The control unit detects a human image from the photographed image, estimates the skeleton of the human image, measures shoulder width as a length of a part of the virtual skeleton associated with the human image, and measures the length of the shoulder width . a human proximity detection method for a human proximity detection device for outputting the proximity notification to the notification unit when the threshold is exceeded.

Images