JP6844508B2 - Safety device - Google Patents

Description

The present invention relates to a safety device.

In order to prevent the moving body from colliding with the obstacle above, the moving body is provided with a safety device for detecting the obstacle above.
The safety device of Patent Document 1 includes a distance detecting device that detects the distance to an obstacle. The distance detection device is arranged so as to face upward. The distance detection device detects the distance to an obstacle by irradiating an irradiation wave such as an ultrasonic wave or a laser. The safety device can detect an obstacle from the distance detected by the distance detection device.

The safety device of Patent Document 2 includes a stereo camera. The safety device can detect an obstacle that a moving object collides with from an image captured by a stereo camera.

Japanese Unexamined Patent Publication No. 2004-91118 Japanese Unexamined Patent Publication No. 2013-114610

By the way, in the safety device of Patent Document 1, the irradiation wave may not hit the obstacle as the moving body moves. In this case, the safety device will not be able to detect obstacles. Further, also with respect to the safety device of Patent Document 2, if an obstacle deviates from the vertical angle of view of the stereo camera due to the movement of the moving body, the obstacle cannot be detected. Then, the safety device cannot grasp the positional relationship between the moving body and the obstacle, and the moving body and the obstacle may collide with each other.

An object of the present invention is to provide a safety device capable of preventing the positional relationship between a moving body and an obstacle from becoming unclear.

The safety device that solves the above problems is a safety device mounted on a moving body, and is imaged by a stereo camera arranged so that the uppermost side of the vertical angle of view is above the horizontal plane and the stereo camera. When the obstacle is no longer imaged by the stereo camera due to the movement of the moving body, the calculation means for calculating the position of the obstacle, the storage means for storing the position information of the obstacle from the image, the said. An estimation means for estimating the position of the obstacle based on the position information stored by the storage means is provided.

According to this, the uppermost side of the vertical angle of view of the stereo camera is above the horizontal plane, so that the stereo camera can image the upper part of the moving body. The position of the obstacle calculated by the calculation means is stored by the storage means as position information. When the obstacle is no longer imaged by the stereo camera, the position of the obstacle is estimated by the estimation means. As a result, the safety device can grasp the positional relationship between the moving object and the obstacle even when the obstacle is no longer imaged by the stereo camera. Therefore, it is possible to prevent the positional relationship between the moving body and the obstacle from becoming unclear.

Regarding the safety device, the estimation means may estimate the position of the obstacle from the speed of the moving body and the traveling direction of the moving body.
According to this, the estimation means can estimate the position of the obstacle from the speed of the moving body and the traveling direction even when the obstacle is no longer imaged by the stereo camera.

Regarding the safety device, the estimation means is imaged together with the obstacle when the obstacle is imaged by the stereo camera, and the obstacle is no longer imaged as the moving body moves. In some cases, the imaged object may be used as a reference object, and the position of the obstacle may be estimated from the change in the relative position between the reference object and the moving object due to the movement of the moving object.

According to this, the estimation means grasps the positional relationship between the obstacle and the reference object when the obstacle and the reference object are imaged by the stereo camera, so that the relative position between the reference object and the moving object can be determined. The position of the obstacle can be estimated from the change.

With respect to the safety device, the moving speed of the obstacle in the vertical direction is detected, and it is determined whether or not the obstacle collides with the moving body when the obstacle continues to move at the moving speed. A collision determination means may be provided.

According to this, it is possible to determine whether or not the obstacle and the moving body collide with each other, taking into consideration the moving speed of the obstacle.

According to the present invention, it is possible to prevent the positional relationship between the moving body and the obstacle from becoming uncertain.

Side view of a forklift equipped with a safety device. Schematic block diagram of a forklift and a safety device. The flowchart which shows the process for detecting an obstacle in 1st Embodiment. A flowchart showing a relative position calculation process. The schematic diagram which shows the forklift and the obstacle which is imaged by the image pickup apparatus. A plot of an object in the world coordinate system. The schematic diagram which shows the forklift and the obstacle which is no longer imaged by the image pickup apparatus. The figure which shows the world coordinate system when the obstacle is not image | photographed by the image pickup apparatus. The flowchart which shows the process for detecting an obstacle in 2nd Embodiment. The flowchart which shows the process for estimating the movement of an obstacle in 2nd Embodiment. The figure for demonstrating the obstacle detection means in the modification.

(First Embodiment)
Hereinafter, the first embodiment of the safety device will be described.
As shown in FIGS. 1 and 2, the forklift 10 as a moving body includes a vehicle body 11 and a cargo handling device 12 provided at a front portion of the vehicle body 11. The forklift 10 includes a drive wheel 13, a steering wheel 14, and a traveling motor M1 that drives the drive wheel 13. The forklift 10 may be one in which the traveling operation and the cargo handling operation are automatically performed, or the forklift 10 may be one in which the traveling operation and the cargo handling operation are performed by the operation by the passenger (operator).

The cargo handling device 12 includes a mast 15. The mast 15 includes an outer mast 16 that is supported so as to be tilted back and forth with respect to the vehicle body 11, and an inner mast 17 that can be raised and lowered with respect to the outer mast 16. The forklift 10 includes a tilt cylinder 18 for tilting the mast 15 and a lift cylinder 19 for raising and lowering the inner mast 17. The tilt cylinder 18 and the lift cylinder 19 are hydraulic cylinders. The cargo handling device 12 includes a cargo handling tool (for example, an attachment such as a fork or a roll clamp device) L that moves up and down together with the inner mast 17.

The forklift 10 includes a main controller 20, a travel control device 23 for controlling the travel motor M1, a vehicle speed sensor 24, a lift sensor 25, and a steering wheel angle sensor 26. The main controller 20 controls the traveling operation and the cargo handling operation. The main controller 20 includes a CPU (calculation unit) 21 and a memory 22 in which programs for performing various controls and the like are stored.

The CPU 21 of the main controller 20 gives a command of the rotation speed of the traveling motor M1 to the traveling control device 23 so that the vehicle speed of the forklift 10 becomes the target speed. The main controller 20 controls the direction (forward or reverse) of the forklift 10 by giving a command to the travel control device 23 in the direction of rotating the travel motor M1.

The travel control device 23 of this embodiment is a motor driver. The vehicle speed sensor 24 of the present embodiment is a rotation speed sensor that detects the rotation speed of the traveling motor M1. The vehicle speed sensor 24 outputs the rotation speed of the traveling motor M1 to the traveling control device 23. The travel control device 23 controls the travel motor M1 so that the rotation speed of the travel motor M1 matches the command based on the command from the main controller 20. The travel control device 23 controls the travel motor M1 so that the rotation direction of the travel motor M1 coincides with a command from the main controller 20.

The lift sensor 25 is, for example, a reel-type lift sensor. The lift sensor 25 includes a wire (not shown) whose one end is connected to the cargo handling tool L, a reel (not shown) around which the wire is wound, and a rotation detector (rotary encoder) for detecting the amount of rotation of the reel. Be prepared. The lift sensor 25 outputs the detection result to the main controller 20.

The steering wheel angle sensor 26 is arranged on the steering wheel and detects the operation amount (angle) of the steering wheel. The steering wheel angle sensor 26 outputs the detection result to the main controller 20. The main controller 20 controls the direction of the steering wheel 14 so that the steering angle is set according to the operation amount (angle) of the steering wheel.

The forklift 10 is equipped with a safety device 30. The safety device 30 includes two imaging devices 31 and 41. One of the two image pickup devices 31 and 41 is the front image pickup device 31, and the other is the rear image pickup device 41. The front image pickup device 31 is a stereo camera that images the front of the forklift 10. The rear imaging device 41 is a stereo camera that photographs the rear of the forklift 10.

The imaging devices 31 and 41 are arranged on the upper part of the vehicle body 11. The front imaging device 31 includes two cameras 32 and 33. One of the two cameras 32 and 33 is the first camera 32, and the other is the second camera 33. As the cameras 32 and 33, for example, a CCD image sensor or a CMOS image sensor is used. The cameras 32 and 33 are arranged so that their optical axes are parallel to each other. The optical axes of the cameras 32 and 33 may or may not be horizontal. The rear imaging device 41 has the same configuration as the front imaging device 31, and includes a first camera 42 and a second camera 43. In the following description, the images captured by the first cameras 32 and 42 will be referred to as the first image, and the images captured by the second cameras 33 and 43 will be referred to as the second image.

The uppermost U1 of the vertical angle of view θ1 of the front imaging device 31 (the first camera 32 and the second camera 33), that is, the uppermost upper side of the imaging range of the front imaging device 31 is above the horizontal plane. Similarly, the uppermost U2 of the vertical angle of view θ2 of the rear imaging device 41 (first camera 42 and second camera 43) is above the horizontal plane. As a result, each of the image pickup devices 31 and 41 can take an image above the mounting position.

The safety device 30 includes an image processing device 50 that detects an obstacle from the images captured by the image pickup devices 31 and 41. The image processing device 50 is composed of a CPU, RAM, ROM, and the like. The image processing device 50 detects an obstacle above. The upper obstacle is an obstacle that is located above the floor on which the forklift 10 travels and may collide with the upper part of the forklift 10. Examples of obstacles above include beams and shutters.

The image processing device 50 repeats the process shown in FIG. 3 at predetermined control cycles. Hereinafter, the processing performed by the image processing device 50 will be described together with the operation of the safety device 30.
As shown in FIG. 3, the image processing device 50 calculates the parallax from the images captured by the image pickup devices 31 and 41 (step S10). For the parallax, the first image and the second image are compared with each of the image pickup devices 31 and 41, and the difference in the image positions (pixels) of the first image and the second image is calculated for the same object appearing in each image. Obtained by doing. The difference in image position is the difference [px] in the number of pixels (pixels) between the object reflected in the first image and the object reflected in the second image. In this embodiment, the parallax is obtained as a parallax image. The parallax image is an image in which an object captured by the images captured by the imaging devices 31 and 41 and the parallax of the object are associated with each other. Parallax is associated with each pixel in the parallax image. The number of pixels of the parallax image is the same as the number of pixels of the first image (second image). The parallax image does not necessarily require display, and indicates data in which parallax is associated with each pixel (X coordinate and Y coordinate in the parallax image) in the parallax image.

The object is a feature portion included in each image, and is a portion that can be recognized as a boundary, such as an edge of an object. The object can be detected from the luminance information and the like, and the image processing device 50 can calculate the parallax from the object.

The image processing device 50 calculates the position of an object existing around the forklift 10 from the parallax (step S11). First, the image processing device 50 calculates the position of the object in the camera coordinate system from the parallax images obtained by the image pickup devices 31 and 41. The camera coordinate system is a three-axis Cartesian coordinate system in which the optical axis is the Z axis and the two axes orthogonal to the optical axis are the X axis and the Y axis, respectively. The position of the object in the camera coordinate system can be represented by the Z coordinate Zc, the X coordinate Xc, and the Y coordinate Yc in the camera coordinate system. The Z coordinate Zc, the X coordinate Xc, and the Y coordinate Yc can be calculated using the following equations (1) to (3), respectively.

Here, B is the inter-eye distance [m] of each of the imaging devices 31 and 41, f is the focal length [m], and d is the parallax [px]. xp is an arbitrary X coordinate (horizontal pixel of an arbitrary pixel) in the parallax image, and x'is the center X coordinate in the parallax image. yp is an arbitrary Y coordinate in the parallax image (vertical pixel of an arbitrary pixel), and y'is the center Y coordinate in the parallax image. The above equations (1) to (3) are calculated for all the objects in the parallax image.

Here, the axis extending in the front-rear direction of the forklift 10 is the Y-axis as the first axis, the axis extending in the vertical direction is the Z-axis as the second axis, and the Y-axis and the axis orthogonal to the Z-axis are the third axes. The coordinates in the 3-axis Cartesian coordinate system (world coordinate system), which is the X-axis, are the coordinates in the real space (world coordinates). The position of the object in the real space can be represented by the X coordinate Xw, the Y coordinate Yw, and the Z coordinate Zw in the real space.

When the image pickup devices 31 and 41 are installed so that the optical axes of the cameras 32, 33, 42, and 43 are horizontal (orthogonal to the vertical direction), Xc = Xw, Yc = Zw, and Zc = Yw. That is, the position of the object in the camera coordinate system is the position of the object in the real space.

On the other hand, when the optical axes of the cameras 32, 33, 42, and 43 are not horizontal (orthogonal to the vertical direction), the image processing device 50 performs coordinate conversion (world coordinate conversion) using the following equation (4). Do.

Here, H is the mounting height of the imaging devices 31 and 41 in the world coordinate system, and θ is the angle + 90 ° between the optical axes of the cameras 32, 33, 42 and 43 and the horizontal plane. In the present embodiment, the image processing device 50 functions as a calculation means.

Next, the image processing device 50 determines whether or not there is an obstacle in the traveling direction (step S12). The traveling direction can be grasped from, for example, the rotation direction of the traveling motor M1 detected by the vehicle speed sensor 24 and the angle detected by the steering wheel angle sensor 26. The presence or absence of an obstacle is determined by plotting the object in the world coordinate system.

As an example, as shown in FIG. 5, when the forklift 10 is moving backward and there is an obstacle (shutter) B having a height that collides with the mast 15 behind the forklift 10, the obstacle detection is performed. explain.

As shown in FIG. 6, the position of the object is plotted in the world coordinate system for the region behind the forklift 10. Let the position of the object plotted in the world coordinate system be the feature point P. The feature point P can be said to be the coordinates obtained by obtaining the parallax (position of the object) in the world coordinate system. In FIG. 6, the lower end of the obstacle is plotted as an object.

The image processing device 50 determines that an object that may collide with the forklift 10 is an obstacle. It should be noted that the determination as to whether or not the object is an obstacle can be determined from the Z coordinate Zw of the object. The image processing device 50 determines whether or not the object is an obstacle by comparing the height of the forklift 10 (for example, the highest reaching point of the inner mast 17) with the height of the object.

As shown in FIG. 3, when the determination result in step S12 is affirmative, the image processing device 50 performs the process in step S13. On the other hand, if the determination result in step S12 is negative, the image processing device 50 performs the process in step S14. First, a case where the determination result in step S12 is affirmative will be described.

The image processing device 50 clusters obstacles and stores (stores) the position information (coordinates) of the clustered obstacles (step S13). Clustering is to regard an aggregate of a plurality of feature points P plotting an object as an obstacle as one obstacle B. For example, as shown in FIG. 6, feature points P densely packed on the X-axis are treated as one obstacle B. The position information (world coordinates) of the obstacle B is temporarily stored (stored) in a rewritable memory such as RAM or EEPROM. The position information of the obstacle B is determined based on the coordinates of the clustered obstacle B. For example, the coordinates of any feature point P from the plurality of clustered feature points P may be used as the position information of the obstacle B, or the center coordinates of the clustered obstacle B may be used as the position information. The image processing device 50 functions as a storage means.

As shown in FIG. 3, the image processing device 50 performs the relative position calculation process (step S20).
As shown in FIG. 4, in the relative position calculation process, the image processing device 50 calculates the relative position between the forklift 10 and the obstacle B (step S21). The relative position can be calculated from the Y coordinate Yw of the obstacle B. The image processing device 50 outputs the calculated relative position to the main controller 20.

The main controller 20 determines whether or not the relative distance between the forklift 10 and the obstacle B is equal to or less than a predetermined value (step S22). The predetermined value is, for example, a value determined based on the maximum speed of the forklift 10 and the braking performance.

If the determination result in step S22 is negative, the main controller 20 ends the process. If the determination result in step S22 is affirmative, the main controller 20 performs deceleration processing (step S23). The deceleration process includes a process of stopping the forklift 10, a process of gradually reducing the speed of the forklift 10 from the current speed, a process of imposing a limit on the speed of the forklift 10 and restricting running at a speed exceeding the speed limit, and the like. , Various processing can be mentioned.

Next, in step S12, processing when the obstacle B is not detected will be described.
As shown in FIG. 3, when the determination result in step S12 is negative, the image processing device 50 performs the process in step S14. The image processing device 50 determines whether or not the obstacle B has been detected in the process one cycle before (step S14).

As shown in FIGS. 7 and 8, when the forklift 10 approaches the obstacle B from the state of FIG. 5, the obstacle B is removed from the vertical angle of view θ2 of the rear imaging device 41. Then, the obstacle B is not imaged by the rear image pickup device 41. When the obstacle B is no longer imaged by the rear imaging device 41, the feature points P that were plotted when the obstacle B was imaged are not plotted. That is, the obstacle B once detected is not detected (detection is lost). If the determination result in step S14 is affirmative, it can be determined that the obstacle B has deviated from the vertical angles of view θ1 and θ2 of the image pickup devices 31 and 41 due to the movement of the forklift 10.

As shown in FIG. 3, when the determination result in step S14 is affirmative, the image processing device 50 performs the process in step S15. The image processing device 50 turns on the position estimation flag (step S15). Next, the image processing device 50 performs the relative position calculation process (step S20). In the relative position calculation process at this time, when calculating the relative position in step S21, the relative position is calculated based on the position information stored in step S13, the traveling direction of the forklift 10, and the speed of the forklift 10. .. The traveling direction of the forklift 10 can be grasped from the rotation direction and the steering wheel angle of the traveling motor M1. If the traveling direction and the speed of the forklift 10 can be grasped, the direction in which the forklift 10 has moved and the distance moved from the position where the obstacle B was last detected can be grasped. Therefore, the relative position between the obstacle B and the forklift 10 can be calculated after adding the moving direction and the moving distance of the forklift 10 to the position of the obstacle B stored in step S13. Considering that the distance traveled by the forklift 10 during one control cycle is small, the relative position between the forklift 10 and the obstacle B is the same as that calculated in the previous control cycle. May be regarded as. In the following description, the traveling direction of the forklift 10 and the speed of the forklift 10 will be described as vehicle information.

If the determination result in step S14 is negative, the image processing device 50 performs the process in step S16. The image processing device 50 determines whether or not the position estimation flag is on (step S16). If the determination result in step S16 is affirmative, it can be determined that the obstacle B once detected is no longer detected due to the movement of the forklift 10. On the other hand, if the determination result in step S16 is negative, it can be determined that the obstacle B has not been detected. If the determination result in step S16 is negative, the image processing device 50 determines that there is no obstacle B with which the forklift 10 collides, and ends the process. If the determination result in step S16 is affirmative, the image processing device 50 performs the process in step S17.

The image processing device 50 estimates the position of the obstacle B (step S17). The image processing device 50 estimates the position of the obstacle B from the position information of the obstacle B and the vehicle information stored in step S13.

The image processing device 50 calculates the direction in which the forklift 10 has moved and the distance traveled from the control cycle in which the obstacle B was last detected to the current control cycle from the vehicle information. When the forklift 10 moves, the position (world coordinates) of the obstacle B also changes. The image processing device 50 estimates the current position (world coordinates) of the obstacle B by adding the moving direction of the forklift 10 obtained by the vehicle information and the moved distance to the position information stored in step S13. To do. As a result, the position of the obstacle B is estimated even if the obstacle B is not imaged by the imaging devices 31 and 41. The image processing device 50 functions as an estimation means.

Next, the image processing device 50 performs the relative position calculation process (step S20). In the relative position calculation process at this time, when the relative position is calculated in step S21, the relative position between the forklift 10 and the obstacle B is calculated from the position of the obstacle B estimated in step S17.

In the present embodiment, the position estimation flag is set when it is estimated that the forklift 10 has passed the detected obstacle B, when deceleration processing is performed, when the determination result in step S12 is affirmative, and the like. Is turned off.

Therefore, according to the above embodiment, the following effects can be obtained.
(1) The image processing device 50 estimates the position of the obstacle B when the obstacle B is no longer imaged by the image pickup devices 31 and 41 due to the movement of the forklift 10. Therefore, even when the obstacle B is no longer imaged by the imaging devices 31 and 41, the relative position between the forklift 10 and the obstacle B can be grasped. Therefore, it is prevented that the positional relationship between the forklift 10 and the obstacle B cannot be grasped.

(2) The image processing device 50 can grasp from the vehicle information the direction in which the forklift 10 has moved from the position where the obstacle B was last detected and the distance in which the forklift 10 has moved. By utilizing this, the image processing device 50 can estimate the position of the obstacle B.

(3) The image pickup devices 31 and 41 take an image of the obstacle B reflected in the range of the vertical angles of view θ1 and θ2. Therefore, the range in which obstacles can be detected is wider than in the case where the distance to an obstacle is detected by a range finder that measures the distance with ultrasonic waves or a laser, and thereby the obstacle is detected.

Further, when an obstacle is detected by the distance meter, it is necessary to arrange the distance meter on the upper part of the forklift 10 in order to irradiate the obstacle with ultrasonic waves or a laser. On the other hand, when the image pickup devices 31 and 41 detect an obstacle, the image pickup devices 31 and 41 can be arranged in addition to the upper part of the forklift 10 because the range in which the obstacle can be detected is wide. That is, there are few restrictions on placement.

(Second Embodiment)
Hereinafter, a second embodiment of the safety device will be described. The safety device of the second embodiment is different from the first embodiment in the processing related to the detection of obstacles. In the following description, the same description as in the first embodiment will be omitted or simplified by adding the same reference numerals.

As shown in FIG. 9, the image processing apparatus 50 calculates the parallax (step S10) and calculates the position of the object using the parallax (step S11), as in the first embodiment. Next, the image processing device 50 acquires the height of the mast 15 (inner mast 17) (step S31). The height of the mast 15 can be grasped by acquiring the detection result of the lift sensor 25. The detection result of the lift sensor 25 may be acquired from the lift sensor 25 or may be acquired via the main controller 20.

Next, the image processing device 50 determines whether or not there is an obstacle B in the traveling direction (step S12), and if there is an obstacle B, clusters and saves it (step S13).
Next, the image processing device 50 estimates the movement of the obstacle B (step S40). The movement estimation of the obstacle B is a process of determining whether or not the obstacle B is moving (moving up and down), and if the obstacle B is moving, calculating the moving speed. Depending on the obstacle B, such as a shutter that automatically opens and closes when the forklift 10 or the like is approaching, the position in the vertical direction may change. In this case, whether or not the forklift 10 collides with the forklift 10 differs depending on the vertical position of the obstacle B, so it is determined whether or not the obstacle B is moving.

As shown in FIG. 10, in estimating the movement of the obstacle B, the image processing device 50 determines whether or not the obstacle B is moving (step S41). The movement determined in the movement estimation of the obstacle B is the presence or absence of vertical movement, and the presence or absence of horizontal movement is not determined. If the determination result in step S41 is negative, the image processing device 50 ends the process of estimating the movement of the obstacle B.

If the determination result in step S41 is affirmative, the image processing device 50 calculates the moving speed of the obstacle B and the moving direction (up or down) of the obstacle B (step S42). The determination of whether or not the obstacle B is moving, the calculation of the moving direction and the moving speed of the obstacle B are performed by the so-called optical flow.

As shown in FIG. 9, when the movement estimation of the obstacle B is completed, the image processing device 50 performs the relative position calculation process (step S20). The relative position calculation process is the same process as that of the first embodiment.

If the determination result in step S12 is negative, the image processing device 50 determines whether or not the obstacle B has been detected in the process one cycle before (step S14). If the determination result in step S14 is negative, the image processing device 50 determines whether or not the position estimation flag is on (step S16). If the determination result in step S16 is affirmative, the image processing device 50 performs the process in step S32.

In step S32, the image processing device 50 estimates the height of the obstacle B. The height of the obstacle B can be estimated from the moving speed calculated in step S40. The image processing device 50 calculates the amount of movement of the obstacle B after the obstacle B is no longer imaged from the time elapsed since the obstacle B is no longer detected and the movement speed and the movement direction calculated in step S40. To do. By adding this movement amount to the height of the obstacle B stored in step S13, the height of the obstacle B can be estimated. If the determination result in step S41 is negative, that is, if it is determined that the obstacle B is not moving, the height of the obstacle B is estimated to be the same as the height memorized in step S13. Will be done.

Next, the image processing device 50 estimates the position of the obstacle B (step S17). Next, the image processing device 50 determines whether or not the height of the obstacle B estimated in step S32 is equal to or less than the height of the mast 15 (step S34).

If the determination result in step S34 is negative, it can be determined that the obstacle B does not collide with the mast 15 because the obstacle B rises. On the other hand, if the determination result in step S34 is affirmative, it can be determined that the obstacle B and the mast 15 collide. If the determination result in step S34 is negative, the image processing device 50 turns off the position estimation flag (step S35) and ends the process. On the other hand, if the determination result in step S34 is affirmative, the image processing device 50 performs the relative position calculation process (step S20). The image processing device 50 functions as a collision determination means.

Therefore, according to the above embodiment, the following effects can be obtained in addition to the effects of the first embodiment.
(2-1) It is determined whether or not the obstacle B moves in the vertical direction, and if it moves, the moving speed is detected. When the obstacle B is no longer imaged by the imaging devices 31 and 41, the height of the obstacle B can be estimated from the moving speed of the obstacle B.

In the case of a moving object in which the obstacle B moves in the vertical direction, the descending obstacle B that was in a position not to collide with the forklift 10 while being imaged by the image pickup devices 31 and 41 is imaged by the image pickup devices 31 and 41. After it disappears, it may have descended to a position where it collides with the forklift 10. In this case, the forklift 10 and the obstacle B may collide with each other even though the position of the obstacle B is estimated. Further, the ascending obstacle B that was in the position of colliding with the forklift 10 while being imaged by the imaging devices 31 and 41 is not imaged by the imaging devices 31 and 41 until the position does not collide with the forklift 10. It may be rising. In this case, the deceleration process may be performed even though the forklift 10 does not collide with the obstacle B, and the work efficiency may decrease.

On the other hand, by estimating the height of the obstacle B in consideration of the moving speed of the obstacle B, it is possible to suppress the collision between the forklift 10 and the obstacle B, and the forklift 10 does not collide with the obstacle B. It is also possible to suppress the deceleration process.

The embodiment may be changed as follows.
○ As shown in FIG. 11, the image processing device 50 may estimate the position of the obstacle B from the correlation between the reference object N and the obstacle B. The reference object N is imaged together with the obstacle B when the obstacle B is imaged by the imaging devices 31 and 41, and is also imaged when the obstacle B is no longer imaged due to the movement of the forklift 10. It is what has been done. The reference object N is preferably one that does not move, such as a white line or a sign, that is, one that does not change the positional relationship with the obstacle B.

The image processing device 50 calculates the positions of the obstacle B and the reference object N when they are imaged. Then, the position of the obstacle B and the position of the reference object N are associated with each other. Even when the obstacle B is no longer imaged by the image pickup devices 31 and 41, the image processing device 50 continues to calculate the position of the reference object N (the position relative to the forklift 10). Further, the image processing device 50 estimates the position of the obstacle B from the positional relationship between the position of the reference object N and the obstacle B. Even when the obstacle B is no longer imaged, the position of the obstacle B can be estimated by continuously monitoring the change in the relative position between the forklift 10 and the reference object N.

The position of the obstacle B may be estimated by combining the position estimation of the obstacle B by the reference object N and the position estimation of the obstacle B by the vehicle information. For example, both the position estimation of the obstacle B by the reference object N and the position estimation by the vehicle information may be performed, and the consistency between the two may be confirmed.

○ In each embodiment, when the relative distance between the forklift 10 and the obstacle B is equal to or less than a predetermined value, the main controller 20 notifies the notification by displaying the fact on the display unit or sounding a buzzer sound. May be good.

-In each embodiment, the imaging devices 31 and 41 including three or more cameras may be used.
-In each embodiment, the moving body may be a vehicle such as a passenger car, a construction machine, an automatic guided vehicle, a truck, or a bipedal robot. The moving body is preferably one that moves indoors.

-In each embodiment, the front image pickup device 31 and the rear image pickup device 41 do not have to have the same configuration.
○ In each embodiment, the traveling direction of the forklift 10 may be determined from the detection result of the vehicle speed sensor 24, the steering angle of the steering wheel 14, and the like.

-In each embodiment, the image processing device 50 is provided as a device that performs image processing of the images captured by the image pickup devices 31 and 41, but the image processing may be performed by the main controller 20 or the like. That is, the image processing may be performed as one function of the main controller 20 by controlling the traveling operation and the cargo handling operation.

-In each embodiment, the image processing device 50 functions as a calculation means, a storage means, an estimation means, and a collision determination means, but individual devices that function as the respective means may be provided.

-In each embodiment, an engine may be used as a traveling drive device. That is, the forklift 10 may travel by driving the engine. In this case, the travel control device is a device that controls the fuel injection amount to the engine and the like.

○ In each embodiment, in the case of a moving body that can only move forward, it is not necessary to provide the rear imaging device 41.
-In each embodiment, when the safety device 30 is mounted on a moving body on which the passenger can board, only obstacles behind the vehicle, which are difficult for the passenger to visually recognize, may be detected. In this case, it is not necessary to provide the front imaging device 31.

○ In each embodiment, the front imaging device 31 may be arranged at any position as long as it can image the front of the forklift 10. The rear imaging device 41 may be arranged at any position as long as it can image the rear of the forklift 10. For example, the front image pickup device 31 may be arranged on the mast 15, and the rear image pickup device 41 may be arranged on a counterweight provided behind the vehicle body 11. When the front image pickup device 31 is arranged on the mast 15, the upper side of the vertical angle of view θ1 of the front image pickup device 31 is above the horizontal plane in the state where the mast 15 is tilted most forward.

○ In each embodiment, the coordinate transformation may be performed by table data instead of the equation (4). The table data is table data in which the Y coordinate Yw is associated with the combination of the Y coordinate Yc and the Z coordinate Zc, and the table data in which the Z coordinate Zw is associated with the combination of the Y coordinate Yc and the Z coordinate Zc. By storing these table data in the ROM of the image processing device 50 or the like, the Y coordinate Yw and the Z coordinate Zw in the world coordinate system can be obtained from the Y coordinate Yc and the Z coordinate Zc in the camera coordinate system. In the embodiment, since the X coordinate Xc in the camera coordinate system and the X coordinate Xw in the world coordinate system match, the table data for obtaining the X coordinate Xw is not stored.

N ... Reference object, θ1, θ2 ... Vertical angle of view, 10 ... Forklift (moving body), 30 ... Safety device, 31,41 ... Imaging device (stereo camera), 50 ... Image processing device (calculation means, storage means, estimation) Means and collision determination means).

Claims (3)

A safety device mounted on a moving body
A stereo camera arranged so that the uppermost vertical angle of view is above the horizontal plane,
A calculation means for calculating the position of an obstacle from an image captured by the stereo camera, and
A storage means for storing the position information of the obstacle and
When the obstacle is no longer imaged by the stereo camera due to the movement of the moving body, an estimation means for estimating the position of the obstacle based on the position information stored by the storage means is provided .
The estimation means is also captured when the obstacle is imaged together with the obstacle when the obstacle is imaged by the stereo camera, and the obstacle is not imaged due to the movement of the moving body. Based on what is
A safety device that estimates the position of an obstacle from a change in the relative position between the reference object and the moving body due to the movement of the moving body. The safety device according to claim 1, wherein the estimation means estimates the position of the obstacle from the speed of the moving body and the traveling direction of the moving body. A collision determining means for detecting the moving speed of the obstacle in the vertical direction and determining whether or not the obstacle collides with the moving body when the obstacle continues to move at the moving speed is provided. The safety device according to claim 1 or 2.