JP6358629B2 - forklift - Google Patents

Description

  The present invention relates to a forklift provided with a camera.

  Forklifts are used in various environments to load and unload luggage. For example, a forklift is used in a platform provided in accordance with the height of the truck bed to load or unload the truck from the truck bed, and the platform height is generally About 1m. In such an environment, if the forklift driver misoperates, the forklift may fall from the platform to the ground.

  Patent Document 1 describes a forklift provided with a camera in order to prevent the forklift from falling from the platform. This forklift detects a mark of an entry restriction area provided on the road surface based on image data relating to an image taken by a camera, and when the distance between the mark of the entry restriction area and the vehicle body is less than a predetermined value Limit the running speed of the vehicle body. The mark of the entry restriction region is configured by a line (tape stuck on the road surface or ink applied to the road surface) having a hue that is easily noticeable with respect to the road surface.

  However, if the line, which is the mark of the entry restriction area, is contaminated, such as dirt or damage, the mark cannot be detected based on the image data, and the forklift may enter the entry restriction area. .

Japanese Patent Laying-Open No. 2015-217738

  The present invention has been made in view of the above circumstances, and it is an object of the present invention to provide a forklift capable of preventing entry into the entry restriction region even when the mark in the entry restriction region is contaminated. To do.

In order to solve the above-described problem, the invention according to claim 1 is provided with a plurality of markers provided in a row at the boundary between the entry restriction area and the travel permission area on the road surface , each having a feature point. A vehicle body that travels on the road surface, and a camera that generates image data by simultaneously capturing a plurality of the markers in a row, and a plurality of the markers that are simultaneously captured based on the generated image data And a vehicle position estimation unit that estimates the position of the vehicle body with respect to each of the plurality of markers based on at least one of the feature points of the plurality of recognized markers. a plurality of based on at least one of the feature points of the marker, and the vehicle posture estimation unit that estimates a posture of the vehicle body for each of the plurality of the markers, estimated the vehicle Based on both the position and attitude of the body, the vehicle body and a deviation travel suppressing unit for performing an operation for preventing the traveling beyond said marker a row, the vehicle position estimating section, by estimating the position of the vehicle body for each of the plurality of the markers to calculate a distance to each of the plurality of the markers from the vehicle body, the vehicle posture estimation unit, wherein for each of the plurality of the marker By estimating the posture of the vehicle main body, an angle formed by the traveling direction of the vehicle main body and the direction in which each of the plurality of markers is located from the vehicle main body is calculated. when the distance to a particular marker is any one of a plurality of said markers is less than the predetermined, and the traveling direction and the specific marker position from the vehicle body of the vehicle body When the angle between the direction in which is less than a predetermined, and performs the operation for suppressing said vehicle body traveling on the traveling allowable region travels past the marker a row.

According to a second aspect of the present invention, in the forklift according to the first aspect, when the vehicle position estimating unit has a recognition accuracy of one or more of the markers by the marker recognizing unit being less than a predetermined value, Based on the above marker, the position of the vehicle main body with respect to each of the markers having a recognition accuracy of a predetermined value or more is estimated.

According to a third aspect of the present invention, in the forklift according to the first or second aspect , when the vehicle posture estimation unit is less than a predetermined recognition accuracy of the one or more markers by the marker recognition unit, Based on the one or more markers, the posture of the vehicle body with respect to each of the markers having a recognition accuracy of a predetermined value or more is estimated.

According to a fourth aspect of the present invention, in the forklift according to any one of the first to third aspects, the deviation travel suppression unit suppresses the vehicle body from traveling beyond the markers in a row. As an operation for this purpose, a notification operation that emits sound or light is performed.

According to a fifth aspect of the present invention, in the forklift according to any one of the first to fourth aspects, the deviation traveling suppression unit suppresses the vehicle body from traveling beyond the marker in a row. As an operation for this purpose, the vehicle is controlled to travel.

  According to the present invention, it is possible to provide a forklift capable of preventing entry into the entry restriction area even when the mark in the entry restriction area is contaminated.

(A) And (B) is a whole block diagram of the fall prevention system containing the forklift which concerns on one Embodiment of this invention. It is a block diagram which shows schematic structure of the forklift which concerns on the embodiment. It is a flowchart which shows the flow of the fall prevention process which concerns on the same embodiment.

An embodiment of the present invention will be described with reference to the drawings. In addition, the front-rear direction X, the left-right direction Y, and the up-down direction Z indicated by arrows in the drawing are directions orthogonal to each other.
As shown in FIG. 1, a counterbalance forklift 1 includes a vehicle body 10 that travels on a road surface S, a cargo handling device 20 that loads and unloads loads (not shown), and an operator (not shown) operates the forklift 1. And a camera 40 for photographing the marker M provided on the road surface S.

  The vehicle main body 10 includes a boarding unit 11 on which an operator boards, a head guard 12 disposed above the boarding unit 11, and a stay 13 that supports the head guard 12. The vehicle body 10 can travel straight in the front-rear direction X, and can also travel while turning in the left-right direction Y.

  The cargo handling device 20 includes a mast 21 that expands and contracts in the vertical direction Z and a fork 22 that moves up and down along the mast 21. The cargo handling device 20 is provided in front of the vehicle body 10 and moves on the road surface S together with the vehicle body 10.

  The operating device 30 includes an accelerator pedal 31 for inputting an instruction for accelerating the traveling of the vehicle main body 10, a steering handle 32 for inputting an instruction for turning the vehicle main body 10, and the like.

  The camera 40 is a monocular camera that captures the road surface S behind the vehicle body 10, and includes, for example, a global shutter type monochrome image sensor and a wide-angle lens. The camera 40 is attached to a stay 13 provided at the rear part of the vehicle body 10. The camera 40 generates image data by simultaneously photographing a plurality of markers M forming a row in a predetermined range.

  The marker M installed on the road surface S is a mark of the entry restriction area, and is configured by, for example, a sticker attached to the road surface S. The markers M are provided side by side at the edge of the platform P, and the plurality of markers M form a line at the boundary between the entry restriction area and the travel allowable area. The marker M has a rectangular outer shape and a characteristic pattern.

With reference to FIG. 2, the structure of the forklift 1 not shown in FIG. 1 will be described.
As shown in FIG. 2, the forklift 1 includes a marker recognition unit 51, a vehicle position estimation unit 52, a vehicle posture estimation unit 53, and a deviation travel suppression unit 60.

  The marker recognizing unit 51 is an image processing circuit that recognizes a plurality of markers M photographed simultaneously by the camera 40 based on image data generated by the camera 40. The marker recognition unit 51 acquires image data from the camera 40, performs various image processing on the image data, and recognizes (detects) the marker M from the image data.

  The vehicle position estimation unit 52 is an arithmetic circuit that estimates the position of the vehicle main body 10 with respect to the marker M based on at least one of the plurality of markers M recognized by the marker recognition unit 51. That is, the vehicle position estimation unit 52 calculates the distance from the vehicle main body 10 to the marker M by estimating the position of the vehicle main body 10 with respect to the marker M. When the marker recognition unit 51 recognizes the plurality of markers M, the vehicle position estimation unit 52 estimates the position of the vehicle main body 10 with respect to each marker M based on each of the markers M. Moreover, the vehicle position estimation part 52 estimates the position of the vehicle main body 10 based on one or more other markers M, when the recognition accuracy of the one or more markers M by the marker recognition part 51 is less than predetermined. . That is, the vehicle position estimation unit 52 does not estimate the position of the vehicle main body 10 with respect to the marker M for the marker M with low recognition accuracy in the marker recognition unit 51.

  The vehicle posture estimation unit 53 is an arithmetic circuit that estimates the posture of the vehicle main body 10 with respect to the marker M based on at least one of the plurality of markers M recognized by the marker recognition unit 51. That is, the vehicle posture estimation unit 53 calculates the angle between the traveling direction of the vehicle main body 10 and the direction in which the marker M is located from the vehicle main body 10 by estimating the posture of the vehicle main body 10 with respect to the marker M. When the marker recognition unit 51 recognizes the plurality of markers M, the vehicle posture estimation unit 53 estimates the posture of the vehicle main body 10 with respect to each marker M based on each of the markers M. The vehicle posture estimation unit 53 estimates the posture of the vehicle main body 10 based on one or more other markers M when the recognition accuracy of the one or more markers M by the marker recognition unit 51 is less than a predetermined value. . That is, the vehicle posture estimation unit 53 does not estimate the posture of the vehicle main body 10 with respect to the marker M with respect to the marker M whose recognition accuracy in the marker recognition unit 51 is low.

  Based on the position of the vehicle main body 10 estimated by the vehicle position estimation unit 52 and the posture of the vehicle main body 10 estimated by the vehicle posture estimation unit 53, the departure travel suppression unit 60 displays the marker M in a row. They are a control circuit and an operation mechanism that perform an operation (hereinafter referred to as “departure travel suppression operation”) for suppressing traveling beyond. Specifically, the deviation travel suppression unit 60 includes a speaker and a warning light that perform a notification operation, and a travel control circuit that controls the travel of the vehicle body 10. The departure travel suppression unit 60 performs a notification operation by emitting sound and light from a speaker and a warning light, and prompts the operator to operate the forklift 1 so that the forklift 1 moves away from the entry restriction region. Further, the deviation travel suppression unit 60 invalidates the operator's instruction input to the operation device 30 and performs control for stopping the travel of the vehicle body 10 to prevent the forklift 1 from entering the entry restriction region.

  Next, the flow of the fall prevention process by the forklift 1 will be described with reference to FIG. Note that it is preferable that the series of flows in FIG. 3 is repeated without taking time during the operation of the forklift 1.

  First, the camera 40 directed to the road surface S captures the marker M and generates image data including an image of the marker M (step S1). Next, the marker recognition unit 51 recognizes one or more markers M based on the image data generated in step S1 (step S2).

  In step S2, the marker recognizing unit 51 performs binarization processing, edge detection (contour extraction) processing, polygon approximation processing, and previously stored marker M on the image data acquired by the camera 40. By performing image processing such as matching processing with the data, the marker M is recognized from the image data. At this time, in a matching process between part of the image data acquired by the camera 40 and the data of the marker M stored in advance, a correlation coefficient indicating the recognition accuracy of the marker M is calculated, and the correlation coefficient is a predetermined threshold value. In the case described above, the marker M is recognized from the image data.

  Next, based on the marker M recognized in step S2, the vehicle position estimation unit 52 estimates the position of the vehicle main body 10, and the vehicle posture estimation unit 53 estimates the posture of the vehicle main body 10 (step S3). When a plurality of markers M are recognized in step S2, the position and orientation of the vehicle body 10 with respect to each marker M are estimated.

In step S3, the camera 40 is used as a reference based on the coordinates of the feature point of the recognized marker M included in the image data (for example, the vertex that is the end point of the line segment forming the outline of the marker M). By calculating a transformation matrix between the coordinate system and the marker coordinate system based on the marker M, the position and orientation of the vehicle body 10 with respect to the marker M are estimated. Specifically, the coordinates (Xc, Yc, Zc) of the camera coordinate system and the coordinates (Xm, Ym, Zm) of the marker coordinate system have the relationship of the following formula (1), and the vehicle position estimation unit 52 estimates the position of the marker M with respect to the camera 40, that is, the position of the vehicle body 10 with respect to the marker M, by calculating the component (translation component) of the translation vector T included in Equation (1). Further, the vehicle posture estimation unit 53 calculates the component of the 3 × 3 rotation matrix R (rotational movement component) included in the mathematical expression (1), so that the direction of the marker M with respect to the camera 40, that is, the vehicle with respect to the marker M The posture of the main body 10 is estimated. Further, the vehicle posture estimation unit 53 corrects the rotation matrix R by the hill-climbing method to accurately estimate the posture of the vehicle main body 10, and the vehicle position estimation unit 52 translates the translation vector based on the corrected rotation matrix R. By correcting T, the position of the vehicle body 10 is accurately estimated.

  Next, based on the position of the vehicle main body 10 estimated in step S3, the control circuit constituting the deviation travel suppression unit 60 determines whether or not the forklift 1 is located in the vicinity of the marker M (step S4). That is, in step S4, when the distance from the vehicle main body 10 to the marker M is less than the predetermined distance, the departure travel suppression unit 60 determines that the forklift 1 is located in the vicinity of the marker M, and from the vehicle main body 10 to the marker M. When the distance is equal to or greater than a predetermined distance, it is determined that the forklift 1 is not located near the marker M. When a plurality of markers M are recognized in step S2, it is determined that the forklift 1 is located in the vicinity of the marker M when the distance from the vehicle body 10 to any one of the plurality of markers M is less than a predetermined value. The

  When it is determined in step S4 that the forklift 1 is not located in the vicinity of the marker M, the departure travel suppression unit 60 determines that there is no possibility that the forklift 1 enters the entry restriction region, and performs the departure travel suppression operation. The fall prevention process ends.

  On the other hand, when it is determined in step S4 that the forklift 1 is located in the vicinity of the marker M, the deviation travel suppression unit 60 determines that the forklift 1 may enter the entry restriction region, and issues a sound and light. An operation is performed (step S5).

  Further, based on the attitude of the vehicle main body 10 estimated in step S3, the control circuit constituting the deviation travel suppressing unit 60 determines whether or not the travel direction of the forklift 1 is a direction approaching the marker M (step). S6). That is, in step S6, when the deviation travel suppression unit 60 determines that the travel direction of the vehicle body 10 and the direction in which the marker M is located from the vehicle body 10 is less than a predetermined angle, the travel direction of the forklift 1 is the marker M. When the angle between the traveling direction of the vehicle main body 10 and the direction in which the marker M is located from the vehicle main body 10 is greater than or equal to a predetermined angle, the traveling direction of the forklift 1 is a direction that does not approach the marker M. Judge that there is. It is preferable to determine whether or not the forklift 1 travels in the direction approaching the marker M only for the marker M determined in step S4 when the forklift 1 is located in the vicinity.

  When it is determined in step S6 that the traveling direction of the forklift 1 is not close to the marker M, the departure traveling suppression unit 60 determines that the probability that the forklift 1 enters the entry restriction region is low, and the vehicle body 10 The fall prevention process is terminated without controlling the traveling.

  On the other hand, when it is determined in step S6 that the traveling direction of the forklift 1 is the direction approaching the marker M, the departure traveling suppression unit 60 determines that the probability that the forklift 1 enters the entry restriction region is high, and the vehicle body 10 The traveling is controlled by stopping the traveling (step S7).

According to the present embodiment, the following effects can be obtained.
(1) A plurality of markers M are arranged along an entry restriction area such as an edge of the platform P, and the forklift 1 enters the entry restriction area by photographing these markers M as a mark of the entry restriction area. It is possible to suppress this. In addition, since the position of the vehicle body 10 with respect to the marker M is estimated based on at least one of the plurality of recognized markers M, for example, even if the recognition accuracy of one marker M is less than a predetermined value, the plurality of markers It is possible to accurately estimate the position of the vehicle body 10 based on M or one or more other markers M determined to have high recognition accuracy. Therefore, even if some of the markers M that are marks of the entry restriction area are contaminated, the forklift 1 does not enter the entry restriction area by accurately recognizing other markers M that are marks of the entry restriction area. It is possible to do so.

  (2) When the accuracy of recognition of one or more markers M by the marker recognizing unit 51 is less than a predetermined value, the vehicle position estimating unit 52 determines the vehicle body 10 relative to the marker M based on one or more other markers M. Estimate the position. For this reason, it is possible to accurately estimate the position of the vehicle main body 10 by excluding the marker M whose recognition accuracy is less than a predetermined value.

  (3) The departure travel suppression unit 60 performs the departure travel suppression operation based on both the estimated position and orientation of the vehicle body 10. For this reason, it is possible to appropriately determine whether or not the forklift 1 is likely to enter the entry restriction region, and to prevent the forklift 1 from entering the entry restriction region.

  (4) When the recognition accuracy of the one or more markers M by the marker recognizing unit 51 is less than a predetermined value, the vehicle posture estimating unit 53 determines the vehicle body 10 with respect to the marker M based on the one or more other markers M. Estimate posture. For this reason, it is possible to accurately estimate the posture of the vehicle main body 10 by excluding the marker M whose recognition accuracy is less than a predetermined value.

  (5) Since the departure travel suppression unit 60 performs a notification operation that emits sound or light as the departure travel suppression operation, the operator audibly or visually confirms that there is a high probability that the forklift 1 enters the entry restriction region. I can know.

  (6) Since the departure travel suppression unit 60 controls the travel of the vehicle main body 10 as the departure travel suppression operation, the forklift 1 can be prevented from entering the entry restriction region regardless of the operator.

  The present invention is not limited to the above embodiment, and the above configuration can be changed as appropriate. For example, the following modifications can be implemented, and the following modifications can be combined.

  The marker M may be composed of ink applied to the road surface S. Moreover, you may change the shape of a marker M, a dimension, a pattern, etc. Further, the markers M may be arranged so as to form two or more rows.

  The configuration and arrangement of the camera 40 can be changed. For example, the camera 40 may be configured by a color image sensor and a fisheye lens. The camera 40 may be attached to the head guard 12.

  The marker recognizing unit 51 may estimate the position of the vehicle main body 10 based on a marker whose recognition accuracy is less than a predetermined value. That is, for example, even when the recognition accuracy of one marker M is low, the position of the vehicle main body 10 can be appropriately estimated based on the plurality of other markers M.

  The vehicle posture estimation unit 53 may estimate the posture of the vehicle body 10 based on a marker whose recognition accuracy is less than a predetermined value. That is, for example, even when the recognition accuracy of one marker M is low, the posture of the vehicle main body 10 can be appropriately estimated based on the plurality of other markers M.

  The departure travel suppression unit 60 may perform the departure travel suppression operation without being based on the attitude of the vehicle body 10. That is, the departure travel suppression unit 60 may perform the departure travel suppression operation based only on the position of the vehicle main body 10, and when the distance from the vehicle main body 10 to the marker M is less than a predetermined value, the attitude of the vehicle main body 10 is determined. Regardless of this, the deviation travel suppression operation for both the notification operation and the travel control may be performed.

  The departure traveling suppression unit 60 may not perform traveling control. That is, the departure travel suppression unit 60 may perform only the notification operation as the departure travel suppression operation. Moreover, you may emit only one of a sound and light as alerting | reporting operation | movement which the deviation driving | running | working suppression part 60 performs. The notification operation may be performed by means other than the speaker and the warning light.

  The departure traveling suppression unit 60 may not perform the notification operation. That is, the departure travel suppression unit 60 may perform only the travel control of the vehicle body 10 as the departure travel suppression operation. In addition, as the travel control performed by the departure travel suppression unit 60, control other than travel stop may be performed. That is, the departure travel suppression unit 60 may perform control for limiting the travel speed of the vehicle main body 10 or control for causing the vehicle main body 10 to travel away from the marker M as the departure travel suppression operation.

DESCRIPTION OF SYMBOLS 1 Forklift 10 Vehicle main body 40 Camera 51 Marker recognition part 52 Vehicle position estimation part 53 Vehicle attitude | position estimation part 60 Departure travel suppression part M Marker (mark of an entry control area)
S road surface

Claims (5)

A plurality of markers provided in a row at the boundary between the entry restriction area and the travel allowable area on the road surface , each having a feature point ;
A vehicle body traveling on the road surface;
A camera that generates image data by simultaneously photographing a plurality of the markers in a row;
Based on the generated image data, a marker recognition unit that recognizes a plurality of the markers photographed simultaneously;
A vehicle position estimation unit that estimates a position of the vehicle body with respect to each of the plurality of markers based on at least one of the feature points of the plurality of recognized markers;
A vehicle posture estimation unit that estimates a posture of the vehicle body with respect to each of the plurality of markers based on at least one feature point of the plurality of recognized markers;
A deviation travel suppression unit that performs an operation for suppressing the vehicle body from traveling beyond the marker in a row based on both the estimated position and orientation of the vehicle body,
The vehicle position estimating section, by estimating the position of the vehicle body for each of the plurality of the markers to calculate a distance to each of the plurality of the markers from the vehicle body,
The vehicle posture estimation unit, by estimating the posture of the vehicle body for each of the plurality of the marker, the angle between the running direction and the direction in which each position of the plurality of the markers from the vehicle body of the vehicle body To calculate
The deviation travel suppression unit is configured such that when the distance from the vehicle main body to a specific marker that is any one of the plurality of markers is less than a predetermined distance, and the travel direction of the vehicle main body and the specific marker from the vehicle main body When the angle formed with the direction in which the vehicle is located is less than a predetermined value, the vehicle body that travels in the travel-permitted region performs an operation for suppressing travel beyond the marker in a row. forklift. When the recognition accuracy of the one or more markers by the marker recognizing unit is less than a predetermined value, the vehicle position estimation unit has a recognition accuracy of a predetermined value or more based on the one or more other markers . The forklift according to claim 1, wherein the position of the vehicle body relative to each is estimated. When the recognition accuracy of the one or more markers by the marker recognizing unit is less than a predetermined value, the vehicle posture estimation unit has a recognition accuracy of a predetermined value or more based on the one or more other markers . The forklift according to claim 1 or 2, wherein a posture of the vehicle main body with respect to each is estimated. The said departure driving | running | working suppression part performs the alerting | reporting operation | movement which emits a sound or light as operation | movement for suppressing that the said vehicle main body drive | works exceeding the said marker which forms a row | line | column. The forklift as described in any one. The deviation driving suppression unit controls the traveling of the vehicle body as an operation for suppressing the vehicle body from traveling beyond the markers in a row. The forklift as described in any one.

Images