WO2020039817A1 - Loading operation assistance device for forklift - Google Patents

Loading operation assistance device for forklift Download PDF

Info

Publication number
WO2020039817A1
WO2020039817A1 PCT/JP2019/028728 JP2019028728W WO2020039817A1 WO 2020039817 A1 WO2020039817 A1 WO 2020039817A1 JP 2019028728 W JP2019028728 W JP 2019028728W WO 2020039817 A1 WO2020039817 A1 WO 2020039817A1
Authority
WO
WIPO (PCT)
Prior art keywords
forklift
unit
cargo handling
pallet
fork
Prior art date
Application number
PCT/JP2019/028728
Other languages
French (fr)
Japanese (ja)
Inventor
岡本浩伸
比嘉孝治
Original Assignee
株式会社豊田自動織機
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 株式会社豊田自動織機 filed Critical 株式会社豊田自動織機
Publication of WO2020039817A1 publication Critical patent/WO2020039817A1/en

Links

Images

Classifications

    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B66—HOISTING; LIFTING; HAULING
    • B66F—HOISTING, LIFTING, HAULING OR PUSHING, NOT OTHERWISE PROVIDED FOR, e.g. DEVICES WHICH APPLY A LIFTING OR PUSHING FORCE DIRECTLY TO THE SURFACE OF A LOAD
    • B66F9/00—Devices for lifting or lowering bulky or heavy goods for loading or unloading purposes
    • B66F9/06—Devices for lifting or lowering bulky or heavy goods for loading or unloading purposes movable, with their loads, on wheels or the like, e.g. fork-lift trucks
    • B66F9/075—Constructional features or details
    • B66F9/20—Means for actuating or controlling masts, platforms, or forks
    • B66F9/24—Electrical devices or systems

Definitions

  • the present invention relates to a cargo handling support device for a forklift.
  • the periphery of the traveling vehicle body at the target position is photographed, the position of the object on the image data is predicted, and a search range of a predetermined range including the predicted position of the object is determined.
  • Feature points of the image data calculate the feature amounts of the extracted matching candidate points, and match the feature amounts of the matching candidate points with the model data to recognize the position of the object on the image data.
  • the object recognition device disclosed in Patent Document 2 detects a feature point in a three-dimensional image including an object, calculates a feature amount at each point, and determines a feature point and a specific geometric condition for the feature point.
  • a group of matching candidate points for pattern matching is defined as interpolation points that satisfy the conditions, and object recognition is performed by comparing similarities with model data based on the feature amounts of the matching candidate points included in each group. I have to.
  • the pallet position information is required, so that it can be applied only to the autonomous vehicle.
  • the technique of Patent Document 2 is used to recognize a pallet from a camera image, erroneous detection may occur during two-dimensional image processing.
  • the method of finding a part that seems to be a pallet in a two-dimensional image before recognizing the image in the three-dimensional image and recognizing the part in a three-dimensional image may cause erroneous detection during two-dimensional image processing.
  • An object of the present invention is to provide a forklift cargo handling work support device capable of detecting a pallet with high accuracy.
  • a forklift cargo handling work support device for solving the above-described problems is a forklift cargo handling work support device for supporting a cargo handling work of a forklift.
  • the forklift cargo handling work support device further includes a fork target height calculating unit that calculates a fork target height based on the position of the pallet recognized by the image recognition unit.
  • the cargo handling work support device for a forklift further includes a notifying unit for notifying the fork target height calculated by the fork target height calculating unit.
  • the cargo handling work support device for a forklift further includes an automatic control unit that automatically controls the forklift based on the position of the pallet recognized by the image recognition unit.
  • the forklift cargo handling support device is used for a forklift remote operation system, wherein the forklift remote operation system includes the forklift and a remote operation device, and the forklift is mounted on a machine base.
  • the remote control device includes a cargo handling device and has a vehicle communication unit, and the remote control device has an operation device communication unit that performs wireless communication with the vehicle communication unit, and remotely controls traveling of the forklift and cargo handling by the cargo handling device. It is preferred to be used.
  • a pallet can be detected with high accuracy.
  • FIG. 2 is a block diagram showing an electric configuration of a forklift remote control system.
  • FIG. 2 is a block diagram showing an electrical configuration of a part of the remote control device.
  • action The figure which shows the display screen for demonstrating an effect
  • FIG. 3 is a schematic plan view for explaining a positional relationship between a forklift and a pallet at a work place.
  • FIG. 3 is a schematic side view for explaining a positional relationship between a forklift and a pallet at a work place.
  • the forklift operation support device is used for a forklift remote operation system.
  • the forklift remote control system 10 includes a forklift 20 and a remote control device 40 used to remotely control the traveling of the forklift 20 and the loading and unloading work by the loading and unloading device.
  • the forklift 20 is arranged in a work place.
  • the remote control device 40 is arranged in an operation room. Then, the forklift 20 at the work place can be remotely controlled from the operation room using the remote control device 40.
  • shelves 50 are installed in the workplace.
  • the pallets 51 to 58 and the load W can be arranged in two stages. That is, the load W is arranged on the pallet 51 in the lower stage, and the load W is arranged on the pallet 52 in the upper stage.
  • the load W is placed on the pallet 53 in the lower stage next to it, and the load W is placed on the pallet 54 in the upper stage.
  • the load W is placed on the pallet 55 in the lower stage next to it, and the load W is placed on the pallet 56 in the upper stage.
  • the load W is placed on the pallet 57 in the lower stage next to it, and the load W is placed on the pallet 58 in the upper stage. From this state, the operator remotely operates the forklift 20 to pick up luggage.
  • a pallet hole (fork insertion hole) 60 is formed in each of the pallets 51 to 58, and a fork is inserted into the pallet hole 60.
  • the forklift 20 is provided with wheels 22 before and after a machine base 21.
  • the machine base 21 is provided with a cargo handling device 23.
  • the cargo handling device 23 can stack or unload cargo.
  • the cargo handling device 23 includes a mast 24, a bracket 25, and a fork 26.
  • a fork 26 is provided on the mast 24 via a bracket 25 so as to be vertically movable.
  • the forklift 20 of the present embodiment is configured so that a driver can sit and operate it, an unmanned forklift having no driver's seat may be used.
  • the forklift 20 may be, for example, an engine type equipped with an engine, an EV type equipped with a power storage device and an electric motor, or an FCV type equipped with a fuel cell and an electric motor. It may be.
  • the forklift 20 may be, for example, an HV type having an engine, a power storage device, and an electric motor.
  • the forklift 20 includes a wireless unit 30 as a vehicle communication unit, a control unit 31, a video signal processing unit 32, a camera 33, and an actuator unit.
  • the camera 33 is mounted on the forklift 20 and captures an image of the front of the forklift 20. Specifically, as shown in FIG. 2, the camera 33 is embedded at the tip of the fork 26 and captures an image of the front of the fork 26. Note that the camera 33 is attached to one of the left and right forks 26.
  • 3 includes a traveling actuator and a cargo handling actuator, and the cargo handling actuator includes a tilt actuator and a lifting / lowering actuator.
  • the wheels 22 are rotated and steered by the traveling actuator.
  • the remote operation device 40 includes a wireless unit 41 as an operation device communication unit, a control unit 42, an operation unit 43, a display unit (monitor) 44, and a video signal processing unit 45.
  • a wireless unit 41 as an operation device communication unit
  • a control unit 42 controls the operation unit 43
  • an operation unit 43 controls the operation unit 43
  • a video signal processing unit 45 As an operation method in the operation unit 43, a touch panel method, a mouse method, a joystick method, or the like is used.
  • the wireless unit 41 can perform wireless communication with the wireless unit 30 of the forklift 20. That is, the wireless unit 30 of the forklift 20 and the wireless unit 41 of the remote control device 40 can wirelessly communicate with each other.
  • the control unit 42 sends the operation content to the forklift 20 via the wireless unit 41.
  • the operation content from the remote control device 40 is received by the wireless unit 30 and the control unit 31 drives the actuator unit 34 to perform a desired operation.
  • an image captured by the camera 33 is sent by the control unit 31 to the remote control device 40 via the video signal processing unit 32 and the wireless unit 30.
  • a camera image from the forklift 20 is received by the wireless unit 41 and displayed on the display unit 44 via the video signal processing unit 45 by the control unit 42. That is, the display unit 44 provided in the remote control device 40 is for displaying an image captured by the camera 33, and has a screen 44a (see FIG. 5). The operator operates while viewing the image captured by the camera 33 on the display unit 44.
  • the forklift remote control system 10 has a function of automatically controlling the forklift 20.
  • a loading operation can be performed using this automatic control function. Specifically, as shown in FIG. 7, the forklift 20 is automatically adjusted in position and the fork 26 is inserted into the pallet hole 60 from a state in which the pallets 51 to 58 selected by the operator are directly opposed as shown in FIG. And the fork 26 inserted into the pallet hole 60 can be lifted by automatic control.
  • the video signal processing unit 45 can detect a pallet selected from a plurality of pallets 51 to 58 by performing image recognition processing on an image captured by the camera 33.
  • the video signal processing unit 45 of the remote control device 40 has an image recognition unit 45a and a fork target height calculation unit 45b. Further, the control unit 42 of the remote operation device 40 has an automatic control unit 42a.
  • the operation unit 43 has a detection area designation unit 43a.
  • the detection area specifying unit 43a includes a mouse, a keyboard, and the like.
  • the detection area specifying unit 43a is for specifying an area Rd (see FIG. 6) in which the pallets 51 to 58 are to be recognized in the entire image captured by the camera 33 that captures the front of the forklift 20.
  • the region designation information is sent from the detection region designation unit 43a to the video signal processing unit 45.
  • the image recognizing unit 45a can recognize the positions of the pallets 51 to 58 in the region Rd specified by the detection region specifying unit 43a.
  • the fork target height calculator 45b calculates the fork target height Hg (see FIG. 8) based on the positions of the pallets 51 to 58 recognized by the image recognizer 45a.
  • the automatic control unit 42a automatically controls the forklift 20 based on the positions of the pallets 51 to 58 recognized by the image recognition unit 45a.
  • the video signal processing unit 45 receives image data and sends the image data to the display unit 44.
  • the recognition result (relative height information) is sent to the control unit 42.
  • the automatic control unit 42a instructs the forklift operation amount to the forklift side in response to an instruction to start fork height automatic adjustment from the operation unit 43.
  • the forklift 20 is located at a position distant from the shelf 50 (pallets 51 to 58). From this state, the forklift 20 travels so as to approach the shelf 50 (pallets 51 to 58). Further, the forklift 20 is directly opposed to the pallets 51 to 58 as shown in FIG. In this state, the left and right forks 26 of the forklift 20 are separated from the left and right pallet holes 60 of the pallets 51 to 58 by a predetermined distance. By moving the left and right forks 26 forward, the left and right forks 26 are inserted into the left and right pallet holes 60.
  • an image captured by the camera 33 is displayed on the display unit 44.
  • an image captured by the camera 33 is displayed as shown in FIG.
  • the operator specifies an area (Rd) including the specific pallet 52 in the image captured by the camera 33 using a cursor or the like.
  • a rectangular area Rd is designated by four designated points P1, P2, P3, and P4, and the designated rectangular area Rd includes the pallet 52.
  • the region Rd in which the pallets 51 to 58 are to be recognized in the entire image captured by the camera 33 is specified.
  • the image recognition unit 45a performs image recognition of the position of the pallet 52 in the specified area Rd. Therefore, the pallet 52 can be detected with high accuracy.
  • pallet recognition using a laser requires laser-related parts, which affects the cost.
  • the pallets 51 to 58 can be recognized from the camera image, there is a cost advantage compared to the case where a laser is used. It is difficult to adjust the height of the pallet hole 60 and the height of the fork 26 by adjusting the height of the fork 26 based on the camera image, which may burden the operator. Therefore, the pallets 51 to 58 are detected from the image information, and the fork height adjustment is supported by using the recognized position information of the pallets 51 to 58.
  • the shapes of the pallets 51 to 58 are formed by straight lines, the amount of characteristics is small in terms of shape, it is difficult to detect the pallets from the entire image, and it is difficult to make a difference in the amount of characteristics with racks and columns in the environment. . That is, the pallets 51 to 58 are formed of straight lines, have few edges, and have a small amount of features. Therefore, as shown by the area P100 in FIG. 2, it is difficult to recognize the division between the pallets, and it is difficult to make a difference in the feature amount from other areas.
  • the detection accuracy of the pallet 52 is improved by specifying a rough detection area (Rd) including the pallet 52 as a target on the image.
  • a region (Rd) from which features are extracted comparison targets are narrowed down.
  • the false recognition rate decreases.
  • the mounting position of the camera 33 is a camera position where the positional relationship between the fork 26 and the pallets 51 to 58 can be understood. Specifically, it is the tip of the fork, which operates in conjunction with the fork 26, and sets the camera position where the fork 26 and the pallets 51 to 58 are simultaneously projected.
  • the area specification (Rd) is specified by the operator on the image on the image display monitor, and the specification is performed using an input device such as a mouse or a keyboard. Judge the specified range (Rd) from the input information.
  • the image recognition unit 45a performs pallet detection by focusing on a specified region Rd (it is also possible in the vicinity).
  • the image recognition unit 45a calculates a distance dp (see FIG. 8) from the detected size of the pallet 52 to the pallet 52.
  • the image recognition unit 45a grasps the relative height ⁇ H (see FIG. 8) between the fork 26 and the pallet 52 based on the calculated distance information (dp).
  • the size of the pallets 51 to 58 is defined for the method of grasping the relative height ⁇ H
  • the camera 33 is located directly opposite to the pallets 51 to 58 as shown in FIG.
  • the approximate distance is calculated from the pallet size (length of side, area, etc.). If the distance dp between the pallets 51 to 58 is known, the installation position of the camera 33 is known in advance (since it is the fork tip in this embodiment), so the relative height between the camera 33 and the pallet can be known.
  • the pallet height (Hg) is known from the position. Thus, the relative height ⁇ H between the fork 26 and the pallets 51 to 58 can be determined.
  • the image recognition unit 45a notifies the automatic control unit 42a of information on the relative height ⁇ H between the fork 26 and the pallets 51 to 58.
  • the automatic control unit 42a determines the lift operation amount from the relative height information and instructs the forklift 20 to be controlled. Thereby, the height of the fork 26 is adjusted to the same height as the pallet.
  • a fork target height calculator 45b for calculating the fork target height Hg based on the positions of the pallets 51 to 58 recognized by the image recognizer 45a is further provided. Therefore, the fork target height can be obtained.
  • An automatic control unit 42a for automatically controlling the forklift 20 based on the position of the pallet 52 recognized by the image recognition unit 45a is further provided. Therefore, the forklift 20 can be automatically controlled based on the recognized position of the pallet 52.
  • the forklift cargo handling work support device is used for the forklift remote operation system 10, and the forklift remote operation system 10 includes the forklift 20 and the remote operation device 40.
  • the platform 21 includes the cargo handling device 23 and a wireless unit 30 as a vehicle communication unit.
  • the remote operation device 40 includes a wireless unit 41 as an operation device communication unit that performs wireless communication with the wireless unit 30 as a vehicle communication unit. It is used to remotely control the traveling of the forklift 20 and the cargo handling by the cargo handling device 23. Therefore, when performing remote control, the pallet 52 can be detected with high accuracy.
  • the embodiment is not limited to the above, and may be embodied as follows, for example. ⁇ If the image display screen has a touch panel function, the area may be specified using the touch panel.
  • ⁇ O ⁇ Operation support may be display of the relative height ⁇ H on the image instead of automatic adjustment. That is, when displaying the lift height which is the height of the fork on the display unit 44 as the notification unit, the relative height ⁇ H is displayed, and the operator adjusts the height of the fork by the relative height ⁇ H. You may.
  • the fork target height Hg is notified by displaying the fork target height Hg calculated by the fork target height calculator 45b. That is, the fork target height Hg may be displayed as it is, or the relative height ⁇ H that is a difference from the current lift height may be displayed.
  • the display unit 44 may be further provided as a notification unit for notifying the fork target height Hg calculated by the fork target height calculation unit 45b.
  • the horizontal direction may be adjusted. That is, as shown in FIG. 7, when the place to be directly opposed to the pallet is shifted in the horizontal direction, the guide may be guided in the horizontal direction.
  • the image recognition unit 45a and the automatic control unit 42a may be integrated.
  • the process performed by the control unit 42 of the remote control device 40 may be performed by the control unit 31 of the forklift 20.
  • the camera is provided at the tip of the fork, but is not limited to this. For example, it may be provided on the bracket 25. In short, the camera only needs to capture an image in front of the forklift.
  • the forklift operation support device is used in a forklift remote operation system, but is not limited to this.
  • it may be used for a manned forklift. That is, the present invention may be applied to, for example, a manned forklift equipped with a camera and a display unit, instead of including an unmanned forklift equipped with a camera and a remote control device having a display unit.

Landscapes

  • Engineering & Computer Science (AREA)
  • Transportation (AREA)
  • Structural Engineering (AREA)
  • Chemical & Material Sciences (AREA)
  • Combustion & Propulsion (AREA)
  • Civil Engineering (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Geology (AREA)
  • Mechanical Engineering (AREA)
  • Forklifts And Lifting Vehicles (AREA)

Abstract

The present invention comprises: a camera (33) mounted on a forklift, the camera capturing an image forward of the forklift; a sensing region designation unit (43a) for designating a region in which a pallet is to be recognized from within the entirety of the image captured by the camera; and an image recognition unit (45a) that conducts image recognition at the position of the pallet in the designated region.

Description

フォークリフト用荷役作業支援装置Cargo handling support equipment for forklifts
　本発明は、フォークリフト用荷役作業支援装置に関するものである。 The present invention relates to a cargo handling support device for a forklift.
　特許文献１に開示の自律走行車においては、目的位置における走行車本体の周辺を撮影して画像データ上における物体の位置を予想し、予想された物体の位置を含む所定範囲の探索領域に対して画像データの特徴点を検出し、抽出されたマッチング候補点の特徴量を算出してマッチング候補点の特徴量とモデルデータとを照合することで物体の画像データ上の位置を認識するようにしている。特許文献２に開示の物体認識装置においては、物体が含まれる三次元画像における特徴点を検出して各点における特徴量を算出し、特徴点と、特徴点に対して特定の幾何的条件を満たす補間点とをパターンマッチングのためのマッチング候補点のグループとし、各グループに含まれるマッチング候補点の特徴量に基づいて、モデルのデータとの類似度を比較することで物体の認識を行うようにしている。 In the autonomous traveling vehicle disclosed in Patent Document 1, the periphery of the traveling vehicle body at the target position is photographed, the position of the object on the image data is predicted, and a search range of a predetermined range including the predicted position of the object is determined. Feature points of the image data, calculate the feature amounts of the extracted matching candidate points, and match the feature amounts of the matching candidate points with the model data to recognize the position of the object on the image data. ing. The object recognition device disclosed in Patent Document 2 detects a feature point in a three-dimensional image including an object, calculates a feature amount at each point, and determines a feature point and a specific geometric condition for the feature point. A group of matching candidate points for pattern matching is defined as interpolation points that satisfy the conditions, and object recognition is performed by comparing similarities with model data based on the feature amounts of the matching candidate points included in each group. I have to.
特開２０１５－２２５４５０号公報JP 2015-225450 A 特開２０１５－２２５４５３号公報JP-A-2005-225453
　ところで、フォークリフトの荷役作業を支援するフォークリフト用荷役作業支援装置において、カメラ画像からパレットを認識するために、特許文献１の技術を用いた場合、パレット位置情報が必要なので自律走行車にしか適用できない。つまり、画像認識領域を決める時に自律走行なので予めパレットのある位置が分かり、そこだけに絞って画像認識することにより負荷の軽減を図る手法はパレット位置情報が必要なので自律走行車にしか適用できない。また、カメラ画像からパレットを認識するために、特許文献２の技術を用いた場合、二次元画像処理時に誤検知することが懸念される。つまり、三次元画像で画像認識する前に二次元画像でパレットらしきものの箇所を見つけておいてそこを三次元で画像認識する手法は二次元画像処理時に誤検知する可能性がある。 By the way, in the forklift cargo handling work support device that supports the cargo handling work of the forklift, when the technology of Patent Document 1 is used to recognize the pallet from the camera image, the pallet position information is required, so that it can be applied only to the autonomous vehicle. . In other words, since the autonomous traveling is performed when the image recognition area is determined, the position of the pallet can be known in advance, and the method of reducing the load by recognizing the image by focusing only on the pallet can be applied only to the autonomous vehicle because the pallet position information is required. Further, when the technique of Patent Document 2 is used to recognize a pallet from a camera image, erroneous detection may occur during two-dimensional image processing. In other words, the method of finding a part that seems to be a pallet in a two-dimensional image before recognizing the image in the three-dimensional image and recognizing the part in a three-dimensional image may cause erroneous detection during two-dimensional image processing.
　本発明の目的は、パレットを高精度に検知することができるフォークリフト用荷役作業支援装置を提供することにある。 An object of the present invention is to provide a forklift cargo handling work support device capable of detecting a pallet with high accuracy.
　上記問題点を解決するためのフォークリフト用荷役作業支援装置は、フォークリフトの荷役作業を支援するフォークリフト用荷役作業支援装置であって、前記フォークリフトに搭載され、前記フォークリフトの前方を撮像するカメラと、前記カメラにて撮像された画像全体のうちパレットを認識させたい領域を指定するための検知領域指定部と、前記検知領域指定部で指定された領域において前記パレットの位置を画像認識する画像認識部と、を備えることを要旨とする。 A forklift cargo handling work support device for solving the above-described problems is a forklift cargo handling work support device for supporting a cargo handling work of a forklift. A detection area designating unit for designating an area in which the pallet is to be recognized in the entire image captured by the camera, and an image recognition unit that recognizes the position of the pallet in the area designated by the detection area designating unit. , Is provided.
　これによれば、カメラにて撮像された画像全体のうちパレットを認識させたい領域が指定され、指定された領域においてパレットの位置が画像認識される。よって、パレットを高精度に検知することができる。 According to this, an area where the pallet is to be recognized in the entire image captured by the camera is specified, and the position of the pallet is recognized in the specified area. Therefore, the pallet can be detected with high accuracy.
　また、フォークリフト用荷役作業支援装置について、前記画像認識部で認識された前記パレットの位置に基づきフォーク目標高さを演算するフォーク目標高さ演算部を更に備えるのが好ましい。 It is preferable that the forklift cargo handling work support device further includes a fork target height calculating unit that calculates a fork target height based on the position of the pallet recognized by the image recognition unit.
　また、フォークリフト用荷役作業支援装置について、前記フォーク目標高さ演算部で演算されたフォーク目標高さを通知する通知部を更に備えるのが好ましい。
　また、フォークリフト用荷役作業支援装置について、前記画像認識部で認識された前記パレットの位置に基づき前記フォークリフトを自動制御する自動制御部を更に備えるのが好ましい。 Preferably, the cargo handling work support device for a forklift further includes a notifying unit for notifying the fork target height calculated by the fork target height calculating unit.
Preferably, the cargo handling work support device for a forklift further includes an automatic control unit that automatically controls the forklift based on the position of the pallet recognized by the image recognition unit.
　また、前記フォークリフト用荷役作業支援装置は、フォークリフト用遠隔操作システムに用いられるものであって、前記フォークリフト用遠隔操作システムは、前記フォークリフトと、遠隔操作装置とを備え、前記フォークリフトは、機台に荷役装置を備えるとともに車両通信部を有し、前記遠隔操作装置は、前記車両通信部と無線通信を行う操作装置通信部を有し、前記フォークリフトの走行及び前記荷役装置による荷役を遠隔操作するのに用いられるのが好ましい。 Further, the forklift cargo handling support device is used for a forklift remote operation system, wherein the forklift remote operation system includes the forklift and a remote operation device, and the forklift is mounted on a machine base. The remote control device includes a cargo handling device and has a vehicle communication unit, and the remote control device has an operation device communication unit that performs wireless communication with the vehicle communication unit, and remotely controls traveling of the forklift and cargo handling by the cargo handling device. It is preferred to be used.
　本発明によれば、パレットを高精度に検知することができる。 According to the present invention, a pallet can be detected with high accuracy.
フォークリフト用遠隔操作システムの概要を示す概略図。The schematic diagram showing the outline of the remote control system for forklifts. 作業場でのフォークリフトとパレットの状況を説明するための概略側面図。The schematic side view for explaining the situation of a forklift and a pallet in a work place. フォークリフト用遠隔操作システムの電気的構成を示すブロック図。FIG. 2 is a block diagram showing an electric configuration of a forklift remote control system. 遠隔操作装置の一部の電気的構成を示すブロック図。FIG. 2 is a block diagram showing an electrical configuration of a part of the remote control device. 作用を説明するための表示画面を示す図。The figure which shows the display screen for demonstrating an effect | action. 作用を説明するための表示画面を示す図。The figure which shows the display screen for demonstrating an effect | action. 作業場でのフォークリフトとパレットとの位置関係を説明するための概略平面図。FIG. 3 is a schematic plan view for explaining a positional relationship between a forklift and a pallet at a work place. 作業場でのフォークリフトとパレットとの位置関係を説明するための概略側面図。FIG. 3 is a schematic side view for explaining a positional relationship between a forklift and a pallet at a work place.
　以下、本発明を具体化した一実施形態を図面に従って説明する。
　本実施形態では、フォークリフト用操作支援装置は、フォークリフト用遠隔操作システムに用いられるものである。 An embodiment of the present invention will be described below with reference to the drawings.
In this embodiment, the forklift operation support device is used for a forklift remote operation system.
　図１に示すように、フォークリフト用遠隔操作システム１０は、フォークリフト２０と、フォークリフト２０の走行及び荷役装置による荷役作業を遠隔操作するのに用いられる遠隔操作装置４０と、を備えている。フォークリフト２０は、作業場に配置される。遠隔操作装置４０は、操作室に配置される。そして、遠隔操作装置４０を用いて操作室から、作業場のフォークリフト２０を遠隔操作することができるようになっている。 As shown in FIG. 1, the forklift remote control system 10 includes a forklift 20 and a remote control device 40 used to remotely control the traveling of the forklift 20 and the loading and unloading work by the loading and unloading device. The forklift 20 is arranged in a work place. The remote control device 40 is arranged in an operation room. Then, the forklift 20 at the work place can be remotely controlled from the operation room using the remote control device 40.
　図２に示すように、作業場においては、棚５０が設置されている。棚５０は２段にわたりパレット５１～５８及び荷物Ｗを配置することができる。つまり、下の段においてパレット５１に荷物Ｗが配置されるとともに、その上の段においてパレット５２に荷物Ｗが配置される。その隣での下の段においてパレット５３に荷物Ｗが配置されるとともに、その上の段においてパレット５４に荷物Ｗが配置される。その隣での下の段においてパレット５５に荷物Ｗが配置されるとともに、その上の段においてパレット５６に荷物Ｗが配置される。その隣での下の段においてパレット５７に荷物Ｗが配置されるとともに、その上の段においてパレット５８に荷物Ｗが配置される。この状態から、作業者はフォークリフト２０を遠隔操作して荷物を取りに行く。 棚 As shown in FIG. 2, shelves 50 are installed in the workplace. On the shelf 50, the pallets 51 to 58 and the load W can be arranged in two stages. That is, the load W is arranged on the pallet 51 in the lower stage, and the load W is arranged on the pallet 52 in the upper stage. The load W is placed on the pallet 53 in the lower stage next to it, and the load W is placed on the pallet 54 in the upper stage. The load W is placed on the pallet 55 in the lower stage next to it, and the load W is placed on the pallet 56 in the upper stage. The load W is placed on the pallet 57 in the lower stage next to it, and the load W is placed on the pallet 58 in the upper stage. From this state, the operator remotely operates the forklift 20 to pick up luggage.
　図２において、各パレット５１～５８にはパレット穴（フォーク挿入用穴）６０が形成されており、このパレット穴６０にフォークが差し込まれる。
　図１に示すように、フォークリフト２０は、機台２１の前後に車輪２２が設けられている。機台２１には荷役装置２３が備えられている。荷役装置２３により荷物の積み上げ又は積み降ろしを行うことができる。 In FIG. 2, a pallet hole (fork insertion hole) 60 is formed in each of the pallets 51 to 58, and a fork is inserted into the pallet hole 60.
As shown in FIG. 1, the forklift 20 is provided with wheels 22 before and after a machine base 21. The machine base 21 is provided with a cargo handling device 23. The cargo handling device 23 can stack or unload cargo.
　荷役装置２３は、マスト２４と、ブラケット２５と、フォーク２６を具備している。マスト２４にブラケット２５を介してフォーク２６が上下方向に移動可能に設けられている。本実施形態のフォークリフト２０は、運転者が着座して操作することが可能に構成されているが、運転席の無い無人フォークリフトであってもよい。 The cargo handling device 23 includes a mast 24, a bracket 25, and a fork 26. A fork 26 is provided on the mast 24 via a bracket 25 so as to be vertically movable. Although the forklift 20 of the present embodiment is configured so that a driver can sit and operate it, an unmanned forklift having no driver's seat may be used.
　なお、フォークリフト２０は、例えばエンジンが搭載されたエンジンタイプであってもよいし、蓄電装置及び電動モータが搭載されたＥＶタイプであってもよいし、燃料電池及び電動モータが搭載されたＦＣＶタイプであってもよい。また、フォークリフト２０は、例えばエンジンと蓄電装置と電動モータとを有するＨＶタイプでもよい。 The forklift 20 may be, for example, an engine type equipped with an engine, an EV type equipped with a power storage device and an electric motor, or an FCV type equipped with a fuel cell and an electric motor. It may be. The forklift 20 may be, for example, an HV type having an engine, a power storage device, and an electric motor.
　図３に示すように、フォークリフト２０は、車両通信部としての無線部３０と、制御部３１と、映像信号処理部３２と、カメラ３３と、アクチュエータ部３４を有する。カメラ３３は、フォークリフト２０に搭載され、フォークリフト２０の前方を撮像するためのものである。具体的には、カメラ３３は、図２に示すように、フォーク２６の先端部に埋め込まれており、フォーク２６の前方を撮像する。なお、カメラ３３は左右のフォーク２６のうちのいずれかに取り付けられている。図３のアクチュエータ部３４は、走行用アクチュエータと、荷役用アクチュエータを含み、荷役用アクチュエータはティルト用アクチュエータと昇降用アクチュエータを含んでいる。走行用アクチュエータにより車輪２２が回転及び操舵される。 フ ォ ー ク As shown in FIG. 3, the forklift 20 includes a wireless unit 30 as a vehicle communication unit, a control unit 31, a video signal processing unit 32, a camera 33, and an actuator unit. The camera 33 is mounted on the forklift 20 and captures an image of the front of the forklift 20. Specifically, as shown in FIG. 2, the camera 33 is embedded at the tip of the fork 26 and captures an image of the front of the fork 26. Note that the camera 33 is attached to one of the left and right forks 26. 3 includes a traveling actuator and a cargo handling actuator, and the cargo handling actuator includes a tilt actuator and a lifting / lowering actuator. The wheels 22 are rotated and steered by the traveling actuator.
　遠隔操作装置４０は、操作装置通信部としての無線部４１と、制御部４２と、操作部４３と、表示部（モニタ）４４と、映像信号処理部４５を有する。操作部４３における操作方式として、タッチパネル方式、マウス方式、ジョイスティック方式等が用いられる。 The remote operation device 40 includes a wireless unit 41 as an operation device communication unit, a control unit 42, an operation unit 43, a display unit (monitor) 44, and a video signal processing unit 45. As an operation method in the operation unit 43, a touch panel method, a mouse method, a joystick method, or the like is used.
　無線部４１はフォークリフト２０の無線部３０と無線通信を行うことができる。つまり、フォークリフト２０の無線部３０と遠隔操作装置４０の無線部４１とは無線通信可能となっている。 The wireless unit 41 can perform wireless communication with the wireless unit 30 of the forklift 20. That is, the wireless unit 30 of the forklift 20 and the wireless unit 41 of the remote control device 40 can wirelessly communicate with each other.
　そして、遠隔操作装置４０において、操作部４３を用いて作業者が所望の操作を行うと制御部４２により操作内容が無線部４１を介してフォークリフト２０側に送られる。フォークリフト２０において、無線部３０で遠隔操作装置４０からの操作内容が受信され、制御部３１によりアクチュエータ部３４が駆動されて所望の動作が実行される。 Then, when the operator performs a desired operation using the operation unit 43 on the remote control device 40, the control unit 42 sends the operation content to the forklift 20 via the wireless unit 41. In the forklift 20, the operation content from the remote control device 40 is received by the wireless unit 30 and the control unit 31 drives the actuator unit 34 to perform a desired operation.
　一方、フォークリフト２０において、カメラ３３により撮像された画像は制御部３１により映像信号処理部３２及び無線部３０を介して遠隔操作装置４０側に送られる。遠隔操作装置４０において、無線部４１でフォークリフト２０からのカメラ画像が受信されて制御部４２により映像信号処理部４５を介して表示部４４で表示される。つまり、遠隔操作装置４０に設けられる表示部４４は、カメラ３３にて撮像された画像を表示するためのものであり、画面４４ａ（図５参照）を有する。作業者はカメラ３３にて撮像された画像を表示部４４で見ながら操作することになる。 On the other hand, in the forklift 20, an image captured by the camera 33 is sent by the control unit 31 to the remote control device 40 via the video signal processing unit 32 and the wireless unit 30. In the remote control device 40, a camera image from the forklift 20 is received by the wireless unit 41 and displayed on the display unit 44 via the video signal processing unit 45 by the control unit 42. That is, the display unit 44 provided in the remote control device 40 is for displaying an image captured by the camera 33, and has a screen 44a (see FIG. 5). The operator operates while viewing the image captured by the camera 33 on the display unit 44.
　本フォークリフト用遠隔操作システム１０は、フォークリフト２０を自動制御する機能を有している。この自動制御機能を用いて荷取り作業を行うことができるようになっている。詳しくは、作業者により選択されたパレット５１～５８に対し図７に示すように正対した状態から図８に示すように自動制御でフォークリフト２０を位置調整してパレット穴６０にフォーク２６を挿入したり、パレット穴６０に挿入したフォーク２６を自動制御でリフトすることができるようになっている。 遠隔 The forklift remote control system 10 has a function of automatically controlling the forklift 20. A loading operation can be performed using this automatic control function. Specifically, as shown in FIG. 7, the forklift 20 is automatically adjusted in position and the fork 26 is inserted into the pallet hole 60 from a state in which the pallets 51 to 58 selected by the operator are directly opposed as shown in FIG. And the fork 26 inserted into the pallet hole 60 can be lifted by automatic control.
　図３において映像信号処理部４５は、カメラ３３にて撮像された画像に対し画像認識処理により複数のパレット５１～５８のうち選択されたパレットを検出することができるようになっている。 (3) In FIG. 3, the video signal processing unit 45 can detect a pallet selected from a plurality of pallets 51 to 58 by performing image recognition processing on an image captured by the camera 33.
　図４に示すように、遠隔操作装置４０の映像信号処理部４５は、画像認識部４５ａとフォーク目標高さ演算部４５ｂを有する。また、遠隔操作装置４０の制御部４２は自動制御部４２ａを有する。 As shown in FIG. 4, the video signal processing unit 45 of the remote control device 40 has an image recognition unit 45a and a fork target height calculation unit 45b. Further, the control unit 42 of the remote operation device 40 has an automatic control unit 42a.
　操作部４３は、検知領域指定部４３ａを有する。検知領域指定部４３ａは、マウス、キーボード等を備えている。検知領域指定部４３ａは、フォークリフト２０の前方を撮像するカメラ３３にて撮像された画像全体のうちパレット５１～５８を認識させたい領域Ｒｄ（図６参照）を指定するためのものである。検知領域指定部４３ａから映像信号処理部４５に領域指定情報が送られる。 The operation unit 43 has a detection area designation unit 43a. The detection area specifying unit 43a includes a mouse, a keyboard, and the like. The detection area specifying unit 43a is for specifying an area Rd (see FIG. 6) in which the pallets 51 to 58 are to be recognized in the entire image captured by the camera 33 that captures the front of the forklift 20. The region designation information is sent from the detection region designation unit 43a to the video signal processing unit 45.
　画像認識部４５ａは、検知領域指定部４３ａで指定された領域Ｒｄにおいてパレット５１～５８の位置を画像認識することができる。フォーク目標高さ演算部４５ｂは、画像認識部４５ａで認識されたパレット５１～５８の位置に基づきフォーク目標高さＨｇ（図８参照）を演算する。自動制御部４２ａは、画像認識部４５ａで認識されたパレット５１～５８の位置に基づきフォークリフト２０を自動制御する。 (4) The image recognizing unit 45a can recognize the positions of the pallets 51 to 58 in the region Rd specified by the detection region specifying unit 43a. The fork target height calculator 45b calculates the fork target height Hg (see FIG. 8) based on the positions of the pallets 51 to 58 recognized by the image recognizer 45a. The automatic control unit 42a automatically controls the forklift 20 based on the positions of the pallets 51 to 58 recognized by the image recognition unit 45a.
　図４の構成について、詳しくは、映像信号処理部４５は画像データを入力するとともに画像データが表示部４４に送られる。認識結果（相対高さ情報）が制御部４２に送られる。操作部４３からのフォーク高さ自動調整の開始指示により自動制御部４２ａからフォークリフト側にフォークリフト操作量が指示される。 4 More specifically, regarding the configuration in FIG. 4, the video signal processing unit 45 receives image data and sends the image data to the display unit 44. The recognition result (relative height information) is sent to the control unit 42. The automatic control unit 42a instructs the forklift operation amount to the forklift side in response to an instruction to start fork height automatic adjustment from the operation unit 43.
　次に、作用について説明する。
　図２に示すように、フォークリフト２０が棚５０（パレット５１～５８）に対し離れた場所に位置している。この状態から、フォークリフト２０が棚５０（パレット５１～５８）に接近するように走行する。さらに、図７に示すようにフォークリフト２０をパレット５１～５８に対し正対させる。この状態では、パレット５１～５８の左右のパレット穴６０に対しフォークリフト２０の左右のフォーク２６が所定距離だけ離間している。左右のフォーク２６を前方に移動させることにより左右のフォーク２６を左右のパレット穴６０に差し込む。 Next, the operation will be described.
As shown in FIG. 2, the forklift 20 is located at a position distant from the shelf 50 (pallets 51 to 58). From this state, the forklift 20 travels so as to approach the shelf 50 (pallets 51 to 58). Further, the forklift 20 is directly opposed to the pallets 51 to 58 as shown in FIG. In this state, the left and right forks 26 of the forklift 20 are separated from the left and right pallet holes 60 of the pallets 51 to 58 by a predetermined distance. By moving the left and right forks 26 forward, the left and right forks 26 are inserted into the left and right pallet holes 60.
　このようにフォークリフト２０を操作する際に以下の操作支援が行われる。
　図５に示すように、表示部４４においてカメラ３３にて撮像された画像が表示される。更にパレット５１～５８に接近すると、図６に示すようにカメラ３３にて撮像された画像が表示される。ここで、作業者は、カメラ３３にて撮像された画像のうち特定のパレット５２を含む領域（Ｒｄ）を、カーソル等を用いて指定する。図６において４つの指定点Ｐ１，Ｐ２，Ｐ３，Ｐ４にて、長方形の領域Ｒｄが指定されており、指定された長方形の領域Ｒｄにパレット５２を含んでいる。 When operating the forklift 20 in this manner, the following operation support is performed.
As shown in FIG. 5, an image captured by the camera 33 is displayed on the display unit 44. When the user further approaches the pallets 51 to 58, an image captured by the camera 33 is displayed as shown in FIG. Here, the operator specifies an area (Rd) including the specific pallet 52 in the image captured by the camera 33 using a cursor or the like. In FIG. 6, a rectangular area Rd is designated by four designated points P1, P2, P3, and P4, and the designated rectangular area Rd includes the pallet 52.
　このようにして、カメラ３３にて撮像された画像全体のうちパレット５１～５８を認識させたい領域Ｒｄが指定される。
　画像認識部４５ａは、指定された領域Ｒｄにおいてパレット５２の位置を画像認識する。よって、パレット５２を高精度に検知することができる。 In this way, the region Rd in which the pallets 51 to 58 are to be recognized in the entire image captured by the camera 33 is specified.
The image recognition unit 45a performs image recognition of the position of the pallet 52 in the specified area Rd. Therefore, the pallet 52 can be detected with high accuracy.
　以下、詳しく説明する。
　従来、レーザーによるパレット認識は、レーザー関連の部品が必要になり、コストに影響がある。遠隔操作システムでは、操作用のカメラがある為、そのカメラ画像からパレット５１～５８を認識できれば、レーザーを用いる場合に比べコスト面のメリットがある。なお、カメラ画像にてフォーク２６の高さを調整してパレット穴６０の高さとフォーク２６の高さを合わせるのは難しく、作業者の負担になる可能性がある。よって、画像情報からパレット５１～５８を検知し、認識したパレット５１～５８の位置情報を用いてフォーク高さ合せを支援する。ただし、パレット５１～５８の形状は、直線で構成されていることにより形状上特徴量が少なく、画像全体からのパレット検知は難しく、環境にあるラックや柱などとの特徴量の差が出にくい。つまり、パレット５１～５８の形状について、直線で構成されており、エッジが少なく、特徴量が少ない。よって、図２において領域Ｐ１００で示すごとくパレット間の区切れが認識しにくく、他の領域との特徴量の差がでにくい。 The details will be described below.
Conventionally, pallet recognition using a laser requires laser-related parts, which affects the cost. In the remote operation system, since there is an operation camera, if the pallets 51 to 58 can be recognized from the camera image, there is a cost advantage compared to the case where a laser is used. It is difficult to adjust the height of the pallet hole 60 and the height of the fork 26 by adjusting the height of the fork 26 based on the camera image, which may burden the operator. Therefore, the pallets 51 to 58 are detected from the image information, and the fork height adjustment is supported by using the recognized position information of the pallets 51 to 58. However, since the shapes of the pallets 51 to 58 are formed by straight lines, the amount of characteristics is small in terms of shape, it is difficult to detect the pallets from the entire image, and it is difficult to make a difference in the amount of characteristics with racks and columns in the environment. . That is, the pallets 51 to 58 are formed of straight lines, have few edges, and have a small amount of features. Therefore, as shown by the area P100 in FIG. 2, it is difficult to recognize the division between the pallets, and it is difficult to make a difference in the feature amount from other areas.
　本実施形態においては、画像上のターゲットとするパレット５２を含む大まかな検知領域（Ｒｄ）を指定することで、パレット５２の検知精度を向上させる。特徴を抽出する領域（Ｒｄ）を指定することで、比較対象が絞られる。その結果、誤認識率が減少する。このように、精度よくパレット５２を検知できることで、精度の高いフォーク高さ合せの支援が可能になる。 In the present embodiment, the detection accuracy of the pallet 52 is improved by specifying a rough detection area (Rd) including the pallet 52 as a target on the image. By specifying a region (Rd) from which features are extracted, comparison targets are narrowed down. As a result, the false recognition rate decreases. As described above, by accurately detecting the pallet 52, it is possible to support the fork height adjustment with high accuracy.
　カメラ３３の取付位置については、フォーク２６とパレット５１～５８の位置関係がわかるカメラ位置とする。具体的には、フォーク先端部でありフォーク２６と連動して動作し、フォーク２６とパレット５１～５８が同時に映るカメラ位置などとする。領域指定（Ｒｄ）の仕方については、領域指定（Ｒｄ）は、操作者が画像表示用モニタの画像上で指定することとし、指定は、マウス、キーボードなどの入力装置を用い、画像認識部４５ａが入力情報から指定範囲（Ｒｄ）を判断する。 取 付 The mounting position of the camera 33 is a camera position where the positional relationship between the fork 26 and the pallets 51 to 58 can be understood. Specifically, it is the tip of the fork, which operates in conjunction with the fork 26, and sets the camera position where the fork 26 and the pallets 51 to 58 are simultaneously projected. As for the method of specifying the area (Rd), the area specification (Rd) is specified by the operator on the image on the image display monitor, and the specification is performed using an input device such as a mouse or a keyboard. Judge the specified range (Rd) from the input information.
　パレット検知については、画像認識部４５ａは、指定された領域Ｒｄ内（付近でも可能）に絞ってパレット検知を行う。ここで、画像認識部４５ａは、検知したパレット５２のサイズからパレット５２までの距離ｄｐ（図８参照）を算出する。そして、画像認識部４５ａは、算出した距離情報（ｄｐ）を基に、フォーク２６とパレット５２の相対高さΔＨ（図８参照）を把握する。 Regarding pallet detection, the image recognition unit 45a performs pallet detection by focusing on a specified region Rd (it is also possible in the vicinity). Here, the image recognition unit 45a calculates a distance dp (see FIG. 8) from the detected size of the pallet 52 to the pallet 52. Then, the image recognition unit 45a grasps the relative height ΔH (see FIG. 8) between the fork 26 and the pallet 52 based on the calculated distance information (dp).
　なお、相対高さΔＨの把握方法について、パレット５１～５８のサイズは規定されているため、図７に示すようにパレット５１～５８に正対する位置にカメラ３３がある場合、認識した画像上のパレットサイズ（辺の長さ、面積等）から概算距離を算出する。パレット５１～５８までの距離ｄｐがわかれば、カメラ３３の設置位置は予め分かっているので（本実施形態ではフォーク先端部であるので）、カメラ３３とパレットの相対高さが分かるのでカメラ３３の位置からパレット高さ（Ｈｇ）が分かる。このように、フォーク２６とパレット５１～５８の相対高さΔＨがわかる。 Since the size of the pallets 51 to 58 is defined for the method of grasping the relative height ΔH, if the camera 33 is located directly opposite to the pallets 51 to 58 as shown in FIG. The approximate distance is calculated from the pallet size (length of side, area, etc.). If the distance dp between the pallets 51 to 58 is known, the installation position of the camera 33 is known in advance (since it is the fork tip in this embodiment), so the relative height between the camera 33 and the pallet can be known. The pallet height (Hg) is known from the position. Thus, the relative height ΔH between the fork 26 and the pallets 51 to 58 can be determined.
　自動制御部４２ａにおけるフォーク高さ自動調整機能について、画像認識部４５ａは、自動制御部４２ａにフォーク２６とパレット５１～５８の相対高さΔＨの情報を通知する。自動制御部４２ａは、相対高さ情報からリフト操作量を決め、制御対象であるフォークリフト２０に指示する。これによりフォーク２６の高さが調整されてパレットと同じ高さにされる。 Regarding the fork height automatic adjustment function in the automatic control unit 42a, the image recognition unit 45a notifies the automatic control unit 42a of information on the relative height ΔH between the fork 26 and the pallets 51 to 58. The automatic control unit 42a determines the lift operation amount from the relative height information and instructs the forklift 20 to be controlled. Thereby, the height of the fork 26 is adjusted to the same height as the pallet.
　以上のごとく、画像認識によるパレット検知精度の向上が図られる。
　上記実施形態によれば、以下のような効果を得ることができる。
　（１）フォークリフト２０の荷役作業を支援するフォークリフト用荷役作業支援装置の構成として、フォークリフト２０に搭載され、フォークリフト２０の前方を撮像するカメラ３３と、カメラ３３にて撮像された画像全体のうちパレット５１～５８を認識させたい領域Ｒｄを指定するための検知領域指定部４３ａと、検知領域指定部４３ａで指定された領域Ｒｄにおいてパレット５２の位置を画像認識する画像認識部４５ａとを備える。よって、パレット５２を高精度に検知することができる。 As described above, pallet detection accuracy by image recognition is improved.
According to the above embodiment, the following effects can be obtained.
(1) As a configuration of a forklift cargo handling work support device for supporting the cargo handling work of the forklift 20, a camera 33 mounted on the forklift 20 and imaging the front of the forklift 20, and a pallet of the entire image captured by the camera 33. A detection area designating section 43a for designating an area Rd in which the user wants to recognize 51 to 58 is provided. Therefore, the pallet 52 can be detected with high accuracy.
　（２）画像認識部４５ａで認識されたパレット５１～５８の位置に基づきフォーク目標高さＨｇを演算するフォーク目標高さ演算部４５ｂを更に備える。よって、フォーク目標高さを求めることができる。 (2) A fork target height calculator 45b for calculating the fork target height Hg based on the positions of the pallets 51 to 58 recognized by the image recognizer 45a is further provided. Therefore, the fork target height can be obtained.
　（３）画像認識部４５ａで認識されたパレット５２の位置に基づきフォークリフト２０を自動制御する自動制御部４２ａを更に備える。よって、認識されたパレット５２の位置に基づきフォークリフト２０を自動制御することができる。 (3) An automatic control unit 42a for automatically controlling the forklift 20 based on the position of the pallet 52 recognized by the image recognition unit 45a is further provided. Therefore, the forklift 20 can be automatically controlled based on the recognized position of the pallet 52.
　（４）フォークリフト用荷役作業支援装置は、フォークリフト用遠隔操作システム１０に用いられるものであって、フォークリフト用遠隔操作システム１０は、フォークリフト２０と、遠隔操作装置４０とを備え、フォークリフト２０は、機台２１に荷役装置２３を備えるとともに車両通信部としての無線部３０を有し、遠隔操作装置４０は、車両通信部としての無線部３０と無線通信を行う操作装置通信部としての無線部４１を有し、フォークリフト２０の走行及び荷役装置２３による荷役を遠隔操作するのに用いられる。よって、遠隔操作する際に、パレット５２を高精度に検知することができる。 (4) The forklift cargo handling work support device is used for the forklift remote operation system 10, and the forklift remote operation system 10 includes the forklift 20 and the remote operation device 40. The platform 21 includes the cargo handling device 23 and a wireless unit 30 as a vehicle communication unit. The remote operation device 40 includes a wireless unit 41 as an operation device communication unit that performs wireless communication with the wireless unit 30 as a vehicle communication unit. It is used to remotely control the traveling of the forklift 20 and the cargo handling by the cargo handling device 23. Therefore, when performing remote control, the pallet 52 can be detected with high accuracy.
　実施形態は前記に限定されるものではなく、例えば、次のように具体化してもよい。
　○　画像の表示画面にタッチパネル機能がある場合、タッチパネルにて領域指定をして
もよい。 The embodiment is not limited to the above, and may be embodied as follows, for example.
○ If the image display screen has a touch panel function, the area may be specified using the touch panel.
　○　操作支援は、自動調整でなく、画像上への相対高さΔＨの表示でもよい。つまり、通知部としての表示部４４において、フォークの高さである揚高を表示する際に、相対高さΔＨを表示し、操作者が相対高さΔＨだけフォークの高さを調整するようにしてもよい。若しくは、フォーク目標高さ演算部４５ｂで演算されたフォーク目標高さＨｇを表示することによりフォーク目標高さＨｇを通知する。つまり、フォーク目標高さＨｇをそのまま表示してもよいし、現揚高との差である相対高さΔＨを表示してもよい。このように、フォーク目標高さ演算部４５ｂで演算されたフォーク目標高さＨｇを通知する通知部としての表示部４４を更に備えた構成としてもよい。 {O} Operation support may be display of the relative height ΔH on the image instead of automatic adjustment. That is, when displaying the lift height which is the height of the fork on the display unit 44 as the notification unit, the relative height ΔH is displayed, and the operator adjusts the height of the fork by the relative height ΔH. You may. Alternatively, the fork target height Hg is notified by displaying the fork target height Hg calculated by the fork target height calculator 45b. That is, the fork target height Hg may be displayed as it is, or the relative height ΔH that is a difference from the current lift height may be displayed. As described above, the display unit 44 may be further provided as a notification unit for notifying the fork target height Hg calculated by the fork target height calculation unit 45b.
　○　横方向も調整してもよい。即ち、図７に示すようにパレットに正対すべきところが横方向にずれている場合に横方向にガイドするようにしてもよい。
　○　画像認識部４５ａと自動制御部４２ａは、統合してもよい。 ○ The horizontal direction may be adjusted. That is, as shown in FIG. 7, when the place to be directly opposed to the pallet is shifted in the horizontal direction, the guide may be guided in the horizontal direction.
The image recognition unit 45a and the automatic control unit 42a may be integrated.
　○　遠隔操作装置４０の制御部４２が行った処理はフォークリフト２０の制御部３１が行ってもよい。
　○　カメラはフォークの先端部に設けたが、これに限るものではない。例えば、ブラケット２５に設けてもよい。要は、カメラは、フォークリフトの前方を撮像するものであればよい。 The process performed by the control unit 42 of the remote control device 40 may be performed by the control unit 31 of the forklift 20.
○ The camera is provided at the tip of the fork, but is not limited to this. For example, it may be provided on the bracket 25. In short, the camera only needs to capture an image in front of the forklift.
　○　フォークリフト用操作支援装置はフォークリフト用遠隔操作システムに用いられるものであったが、これに限るものではない。例えば、有人フォークリフトに用いてもよい。つまり、カメラを搭載した無人フォークリフトと、表示部を有する遠隔操作装置とを備えるではなく、例えば、カメラと表示部を搭載した有人フォークリフトに適用してもよい。 {Circle around (1)} The forklift operation support device is used in a forklift remote operation system, but is not limited to this. For example, it may be used for a manned forklift. That is, the present invention may be applied to, for example, a manned forklift equipped with a camera and a display unit, instead of including an unmanned forklift equipped with a camera and a remote control device having a display unit.
　１０　　フォークリフト用遠隔操作システム
　２０　　フォークリフト
　２１　　機台
　２３　　荷役装置
　３０　　無線部
　３３　　カメラ
　４０　　遠隔操作装置
　４１　　無線部
　４２ａ　　自動制御部
　４３ａ　　検知領域指定部
　４４　　表示部
　４５ａ　　画像認識部
　４５ｂ　　フォーク目標高さ演算部
　５１～５８　　パレット
　Ｈｇ　　フォーク目標高さ
　Ｒｄ　　指定された領域 REFERENCE SIGNS LIST 10 remote control system for forklift 20 forklift 21 machine stand 23 cargo handling device 30 wireless unit 33 camera 40 remote control device 41 wireless unit 42a automatic control unit 43a detection area designation unit 44 display unit 45a image recognition unit 45b fork target height calculation unit 51 ~ 58 pallets Hg Fork target height Rd Specified area

Claims (5)

  1. 　フォークリフトの荷役作業を支援するフォークリフト用荷役作業支援装置であって、
    　前記フォークリフトに搭載され、前記フォークリフトの前方を撮像するカメラと、
    　前記カメラにて撮像された画像全体のうちパレットを認識させたい領域を指定するための検知領域指定部と、
    　前記検知領域指定部で指定された領域において前記パレットの位置を画像認識する画像認識部と、
    を備えることを特徴とするフォークリフト用荷役作業支援装置。     A forklift cargo handling work support device for supporting forklift cargo handling work,
    A camera mounted on the forklift and imaging the front of the forklift;
    A detection area designation unit for designating an area in which the pallet is to be recognized in the entire image captured by the camera,
    An image recognition unit that recognizes the position of the pallet in an area specified by the detection area specification unit;
    A cargo handling work support device for a forklift, comprising:
  2. 　前記画像認識部で認識された前記パレットの位置に基づきフォーク目標高さを演算するフォーク目標高さ演算部を更に備えることを特徴とする請求項１に記載のフォークリフト用荷役作業支援装置。 The forklift cargo handling work support apparatus according to claim 1, further comprising a fork target height calculating unit that calculates a fork target height based on the position of the pallet recognized by the image recognition unit.
  3. 　前記フォーク目標高さ演算部で演算されたフォーク目標高さを通知する通知部を更に備えることを特徴とする請求項２に記載のフォークリフト用荷役作業支援装置。 3. The cargo handling work support device for a forklift according to claim 2, further comprising: a notifying unit for notifying the fork target height calculated by the fork target height calculating unit.
  4. 　前記画像認識部で認識された前記パレットの位置に基づき前記フォークリフトを自動制御する自動制御部を更に備えることを特徴とする請求項１～３のいずれか１項に記載のフォークリフト用荷役作業支援装置。 The cargo handling work support device for a forklift according to any one of claims 1 to 3, further comprising an automatic control unit that automatically controls the forklift based on the position of the pallet recognized by the image recognition unit. .
  5. 　前記フォークリフト用荷役作業支援装置は、フォークリフト用遠隔操作システムに用いられるものであって、
    　前記フォークリフト用遠隔操作システムは、前記フォークリフトと、遠隔操作装置とを備え、
    　前記フォークリフトは、機台に荷役装置を備えるとともに車両通信部を有し、
    　前記遠隔操作装置は、前記車両通信部と無線通信を行う操作装置通信部を有し、前記フォークリフトの走行及び前記荷役装置による荷役を遠隔操作するのに用いられることを特徴とする請求項１～４のいずれか１項に記載のフォークリフト用荷役作業支援装置。     The forklift cargo handling support device is used for a forklift remote operation system,
    The forklift remote control system includes the forklift and a remote control device,
    The forklift has a vehicle communication unit while including a cargo handling device on a machine base,
    The remote control device includes an operating device communication unit that performs wireless communication with the vehicle communication unit, and is used to remotely control traveling of the forklift and cargo handling by the cargo handling device. 4. The cargo handling work support device for a forklift according to any one of 4.
PCT/JP2019/028728 2018-08-24 2019-07-23 Loading operation assistance device for forklift WO2020039817A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2018-157339 2018-08-24
JP2018157339A JP2020029361A (en) 2018-08-24 2018-08-24 Cargo handling work supporting device for fork lift

Publications (1)

Publication Number Publication Date
WO2020039817A1 true WO2020039817A1 (en) 2020-02-27

Family

ID=69592525

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2019/028728 WO2020039817A1 (en) 2018-08-24 2019-07-23 Loading operation assistance device for forklift

Country Status (2)

Country Link
JP (1) JP2020029361A (en)
WO (1) WO2020039817A1 (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2021195195A (en) * 2020-06-10 2021-12-27 三菱ロジスネクスト株式会社 Forklift, control method of forklift, and program
WO2022199644A1 (en) * 2021-03-26 2022-09-29 未来机器人（深圳）有限公司 Pallet stacking tilt recognition device

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR102598941B1 (en) * 2021-07-19 2023-11-07 (주)러셀로보틱스 Method and server for controlling forking height of autonomous forklift
JP7347910B2 (en) * 2021-09-10 2023-09-20 三菱ロジスネクスト株式会社 Remote control system for cargo handling vehicles

Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5812395A (en) * 1994-11-16 1998-09-22 Masciangelo; Stefano Vision based forklift control system for autonomous pallet loading
JP2002241094A (en) * 2001-02-16 2002-08-28 Toyota Industries Corp Camera lifting device in industrial vehicle
JP2003155199A (en) * 2001-11-19 2003-05-27 Toyota Industries Corp Voice annunciator for industrial vehicle and industrial vehicle
JP2009061795A (en) * 2007-09-04 2009-03-26 Fujitsu Ten Ltd Parking assist control device
JP2010205121A (en) * 2009-03-05 2010-09-16 Sony Ericsson Mobile Communications Ab Information processor and portable terminal
KR20110027460A (en) * 2009-09-10 2011-03-16 부산대학교 산학협력단 A method for positioning and orienting of a pallet based on monocular vision
JP2013086959A (en) * 2011-10-21 2013-05-13 Sumitomo Heavy Ind Ltd Support device and method for positioning fork of forklift
JP2013218571A (en) * 2012-04-10 2013-10-24 Toyota Motor Corp Image recognition device and driving support device
JP2018072945A (en) * 2016-10-25 2018-05-10 株式会社豊田自動織機 Remote operation system for industrial vehicle, remote controller, remote operation program for industrial vehicle, and industrial vehicle

Patent Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5812395A (en) * 1994-11-16 1998-09-22 Masciangelo; Stefano Vision based forklift control system for autonomous pallet loading
JP2002241094A (en) * 2001-02-16 2002-08-28 Toyota Industries Corp Camera lifting device in industrial vehicle
JP2003155199A (en) * 2001-11-19 2003-05-27 Toyota Industries Corp Voice annunciator for industrial vehicle and industrial vehicle
JP2009061795A (en) * 2007-09-04 2009-03-26 Fujitsu Ten Ltd Parking assist control device
JP2010205121A (en) * 2009-03-05 2010-09-16 Sony Ericsson Mobile Communications Ab Information processor and portable terminal
KR20110027460A (en) * 2009-09-10 2011-03-16 부산대학교 산학협력단 A method for positioning and orienting of a pallet based on monocular vision
JP2013086959A (en) * 2011-10-21 2013-05-13 Sumitomo Heavy Ind Ltd Support device and method for positioning fork of forklift
JP2013218571A (en) * 2012-04-10 2013-10-24 Toyota Motor Corp Image recognition device and driving support device
JP2018072945A (en) * 2016-10-25 2018-05-10 株式会社豊田自動織機 Remote operation system for industrial vehicle, remote controller, remote operation program for industrial vehicle, and industrial vehicle

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2021195195A (en) * 2020-06-10 2021-12-27 三菱ロジスネクスト株式会社 Forklift, control method of forklift, and program
JP7132976B2 (en) 2020-06-10 2022-09-07 三菱ロジスネクスト株式会社 FORKLIFT, FORKLIFT CONTROL METHOD, AND PROGRAM
WO2022199644A1 (en) * 2021-03-26 2022-09-29 未来机器人（深圳）有限公司 Pallet stacking tilt recognition device

Also Published As

Publication number Publication date
JP2020029361A (en) 2020-02-27

Similar Documents

Publication Publication Date Title
WO2020039817A1 (en) Loading operation assistance device for forklift
EP1408001B1 (en) Industrial vehicle equipped with material handling work controller
US9667923B2 (en) Camera attitude detection device and work region line display device
US7010404B2 (en) Position control apparatus and position control method for cargo carrying apparatus in industrial vehicle
US10981764B2 (en) Industrial truck having a fork and a fork arm camera and method for operating such an industrial truck
WO2023272985A1 (en) Smart stacker crane, and method, apparatus, and device for recognizing anomalous pallet position
JP2013086959A (en) Support device and method for positioning fork of forklift
JP3921968B2 (en) Position detection method and position detection apparatus
JP7215394B2 (en) Operation support device for cargo handling vehicle
JP2020132431A (en) Travel support device for industrial vehicle
US11939197B2 (en) Forklift-truck remote operation system
JP2020109030A (en) Determination device, determination method and program
WO2021066138A1 (en) Operation assistance device for cargo handling vehicle
JP7268575B2 (en) Operation support device for cargo handling vehicle
JP3900942B2 (en) Automatic position control device for cargo handling equipment in industrial vehicles, industrial vehicle and automatic position control method for cargo handling equipment
JP4961643B2 (en) Cargo handling control device and industrial vehicle in industrial vehicle
JP2003128395A (en) Cargo handling work support device in industrial vehicle and industrial vehicle
JP7087737B2 (en) Remote control system for forklifts
JP2021020793A (en) Operation supporting device for cargo handling vehicle
JP2020043382A (en) Forklift remote control system
JP2020001863A (en) Remote operation system for industrial vehicle
WO2020095602A1 (en) Remote operating system for forklift
JP3858703B2 (en) Cargo work support device for industrial vehicle and industrial vehicle
WO2020085068A1 (en) Remote control system for forklift
JP2003155199A (en) Voice annunciator for industrial vehicle and industrial vehicle

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19852970

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 19852970

Country of ref document: EP

Kind code of ref document: A1