WO2021171618A1 - 搬送システム、制御装置、制御方法 - Google Patents

搬送システム、制御装置、制御方法 Download PDF

Info

Publication number
WO2021171618A1
WO2021171618A1 PCT/JP2020/008528 JP2020008528W WO2021171618A1 WO 2021171618 A1 WO2021171618 A1 WO 2021171618A1 JP 2020008528 W JP2020008528 W JP 2020008528W WO 2021171618 A1 WO2021171618 A1 WO 2021171618A1
Authority
WO
WIPO (PCT)
Prior art keywords
transport
information
transported object
contact position
height
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/JP2020/008528
Other languages
English (en)
French (fr)
Japanese (ja)
Inventor
安田　真也
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
NEC Corp
Original Assignee
NEC Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by NEC Corp filed Critical NEC Corp
Priority to PCT/JP2020/008528 priority Critical patent/WO2021171618A1/ja
Priority to US17/800,413 priority patent/US12103786B2/en
Priority to JP2022503056A priority patent/JP7380829B2/ja
Publication of WO2021171618A1 publication Critical patent/WO2021171618A1/ja
Anticipated expiration legal-status Critical
Priority to JP2023188751A priority patent/JP7548401B2/ja
Ceased legal-status Critical Current

Links

Images

Classifications

    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B65—CONVEYING; PACKING; STORING; HANDLING THIN OR FILAMENTARY MATERIAL
    • B65G—TRANSPORT OR STORAGE DEVICES, e.g. CONVEYORS FOR LOADING OR TIPPING, SHOP CONVEYOR SYSTEMS OR PNEUMATIC TUBE CONVEYORS
    • B65G47/00—Article or material-handling devices associated with conveyors; Methods employing such devices
    • B65G47/34—Devices for discharging articles or materials from conveyor 
    • B65G47/46—Devices for discharging articles or materials from conveyor  and distributing, e.g. automatically, to desired points
    • B65G47/50—Devices for discharging articles or materials from conveyor  and distributing, e.g. automatically, to desired points according to destination signals stored in separate systems
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B66—HOISTING; LIFTING; HAULING
    • B66F—HOISTING, LIFTING, HAULING OR PUSHING, NOT OTHERWISE PROVIDED FOR, e.g. DEVICES WHICH APPLY A LIFTING OR PUSHING FORCE DIRECTLY TO THE SURFACE OF A LOAD
    • B66F9/00—Devices for lifting or lowering bulky or heavy goods for loading or unloading purposes
    • B66F9/06—Devices for lifting or lowering bulky or heavy goods for loading or unloading purposes movable, with their loads, on wheels or the like, e.g. fork-lift trucks
    • B66F9/063—Automatically guided
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B66—HOISTING; LIFTING; HAULING
    • B66F—HOISTING, LIFTING, HAULING OR PUSHING, NOT OTHERWISE PROVIDED FOR, e.g. DEVICES WHICH APPLY A LIFTING OR PUSHING FORCE DIRECTLY TO THE SURFACE OF A LOAD
    • B66F9/00—Devices for lifting or lowering bulky or heavy goods for loading or unloading purposes
    • B66F9/06—Devices for lifting or lowering bulky or heavy goods for loading or unloading purposes movable, with their loads, on wheels or the like, e.g. fork-lift trucks
    • B66F9/075—Constructional features or details
    • B66F9/0755—Position control; Position detectors
    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
    • G05D1/02—Control of position or course in two dimensions
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B65—CONVEYING; PACKING; STORING; HANDLING THIN OR FILAMENTARY MATERIAL
    • B65G—TRANSPORT OR STORAGE DEVICES, e.g. CONVEYORS FOR LOADING OR TIPPING, SHOP CONVEYOR SYSTEMS OR PNEUMATIC TUBE CONVEYORS
    • B65G2203/00—Indexing code relating to control or detection of the articles or the load carriers during conveying
    • B65G2203/02—Control or detection
    • B65G2203/0208—Control or detection relating to the transported articles
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B65—CONVEYING; PACKING; STORING; HANDLING THIN OR FILAMENTARY MATERIAL
    • B65G—TRANSPORT OR STORAGE DEVICES, e.g. CONVEYORS FOR LOADING OR TIPPING, SHOP CONVEYOR SYSTEMS OR PNEUMATIC TUBE CONVEYORS
    • B65G2203/00—Indexing code relating to control or detection of the articles or the load carriers during conveying
    • B65G2203/02—Control or detection
    • B65G2203/0266—Control or detection relating to the load carrier(s)
    • B65G2203/0283—Position of the load carrier
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B65—CONVEYING; PACKING; STORING; HANDLING THIN OR FILAMENTARY MATERIAL
    • B65G—TRANSPORT OR STORAGE DEVICES, e.g. CONVEYORS FOR LOADING OR TIPPING, SHOP CONVEYOR SYSTEMS OR PNEUMATIC TUBE CONVEYORS
    • B65G2203/00—Indexing code relating to control or detection of the articles or the load carriers during conveying
    • B65G2203/04—Detection means
    • B65G2203/041—Camera
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B65—CONVEYING; PACKING; STORING; HANDLING THIN OR FILAMENTARY MATERIAL
    • B65G—TRANSPORT OR STORAGE DEVICES, e.g. CONVEYORS FOR LOADING OR TIPPING, SHOP CONVEYOR SYSTEMS OR PNEUMATIC TUBE CONVEYORS
    • B65G61/00—Use of pick-up or transfer devices or of manipulators for stacking or de-stacking articles not otherwise provided for

Definitions

  • the present invention relates to a transfer system, a control device, and a control method.
  • Patent Document 1 discloses a technique of a transport system for transporting a transported object using a robot.
  • Patent Document 1 discloses an example as an algorithm capable of efficiently and stably executing push and transport of a robot.
  • the technique of Patent Document 1 discloses a technique of controlling the moving direction of the robot.
  • an object of the present invention is to provide a transport system, a control device, and a control method that solve the above-mentioned problems.
  • the transport system is a transport system including a transport vehicle that transports the transported object, a sensor that acquires information about the transported object, and a control device that controls the transported vehicle. Therefore, the contact position with respect to the transported object that the transport vehicle comes into contact with when transporting the transported object is specified based on the information about the transported object, and the transport vehicle is controlled based on the contact position.
  • the control device is communicatively connected to a transport vehicle for transporting the transported object and a sensor for acquiring information on the transported object, and the transport is based on the information on the transported object.
  • the contact position with respect to the conveyed object that the vehicle comes into contact with when the conveyed object is conveyed is specified, and the conveyed vehicle is controlled based on the contact position.
  • the control method communicates with a transport vehicle for transporting the transported object and a sensor for acquiring information on the transported object, and the transport is based on the information on the transported object.
  • the contact position with respect to the conveyed object that the vehicle comes into contact with when the conveyed object is conveyed is specified, and the conveyed vehicle is controlled based on the contact position.
  • FIG. 1 It is a figure which shows the photographing state of the conveyed object in the case where the position of the circumscribed frame R by one Embodiment of this invention is a 2nd to 3rd pattern. It is a figure which shows the specific outline of the contact position by one Embodiment of this invention. It is a figure which shows the example of the display information by one Embodiment of this invention. It is a figure which shows the outline of calculation of the position of the conveyed object at a predetermined height by one Embodiment of this invention. It is a figure which shows the relationship between the circumscribed frame R and a specific area in the 1st pattern by one Embodiment of this invention.
  • FIG. 1 is a first diagram showing the relationship between the circumscribed frame R and the specific region in the second pattern according to the embodiment of the present invention. It is a second figure which shows the relationship between the circumscribed frame R and a specific area in the 2nd pattern by one Embodiment of this invention.
  • FIG. 1 is a first diagram showing the relationship between the circumscribed frame R and the specific region in the third pattern according to the embodiment of the present invention.
  • FIG. 2 is a second diagram showing the relationship between the circumscribed frame R and the specific region in the third pattern according to the embodiment of the present invention.
  • It is a schematic block diagram of the control system provided with the control device in the 2nd Embodiment of this invention. It is a schematic configuration of the transport system according to the third embodiment of the present invention. It is a schematic block diagram of the control device which concerns on 3rd Embodiment of this invention. It is a figure which shows the processing flow by the control apparatus which concerns on 3rd Embodiment of this invention.
  • FIG. 1 is a schematic configuration diagram of a transport system according to the present embodiment.
  • the transport system 100 includes a control device 1, a sensor 2, and a transport vehicle 3.
  • the sensor 2 measures information about the transported object 4.
  • the sensor 2 transmits the measured information about the transported object 4 to the control device 1.
  • the sensor 2 is a device that images the inside of a field in which the transport vehicle 3 can move.
  • the sensor 2 is, for example, a depth camera or a stereo camera.
  • the sensor 2 photographs the floor surface on which the transport vehicle 3 travels.
  • the senor 2 measures image information and distance information in a range centered on the lower axis from the vicinity of the ceiling where the sensor 2 is installed toward the floor surface.
  • the sensor 2 generates image information obtained by capturing the measurement range of the sensor 2 and distance information indicating the distance to each position of the measurement range of the sensor 2.
  • the distance information indicates, for example, the distance from the sensor 2 corresponding to each pixel of the image information in the measurement range.
  • the control device 1 controls the transport vehicle 3.
  • the control device 1 controls the transport vehicle 3 based on the acquired information.
  • the control device 1 communicates with the sensor 2 for measuring the transported object 4 and the transport vehicle 3.
  • the control device 1 acquires image information and distance information from the sensor 2.
  • the control device 1 identifies a position where the transport vehicle 3 contacts the transport object 4 when the transport vehicle 4 is transported based on the information (image information and distance information) about the transport object 4, and based on the contact position.
  • the control device 1 may control one transport vehicle 3 or may control a plurality of transport vehicles 3.
  • the transport vehicle 3 is an aspect of a robot.
  • FIG. 2 is a hardware configuration diagram of the control device according to the present embodiment.
  • the control device 1 is a computer server provided with hardware such as an arithmetic processing unit 101, a ROM (Read Only Memory) 102, a RAM (Random Access Memory) 103, a storage unit 104, and a communication module 105.
  • the arithmetic processing unit 101 is, for example, a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), or the like.
  • the storage unit 104 is, for example, an HDD (Hard Disk Drive), an SSD (Solid State Drive), a memory card, or the like. Further, the storage unit 104 may be a memory such as a RAM or a ROM.
  • the communication module 105 transmits and receives data to and from an external device. For example, the communication module 105 communicates with an external device via a wired or wireless communication path.
  • FIG. 3 is a functional block diagram of the control device according to the present embodiment.
  • the control device 1 is activated when the power is turned on, and executes a control program stored in advance. As a result, the control device 1 exerts the functions of the image information acquisition unit 11, the distance information acquisition unit 12, the difference detection unit 13, the survey unit 14, the contact position identification unit 15, the transport control unit 16, and the display unit 17.
  • the transported object 4 is an object to be transported, and an example is a cart or a trolley on which luggage is placed.
  • the transport vehicle 3 transports the transported object 4 based on the control of the control device 1.
  • the transport vehicle 3 receives information on the contact position indicating the position where the transport vehicle 3 contacts the transport object 4 from the control device 1, and conveys the transport object 4 by pushing or pulling the contact position.
  • the side surface of the transported object 4 may or may not be included in the captured image depending on the positional relationship between the sensor 2 and the transported object 4. be.
  • the conveyed object 4 is near the center of the measurement range of the sensor 2, the upper surface of the conveyed object 4 is reflected in the captured image, and the side surface is not reflected.
  • the conveyed object 4 is located at a position away from the center of the measurement range of the sensor 2, the conveyed object 4 is photographed from diagonally above, so that the upper surface and the side surface of the conveyed object 4 are reflected in the photographed image.
  • the control device 1 detects a region in the captured image in which the transported object 4 is projected using the captured image, the region may or may not include the side surface depending on the position of the transported object 4. do. Further, the closer the distance to the conveyed object 4 is to the sensor 2, the wider the upper surface area is reflected in the captured image. Therefore, even if the entire area of the transported object 4 displayed in the captured image differs depending on the height of the transported object 4 (distance from the sensor 2) and the positional relationship between the transported object 4 and the sensor 2, control is performed. The device needs to accurately calculate the contact position at a predetermined height at which the transport vehicle 3 can contact the transport object 4.
  • FIG. 4 is a diagram showing a processing flow of the transport system according to the present embodiment.
  • the sensor 2 transmits a number of image information such as 30 frames per second to the control device 1. Further, the sensor 2 transmits a number of distance information such as 30 frames per second to the control device 1. It is assumed that the timings of the image information transmitted by the sensor 2 and the distance information match. The image information and the distance information indicate information in the same area.
  • the image information acquisition unit 11 of the control device 1 acquires the image information (step S101).
  • the distance information acquisition unit 12 of the control device 1 acquires the distance information (step S102).
  • the image information acquisition unit 11 generates a background image based on the image information and records it in a storage unit such as a RAM 103.
  • the generation and recording of the background image may be performed before detecting the conveyed object 4.
  • the image information acquisition unit 11 may be performed at the time when the transfer system 100 starts operating, or may be performed at the timing when the administrator of the transfer system instructs the recording.
  • the background image is image information when the transport vehicle 3, the transport object 4, and other foreign substances are not included in the measurement range.
  • the image information acquisition unit 11 outputs the image information received from the sensor 2 to the difference detection unit 13.
  • the distance information acquisition unit 12 records the distance information received from the sensor 2 in a storage unit such as the RAM 103.
  • the image information acquisition unit 11 and the distance information acquisition unit 12 may each be assigned an ID or the like so that the relationship between the image information and the distance information corresponding to the transmission timing of the sensor 2 is linked.
  • the difference detection unit 13 generates difference information indicating the difference between the image information received from the image information acquisition unit 11 and the background image. Specifically, when the difference detection unit 13 acquires the image information, the difference detection unit 13 compares the image information with the background image. The difference detection unit 13 generates difference information indicating a region in which a change occurs between the image information and the background image (step S103). For example, the difference detection unit 13 binarizes the image information and the background image into pixels indicating "0" and "1" based on the brightness of each pixel, and binarizes the binarized image information and the binarized image information. Generates difference information indicating the difference between each pixel and the background image. A pixel whose difference is "1" in this difference information indicates that some object is located in the measurement range. The difference detection unit 13 outputs the difference information to the surveying unit 14.
  • the surveying unit 14 determines whether the acquired difference information includes the transported object 4 (step 104). For example, the surveying unit 14 determines whether the transport vehicle 3 is located in the measurement range, and if the difference information includes information other than the transport vehicle 3, it determines that the transport vehicle 4 is included.
  • the position information of the transport vehicle 3 may be detected by the transport vehicle 3 and transmitted to the control device 1, or may be detected by the sensor 2 and transmitted to the control device 1.
  • the measuring unit 14 determines whether or not the transport vehicle 3 is located in the measurement range by comparing the position information of the transport vehicle 3 with the position information of the measurement range measured by the sensor 2 stored in advance.
  • the measuring unit 14 detects the position of the transport vehicle 3 from the image information by using the characteristics (brightness, size, etc.) of the transport vehicle 3 stored in advance, and identifies whether or not the vehicle 3 is located within the measurement range. You may.
  • the method by which the measuring unit 14 detects the position of the transport vehicle 3 is not limited to the above.
  • the surveying unit 14 may mask the region of the transport vehicle 3 in the measurement range indicated by the difference information to generate the difference information.
  • the measuring unit 14 determines whether the transport vehicle 3 is located in the measurement range, and when the difference information acquired from the difference detection unit 13 includes information other than the transport vehicle 3, the transported object 4 is included.
  • the specification of whether or not the transported object 4 is included is not limited to the above.
  • the measuring unit 14 includes the transported object 4 in the difference information based on the information of the specified size of the transported object 4 even when the transported object 4 or the transport vehicle 3 is included in the measurement range. It may be determined whether or not.
  • the surveying unit 14 identifies an inclusion area including the transported object 4. For example, the surveying unit 14 determines the size of a region of a group of pixels indicating that there is a difference in the difference information.
  • the surveying unit 14 identifies the outer frame of the area as the circumscribed frame R of the conveyed object 4 (step S105). ).
  • the circumscribed frame R indicates a frame of an inclusion region including the upper surface and the side surface of the conveyed object 4 to be measured.
  • the method of specifying the outline frame R is not limited to the above method, and the surveying unit 14 may determine the circumscribed frame R of the region of the transported object 4 included in the difference information by another method.
  • FIG. 5 is a diagram showing a pattern of positions where the circumscribed frame R is detected in the measurement range.
  • it is determined which of the first pattern and the third pattern the position pattern of the circumscribed frame R detected by the surveying unit 14 in the captured image is (step S106).
  • the measurement range by the sensor 2 is divided into a vertical line 51 and a horizontal line 52 passing through the center of the measurement range, the upper right is the first region, the upper left is the second region, the lower left is the third region, and the lower right is the fourth region.
  • FIG. 5 (1) shows the first pattern of the position of the circumscribed frame R.
  • the first pattern is a pattern in which the four vertices of the circumscribed frame R are included in each of the first to fourth regions.
  • the first pattern is a pattern that appears when the circumscribed frame R including the conveyed object 4 is located at the center.
  • FIG. 5 (2) shows the second pattern of the position of the circumscribed frame R.
  • the second pattern is a pattern in which all four vertices of the circumscribed frame R are included in one of the first to fourth regions.
  • the second pattern is a pattern in which the circumscribed frame R including the conveyed object 4 appears only in any one of the first to fourth regions.
  • FIGS. 5 (3) and 5 (4) show the third pattern of the position of the circumscribed frame R.
  • the third pattern is a pattern in which the four vertices of the circumscribed frame R are located in the two regions, respectively.
  • the third pattern shown in FIG. 5 (3) includes a case where the circumscribed frame R including the transported object 4 straddles the first region and the second region, and a case where the circumscribed frame R straddles the third region and the fourth region.
  • the third pattern shown in FIG. 5 (4) includes a case where the circumscribed frame R including the transported object 4 straddles the second region and the third region, and a case where the circumscribed frame R straddles the first region and the fourth region.
  • FIG. 6 is a diagram showing a photographing state of the transported object when the position of the circumscribed frame R is the first pattern.
  • the position of the circumscribed frame R detected by the surveying unit 14 is the first pattern
  • the upper surface of the conveyed object 4 is reflected in the captured image, and the side surface is not reflected in the captured image. Further, the closer the upper surface of the conveyed object 4 is to the sensor 2, the wider the area of the upper surface in the captured image.
  • FIG. 7 is a diagram showing a photographing state of the transported object when the position of the circumscribed frame R is the second pattern or the third pattern.
  • the position of the circumscribed frame R detected by the measuring unit 14 is the second pattern, as shown in FIG. 7 (2), the conveyed object 4 that can be connected to the upper surface of the conveyed object 4 and the position of the sensor 2 in a straight line. The surface is reflected in the captured image.
  • the position of the circumscribed frame R detected by the surveying unit 14 is the third pattern, it is reflected in the captured image as shown in FIGS. 7 (3) and 7 (4).
  • the surveying unit 14 first Judge as a pattern.
  • the measuring unit 14 is one of four regions, the first region to the fourth region, in which all the coordinates of the pixels within the range of the circumscribed frame R are divided by a vertical line and a horizontal line passing through the center of the measurement range. If it is included only in, it is determined that it is the second pattern.
  • the measuring unit 14 when the coordinates of the pixels within the range of the circumscribed frame R are located in two regions separated by a vertical line 51 passing through the center of the measurement range, or when the coordinates of the pixels within the range of the circumscribed frame R are determined. When it is located in two regions separated by a horizontal line 52 passing through the center of the measurement range, it is determined to be the third pattern.
  • the surveying unit 14 specifies the first specific region R1 indicating the upper surface of the transported object 4 in the captured image (step S107). Specifically, the surveying unit 14 determines the relationship between the feature points appearing in the captured images of the plurality of circumscribed frames R specified in the circumscribed frame R and the height information of the feature points of the plurality of circumscribed frames R, and the circumscribed circle. Corresponding points appearing in captured images at predetermined heights that match the horizontal coordinates (horizontal position) of the feature points of the frame R and have different height coordinates, and the height information indicated by the predetermined height. Based on the relationship between, and, a plurality of corresponding points constituting the surface of the object at a predetermined height are calculated.
  • the plurality of corresponding points constituting the surface of the object at a predetermined height include estimated corresponding points that do not appear in the captured image.
  • the surveying unit 14 identifies the first specific region R1 indicating the upper surface of the transported object 4 in the captured image based on the plurality of corresponding points.
  • the surveying unit 14 identifies the second specific region R2 that shows the region of the transported object 4 at the height at which the transport vehicle 3 contacts the transported object 4 in the photographed image (step S108).
  • the contact position specifying unit 15 acquires the information of the second specific region R2 showing the region of the transported object 4 at the height h'when the transport vehicle 3 contacts the transported object 4 in the photographed image.
  • the contact position specifying unit 15 acquires information on the feature points indicated by the second specific area R2.
  • the contact position specifying unit 15 specifies a position in the captured image in which the transport vehicle 3 contacts the transported object 4 based on the information of the feature points indicated by the second specific region R2 (step S109).
  • FIG. 8 is a diagram showing a specific outline of the contact position.
  • the contact position specifying unit 15 when the rectangle of the second specific area R2 indicates the feature points P21, P22, P23, P24, the transport vehicle 3 transports the center of any side of the second specific area R2. It is specified as the contact position T1 in the captured image in contact with the object 4.
  • the contact position specifying unit 15 identifies the first side connecting the feature points P21 and P22 on the transport direction D side and the second side connecting the feature points P22 and P23, and among the normals of those sides.
  • the center of the second side having a small angle with the transport direction D is designated as the contact position T1 in the photographed image in which the transport vehicle 3 contacts the transport object 4.
  • the contact position specifying unit 15 outputs the contact position T1 to the transport control unit 16. For example, the transport vehicle 3 contacts the transport object 4 at the contact position T1 of the transport object 4 and pulls the transport object 4 in the transport direction D.
  • the contact position specifying portion 15 is the side of the second specific region R2, which is on the opposite side of the transport direction D and has a normal line formed by a small angle with the transport direction. It may be specified as the contact position P in the captured image in contact with 4. That is, the contact position specifying portion 15 identifies a third side connecting the feature points P21 and P24 on the opposite side of the transport direction D and a fourth side connecting the feature points P24 and P23, and among the sides, The center of the third side having a normal having a small angle with the transport direction D is specified as the contact position T2 in the captured image in which the transport vehicle 3 contacts the transport object 4.
  • the contact position specifying unit 15 outputs the contact position T2 to the transport control unit 16. In this case, the transport vehicle 3 comes into contact with the transport object 4 at the contact position of the transport object 4 and pushes the transport object 4 in the transport direction D.
  • the transport control unit 16 converts the contact positions T1 and T2 in the captured image into the contact positions T1'and T2'in the real space (step S110). For example, the transport control unit 16 stores in advance the relationship between the coordinates in the virtual space indicated by the captured image and the coordinates in the real space, and based on the correspondence, the contact positions T1 and T2 are set to the contact positions in the real space. Convert to T1'and T2'.
  • the transport control unit 16 transmits the contact positions T1'and T2'in the real space and the transport direction of the transport object 4 to the transport vehicle 3 (step S111).
  • the transport vehicle 3 moves toward the contact positions T1'and T2', contacts the contact positions T1'and T2', and transports the transported object in the transport direction.
  • the transported object 4 is conveyed by two transport vehicles 3, but a plurality of transport vehicles 3 are connected to the transport system 100 by communication, and the plurality of transport vehicles 3 connect the transported object 4 to the transport system 100.
  • One transport vehicle may be connected to the transport system 100 by communication, and one transport vehicle 3 may transport the transported object 4.
  • the transported object 4 may be transported by contacting the transported object 4 from all sides with four transport vehicles 3, or one transport vehicle 3 may contact and tow any surface of the transported object 4.
  • it may be conveyed by pushing it in.
  • the transport control unit 16 transmits the contact position T1 to the first transport vehicle 3 and the contact position T3 to the second transport vehicle 3.
  • the transport control unit 16 transmits the transport direction D to the first transport vehicle 3 and the second transport vehicle 3.
  • the first transport vehicle 3 comes into contact with the contact position T1 of the transport object 4.
  • the second transport vehicle 3 comes into contact with the contact position T2 of the transport object 4.
  • the first transport vehicle 3 and the second transport vehicle 3 sandwich the transport object 4 with each other and transport the transport object 4 in the transport direction.
  • the transport vehicle 3 comes into contact with the transport object 4, but the method of the transport vehicle 3 acting on the transport object 4 is not limited.
  • the transport vehicle 3 may, for example, press the equipment of the transport vehicle against the transport object 4, connect or push (fit) the equipment into the dents or protrusions of the transport object 4, or apply an impact from the transport object 4. You may try to accept it.
  • the transport vehicle 3 may grab the transport object 4 with an instrument that sandwiches the transport object 4 from two directions and tow it. For example, when the first transport vehicle 3 and the second transport vehicle 3 each come into contact with the transport object 4 to transport the transport object 4, the second transport vehicle 3 has the first force F1 at the contact position T2. And proceed in the direction of travel.
  • the first transport vehicle 3 applies a second force F2 smaller than the first force F1 to the contact position T1 and advances in the same transport direction at the same speed as the first transport vehicle 3.
  • the transported object 4 is transported by the second transport vehicle 3 of the first transport vehicle 3 and the second transport vehicle 3.
  • the first transport vehicle 3 is connected to the contact position T1 and towed, and the second transport vehicle 3 applies a force to the contact position T2 to move in the transport direction while controlling the transported object 4 so as not to wobble.
  • the second transport vehicle 3 is connected to the contact position T2 and pushes forward in the transport direction, and the first transport vehicle 3 applies a force to the contact position T1 to control the transport object 4 so as not to wobble in the transport direction. You may proceed.
  • the position where the transport vehicle comes into contact with the transported object can be specified. Further, according to the above-mentioned processing of the transport system, the transport vehicle can specify the contact position with the transported object with higher accuracy. Further, according to the processing of the transport system 100 described above, the region of the transport object 4 reflected in the captured image is determined according to the height of the transport object 4 (distance from the sensor 2) and the positional relationship between the transport object 4 and the sensor 2. Even if they are different, the contact position at a predetermined height at which the transport vehicle 3 can contact the transport object 4 can be calculated with higher accuracy.
  • FIG. 9 is a diagram showing an example of display information.
  • the display unit 17 outputs the information specified by the control device 1 to a predetermined output destination.
  • the display unit 17 acquires, for example, the circumscribed frame R, the first specific area R1, and the second specific area R2 from the photographed image to be processed and the surveying unit 14. Further, the display unit 17 acquires the contact position calculated based on the second specific area R2.
  • the display unit 17 generates the captured image, the circumscribed frame R, the first specific area R1, the second specific area R2, the contact position information, and the display information to be displayed.
  • the display unit 17 outputs display information to a predetermined output destination.
  • the display unit 17 outputs display information to an LCD (Liquid Crystal Display), a CRT (Cathode Ray Tube) display, a monitor provided in the control device 1, and a terminal for communication connection to the control device 1.
  • LCD Liquid Crystal Display
  • CRT Cathode Ray Tube
  • the display unit 17 may superimpose and display information on the circumscribed frame R, the first specific area R1, the second specific area R2, and the contact position as display information, or display each information separately.
  • Display information may be generated such that arbitrary information selected by the worker involved in the work is superimposed and displayed.
  • the processing of the surveying unit 14 for calculating the first specific area and the second specific area will be described in detail below.
  • the measuring unit 14 determines that the circumscribed frame R is the upper surface of the conveyed object 4.
  • the surveying unit 14 identifies a specific region (first specific region, second specific region) at a predetermined height of the transported object 4 included in the region of the external frame R based on the external frame R and its height information. ..
  • FIG. 10 is a diagram showing an outline of calculation of the position of the transported object at a predetermined height.
  • the focal length f when the sensor 2 captures image information an arbitrary feature point P1 on the upper surface of the transport object 4 in the real space, and the transport object whose coordinates X in the horizontal direction coincide with the feature point P1.
  • the distance of the feature point P1 in the real space in the height direction from the sensor 2 is z
  • the distance of the corresponding point P2 in the height direction from the sensor 2 is h
  • the distance of the feature point P1 and the corresponding point P2 in the horizontal direction from the sensor 2 is Let X be the distance. In this case, it is based on the relationship between the height distance from the sensor 2 and the horizontal distance in the real space, and the relationship between the focal length in the captured image and the horizontal distance of each point from the center point of the captured image. Therefore, the following two equations (1) and (2) can be derived.
  • Equations (3) and (4) are obtained as described above. Therefore,
  • Equation (5) represented by The point x1 corresponding to the feature point P1 on the upper surface in the captured image, z in the real space, and the height h can be acquired by the sensor 2. Therefore, it is possible to calculate the point x2 in the captured image corresponding to an arbitrary corresponding point P2 in the real space by using the equation (5).
  • the value of the height h may be measured by the sensor 2 at an arbitrary timing, or may be set as an initial value when the sensor 2 is installed, and the acquisition method thereof is not limited.
  • "/" indicates division.
  • ⁇ indicates multiplication.
  • FIG. 11 is a diagram showing the relationship between the circumscribed frame R and the specific region in the first pattern.
  • FIG. 11 (1) shows the transported object 4 shown in the captured image
  • FIG. 11 (2) shows a perspective view of the corresponding transported object 4.
  • the predetermined height h'of the transported object 4 is defined as the height at which the transport vehicle 3 comes into contact with the transported object 4.
  • the height at which the transport vehicle 3 contacts the transported object 4 is a value known by the standard of the transport vehicle 3.
  • the circumscribed frame R on the upper surface of the conveyed object 4 is specified in the photographed image.
  • the circumscribed frame R specifies the circumscribed frame R as the first specific region R1.
  • the first specific region R1 in the first pattern is a region presumed to indicate the upper surface of the transported object 4.
  • the surveying unit 14 acquires a distance image corresponding to the captured image used to identify the circumscribed frame R of the transported object 4 from the storage unit or the like.
  • the surveying unit 14 acquires the height information z (height information of the first specific region R1) of the feature points P11, P12, P13, and P14 of the circumscribed frame R in the distance image.
  • the surveying unit 14 sets the position x1 of the feature points P11, P12, P13, and P14 specified in the circumscribed frame R in the equation (5), the height h'to h in the equation (5), and the height z to the equation (5).
  • Z and the points x2 in the captured image corresponding to the corresponding points P21, P22, P23, and P24 constituting the region of the transported object 4 at the height h'are calculated.
  • the surveying unit 14 identifies four vertices as feature points P11, P12, P13, and P14, and has a height h'corresponding to feature points P11, P12, P13, and P14.
  • Corresponding points P21, P22, P23, P24 of are calculated using the equation (5).
  • the heights of the feature points P11, P12, P13, and P14 are obtained from the distance information of each pixel of the circumscribed frame R.
  • the heights of the corresponding points P21, P22, P23, and P24 are the heights h'where the predetermined transport vehicle 3 comes into contact with the transport object 4.
  • the surveying unit 14 uses the region connecting the corresponding points P21, P22, P23, and P24 as the second specific region R2 showing the region of the transported object 4 at the height h'when the transport vehicle 3 contacts the transported object 4 in the photographed image. calculate.
  • the surveying unit 14 identifies a plurality of feature points of the circumscribing frame R based on the circumscribing frame R having a shape other than the rectangular shape, and secondly identifies a region connecting the corresponding points corresponding to the plurality of feature points. It may be calculated as region R2.
  • the surveying unit 14 outputs the information of the second specific region R2 showing the region of the transported object 4 at the height h'where the transport vehicle 3 contacts the transported object 4 in the photographed image to the contact position specifying unit 15.
  • the surveying unit 14 determines that the position of the circumscribed frame R of the transported object 4 is the second pattern, the circumscribed frame R includes the upper surface and the side surface of the transported object 4. Therefore, the surveying unit 14 specifies the first specific region R1 indicating the upper surface of the transported object 4 included in the region indicated by the circumscribed frame R as follows.
  • FIG. 12 is a first diagram showing the relationship between the circumscribed frame R and the specific region in the second pattern.
  • the surveying unit 14 specifies the position of the feature point in the circumscribed frame R by a predetermined method based on the shape of the circumscribed frame R and the pattern corresponding to the position of the circumscribed frame R. For example, when the circumscribed frame R is rectangular and the position of the circumscribed frame R is the second pattern, the four vertices P11, P12, P13, and P14 of the circumscribed frame R are specified as feature points.
  • the surveying unit 14 has a distance h (the height of the feature point P13 in the real space and the sensor) indicating the height information of the feature point P13 closest to the center of the captured image among the feature points P11, P12, P13, and P14 specified in the external frame R.
  • the distance from the height of 2) is acquired from the distance image, the background image, or the storage unit.
  • the information in the height direction indicated by the pixels other than the region of the conveyed object 4 or the conveyed vehicle 3 in the distance image or the background image indicates the vertical distance between the height of the floor surface and the height of the sensor 2.
  • the measuring unit 14 sets the distance z, which is the difference between the height of the feature point P11 farthest from the center of the captured image and the height of the sensor 2, among the feature points P11, P12, P13, and P14 specified in the circumscribed frame R. Is acquired from the distance image corresponding to the specified captured image.
  • the distance z is also information indicating the height of each point on the upper surface of the conveyed object 4 including the points in the real space corresponding to the feature points P11.
  • the surveying unit 14 sets the feature point P13 specified in the circumscribed frame R in the captured image as x2 of the equation (5), and sets the height information of the feature point P13 in the real space as h of the equation (5).
  • the height information of the unknown corresponding point P23 on the upper surface corresponding to the horizontal position of the feature point P13 in the real space (corresponding to the information indicating the height of the feature point P11) is set to z in the equation (5), and the photograph is taken.
  • the unknown corresponding point P23 (corresponding to x1 of the equation (5)) in the image is calculated.
  • the surveying unit 14 sets the coordinates of the feature point P12 as x2 of the equation (5), sets the height information of the feature point P12 in the real space as h of the equation (5), and places the feature in the real space.
  • the coordinates of the point P20 (corresponding to x1 of the equation (5)) of the height z corresponding to the horizontal position of the point P12 are obtained, and the line segment connecting P20 and P23 and the line segment connecting P11 and P12 are obtained.
  • the coordinates of the point P22 are calculated as the intersection of the points.
  • the remaining one point of the rectangle stretched by the points P11, P22, and P23 is obtained as the point P24.
  • the surveying unit 14 includes a corresponding point P23 in a captured image indicating a point corresponding to the feature point P13 and a point on the upper surface corresponding to the horizontal position in the real space, and the external frame R connecting the feature points P12, P13, and P14.
  • the intersections (P22, P24) connecting the straight line parallel to the side (the side connecting P12 and P13, the side connecting P13 and P14) and the circumscribing frame are specified as corresponding points.
  • the surveying unit 14 identifies the rectangular region connecting the corresponding points 22, 23, 24 and the feature point P11 indicating the upper surface as the first specific region R1 presumed to indicate the upper surface of the conveyed object 4 in the second pattern. do.
  • the surveying unit sets P11, P22, P23, and P24 as feature points (first corresponding points) of the first specific region R1.
  • FIG. 13 is a second diagram showing the relationship between the external frame R and the specific region in the second pattern.
  • the surveying unit 14 calculates the corresponding points P31, P32, P33, and P34 at the height h'where the transport vehicle 3 comes into contact with the transport object 4, and specifies the second specific region.
  • the surveying unit 14 acquires a distance image corresponding to the captured image used for specifying the circumscribed frame R of the transported object 4 from the storage unit or the like.
  • the surveying unit 14 acquires the height information z of the feature point P11 of the first specific region R1 in the distance image.
  • the surveying unit 14 sets the position of the captured image corresponding to the feature point P11 specified in the first specific region R1 to x1 in the equation (5), and sets the height information of the feature point P11 indicating the upper surface of the transported object 4 as x1.
  • the z of the formula (5) is set, the height h'of the transport vehicle 3 in contact with the transported object 4 is set to h of the formula (5), and they are input to the formula (5).
  • the surveying unit 14 can calculate the position x2 in the captured image corresponding to the corresponding point P31 whose horizontal position corresponds to the feature point P11 in the real space.
  • the surveying unit 14 similarly calculates the corresponding points P32, P33, and P34 of the height h'corresponding to the feature points P22, P23, and P24 when the transport vehicle 3 contacts the transported object 4, using the equation (5). do.
  • the surveying unit 14 shows the region connecting the corresponding points P31, P32, P33, and P34 (second corresponding point) in the photographed image, and shows the region of the transported object 4 at the height h'when the transport vehicle 3 contacts the transported object 4. Calculated as the second specific region R2.
  • the surveying unit 14 identifies a plurality of feature points of the circumscribing frame R based on the circumscribing frame R having a shape other than the rectangular shape, and secondly identifies a region connecting the corresponding points corresponding to the plurality of feature points. It may be calculated as region R2.
  • the surveying unit 14 outputs the information of the second specific region R2 showing the region of the transported object 4 at the height h'where the transport vehicle 3 contacts the transported object 4 in the photographed image to the contact position specifying unit 15.
  • the surveying unit 14 determines that the position of the circumscribed frame R of the transported object 4 is the third pattern, the circumscribed frame R includes the upper surface and the side surface of the transported object 4. Therefore, the surveying unit 14 specifies the first specific region R1 indicating the upper surface of the transported object 4 included in the region indicated by the circumscribed frame R as follows.
  • FIG. 14 is a first diagram showing the relationship between the circumscribed frame R and the specific region in the third pattern.
  • the surveying unit 14 specifies the position of the feature point in the circumscribed frame R by a predetermined method based on the shape of the circumscribed frame R and the pattern corresponding to the position of the circumscribed frame R. For example, when the circumscribed frame R is rectangular and the position of the circumscribed frame R is the third pattern, the four vertices P11, P12, P13, and P14 of the circumscribed frame R are specified as feature points.
  • the surveying unit 14 indicates a distance h (real space of the feature point P13 or P14) indicating the height information of the feature point P13 or P14 closest to the center of the captured image among the feature points P11, P12, P13, and P14 specified in the external frame R.
  • the distance between the height of the sensor 2 and the height of the sensor 2) is acquired from a distance image, a background image, or a storage unit.
  • the information in the height direction indicated by the pixels other than the region of the conveyed object 4 or the conveyed vehicle 3 in the distance image or the background image indicates the vertical distance between the height of the floor surface and the height of the sensor 2.
  • the surveying unit 14 externally attaches a distance z, which is the difference between the height of the feature point P11 or P12 farthest from the center of the captured image and the height of the sensor 2, among the feature points P11, P12, P13, and P14 specified in the external frame R.
  • the frame R is acquired from the distance image corresponding to the specified captured image.
  • the distance z is also information indicating the height of each point on the upper surface of the conveyed object 4 including the points in the real space corresponding to the feature points P11 or P12.
  • the surveying unit 14 sets the feature point P13 specified in the circumscribed frame R in the captured image as x2 of the equation (5), and sets the height information of the feature point P13 in the real space as h of the equation (5).
  • the height information of the unknown corresponding point P23 on the upper surface corresponding to the horizontal position of the feature point P13 in the real space (corresponding to the information indicating the height of the feature point P11) is set to z in the equation (5), and the photograph is taken.
  • the unknown corresponding point P23 (corresponding to x1 of the equation (5)) in the image is calculated.
  • the surveying unit 14 sets the feature point P14 specified in the circumscribed frame R in the captured image as x2 of the equation (5), and sets the height information of the feature point P14 in the real space as h of the equation (5).
  • the height information of the unknown corresponding point P24 on the upper surface corresponding to the horizontal position of the feature point P14 in the real space (corresponding to the information indicating the height of the feature point P11) is set to z in the equation (5).
  • An unknown corresponding point P24 (corresponding to x1 of the equation (5)) in the captured image is calculated.
  • the surveying unit 14 identifies the corresponding points P23'and P24' at which the straight lines of the corresponding points P23 and the corresponding points P24 intersect the circumscribed frame R as the corresponding points in the captured image corresponding to the points on the upper surface of the conveyed object 4.
  • the surveying unit 14 defines a rectangular region connecting the corresponding points 23'and 24'and the feature points P11 and P12 indicating the upper surface as the first specific region R1 presumed to indicate the upper surface of the conveyed object 4 in the third pattern. To specify.
  • the surveying unit sets P11, P12, P23', and P24'as feature points (first corresponding points) of the first specific region R1.
  • the surveying unit 14 calculates the corresponding points P31, P32, P33, and P34 at the height h'where the transport vehicle 3 comes into contact with the transport object 4, and specifies the second specific region.
  • the surveying unit 14 acquires a distance image corresponding to the captured image used for specifying the circumscribed frame R of the transported object 4 from the storage unit or the like.
  • the surveying unit 14 acquires the height information z of the feature point P11 of the first specific region R1 in the distance image.
  • the surveying unit 14 sets the position of the captured image corresponding to the feature point P11 specified in the first specific region R1 to x1 in the equation (5), and sets the height information of the feature point P11 indicating the upper surface of the transported object 4 as x1.
  • the z of the formula (5) is set, the height h'of the transport vehicle 3 in contact with the transported object 4 is set to h of the formula (5), and they are input to the formula (5).
  • the surveying unit 14 can calculate the position x2 in the captured image corresponding to the corresponding point P31 whose horizontal position corresponds to the feature point P11 in the real space.
  • the surveying unit 14 similarly uses the equation (5) for the corresponding points P32, P33, and P34 of the height h'corresponding to the feature points P12, P23', and P24', in which the transport vehicle 3 contacts the transported object 4. To calculate.
  • the surveying unit 14 shows the region connecting the corresponding points P31, P32, P33, and P34 (second corresponding point) in the photographed image, and shows the region of the transported object 4 at the height h'when the transport vehicle 3 contacts the transported object 4. Calculated as the second specific region R2.
  • the surveying unit 14 identifies a plurality of feature points of the circumscribing frame R based on the circumscribing frame R having a shape other than the rectangular shape, and secondly identifies a region connecting the corresponding points corresponding to the plurality of feature points. It may be calculated as region R2.
  • the surveying unit 14 outputs the information of the second specific region R2 showing the region of the transported object 4 at the height h'where the transport vehicle 3 contacts the transported object 4 in the photographed image to the contact position specifying unit 15.
  • the surveying unit 14 determines that the position of the circumscribed frame R of the transported object 4 is the third pattern, the circumscribed frame R includes the upper surface and the side surface of the transported object 4. Therefore, the surveying unit 14 specifies the first specific region R1 indicating the upper surface of the transported object 4 included in the region indicated by the circumscribed frame R as follows.
  • FIG. 15 is a second diagram showing the relationship between the circumscribed frame R and the specific region in the third pattern.
  • the surveying unit 14 specifies the position of the feature point in the circumscribed frame R by a predetermined method based on the shape of the circumscribed frame R and the pattern corresponding to the position of the circumscribed frame R. For example, when the circumscribed frame R is rectangular and the position of the circumscribed frame R is the third pattern, the four vertices P11, P12, P13, and P14 of the circumscribed frame R are specified as feature points.
  • the surveying unit 14 indicates a distance h (real space of the feature point P12 or P13) indicating the height information of the feature point P12 or P13 closest to the center of the captured image among the feature points P11, P12, P13, and P14 specified in the external frame R.
  • the distance between the height of the sensor 2 and the height of the sensor 2) is acquired from a distance image, a background image, or a storage unit.
  • the information in the height direction indicated by the pixels other than the region of the conveyed object 4 or the conveyed vehicle 3 in the distance image or the background image indicates the vertical distance between the height of the floor surface and the height of the sensor 2.
  • the surveying unit 14 circumscribes the distance z, which is the difference between the height of the feature points P11 or P14 farthest from the center of the captured image and the height of the sensor 2 among the feature points P11, P12, P13, and P14 specified in the circumscribed frame R.
  • the frame R is acquired from the distance image corresponding to the specified photographed image.
  • the distance z is also information indicating the height of each point on the upper surface of the conveyed object 4 including the points in the real space corresponding to the feature points P11 or P14.
  • the surveying unit 14 sets the feature point P12 specified in the circumscribed frame R in the captured image as x2 of the equation (5), and sets the height information of the feature point P12 in the real space as h of the equation (5).
  • the height information of the unknown corresponding point P24 on the upper surface corresponding to the horizontal position of the feature point P14 in the real space (corresponding to the information indicating the height of the feature point P11) is set to z in the equation (5), and the photograph is taken.
  • the unknown corresponding point P22 (corresponding to x1 of the equation (5)) in the image is calculated.
  • the surveying unit 14 sets the feature point P13 specified in the circumscribed frame R in the captured image as x2 of the equation (5), and sets the height information of the feature point P13 in the real space as h of the equation (5).
  • the height information of the unknown corresponding point P23 on the upper surface corresponding to the horizontal position of the feature point P13 in the real space (corresponding to the information indicating the height of the feature point P11) is set to z in the equation (5).
  • An unknown corresponding point P23 (corresponding to x1 of the equation (5)) in the captured image is calculated.
  • the surveying unit 14 identifies the corresponding points P22'and P23' at which the straight lines of the corresponding points P22 and the corresponding points P23 intersect the circumscribed frame R as the corresponding points in the captured image corresponding to the points on the upper surface of the conveyed object 4.
  • the surveying unit 14 defines a rectangular region connecting the corresponding points P22'and P23'and the feature points P11 and P14 indicating the upper surface as the first specific region R1 presumed to indicate the upper surface of the conveyed object 4 in the third pattern. To specify.
  • the surveying unit 14 sets P11, P22', P23', and P14 as feature points (first corresponding points) of the first specific region R1.
  • the surveying unit 14 acquires a distance image corresponding to the captured image used to identify the circumscribed frame R of the transported object 4 from the storage unit or the like.
  • the surveying unit 14 acquires the height information z of the feature point P11 of the first specific region R1 in the distance image.
  • the surveying unit 14 sets the position of the captured image corresponding to the feature point P11 specified in the first specific region R1 to x1 in the equation (5), and sets the height information of the feature point P11 indicating the upper surface of the transported object 4 as x1.
  • the z of the formula (5) is set, the height at which the transport vehicle 3 contacts the transported object 4 is set to h of the formula (5), and they are input to the formula (5).
  • the surveying unit 14 can calculate the position x2 in the captured image corresponding to the corresponding point P31 whose horizontal position corresponds to the feature point P11 in the real space.
  • the surveying unit 14 similarly uses the equation (5) for the corresponding points P32, P33, and P34 of the height h'corresponding to the feature points P22', P23', and P14, where the transport vehicle 3 contacts the transported object 4. To calculate.
  • the surveying unit 14 shows the region connecting the corresponding points P31, P32, P33, and P34 (second corresponding point) in the photographed image, and shows the region of the transported object 4 at the height h'when the transport vehicle 3 contacts the transported object 4. Calculated as the second specific region R2.
  • the surveying unit 14 identifies a plurality of feature points of the circumscribing frame R based on the circumscribing frame R having a shape other than the rectangular shape, and secondly identifies a region connecting the corresponding points corresponding to the plurality of feature points. It may be calculated as region R2.
  • the surveying unit 14 outputs the information of the second specific region R2 showing the region of the transported object 4 at the height h'where the transport vehicle 3 contacts the transported object 4 in the photographed image to the contact position specifying unit 15.
  • the processing of the first specific region R1 of the surveying unit 14 in the second pattern and the third pattern described above is the relationship between the four feature points in the circumscribed frame R (object inclusion region) and the height information of the four feature points.
  • the four first correspondence points at the height of the upper surface are calculated based on the above, and the first specification indicating the area of the object corresponding to the height of the upper surface of the specific regions based on the four first correspondence points. This is one aspect of the process of specifying the region.
  • the horizontal coordinates of the first corresponding point at the height of the contact position of the object, which is the second predetermined height, and the coordinates of the height indicate the coordinates of the height of the contact position.
  • four second corresponding points at the height of the contact position are calculated, and among the specific regions based on the four second corresponding points. It is one aspect of the process of specifying the second specific area indicating the area of the object corresponding to the height of the contact position of.
  • FIG. 16 is a schematic configuration diagram of a control system including the control device according to the second embodiment.
  • the conveyed object 4 is placed and moved on the belt conveyor 5 provided in the control system, and the control device included in the control system identifies the contact position with respect to the conveyed object 4, and the robot 6 provided in the control system.
  • the control may be performed based on the contact position of the conveyed object 4.
  • the control device 1 obtains the image information including the conveyed object 4 based on the change of the image information including the conveyed object 4 and the image information including the conveyed object 4.
  • the circumscribed frame R object inclusion area indicating the area where the pixel information including the conveyed object 4 has changed is specified.
  • the control device 1 identifies the circumscribed frame R of the conveyed object 4, the first specific area R1 (the area on the upper surface), and the second specific area R2 (the area at the height of the contact position).
  • the contact position is specified based on the second specific area R2.
  • the contact position of the conveyed object 4 of the robot 6 is such that the traveling direction of the belt conveyor 5 and the opposite sides formed by a small angle between the rectangular sides are opposed to each other. It may be specified, and the center of each of the two sides may be specified as the contact position.
  • the control device 1 may control the robot 6 to grasp its contact position.
  • the contact position of the transported object 4 is specified in the above-described embodiment, the contact position of another object may be specified.
  • the other object is a welding object or a decorative object, and the control device 1 specifies the contact position (welding position) of the welding object and the contact position (decorative position) of the decorative object. May be good.
  • the object surveying system includes a surveying device that is communicatively connected to a sensor 2 that acquires height information of an object and image information of the object. Similar to each of the above-described embodiments, the surveying device is a region in which the pixel information including the object is changed in the image information including the object based on the change in the image information not including the object and the image information including the object.
  • the object inclusion area indicating the above is specified, and the specific area at a predetermined height of the object included in the object inclusion area is specified based on the object inclusion area and the height information.
  • FIG. 17 is a schematic configuration of a transport system according to a third embodiment.
  • the transport system includes a control device 1, a sensor 2, a transport vehicle 3, and a transport 4.
  • the control device 1 is communicably connected to each of the sensor 2 and the transport vehicle 3 via a network.
  • FIG. 18 is a schematic configuration diagram of the control device 1 according to the third embodiment.
  • the control device 1 controls the transport vehicle 3.
  • the control device 1 is an information processing device such as a computer.
  • the control device 1 may be realized by cloud computing.
  • the control device 1 includes a contact position specifying unit 151 and a control unit 152.
  • the contact position specifying unit 151 specifies the contact position with respect to the transported object 4 that the transport vehicle 3 contacts when transporting the transported object 4 based on the information regarding the transported object 4.
  • the control unit 152 controls the transport vehicle 3 based on the contact position of the transport object 4.
  • the sensor acquires information about the transported object 4.
  • FIG. 19 is a diagram showing a processing flow by the control device 1 according to the third embodiment.
  • the contact position specifying unit 151 specifies the contact position of the transported object 4 by the transport vehicle 3 based on the information regarding the transported object 4 (step S1611).
  • the control unit 152 controls the transport vehicle 3 based on the contact position of the transport object 4 (step S1612).
  • the above-mentioned control device 1 has a computer system inside.
  • the process of each process described above is stored in a computer-readable recording medium in the form of a program, and the process is performed by the computer reading and executing this program.
  • the computer-readable recording medium refers to a magnetic disk, a magneto-optical disk, a CD-ROM, a DVD-ROM, a semiconductor memory, or the like.
  • this computer program may be distributed to a computer via a communication line, and the computer receiving the distribution may execute the program.
  • the above program may be for realizing a part of the above-mentioned functions.
  • a so-called difference file difference program
  • difference program difference program
  • a transport system including a transport vehicle for transporting a transported object, a sensor for acquiring information on the transported object, and a control device for controlling the transported vehicle. Based on the information about the transported object, the contact position with respect to the transported object that the transport vehicle comes into contact with when transporting the transported object is specified. A transport system that controls the transport vehicle based on the contact position.
  • Appendix 2 The transport system according to Appendix 1, wherein the information regarding the transported object includes image information obtained by photographing the transported object and height information of the transported object.
  • Appendix 3 The transport system according to Appendix 1, wherein the control device controls the transport vehicle based on a contact position indicating the center of a side surface of the transport object.
  • the sensor measures the image information and the distance information of each position of the conveyed object, and measures the image information.
  • Appendix 6 The transport system according to Appendix 5, wherein the control device identifies a side surface for sandwiching the transported object by the plurality of transport vehicles, and controls each of the plurality of transport vehicles based on a contact position corresponding to the side surface. ..
  • a communication connection is made between the transport vehicle that transports the transported object and the sensor that acquires information about the transported object. Based on the information about the transported object, the contact position with respect to the transported object that the transport vehicle comes into contact with when transporting the transported object is specified.
  • a control device that controls the transport vehicle based on the contact position.
  • Appendix 8 The control device according to Appendix 7, wherein the information regarding the transported object includes image information obtained by photographing the transported object and height information of the transported object.
  • Appendix 9 The control device according to Appendix 7, which controls the transport vehicle based on a contact position indicating the center of a side surface of the transport object.
  • the sensor measures the image information and the distance information of each position of the conveyed object, and measures the image information.
  • the control device which specifies the contact position based on the image information and the distance information.
  • Appendix 12 The control device according to Appendix 11, wherein a side surface for sandwiching the transported object is specified by the plurality of transport vehicles, and each of the plurality of transport vehicles is controlled based on a contact position corresponding to the side surface.
  • a communication connection is made between the transport vehicle that transports the transported object and the sensor that acquires information about the transported object. Based on the information about the transported object, the contact position with respect to the transported object that the transport vehicle comes into contact with when transporting the transported object is specified. A control method for controlling the transport vehicle based on the contact position.
  • Appendix 14 The control method according to Appendix 13, wherein the information regarding the transported object includes image information obtained by photographing the transported object and height information of the transported object.
  • Appendix 15 The control method according to Appendix 13 for controlling the transport vehicle based on a contact position indicating the center of a side surface of the transport object.
  • the sensor measures the image information and the distance information of each position of the conveyed object, and measures the image information.
  • Appendix 18 The control method according to Appendix 17, wherein a side surface for sandwiching the transported object is specified by the plurality of transport vehicles, and each of the plurality of transport vehicles is controlled based on a contact position corresponding to the side surface.
  • a control system including a robot that comes into contact with an object, a sensor that acquires information about the object, and a control device that controls the robot. Based on the information about the object, the robot identifies the contact position with respect to the object in contact with the object, and determines the contact position with respect to the object. A control system that controls the robot based on the contact position.
  • Appendix 20 Communicate and connect to a robot that comes into contact with an object and a sensor that acquires information about the object. Based on the information about the object, the robot identifies the contact position with respect to the object in contact with the object, and determines the contact position with respect to the object. A control device that controls the robot based on the contact position.
  • a control device that communicates and connects to a robot that comes into contact with an object and a sensor that acquires information about the object. Based on the information about the object, the robot identifies the contact position with respect to the object in contact with the object, and determines the contact position with respect to the object. A control method for controlling the robot based on the contact position.
  • a surveying device that communicates and connects to a sensor that acquires the height information of the object and the image information of the object An object inclusion area indicating a region in which the pixel information including the object is changed in the image information including the object based on the change between the image information not including the object and the image information including the object. Identify and An object surveying system that identifies a specific region at a predetermined height of the object included in the object inclusion region based on the object inclusion region and the height information.
  • the surveying device is The relationship between the feature point in the object inclusion region and the height information of the feature point, and the corresponding point at the predetermined height corresponding to the feature point and the horizontal position and the height information indicated by the predetermined height.
  • the surveying device is The relationship between the plurality of feature points in the object inclusion region and the height information of the plurality of feature points, and the plurality of corresponding points and the predetermined heights at the predetermined heights whose horizontal positions correspond to the feature points
  • the object survey system according to Appendix 22 which calculates a plurality of the corresponding points at the predetermined height based on the relationship with the indicated height information, and specifies the specific area based on the plurality of corresponding points.
  • the surveying device is The relationship between the plurality of feature points in the object inclusion region and the height information of the plurality of feature points, and the horizontal position at the feature points at the height of the upper surface of the object, which is the first predetermined height. Based on the relationship between the plurality of corresponding first correspondence points and the height information indicated by the height of the upper surface, the plurality of first correspondence points at the height of the upper surface are calculated, and the plurality of first correspondence points are calculated.
  • the first specific region indicating the region of the object corresponding to the height of the upper surface of the specific region is specified based on the above.
  • the relationship between the height information of the plurality of first correspondence points and the plurality of first correspondence points in the first specific region, and the second height of the contact position of the object, which is the second predetermined height, is the first.
  • a plurality of the second corresponding points at the height of the contact position are calculated based on the relationship between the plurality of second corresponding points whose horizontal positions correspond to one corresponding point and the height information indicated by the height of the contact position.
  • a communication connection is made to a sensor that acquires the height information of the object and the image information of the object.
  • An object inclusion area indicating a region in which the pixel information including the object is changed in the image information including the object based on the change between the image information not including the object and the image information including the object.
  • Identify and A surveying device that identifies a specific region at a predetermined height of the object included in the object inclusion region based on the object inclusion region and the height information.
  • a surveying device that communicates and connects to a sensor that acquires the height information of the object and the image information of the object An object inclusion area indicating a region in which the pixel information including the object is changed in the image information including the object based on the change between the image information not including the object and the image information including the object. Identify and An object surveying method for specifying a specific region at a predetermined height of the object included in the object inclusion region based on the object inclusion region and the height information.
  • Control device Surveying device 2 ... Sensor 3 ... Transport vehicle 4 ... Transport (object) 5 ... Belt conveyor 6 ... Robot 11 ... Image information acquisition unit 12 ... Distance information acquisition unit 13 ... Difference detection unit 14 ... Survey unit 15 ... Contact position identification unit 16 ... ⁇ ⁇ Conveyor control unit 17 ⁇ ⁇ ⁇ Display unit 100 ⁇ ⁇ ⁇ Conveyor system (object survey system) 151 ... Contact position specifying unit 152 ... Control unit 200 ... Control system (object surveying system)

Landscapes

  • Engineering & Computer Science (AREA)
  • Transportation (AREA)
  • Structural Engineering (AREA)
  • Mechanical Engineering (AREA)
  • Civil Engineering (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Geology (AREA)
  • Radar, Positioning & Navigation (AREA)
  • Aviation & Aerospace Engineering (AREA)
  • Remote Sensing (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Automation & Control Theory (AREA)
  • Control Of Position, Course, Altitude, Or Attitude Of Moving Bodies (AREA)
  • Warehouses Or Storage Devices (AREA)
  • Image Analysis (AREA)
PCT/JP2020/008528 2020-02-28 2020-02-28 搬送システム、制御装置、制御方法 Ceased WO2021171618A1 (ja)

Priority Applications (4)

Application Number Priority Date Filing Date Title
PCT/JP2020/008528 WO2021171618A1 (ja) 2020-02-28 2020-02-28 搬送システム、制御装置、制御方法
US17/800,413 US12103786B2 (en) 2020-02-28 2020-02-28 Conveyance system, control device, and control method
JP2022503056A JP7380829B2 (ja) 2020-02-28 2020-02-28 搬送システム、制御装置、制御方法
JP2023188751A JP7548401B2 (ja) 2020-02-28 2023-11-02 搬送システム、搬送制御装置、搬送方法

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/JP2020/008528 WO2021171618A1 (ja) 2020-02-28 2020-02-28 搬送システム、制御装置、制御方法

Publications (1)

Publication Number Publication Date
WO2021171618A1 true WO2021171618A1 (ja) 2021-09-02

Family

ID=77490857

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2020/008528 Ceased WO2021171618A1 (ja) 2020-02-28 2020-02-28 搬送システム、制御装置、制御方法

Country Status (3)

Country Link
US (1) US12103786B2 (https=)
JP (2) JP7380829B2 (https=)
WO (1) WO2021171618A1 (https=)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2023150646A (ja) * 2022-03-31 2023-10-16 トヨタ自動車株式会社 自動搬送機の制御方法
WO2024190314A1 (ja) * 2023-03-15 2024-09-19 日本電気株式会社 制御システム、制御装置、制御方法及びプログラム

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP7439042B2 (ja) * 2021-12-24 2024-02-27 楽天グループ株式会社 画像処理装置、画像処理方法及びプログラム

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2009539727A (ja) * 2006-06-09 2009-11-19 キヴァ システムズ，インコーポレイテッド 在庫品目を輸送する方法及びシステム
JP2011022700A (ja) * 2009-07-14 2011-02-03 Japan Science & Technology Agency ロボット制御システム及びロボット制御プログラム
JP2018092393A (ja) * 2016-12-05 2018-06-14 株式会社ディスコ 自動搬送車コントロールシステム
WO2020022479A1 (ja) * 2018-07-27 2020-01-30 日本電気株式会社 搬送ロボット、搬送システム、及び搬送方法

Family Cites Families (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH01242392A (ja) 1988-03-18 1989-09-27 Kawasaki Steel Corp スラブ荷姿の検出方法
JP4952959B2 (ja) 2009-11-18 2012-06-13 独立行政法人科学技術振興機構 搬送システム、ロボットの制御方法
JP2014144836A (ja) 2013-01-28 2014-08-14 Mitsubishi Heavy Industries Machinery Technology Corp コンテナクレーン
US9233470B1 (en) * 2013-03-15 2016-01-12 Industrial Perception, Inc. Determining a virtual representation of an environment by projecting texture patterns
US9152149B1 (en) * 2014-06-06 2015-10-06 Amazon Technologies, Inc. Fiducial markers with a small set of values
US10583982B2 (en) * 2015-08-03 2020-03-10 Hitachi, Ltd. Shelf transport system, shelf transport vehicle, and shelf transport method
US10683171B2 (en) * 2016-09-30 2020-06-16 Staples, Inc. Hybrid modular storage fetching system
DE102016121769A1 (de) * 2016-11-14 2018-05-17 Cl Schutzrechtsverwaltungs Gmbh Anlage zur additiven Herstellung dreidimensionaler Objekte
US11797011B2 (en) * 2018-02-13 2023-10-24 Seiko Epson Corporation Traveling control system for transport vehicle and traveling control method for transport vehicle
EP3807188A1 (en) * 2018-06-12 2021-04-21 Autostore Technology As Automated storage system
JP7500484B2 (ja) * 2021-03-22 2024-06-17 株式会社東芝 無人搬送車、無人搬送システム及び搬送プログラム

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2009539727A (ja) * 2006-06-09 2009-11-19 キヴァ システムズ，インコーポレイテッド 在庫品目を輸送する方法及びシステム
JP2011022700A (ja) * 2009-07-14 2011-02-03 Japan Science & Technology Agency ロボット制御システム及びロボット制御プログラム
JP2018092393A (ja) * 2016-12-05 2018-06-14 株式会社ディスコ 自動搬送車コントロールシステム
WO2020022479A1 (ja) * 2018-07-27 2020-01-30 日本電気株式会社 搬送ロボット、搬送システム、及び搬送方法

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2023150646A (ja) * 2022-03-31 2023-10-16 トヨタ自動車株式会社 自動搬送機の制御方法
JP7693603B2 (ja) 2022-03-31 2025-06-17 トヨタ自動車株式会社 自動搬送機の制御方法
WO2024190314A1 (ja) * 2023-03-15 2024-09-19 日本電気株式会社 制御システム、制御装置、制御方法及びプログラム

Also Published As

Publication number Publication date
JP7548401B2 (ja) 2024-09-10
JPWO2021171618A1 (https=) 2021-09-02
US20230107626A1 (en) 2023-04-06
JP7380829B2 (ja) 2023-11-15
US12103786B2 (en) 2024-10-01
JP2024012512A (ja) 2024-01-30

Similar Documents

Publication Publication Date Title
JP7548401B2 (ja) 搬送システム、搬送制御装置、搬送方法
JP6649796B2 (ja) 物体状態特定方法、物体状態特定装置、および、搬送車
US9736460B2 (en) Distance measuring apparatus and distance measuring method
JP6646854B2 (ja) 投影指示装置、荷物仕分けシステムおよび投影指示方法
US20160117824A1 (en) Posture estimation method and robot
JP2004090183A (ja) 物品の位置姿勢検出装置及び物品取出し装置
JP6590153B2 (ja) 投影指示装置、荷物仕分けシステムおよび投影指示方法
JPH10143659A (ja) 物体検出装置
JP6646853B2 (ja) 投影指示装置、荷物仕分けシステムおよび投影指示方法
JP7689308B2 (ja) 測位システム
JP7486095B2 (ja) 車両監視方法、車両監視装置、車両、及び車両監視システム
JP2017171444A (ja) 投影指示装置、荷物仕分けシステムおよび投影指示方法
JP3842988B2 (ja) 両眼立体視によって物体の３次元情報を計測する画像処理装置およびその方法又は計測のプログラムを記録した記録媒体
JP7364033B2 (ja) 測量方法、測量装置、測量システム、プログラム
JP2020114778A (ja) 投影指示装置、荷物仕分けシステムおよび投影指示方法
JP7631913B2 (ja) 搬送制御システム、搬送制御方法、プログラム
JP6628039B2 (ja) 投影指示装置、荷物仕分けシステムおよび投影指示方法
WO2024190314A1 (ja) 制御システム、制御装置、制御方法及びプログラム
JP3055721B2 (ja) 左，右カメラの撮像画像の対応点検索方法
JP6628038B2 (ja) 投影指示装置、荷物仕分けシステムおよび投影指示方法
JPH07151844A (ja) ３次元追尾装置
JPS6228613A (ja) ３次元位置入力装置
JPH03165203A (ja) 移動体用距離・姿勢測定装置
Coleman et al. Image-based Vision Processing for the Smart Crane Ammunition Transfer System
JP2020146825A (ja) 制御装置、検査システム、制御方法、プログラム

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20921650

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2022503056

Country of ref document: JP

Kind code of ref document: A

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20921650

Country of ref document: EP

Kind code of ref document: A1