WO2024057800A1

WO2024057800A1 - Method for controlling mobile object, transport device, and work system

Info

Publication number: WO2024057800A1
Application number: PCT/JP2023/029395
Authority: WO
Inventors: 慎司今井; 一真 ▲高▼原
Original assignee: 株式会社島津製作所
Priority date: 2022-09-12
Filing date: 2023-08-14
Publication date: 2024-03-21

Abstract

A management device (100) performs, as a method for controlling a mobile object: a step for generating a machine learning model for generating an operation plan for a mobile object for transporting an object, on the basis of the position of a first feature portion in a simulation space; a step for identifying the position of a second feature portion in the real space; a step for acquiring the relationship between the position of the second feature portion in the real space and a position in the real space corresponding to the position of the first feature portion in the simulation space; a step for applying the position of the second feature portion in the real space and the relationship to the machine learning model, to thereby generate an operation plan for the mobile object; and a step for controlling the operation of the mobile object in accordance with the generated operation plan.

Description

Mobile object control method, transport device, and work system

The present invention relates to controlling the operation of a moving body for transporting objects.

Conventionally, moving objects such as robots have been used to transport objects. Various studies have been made on controlling the motion of such moving objects. For example, Japanese Patent No. 6457421 (Patent Document 1) discloses a mechanical system including a machine learning device for removing a workpiece from a basket by a robot. In this machine system, the machine learning device uses simulation by a simulator to train a machine learning model.

Patent No. 6457421

In controlling the operation of a moving body, a given position such as a target position is used as a reference point to control the position of the moving body. In the simulation space, the control device can recognize the positional relationship between the moving body and the reference point. On the other hand, in real space, in order for a control device to recognize the positional relationship between a moving object and a reference point, the position of a marker installed in real space or the position or pattern of a given location of an existing device is used as a reference point. used. When a marker is installed, the control device detects the marker and controls the motion of the moving body based on the detected position of the marker to control the movement of the moving body in real space.

In real space, there may be obstacles to detecting a reference point used as a position reference, such as the presence of objects other than objects and moving objects. If the object is not detected in real space, a position reference cannot be obtained, making it difficult to control the movement of the moving object.

The present invention was devised in view of the above circumstances, and its purpose is to provide a technique for reliably realizing control of the motion of a moving body using simulation results.

A method for controlling a moving body according to an aspect of the present disclosure includes a step of generating a machine learning model for generating a motion plan of the moving body for transporting a target object based on the position of the first feature in the simulation space. a step of identifying the position of the second feature in the real space; and a relationship between the position of the second feature in the real space and the position in the real space corresponding to the position of the first feature in the simulation space. and generating a motion plan for the moving object by applying the position and relationship in real space of the second feature to a machine learning model, and controlling the motion of the moving object according to the generated motion plan. and a step of doing so.

A conveying device according to an aspect of the present disclosure generates a machine learning model for generating a motion plan for the moving body based on the moving body for conveying a target object and the position of the first feature in the simulation space. a controller; a data acquisition unit that acquires data for specifying the position of the second feature in the real space; and a data acquisition unit that obtains data for specifying the position of the second feature in the real space; and a memory for storing a relationship between the position in the space and the position in the space, the controller uses the data acquired by the data acquisition unit to specify the position in the real space of the second feature, and the controller identifies the position in the real space of the second feature. A motion plan for the moving body is generated by applying the positions and relationships of the parts in real space to a machine learning model, and the moving body is operated according to the generated motion plan.

According to a certain aspect of the present disclosure, control of the motion of a moving object using simulation results is reliably realized.

FIG. 1 is a diagram for explaining an overview of a work system in an embodiment. FIG. 1 is a diagram showing an example of a hardware configuration of a work system in an embodiment. 3 is a flowchart of processing executed by the management device 100. 3 is a diagram schematically showing an example of a simulation space defined by a simulator unit 112. FIG. 3 is a flowchart of a subroutine of the transport process in step S3.

Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings. Note that the same or corresponding parts in the figures are designated by the same reference numerals, and the description thereof will not be repeated. The embodiments and modifications described below may be selectively combined as appropriate.

[Overview of work system]
FIG. 1 is a diagram for explaining an overview of a work system in an embodiment. The work system 1 in the embodiment includes a controlled device group 200 and a management device 100. The controlled device group 200 includes a transfer robot 230, a working device 220, and a working device 240.

The transport robot 230 uses the arm 232 to transport the well plate 5 from the working device 220 to the target position Tp in the working device 240. Arm 232 is an example of a "moving body" in the present disclosure. The arm 232 is configured as a part of the transfer robot 230, and the arm 232 moves as the transfer robot 230 moves. In this sense, the transport robot 230 itself can be interpreted as an example of a "moving object." The well plate 5 accommodates samples used by the working device 220 and the working device 240. The well plate 5 containing the sample is an example of the "object" in the present disclosure. Each of the work device 220 and the work device 240 is an example of a device for work using a sample transported by the transport robot 230. The working device 220 is a rack that holds the well plate 5. The working device 240 is a dispensing device for dispensing the sample accommodated in the well plate 5.

In this embodiment, a system for a bacterial culture experiment is adopted as an example of the work system 1, an experiment is adopted as an example of work on a target object, and an apparatus for experiment is adopted as the work device. . Note that the work on the object is not limited to experiments. Any type of work, such as metal processing, can be employed as the work on the object. Therefore, as the working device, any type of device that performs work on a target object can be employed, in addition to the device for experiments.

The management device 100 controls the operations of the transport robot 230, the work device 220, and the work device 240. Management device 100 is an example of a controller. The transport device 300 is a device that includes a management device 100 and a transport robot 230.

The transfer robot 230 is a six-axis vertically articulated robot with one arm. More specifically, the transfer robot 230 includes a main body 231, an arm 232, a gripper 233, and an imaging section 234. The main body portion 231 holds an arm 232. The arm 232 transports the well plate 5 from the working device 220 to the target position Tp in the working device 240. The gripper 233 is provided at the tip of the arm 232 and grips the well plate 5. Arm 232 includes one or more movable parts 8. By moving one or more movable parts 8, the arm 232 can move up and down, left and right, and back and forth. Thereby, the well plate 5 is transported by the arm 232 from the working device 220 to the target position Tp in the working device 240.

In the example of FIG. 1, the controlled device group 200 includes a plurality of work devices, and the transport robot 230 transports an object (well plate) from one work device among the plurality of work devices to another work device. do. Note that the controlled device group 200 may be composed of a single working device, and the transport robot 230 may transport the object from one position to another within the single working device. .

The imaging unit 234 is provided on the arm 232 and acquires an image of the imaging target. The operations of the arm 232, the gripper 233, and the imaging section 234 are controlled by the management device 100.

In the work system 1, the mark Tm indicates the target position Tp. Furthermore, marks Rm1, Rm2, and Rm3 are pasted on objects around the transport robot 230. The mark Rm1 indicates the reference position Rp1, the mark Rm2 indicates the reference position Rp2, and the mark Rm3 indicates the reference position Rp3. Each of the reference positions Rp1, Rp2, and Rp3 is located at a different location from the target position Tp.

In the following description, mark Rm1, mark Rm2, and mark Rm3 will be referred to as "mark Rm" when attention is focused on their common property (indicating a reference position) and they do not need to be distinguished from each other. Ru. The mark Rm may have any pattern as long as it can identify the posture of the mark Rm, such as an AR marker or a QR code (registered trademark). In this embodiment, "posture" includes position (relative position of the marker with respect to "imaging unit 234" described later) and orientation. Mark Rm1, mark Rm2, and mark Rm3 are distinguishable from each other. For example, when mark Rm1, mark Rm2, and mark Rm3 are realized as QR codes, they are realized as QR codes having mutually different information.

The reference position Rp is located at a different location from the target position Tp. In the following explanation, the reference position Rp1, the reference position Rp2, and the reference position Rp3 will be focused on the common property (that they are reference positions), and if they do not need to be distinguished from each other, they will be referred to as "reference position Rp". It is called.

In this embodiment, the target position Tp is an example of the first characteristic part, and the reference position Rp is an example of the second characteristic part.

[Hardware configuration of work system]
FIG. 2 is a diagram showing an example of the hardware configuration of the work system in the embodiment. The work system 1 includes a transport device 300, a work device 220, and a work device 240. The transport device 300 includes a transport robot 230 and a management device 100.

The management device 100 is configured by, for example, a general-purpose computer. In the example of FIG. 2, the management device 100 includes a processor 101, a memory 102, a storage 103, an interface 104, a display 105, and an input device 106.

The processor 101 executes various programs in order for the management device 100 to perform various processes. The processor 101 is composed of hardware elements such as a CPU (Central Processing Unit) and an MPU (Micro-Processing Unit).

The processor 101 functions as a machine learning section 111, a simulator section 112, an information generation section 113, and a transport control section 114 by executing a given program.

The machine learning unit 111 performs processing for machine learning of a model for generating a motion plan for the arm 232. In one implementation, the motion plan for arm 232 includes information that defines motion (eg, distance and/or speed of movement in each of three axes) of arm 232. Note that the operation plan of the arm 232 is the operation of elements other than the arm 232 (the gripper 233, the motor for rotating the wheels attached to the main body 231, etc.), and the operation plan of the arm 232 for conveying the object. may include information specifying actions that contribute to the actions of the user.

The simulator unit 112 performs a simulation of the operation of the arm 232 in the simulation space. In one implementation example, the simulator unit 112 generates time series data of the position of the arm 232. The information generation unit 113 generates "positional relationship information" which will be described later. The transport control unit 114 controls the operation of the transport robot 230 for transporting the well plate 5.

The memory 102 functions as a main storage device, and is configured with a volatile storage device such as a DRAM (Dynamic Random Access Memory) or an SRAM (Static Random Access Memory).

The storage 103 stores programs executed by the processor 101 and various data necessary for executing the programs, and is configured with a nonvolatile storage device such as an SSD (Solid State Drive) and/or a flash memory.

Note that the program may be provided not as a standalone program but as a part of any program. In this case, the processing according to this embodiment is realized in cooperation with an arbitrary program. Even if the program does not include such a part of the module, it does not depart from the spirit of the management device 100 according to the present embodiment. Further, some or all of the functions provided by the program may be realized by dedicated hardware.

The data stored in the storage 103 includes a machine learning model 131. Machine learning model 131 includes data (for example, values of one or more parameters that are learning results of the machine learning model) that constitute a machine learning model for generating a motion plan for arm 232. The data stored in the storage 103 further includes positional relationship information 132. The positional relationship information 132 indicates the positional relationship between two or more marks in real space.

The interface 104 relays communication between the management device 100 and external devices (for example, the transfer robot 230, the work device 220, the work device 240, etc.).

The display 105 displays the results of the arithmetic processing by the processor 101, and the input device 106 (eg, mouse, keyboard, touch sensor, etc.) accepts data input operations to the processor 101.

The transport robot 230 includes an imaging section 234, an interface 235, a motor unit 236, and a driver unit 237. The interface 235 relays communication between the transport robot 230 and the management device 100. Motor unit 236 includes a motor associated with each of one or more movable parts 8. Driver unit 237 includes a driver that drives each of the plurality of motors included in motor unit 236.

[Processing flow]
FIG. 3 is a flowchart of processing executed by the management device 100. In one implementation, the process of FIG. 3 is accomplished by processor 101 executing a given program.

As shown in FIG. 3, the management device 100 performs a learning process for the machine learning model 131 in step S1, performs a positional relationship information generation process (generates positional relationship information 132) in step S2, Then, in step S3, a transport process for controlling the transport operation of the well plate 5 by the arm 232 is performed. The contents of each step will be explained in detail below.

(1) Step S1: Learning Process In an example of the learning process, a reinforcement learning algorithm is used. In this example, machine learning model 131 is trained according to a reinforcement learning algorithm.

In reinforcement learning, an agent selects various actions a under a certain state s, and is rewarded for the action a at that time. Thereby, the machine learning model 131 is trained so that the agent learns a better action choice, ie, the correct value Q(s,a).

In this embodiment, arm 232 is employed as the agent.
The motion plan of arm 232 is adopted as state s. An example of data constituting the operation plan is the position where the transport robot 230 starts gripping the well plate 5, the movement path of the arm 232, and/or the position where the transport robot 230 ends gripping the well plate 5 (well plate 5 mounting positions).

As a reward, the result of transporting the well plate 5 (success/failure) is adopted. In one implementation example, if the position of the well plate 5 after the arm 232 operates according to the operation plan (the placement position of the well plate 5) is within a given range, the machine learning unit 111 5. Specify "success" as the result of transport. If the position of the well plate 5 after the arm 232 operates according to the operation plan (the placement position of the well plate 5) is outside the given range, the machine learning unit 111 determines the result of transporting the well plate 5. ``Failure'' is identified as ``failure''.

In the learning process, the machine learning unit 111 causes the simulator unit 112 to simulate the motion of the arm 232 according to each of a plurality of motion plans (state s).

FIG. 4 is a diagram schematically showing an example of a simulation space defined by the simulator section 112. Simulation space 900 includes

elements

905, 920, 930, and 940.

Element 905 corresponds to well plate 5 in real space. Element 920 corresponds to work device 220 in real space. Element 930 corresponds to transfer robot 230 in real space. Element 940 corresponds to work device 240 in real space. Element 930 includes

elements

931, 932, and 933. Each of the

elements

931, 932, and 933 corresponds to the main body 231, the arm 232, and the gripper 233 in real space. Element Xp corresponds to target position Tp in real space. In the simulation described above, the motion plan causes element 930 to transport element 905 from element 920 to element 940 (more specifically to the location specified by element Xp). Thus, in the simulation, element 930 transports element 905 from element 920 to element 940.

The machine learning unit 111 obtains the results (rewards) of each movement (simulation) according to a plurality of movement plans. Then, the machine learning unit 111 performs a learning process on the machine learning model 131 using the combination of the state s and reward regarding each of the plurality of motion plans.

Note that the method for generating the machine learning model 131 is not limited to using the reinforcement learning algorithm described above. The machine learning model 131 searches for a trajectory according to a rule-based method using information that specifies a movement route prepared using a given method (e.g., grasp start point, grasp end point, and passing point). (For example, a search for a trajectory connecting each of the above points) may be performed.

(2) Step S2: Positional relationship information generation process In the positional relationship information generation process, information (positional relationship information) specifying the positional relationship between at least one of the reference positions Rp1, Rp2, and Rp3 and the target position Tp is generated. generated.

In one implementation example, the information generation unit 113 images the mark Rm using the imaging unit 234, identifies the orientation of the mark Rm in the captured image, and identifies the position of the reference position Rp in real space based on the orientation of the mark Rm. . Furthermore, the information generation unit 113 images the mark Tm using the imaging unit 234, identifies the orientation of the mark Tm in the captured image, and identifies the position of the target position Tp in the real space based on the orientation of the mark Tm. The information generation unit 113 then generates positional relationship information as data representing the relationship between the reference position Rp and the target position Tp in the real space. The generated positional relationship information is stored in the storage 103 as positional relationship information 132. An example of positional relationship information is the difference between the coordinates of the reference position Rp in real space and the coordinates of target position Tp in real space.

Note that the information generation unit 113 specifies the respective positions of the reference positions Rp1, Rp2, and Rp3 in the real space, and determines the relationship between the position of the reference position Rp1 and the position of the target position Tp as positional relationship information. data representing the relationship between the reference position Rp2 and the target position Tp, and data representing the relationship between the reference position Rp3 and the target position Tp may be generated. .

(3) Step S3: Conveyance Processing FIG. 5 is a flowchart of the subroutine of the conveyance process in Step S3.

In step S31, the processor 101 instructs the transport robot 230 to start searching for the mark Rm. More specifically, the processor 101 instructs the imaging unit 234 to start imaging, and instructs the drivers corresponding to each of the one or more movable parts 8 to start driving the motor. Thereby, imaging by the imaging unit 234 is started, and the arm 232 moves up and down, left and right, and back and forth. The imaging unit 234 transmits the acquired image to the processor 101.

In step S32, the processor 101 determines whether the mark Rm has been found. The processor 101 determines that the mark Rm has been found when the image transmitted from the imaging unit 234 includes pixels indicating the mark Rm. If the mark Rm is found (YES in step S32), the processor 101 advances the control to step S33.

In step S33, the processor 101 instructs the transport robot 230 to end the search for the mark Rm. More specifically, the processor 101 instructs the imaging unit 234 to end imaging, and instructs the drivers corresponding to each of the one or more movable parts 8 to stop driving the motor. As a result, the arm 232 stops at the attitude (position and orientation) at the timing when the mark Rm was found.

In step S34, the processor 101 instructs the imaging unit 234 to take an image. The captured image includes pixels indicating the mark Rm. Note that the processor 101 may bring the imaging unit 234 (arm 232) closer to the mark Rm before instructing the imaging unit 234 to take an image in step S34. Thereby, the captured image can include more detailed information about the mark Rm.

In step S35, the processor 101 identifies the position of the reference position Rp in real space using the image captured according to the command in step S34. More specifically, the processor 101 identifies the orientation of the mark Rm in the image, and identifies the position of the reference position Rp in the real space based on the identified orientation and the position of the imaging unit 234 in the real space. .

In step S36, the processor 101 reads the positional relationship information 132 from the storage 103.

In step S37, the processor 101 derives the position of the target position in real space.

In order to derive the position of the target position in the real space, the processor 101 uses the positional relationship information and the position of the reference position Rp (identified in step S35) in the real space. In one example, the processor 101 derives the coordinates of the target position in the real space by adding the coordinates stored as positional relationship information to the coordinates of the reference position Rp in the real space, and The coordinates are treated as the location of the target location in real space. Theoretically, the derived target position coincides with the target position Tp shown in FIG.

In step S38, the processor 101 generates a motion plan for the transport robot 230 using the position derived in step S37 (the position of the target position in real space). More specifically, processor 101 generates a motion plan using machine learning model 131. More specifically, the processor 101 inputs the position derived in step S37 (the position of the target position in real space) as the target position for transporting the well plate 5 to the machine learning model 131, and generate a motion plan.

In step S39, the processor 101 operates the transfer robot 230 (arm 232) according to the operation plan generated in step S38. More specifically, the processor 101 instructs the transfer robot 230 to operate according to the operation plan. Thereafter, processor 101 returns control to the process of FIG. 3.

In the present embodiment described above, the position of the second feature part in real space (the position of the reference position Rm) is identified. In addition, a relationship (positional relationship information) between the position of the second feature part in real space and the position in real space corresponding to the position of the first feature part in the simulation space (the position identified by the element Xp) is acquired. Then, the machine learning model generates a motion plan for the moving body using the position of the second feature part in real space and the above relationship. That is, when generating the motion plan, the position in real space corresponding to the position of the first feature part in the simulation space is indirectly identified, so there is no need to directly detect the position of the first feature part in real space. Therefore, even if there is something that may be an obstacle to detecting the position of the first feature part in real space, a situation in which it becomes difficult to control the motion of the moving body is avoided. Therefore, control of the motion of the moving body is reliably realized.

[Modified example]
The reference position Rp used together with the positional relationship information in step S37 may be singular or plural. That is, the marker indicating the reference position Rp may be composed of one element (for example, any one of the marks Rm1, Rm2, Rm3), or may be composed of a plurality of elements (for example, the marks Rm1, Rm2, Rm3). (two or more of the following).

The flow of processing when the three reference positions Rp1, Rp2, and Rp3 are used together with the positional relationship information in step S37 will be specifically explained below.

In this case, the control in steps S31 to S35 is performed for each of the marks Rm1, Rm2, and Rm3. As a result, in step S34, the processor 101 performs control to capture an image including pixels indicating the mark Rm1, an image including pixels indicating the mark Rm2, and an image including pixels indicating the mark Rm3. As step S35, control is performed to specify the positions of the reference positions Rp1, Rp2, and Rp3 in the real space. Then, in step S37, the processor 101 derives a first temporary target position using the positional relationship information and the position of the reference position Rp1 in the real space, and derives the first temporary target position using the positional relationship information and the position of the reference position Rp2 in the real space. A second temporary target position is derived using the above, and a third temporary target position is derived using the positional relationship information and the position of the reference position Rp3 in the real space. Then, the processor 101 derives the final position of the target position in real space as the average value of the first to third temporary target positions (for example, the average value of coordinates).

Furthermore, in this embodiment, the mark Rm indicating the reference position Rp is affixed to the

work devices

220 and 240. That is, the mark Rm is configured separately from the working

devices

220 and 240. The processor 101 recognizes pixels corresponding to the image information registered as the mark Rm from the image captured by the imaging unit 234. Note that the mark Rm may be formed by a portion of the work device 220 and/or the work device 240 (for example, a logo portion of the manufacturer of the work device attached to the work device). In this case, the storage 103 may store image information for identifying part of the work device 220 and/or the work device 240 as the mark Rm. The processor 101 uses the image captured by the imaging unit 234 and the image information stored in the storage 103 to identify the posture of the portion, and the position of the portion in real space. Good too.

Furthermore, in this embodiment, an image captured by the imaging unit 234 is used as a method for specifying the position of the second feature point in real space. In this sense, the imaging unit 234 is an example of a data acquisition unit that acquires data for specifying the position of the second feature in real space. Note that the method for specifying the position of the second feature point in real space may be a method other than the method using a captured image, such as a method using a beacon. For example, a beacon receiver may be installed in the transport robot 230. The second feature may be comprised of a plurality of beacons. The processor 101 may identify the position of the second characteristic portion in real space based on the reception strength of the signal from each of a plurality of (three or more) beacons. In this case, the receiver that receives the signal from the beacon constitutes an example of a data acquisition unit that acquires data for specifying the position of the second feature in real space.

[Mode]
It will be appreciated by those skilled in the art that the exemplary embodiments described above are specific examples of the following aspects.

(Section 1) A method for controlling a moving object according to one aspect includes a machine learning model for generating a motion plan for the moving object for transporting a target object based on the position of the first feature in the simulation space. a step of identifying a position of the second feature in real space; a position of the second feature in real space; and a position in real space corresponding to the position of the first feature in simulation space. , and generating a motion plan for the moving body by applying the position of the second feature in real space and the positional relationship to the machine learning model; The method may further include the step of controlling the motion of the mobile body according to the motion plan determined.

According to the method for controlling a moving body described in item 1, the operation of the moving body can be reliably controlled using the results of the simulation.

(Section 2) In the method for controlling a moving body according to Item 1, the second characteristic portion may include a part of a working device for working using the target object.

According to the method for controlling a moving body described in item 2, the number of types of components used to implement the control method is minimized.

(Section 3) In the method for controlling a moving object according to Item 1, the second characteristic portion includes a marker configured separately from a work device for work using the target object. You can stay there.

According to the method for controlling a moving body described in item 3, an element having a structure or form suitable for use as the second feature can be used as the second feature.

(Section 4) In the method for controlling a moving body according to any one of Items 1 to 3, the second characteristic portion may include a plurality of mutually distinguishable elements.

According to the method for controlling a moving object described in item 4, the position of the second characteristic part in the real space can be specified using the position of each of the plurality of elements, and thereby the position of the second characteristic part The accuracy with which a position in real space is specified can be improved.

(Section 5) In the method for controlling a moving body according to any one of Items 1 to 4, a reinforcement learning algorithm may be used to generate the machine learning model.

According to the method for controlling a moving object described in Section 5, the learning process of a machine learning model for generating a complex motion plan can be easily performed.

(Section 6) In the method for controlling a moving object according to any one of Items 1 to 5, the position of the second feature in real space and the positional relationship are applied to the machine learning model. The method may include deriving a position in the real space corresponding to a position of the first feature in the simulation space from a position of the second feature in the real space using the positional relationship.

According to the method for controlling a moving object described in item 6, the position of the second characteristic part in real space and the method of using the above relationship can be specifically presented.

(Section 7) The conveying device according to one embodiment includes a moving body for conveying a target object, and machine learning for generating a motion plan for the moving body based on the position of the first characteristic part in the simulation space. a controller that generates a model; a data acquisition unit that acquires data for specifying the position of the second feature in real space; and a simulation space of the position of the second feature in real space and the first feature. and a memory for storing a relationship between a position in the real space corresponding to a position in the second feature, and a memory for storing a relationship between the position in the real space corresponding to the position in the second feature, and the controller using the data acquired by the data acquisition unit to A motion plan of the moving body is generated by specifying the position of the second feature in the real space and the positional relationship to the machine learning model, and the movement is performed according to the generated motion plan. You can also move your body.

According to the transport device described in item 7, control of the movement of the moving body using the simulation results is reliably realized.

(Section 8) In the conveyance device according to Item 7, the controller may recognize, as the second characteristic part, a part of a working device for work using the target object.

According to the transport device described in item 8, the number of types of components used for transporting the moving body can be minimized.

(Section 9) In the conveyance device according to Item 7, the controller recognizes, as the second characteristic part, a marker configured separately from the work device for work using the target object. You may.

According to the conveyance device described in item 9, an element having a structure and form suitable for use as the second feature can be used as the second feature.

(Section 10) In the conveyance device according to any one of Items 7 to 9, the controller may recognize each of a plurality of mutually distinguishable elements as the second characteristic portion. .

According to the conveyance device described in item 10, the position of the second characteristic part in the real space can be specified using the position of each of the plurality of elements, and thereby, the position of the second characteristic part in the real space can be specified. The accuracy with which the location is determined may be improved.

(Section 11) In the transport device according to any one of Items 7 to 10, a reinforcement learning algorithm may be used to generate the machine learning model.

According to the transport device described in item 11, learning processing of a machine learning model for generating a complicated motion plan can be easily performed.

(Section 12) In the conveying device according to any one of Items 7 to 11, applying the position of the second feature in real space and the positional relationship to the machine learning model may include The method may include deriving a position in the real space corresponding to a position of the first feature in the simulation space from a position of the second feature in the real space using a positional relationship.

According to the conveyance device described in item 12, the position of the second characteristic portion in real space and the method of utilizing the above relationship can be specifically presented.

(Paragraph 13) A work system according to one embodiment includes the transport device described in any one of Paragraphs 7 to 12, and a work device for work using an object transported by the transport device. , may be provided.

According to the work system described in item 13, control of the movement of a moving body using the simulation results is reliably realized.

(Section 14) The work system according to Item 13 may further include one or more elements constituting the second feature, which are configured separately from the work device.

According to the work system described in Section 14, the work system can be constructed more reliably.
The embodiments disclosed this time should be considered to be illustrative in all respects and not restrictive. The scope of the present disclosure is indicated by the claims rather than the description of the embodiments described above, and it is intended that all changes within the meaning and range equivalent to the claims are included. Furthermore, it is intended that each technique in the embodiments can be implemented alone or in combination with other techniques in the embodiments as necessary.

1 Work system, 5 Well plate, 8 Moving part, 100 Management device, 101 Processor, 102 Memory, 103 Storage, 104, 235 Interface, 105 Display, 106 Input device, 111 Machine learning section, 112 Simulator section, 113 Information generation section , 114 Transfer control unit, 131 Machine learning model, 132 Positional relationship information, 200 Controlled device group, 220, 240 Working device, 230 Transfer robot, 231 Main body, 232 Arm, 233 Gripper, 234 Imaging unit, 236 Motor unit, 237 driver unit, 300 transport device, 900 simulation space, 905, 920, 930, 931, 932, 933, 940, Xp element.

Claims

Generating a machine learning model for generating a motion plan for the moving body to transport the object based on the position of the first feature in the simulation space;
identifying the position of the second feature in real space;
obtaining a positional relationship between a position of the second feature in real space and a position in real space corresponding to a position of the first feature in simulation space;
generating a motion plan for the moving body by applying the position of the second feature in real space and the positional relationship to the machine learning model;
A method for controlling a moving body, comprising: controlling the motion of the moving body according to the generated motion plan.
The method of controlling a moving body according to claim 1, wherein the second characteristic part includes a part of a work device for work using the target object.
2. The method of controlling a moving body according to claim 1, wherein the second characteristic part includes a marker configured separately from a work device for work using the target object.
The method for controlling a moving body according to claim 1, wherein the second characteristic portion includes a plurality of mutually distinguishable elements.
The method for controlling a moving body according to claim 1, wherein a reinforcement learning algorithm is used to generate the machine learning model.
The method for controlling a moving object according to claim 1, wherein applying the position in real space of the second feature and the positional relationship to the machine learning model includes deriving a position in real space that corresponds to the position in simulation space of the first feature from the position in real space of the second feature by using the positional relationship.
a moving body for transporting the object;
a controller that generates a machine learning model for generating a motion plan for the mobile body based on the position of the first feature in the simulation space;
a data acquisition unit that acquires data for identifying the position of the second feature in real space;
a memory that stores a positional relationship between a position of the second feature in real space and a position in real space that corresponds to a position of the first feature in simulation space;
The controller includes:
identifying the position of the second feature in real space using the data acquired by the data acquisition unit;
Generating a motion plan for the moving object by applying the position of the second feature in real space and the positional relationship to the machine learning model;
A transport device that operates the moving body according to the generated operation plan.
The conveying device according to claim 7, wherein the controller recognizes, as the second characteristic part, a part of a working device for work using the target object.
The conveying device according to claim 7, wherein the controller recognizes, as the second characteristic part, a marker configured separately from a working device for work using the target object.
The conveying device according to claim 7, wherein the controller recognizes each of a plurality of mutually distinguishable elements as the second characteristic portion.
The conveyance device according to claim 7, wherein a reinforcement learning algorithm is used to generate the machine learning model.
Applying the position of the second feature in the real space and the positional relationship to the machine learning model means that the position of the second feature in the real space is used to calculate the first feature from the position of the second feature in the real space. 8. The conveying device according to claim 7, further comprising deriving a position in real space corresponding to a position in simulation space.
A work system comprising: the transport device according to any one of claims 7 to 9; and a work device for performing work using an object transported by the transport device.
The work system according to claim 13, further comprising one or more elements constituting the second feature, which are configured separately from the work device.