WO2018139292A1 - Control content determination device and control content determination method - Google Patents

Control content determination device and control content determination method Download PDF

Info

Publication number
WO2018139292A1
WO2018139292A1 PCT/JP2018/001123 JP2018001123W WO2018139292A1 WO 2018139292 A1 WO2018139292 A1 WO 2018139292A1 JP 2018001123 W JP2018001123 W JP 2018001123W WO 2018139292 A1 WO2018139292 A1 WO 2018139292A1
Authority
WO
WIPO (PCT)
Prior art keywords
control content
determination
unit
content determination
operation support
Prior art date
Application number
PCT/JP2018/001123
Other languages
French (fr)
Japanese (ja)
Inventor
伸裕 見市
智彦 藤田
真梨奈 大野
Original Assignee
パナソニックＩｐマネジメント株式会社
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by パナソニックＩｐマネジメント株式会社 filed Critical パナソニックＩｐマネジメント株式会社
Publication of WO2018139292A1 publication Critical patent/WO2018139292A1/en

Links

Images

Classifications

    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61H—PHYSICAL THERAPY APPARATUS, e.g. DEVICES FOR LOCATING OR STIMULATING REFLEX POINTS IN THE BODY; ARTIFICIAL RESPIRATION; MASSAGE; BATHING DEVICES FOR SPECIAL THERAPEUTIC OR HYGIENIC PURPOSES OR SPECIFIC PARTS OF THE BODY
    • A61H3/00—Appliances for aiding patients or disabled persons to walk about
    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05B—CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
    • G05B13/00—Adaptive control systems, i.e. systems automatically adjusting themselves to have a performance which is optimum according to some preassigned criterion
    • G05B13/02—Adaptive control systems, i.e. systems automatically adjusting themselves to have a performance which is optimum according to some preassigned criterion electric
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N99/00—Subject matter not provided for in other groups of this subclass

Definitions

  • the present invention relates to a control content determination device and a control content determination method for determining control content of an operation support device that mechanically supports a user's operation.
  • Patent Document 1 an operation support system that supports the standing motion of a cared person who is difficult to walk on his own.
  • the operation support apparatus is controlled based on a predetermined control pattern. Therefore, even if comfortable operation support can be performed for one user, only operation support that is uncomfortable for another user is possible. There are cases where it is not possible. That is, it is difficult for the conventional technology to realize operation support adapted to individual users.
  • the present invention provides a control content determination device and a control content determination method that can determine the control content of an operation support device for operation support adapted to individual users.
  • a control content determination apparatus includes: a first acquisition unit that acquires operation state information indicating an operation state of an operation support apparatus that mechanically supports a user's operation; and (i) a control content determination rule Determining the control content of the operation support device from the operation state information; or (ii) a determination unit that randomly determines the control content of the operation support device; and an output unit that outputs the determined control content; A second acquisition unit that acquires comfort information indicating comfort of operation support by the operation support device; and an update unit that updates the control content determination rule based on the operation state information and the comfort information.
  • the determination unit selects the determination of (ii) with a probability ⁇ .
  • a control content determination method includes a first acquisition step of acquiring operation state information indicating an operation state of an operation support apparatus that mechanically supports a user's operation, and (i) according to a control content determination rule. Determining the control content of the operation support device from the operation state information, or (ii) determining the control content of the operation support device at random; and outputting the determined control content; A second acquisition step of acquiring comfort information indicating comfort of operation support by the operation support device; and an update step of updating the control content determination rule based on the operation state information and the comfort information. In the determination step, the determination of (ii) is selected with probability ⁇ .
  • a recording medium such as a system, an integrated circuit, a computer program, or a computer-readable CD-ROM, and the system, the integrated circuit, the computer program, and the recording medium. You may implement
  • the control content determination device can determine the control content of the operation support device for operation support adapted to each user.
  • FIG. 1 is a block diagram illustrating a configuration of the operation support system according to the first embodiment.
  • FIG. 2 is a perspective view of the main body of the operation support apparatus according to the first embodiment.
  • FIG. 3A is a diagram for explaining the operation of the main body of the operation support apparatus according to the first embodiment.
  • FIG. 3B is a diagram for explaining the operation of the main body of the operation support apparatus according to the first embodiment.
  • FIG. 3C is a diagram for explaining the operation of the main body of the operation support apparatus according to Embodiment 1.
  • FIG. 4 is a conceptual diagram illustrating an example of a neural network in the operation support apparatus according to the first embodiment.
  • FIG. 5 is a diagram illustrating an example of control contents in the first embodiment.
  • FIG. 6 is a flowchart showing processing of the control content determination apparatus according to the first embodiment.
  • FIG. 7 is a diagram showing an example of a graphical user interface for inputting comfort information according to the first embodiment.
  • FIG. 8 is a block diagram illustrating a functional configuration of the control content determination apparatus according to the second embodiment.
  • FIG. 9 is a flowchart illustrating processing of the control content determination apparatus according to the second embodiment.
  • FIG. 1 is a block diagram illustrating a configuration of an operation support system 10 according to the first embodiment.
  • the operation support system 10 according to the present embodiment includes an operation support device 100, a control content determination device 200, and an input device 300.
  • an operation support device 100 includes an operation support device 100, a control content determination device 200, and an input device 300.
  • an operation support device 100 includes an operation support device 100, a control content determination device 200, and an input device 300.
  • the configuration of each apparatus will be specifically described with reference to the drawings.
  • the operation support apparatus 100 mechanically supports a user's operation. That is, the operation support apparatus 100 supports the user's operation by physically assisting the user. As illustrated in FIG. 1, the operation support apparatus 100 includes a main body 110, a sensor 120, a control unit 130, and a drive unit 140.
  • the main body 110 mechanically supports the user's operation by applying force to the user according to the user's operation.
  • the main body 110 includes a plurality of mechanical parts that move in accordance with a user's standing motion or walking motion. A specific example of the main body 110 will be described later with reference to FIGS. 2 to 3C.
  • the sensor 120 detects the operation state of the operation support apparatus 100 and outputs the detection result as operation state information.
  • the operation state includes a position, an angle, a trajectory or a moving speed of a machine part included in the main body 110, or a force or pressure received from the user by the machine part.
  • the operation state may include the state of the driving unit 140.
  • the operation state may include an output (power level) of the drive unit 140 and the like.
  • the senor 120 is, for example, a rotary encoder. In this case, the sensor 120 detects the rotation speed (rotation angle) of the drive unit 140.
  • the control unit 130 controls the drive unit 140 based on the control content determined by the control content determination device 200. For example, when the drive unit 140 is an electric motor, the control unit 130 transmits a pulse signal corresponding to the rotation speed of the electric motor to the drive unit 140 based on the control content.
  • the driving unit 140 drives the main body unit 110.
  • the drive unit 140 includes an electric motor, a pulley, a belt, and the like.
  • the main body 110 is driven by the driving unit 140, the operation of the user is mechanically supported.
  • FIG. 2 is a perspective view of the main body 110 of the motion support apparatus 100 according to the first embodiment.
  • the X-axis direction represents the direction opposite to the traveling direction of the motion support apparatus 100.
  • the Z-axis direction is upward in the vertical direction and represents a direction orthogonal to the X-axis.
  • the Y axis direction represents a direction orthogonal to each of the X axis and the Z axis.
  • the main body 110 of the motion support device 100 is configured to support a motion of the user sitting on the seat surface from standing up to walking. And a base 113.
  • the holding unit 111 holds the upper body of the user by being attached to the user.
  • the holding part 111 is detachably connected to one end of the arm part 112.
  • the arm unit 112 is a robot arm with two degrees of freedom.
  • the arm part 112 includes a first joint 112a, a first arm 112b, a second joint 112c, a second arm 112d, and a connection part 112e.
  • One end of the first arm 112b is rotatably connected to the base 113 via the first joint 112a, and the other end of the first arm 112b is rotatably connected to the second arm 112d via the second joint 112c. Connected.
  • the first arm 112b is driven by the driving unit 140 and rotates around the first joint 112a.
  • One end of the second arm 112d is rotatably connected to the first arm 112b via the second joint 112c, and the other end of the second arm 112d is connected to the connecting portion 112e.
  • the second arm 112d is driven independently of the first arm 112b by the driving unit 140, and rotates about the second joint 112c.
  • the base portion 113 supports the arm portion 112.
  • the base 113 has a wheel for moving the floor surface, and the rotation of the wheel is controlled by the drive unit 140 to move the floor surface in the negative direction of the X axis.
  • the movement of the base 113 can support the user's walking motion.
  • the arm portion 112 is connected to the base portion 113.
  • FIGS. 3A to 3C are diagrams for explaining the operation of the main body 110 of the operation support apparatus 100 according to Embodiment 1.
  • FIG. 3A to 3C are diagrams for explaining the operation of the main body 110 of the operation support apparatus 100 according to Embodiment 1.
  • the user is sitting on a seat surface (for example, a bed, a chair, or a toilet seat) while being held by the holding unit 111.
  • a seat surface for example, a bed, a chair, or a toilet seat
  • the driving unit 140 supports the motion from the sitting posture to the forward leaning posture by driving the arm unit 112 as shown in FIG. 3B.
  • the driving unit 140 drives the arm unit 112 to support the operation from the forward leaning posture to the standing posture.
  • control content determination device 200 determines the control content for the operation support device 100 and outputs the control content to the operation support device 100.
  • the control content determination device 200 includes a first acquisition unit 210, a determination unit 220, an output unit 230, a second acquisition unit 240, and an update unit 250.
  • the control content determination device 200 is realized by, for example, a processor and a memory.
  • the processor executes a software program stored in the memory
  • the processor functions as the first acquisition unit 210, the determination unit 220, the output unit 230, the second acquisition unit 240, and the update unit 250.
  • the control content determination apparatus 200 may be realized as one or more dedicated electronic circuits corresponding to the first acquisition unit 210, the determination unit 220, the output unit 230, the second acquisition unit 240, and the update unit 250.
  • the first acquisition unit 210 acquires operation state information from the sensor 120 of the operation support apparatus 100.
  • the 1st acquisition part 210 acquires relative position information to base 113 of connection part 112e as operation state information by processing an output signal of sensor 120.
  • the determining unit 220 determines the control content of the operation support apparatus 100 from the operation state information according to the control content determination rule, or (ii) determines the control content of the operation support apparatus 100 at random. That is, the determination unit 220 selectively executes one determination from among a plurality of determinations including the determination of (i) and the determination of (ii). At this time, the determination unit 220 selects the determination of (ii) with the probability ⁇ .
  • ⁇ is a predetermined value larger than 0 and smaller than 1. For example, the determination unit 220 selects the determination of (i) with a probability of 1 ⁇ , and selects the determination of (ii) with a probability of ⁇ .
  • the control content determination rule is expressed by, for example, a neural network for estimating each value of a plurality of control content from the operation state information.
  • the control content determination rule is stored in a storage unit (not shown). The neural network will be described later with reference to FIG.
  • the output unit 230 outputs the control content determined by the determination unit 220.
  • the output unit 230 outputs the control content to the operation support apparatus 100.
  • the second acquisition unit 240 acquires comfort information indicating the comfort of the user by the operation support of the operation support apparatus 100.
  • This comfort information includes information input from the user via the input device 300.
  • the second acquisition unit 240 acquires a value indicating comfort of operation support input by the user from the input device 300.
  • the second acquisition unit 240 may acquire the comfort information by receiving a voice signal from the input device 300 and detecting a speech of a predetermined keyword by voice recognition.
  • the predetermined keyword is a predetermined keyword indicating the comfort of the user.
  • the predetermined keyword is “painful” or “slow”.
  • the update unit 250 updates the control content determination rule used by the determination unit 220 based on the operation state information acquired by the first acquisition unit 210 and the comfort information acquired by the second acquisition unit 240. Specifically, the updating unit 250 updates the values of the plurality of control contents using a value based on the comfort information as a reward. Then, the updating unit 250 updates the neural network parameters (for example, the weight w) based on the updated value. That is, the update unit 250 learns the determination of the control content adapted to the user by reinforcement learning based on the values of the plurality of control content.
  • the neural network parameters for example, the weight w
  • FIG. 4 is a conceptual diagram showing an example of a neural network in the control content determination apparatus 200 according to the first embodiment.
  • FIG. 5 is a diagram illustrating an example of control contents in the first embodiment.
  • the control content is information indicating the time change of the position of the connection unit 112e.
  • the control content indicates the movement trajectory of the connecting portion 112e on the XZ plane.
  • the value in each environment s (operation state) of the plurality of control contents a1 to an including such control contents is estimated by the neural network.
  • the input device 300 receives comfort information indicating the comfort of operation support from the user.
  • the input device 300 is provided in the operation support device 100 and receives comfort information input via a touch display, a mechanical push button, or the like.
  • the input device 300 may be a microphone. In this case, the input device 300 receives voice input from the user.
  • FIG. 6 is a flowchart showing processing of the control content determination apparatus 200 according to Embodiment 1. This process may be executed based on an instruction from the user, for example.
  • the first acquisition unit 210 acquires operation state information (S110).
  • the determination unit 220 estimates the value of each control content from the operation state information based on the neural network (S120).
  • the determination unit 220 performs branch processing using the probability ⁇ (S130).
  • the determination unit 220 selects the determination of (i) with a probability of 1 ⁇ , and selects the determination of (ii) with a probability of ⁇ .
  • the determination unit 220 determines the control content at random (S140). That is, the determination unit 220 randomly selects a control content from a plurality of control details. In other words, the determination unit 220 determines the control content without depending on the value estimated based on the neural network.
  • the determination unit 220 determines the control content based on the estimated value (S150). For example, the determination unit 220 selects the control content having the highest value from the plurality of control contents.
  • the output unit 230 outputs the control content determined in step S140 or step S150 (S160). Thereby, the motion support apparatus 100 is controlled based on the determined control content.
  • the second acquisition unit 240 acquires comfort information (S170). For example, when the input device 300 includes a display, the second acquisition unit 240 acquires a value indicating the comfort of the user indoors via a graphical user interface (GUI) as illustrated in FIG. In the GUI of FIG. 7, the comfort value is input using a slider, but it is not necessary to be limited to this.
  • the GUI may include a text box in which a numerical value is directly input, a numerical value increase / decrease button, or a combination thereof.
  • the updating unit 250 updates the values of the plurality of control contents based on the operation state information and the comfort information (S180).
  • a value based on the comfort information is used as a reward in reinforcement learning.
  • the value based on the comfort information is a value indicating comfort, for example, a value that increases as the comfort increases.
  • the update unit 250 updates the parameters of the neural network based on the updated value (S190). That is, the update unit 250 learns the parameters of a neural network having a plurality of layers by inputting the value of each updated control content as a teacher signal.
  • the so-called deep reinforcement learning is performed by internally repeating the processes in steps S180 and S190.
  • the deep reinforcement learning is not particularly limited, and a conventional technique may be used. Therefore, detailed description of the deep reinforcement learning is omitted.
  • step S170 may be skipped.
  • the update unit 250 may learn the value of each control content using a predetermined value (for example, 0) as a reward.
  • the control content determination device 200 operates according to the first acquisition unit 210 that acquires the operation state information indicating the operation state of the operation support device 100, and (i) the control content determination rule.
  • the control content of the support device 100 is determined from the operation state information, or (ii) the control content of the motion support device 100 is determined at random, the output unit 230 that outputs the determined control content, and the motion support
  • a second acquisition unit 240 that acquires comfort information indicating comfort of operation support by the apparatus 100; and an update unit 250 that updates a control content determination rule based on the operation state information and comfort information.
  • the unit 220 selects the determination of (ii) with the probability ⁇ .
  • the update unit 250 can update the control content determination rule based on the comfort information. Therefore, the control content determination apparatus 200 can learn a control content determination rule suitable for improving the comfort of the user, and can realize operation support adapted to each user. Furthermore, since the determination unit 220 selects a random determination with the probability ⁇ , the optimal control content can be searched without being bound by the current control content determination rule. That is, the control content determination device 200 can balance the search and use of the learning result, and can effectively update the control content determination rule.
  • control content determination rule is represented by a neural network for estimating the value of each of the plurality of control content from the operation state information, and the updating unit 250 is comfortable
  • a value based on sex information is used as a reward to update the value of a plurality of control contents, and a parameter of the neural network is updated based on the updated value.
  • control content determination device 200 can construct a control content determination rule more suitable for the user.
  • control content determination apparatus 200 can realize operation support suitable for each user.
  • the second acquisition unit 240 may acquire the comfort information by detecting an utterance of a predetermined keyword by voice recognition.
  • control content determination apparatus 200 can reduce the burden of input of user comfort information, and can improve user convenience.
  • Embodiment 2 Next, a second embodiment will be described.
  • one or more control contents are extracted from a plurality of control contents based on the safety level indicating the safety of the user when the operation support is performed, and the extracted one or more control contents
  • the main difference from Embodiment 1 is that the contents of control are determined at random from the inside.
  • the second embodiment will be described below with a focus on differences from the first embodiment.
  • FIG. 8 is a block diagram illustrating a functional configuration of the control content determination apparatus 200A according to the second embodiment.
  • the control content determination device 200A includes a first acquisition unit 210, a determination unit 220A, an output unit 230, a second acquisition unit 240, an update unit 250A, and a detection unit 260A. .
  • the determining unit 220A determines the control content of the operation support apparatus 100 from the operation state information according to the control content determination rule, or (ii) determines the control content at random.
  • the determination unit 220A refers to the safety level information, and randomly determines the control content from among one or more control content in which the safety level satisfies a predetermined condition.
  • Safety level information is information in which a safety level is associated with each of a plurality of control contents.
  • the safety level is a value indicating the safety of the user when the operation support is performed.
  • the safety level information is a table in which a value representing safety is associated with each of a plurality of control contents.
  • the safety level information is stored in a storage unit (not shown).
  • the predetermined condition is a condition for determining the control content with high safety.
  • the predetermined condition is that the safety value is larger than a predetermined threshold value.
  • the determination unit 220A refers to the safety degree information and extracts one or more control contents having a safety degree value larger than the threshold value from a plurality of control contents. Then, the determination unit 220A randomly determines the control contents from the extracted one or more control contents.
  • the detection unit 260A detects whether the user is safe. That is, when the control content is randomly determined, the detection unit 260A determines whether the operation support based on the determined control content is safe. For example, when the user falls, the detection unit 260A detects that the user is not safe (that is, dangerous).
  • the update unit 250A updates the control content determination rule based on the operation state information and the comfort information as in the first embodiment.
  • the update unit 250A according to the present embodiment further updates the safety level information based on the detection result by the detection unit 260A when the control content is randomly determined. For example, when it is detected that the user is not safe, the updating unit 250A decreases the value of the safety level of the determined control content. Conversely, for example, when it is detected that the user is safe, the update unit 250A increases the value of the safety level of the determined control content.
  • FIG. 9 is a flowchart showing processing of the control content determination apparatus 200A according to the second embodiment.
  • the determination unit 220A extracts one or more control contents from a plurality of control contents based on the safety degree information (S132A). For example, the determination unit 220A refers to the safety level information, and extracts control content having a safety level value greater than a predetermined threshold value (for example, 50) from the plurality of control content levels a1 to an.
  • a predetermined threshold value for example, 50
  • the determination unit 220A randomly determines the control content from the extracted control content (S140A).
  • Steps S160 to S190 are executed, and when the control content is not determined at random (No in S192A), the process is terminated as it is.
  • the detection unit 260A detects whether the operation support is safe (S194A).
  • the update unit 250A updates the safety level information based on the detection result by the detection unit 260A (S196A).
  • the determination unit 220A when the control content is randomly determined, the determination unit 220A provides operation support for each of the plurality of control content based on the control content.
  • the control level is randomly determined from one or more control levels that satisfy the predetermined level.
  • the determination unit 220A can reduce the possibility of danger to the user when determining the control content at random. That is, the determination unit 220 ⁇ / b> A can suppress the determination of the control content causing danger to the user in the random determination.
  • control content determination device 200A further includes a detection unit 260A that detects whether the operation support is dangerous, and the update unit 250A further includes a control content when the control content is randomly determined.
  • the safety degree information is updated based on the detection result by the detection unit 260A.
  • the updating unit 250A can update the risk level information based on the detection result as to whether or not the user is at risk due to the operation support, and can improve the risk level information. Therefore, the determination unit 220 ⁇ / b> A can suppress the determination of the control content causing danger to the user in the random determination.
  • control content determination device has been described based on the embodiment, but the present invention is not limited to this embodiment. Unless it deviates from the gist of the present invention, one or more of the present invention may be applied to various modifications that can be conceived by those skilled in the art, or forms constructed by combining components in different embodiments. It may be included within the scope of the embodiments.
  • the second acquisition unit 240 acquires the comfort information based on the information received from the input device 300.
  • the second acquisition unit 240 acquires not only the input device 300 but also the information received from the sensor 120. Based on this, comfort information may be acquired.
  • the second acquisition unit 240 may acquire the comfort information by correcting the information received from the input device 300 using the information received from the sensor 120.
  • the second acquisition unit 240 may correct the information received from the input device 300 based on the user's facial expression, brain wave, or heart rate.
  • the sensor 120 may include an image sensor, an electroencephalogram sensor, or a heart rate sensor.
  • the determination of the control content adapted to the user is learned using the deep reinforcement learning.
  • the embodiment is not limited to the deep reinforcement learning.
  • the control content determination rule may be represented not by a multi-layer neural network but by a single-layer neural network.
  • the control content determination rule may be expressed not by a neural network but by another mathematical model (for example, linear regression, support vector machine, etc.).
  • the control content of the motion support apparatus 100 is determined from the motion state information mainly according to two determinations ((i) control content determination rules, or (ii) the motion support apparatus 100 is randomly selected.
  • the control content is determined), it is not necessarily limited to two determinations.
  • one decision may be selected from three or more decisions.
  • the determination unit only needs to selectively execute one of a plurality of determinations including the determination of (i) and the determination of (ii). At this time, the determination of (ii) is selected with the probability ⁇ . Just do it.
  • control content determination apparatus was implement
  • the control content determination device may be realized by cloud computing.
  • the safety level information is updated.
  • the safety level information is not necessarily updated.
  • the control content determination device 200A may not include the detection unit 260A.
  • control content determination apparatus 200 may be configured by a system LSI having a first acquisition unit 210, a determination unit 220, an output unit 230, a second acquisition unit 240, and an update unit 250.
  • the system LSI is an ultra-multifunctional LSI manufactured by integrating a plurality of components on one chip. Specifically, a microprocessor, a ROM (Read Only Memory), a RAM (Random Access Memory), etc. It is a computer system comprised including. A computer program is stored in the ROM. The system LSI achieves its functions by the microprocessor operating according to the computer program.
  • system LSI may be called IC, LSI, super LSI, or ultra LSI depending on the degree of integration.
  • method of circuit integration is not limited to LSI's, and implementation using dedicated circuitry or general purpose processors is also possible.
  • An FPGA Field Programmable Gate Array
  • a reconfigurable processor that can reconfigure the connection and setting of the circuit cells inside the LSI may be used.
  • one aspect of the present invention may be a control content determination method that uses not only such a control content determination device but also a characteristic component included in the control content determination device as a step. Further, one aspect of the present invention may be a computer program that causes a computer to execute each characteristic step included in the control content determination method. One embodiment of the present invention may be a computer-readable non-transitory recording medium in which such a computer program is recorded.
  • each component may be configured by dedicated hardware or may be realized by executing a software program suitable for each component.
  • Each component may be realized by a program execution unit such as a CPU or a processor reading and executing a software program recorded on a recording medium such as a hard disk or a semiconductor memory.
  • the software that realizes the control content determination device of each of the above embodiments is a program as follows.
  • the program includes a first acquisition step of acquiring operation state information indicating an operation state of an operation support apparatus that mechanically supports a user operation, and (i) the operation support according to a control content determination rule. Determining a control content of a device from the operation state information; or (ii) determining a control content of the motion support device at random; an output step of outputting the determined control content; and the motion support device Control content determination including a second acquisition step of acquiring comfort information indicating the comfort of the operation support by, and an update step of updating the control content determination rule based on the operation state information and the comfort information In the determination step, the determination of (ii) is selected with probability ⁇ .

Abstract

A control content determination device (200) is provided with: a first acquisition unit (210) that acquires operational state information indicating the operational state of an operation assistance device (100) that mechanically assists a user operation; a determination unit (220) that (i) determines, on the basis of the operational state information, the control content for the operation assistance device (100) in accordance with a control content determination rule, or (ii) randomly determines the control content for the operation assistance device (100); an output unit (230) that outputs the determined control content; a second acquisition unit (240) that acquires comfort information indicating the comfort of the operation assistance by the operation assistance device (100); and an updating unit (250) that updates the control content determination rule on the basis of the operational state information and the comfort information. The determination unit (220) selects the determination of (ii) with a probability ε.

Description

制御内容決定装置及び制御内容決定方法Control content determination apparatus and control content determination method
　本発明は、ユーザの動作を機械的に支援する動作支援装置の制御内容を決定する制御内容決定装置及び制御内容決定方法に関する。 The present invention relates to a control content determination device and a control content determination method for determining control content of an operation support device that mechanically supports a user's operation.
　従来、自力で歩行することが困難な被介護者の起立動作を支援する動作支援システムが提案されている（例えば、特許文献１を参照）。 Conventionally, there has been proposed an operation support system that supports the standing motion of a cared person who is difficult to walk on his own (see, for example, Patent Document 1).
特開２０１６－６４１２４号公報JP 2016-64124 A
　しかしながら、従来技術では、予め定められた制御パターンに基づいて動作支援装置が制御されるため、あるユーザに対しては快適な動作支援ができたとしても、別のユーザには不快な動作支援しかできない場合がある。つまり、従来技術では、個々のユーザに適応した動作支援を実現することが難しい。 However, in the related art, the operation support apparatus is controlled based on a predetermined control pattern. Therefore, even if comfortable operation support can be performed for one user, only operation support that is uncomfortable for another user is possible. There are cases where it is not possible. That is, it is difficult for the conventional technology to realize operation support adapted to individual users.
　そこで、本発明は、個々のユーザに適応する動作支援のための動作支援装置の制御内容を決定することができる制御内容決定装置及び制御内容決定方法を提供する。 Therefore, the present invention provides a control content determination device and a control content determination method that can determine the control content of an operation support device for operation support adapted to individual users.
　本発明の一態様に係る制御内容決定装置は、ユーザの動作を機械的に支援する動作支援装置の動作状態を示す動作状態情報を取得する第１取得部と、（i）制御内容決定ルールに従って、前記動作支援装置の制御内容を前記動作状態情報から決定する、又は（ii）ランダムに前記動作支援装置の制御内容を決定する決定部と、決定された前記制御内容を出力する出力部と、前記動作支援装置による動作支援の快適さを示す快適性情報を取得する第２取得部と、前記動作状態情報及び前記快適性情報に基づいて、前記制御内容決定ルールを更新する更新部と、を備え、前記決定部は、確率εで前記（ii）の決定を選択する。 A control content determination apparatus according to an aspect of the present invention includes: a first acquisition unit that acquires operation state information indicating an operation state of an operation support apparatus that mechanically supports a user's operation; and (i) a control content determination rule Determining the control content of the operation support device from the operation state information; or (ii) a determination unit that randomly determines the control content of the operation support device; and an output unit that outputs the determined control content; A second acquisition unit that acquires comfort information indicating comfort of operation support by the operation support device; and an update unit that updates the control content determination rule based on the operation state information and the comfort information. The determination unit selects the determination of (ii) with a probability ε.
　本発明の一態様に係る制御内容決定方法は、ユーザの動作を機械的に支援する動作支援装置の動作状態を示す動作状態情報を取得する第１取得ステップと、（i）制御内容決定ルールに従って、前記動作支援装置の制御内容を前記動作状態情報から決定する、又は（ii）ランダムに前記動作支援装置の制御内容を決定する決定ステップと、決定された前記制御内容を出力する出力ステップと、前記動作支援装置による動作支援の快適さを示す快適性情報を取得する第２取得ステップと、前記動作状態情報及び前記快適性情報に基づいて、前記制御内容決定ルールを更新する更新ステップと、を含み、前記決定ステップでは、確率εで前記（ii）の決定を選択する。 A control content determination method according to an aspect of the present invention includes a first acquisition step of acquiring operation state information indicating an operation state of an operation support apparatus that mechanically supports a user's operation, and (i) according to a control content determination rule. Determining the control content of the operation support device from the operation state information, or (ii) determining the control content of the operation support device at random; and outputting the determined control content; A second acquisition step of acquiring comfort information indicating comfort of operation support by the operation support device; and an update step of updating the control content determination rule based on the operation state information and the comfort information. In the determination step, the determination of (ii) is selected with probability ε.
　なお、これらの包括的又は具体的な態様は、システム、集積回路、コンピュータプログラム又はコンピュータ読み取り可能なＣＤ－ＲＯＭなどの記録媒体で実現されてもよく、システム、集積回路、コンピュータプログラム及び記録媒体の任意な組み合わせで実現されてもよい。 Note that these comprehensive or specific modes may be realized by a recording medium such as a system, an integrated circuit, a computer program, or a computer-readable CD-ROM, and the system, the integrated circuit, the computer program, and the recording medium. You may implement | achieve in arbitrary combinations.
　本発明の一態様に係る制御内容決定装置は、個々のユーザに適応する動作支援のための動作支援装置の制御内容を決定することができる。 The control content determination device according to an aspect of the present invention can determine the control content of the operation support device for operation support adapted to each user.
図１は、実施の形態１に係る動作支援システムの構成を示すブロック図である。FIG. 1 is a block diagram illustrating a configuration of the operation support system according to the first embodiment. 図２は、実施の形態１に係る動作支援装置の本体部の斜視図である。FIG. 2 is a perspective view of the main body of the operation support apparatus according to the first embodiment. 図３Ａは、実施の形態１に係る動作支援装置の本体部の動作を説明するための図である。FIG. 3A is a diagram for explaining the operation of the main body of the operation support apparatus according to the first embodiment. 図３Ｂは、実施の形態１に係る動作支援装置の本体部の動作を説明するための図である。FIG. 3B is a diagram for explaining the operation of the main body of the operation support apparatus according to the first embodiment. 図３Ｃは、実施の形態１に係る動作支援装置の本体部の動作を説明するための図である。FIG. 3C is a diagram for explaining the operation of the main body of the operation support apparatus according to Embodiment 1. 図４は、実施の形態１に係る動作支援装置におけるニューラルネットワークの一例を示す概念図である。FIG. 4 is a conceptual diagram illustrating an example of a neural network in the operation support apparatus according to the first embodiment. 図５は、実施の形態１における制御内容の一例を示す図である。FIG. 5 is a diagram illustrating an example of control contents in the first embodiment. 図６は、実施の形態１に係る制御内容決定装置の処理を示すフローチャートである。FIG. 6 is a flowchart showing processing of the control content determination apparatus according to the first embodiment. 図７は、実施の形態１に係る快適性情報の入力のためのグラフィカルユーザーインターフェースの一例を示す図である。FIG. 7 is a diagram showing an example of a graphical user interface for inputting comfort information according to the first embodiment. 図８は、実施の形態２に係る制御内容決定装置の機能構成を示すブロック図である。FIG. 8 is a block diagram illustrating a functional configuration of the control content determination apparatus according to the second embodiment. 図９は、実施の形態２に係る制御内容決定装置の処理を示すフローチャートである。FIG. 9 is a flowchart illustrating processing of the control content determination apparatus according to the second embodiment.
　以下、実施の形態について、図面を参照しながら具体的に説明する。 Hereinafter, embodiments will be specifically described with reference to the drawings.
　なお、以下で説明する実施の形態は、いずれも包括的または具体的な例を示すものである。以下の実施の形態で示される数値、形状、材料、構成要素、構成要素の配置位置及び接続形態、ステップ、ステップの順序などは、一例であり、請求の範囲を限定する主旨ではない。また、以下の実施の形態における構成要素のうち、最上位概念を示す独立請求項に記載されていない構成要素については、任意の構成要素として説明される。 It should be noted that each of the embodiments described below shows a comprehensive or specific example. Numerical values, shapes, materials, components, arrangement positions and connection forms of components, steps, order of steps, and the like shown in the following embodiments are merely examples, and are not intended to limit the scope of the claims. In addition, among the constituent elements in the following embodiments, constituent elements that are not described in the independent claims indicating the highest concept are described as optional constituent elements.
　また、各図は、模式図であり、必ずしも厳密に図示されたものではない。また、各図において、同一又は類似の構成部及び処理ステップについては同じ符号を付している。 Each figure is a schematic diagram and is not necessarily shown strictly. Moreover, in each figure, the same code | symbol is attached | subjected about the same or similar component and process step.
　（実施の形態１）
　［動作支援システムの構成］
　まず、動作支援システムの全体構成について説明する。図１は、実施の形態１に係る動作支援システム１０の構成を示すブロック図である。本実施の形態に係る動作支援システム１０は、動作支援装置１００と、制御内容決定装置２００と、入力装置３００と、を備える。以下に、各装置の構成について図面を参照しながら具体的に説明する。 (Embodiment 1)
[Operation support system configuration]
First, the overall configuration of the operation support system will be described. FIG. 1 is a block diagram illustrating a configuration of an operation support system 10 according to the first embodiment. The operation support system 10 according to the present embodiment includes an operation support device 100, a control content determination device 200, and an input device 300. Hereinafter, the configuration of each apparatus will be specifically described with reference to the drawings.
　［動作支援装置の構成］
　動作支援装置１００は、ユーザの動作を機械的に支援する。つまり、動作支援装置１００は、ユーザを物理的に補助することによりユーザの動作を支援する。図１に示すように、動作支援装置１００は、本体部１１０と、センサ１２０と、制御部１３０と、駆動部１４０と、を備える。 [Operation support device configuration]
The operation support apparatus 100 mechanically supports a user's operation. That is, the operation support apparatus 100 supports the user's operation by physically assisting the user. As illustrated in FIG. 1, the operation support apparatus 100 includes a main body 110, a sensor 120, a control unit 130, and a drive unit 140.
　本体部１１０は、ユーザの動作にあわせてユーザに力を付加することで、ユーザの動作を機械的に支援する。例えば、本体部１１０は、ユーザの起立動作あるいは歩行動作などにあわせて動く複数の機械部品を含む。この本体部１１０の具体例については、図２～図３Ｃを用いて後述する。 The main body 110 mechanically supports the user's operation by applying force to the user according to the user's operation. For example, the main body 110 includes a plurality of mechanical parts that move in accordance with a user's standing motion or walking motion. A specific example of the main body 110 will be described later with reference to FIGS. 2 to 3C.
　センサ１２０は、動作支援装置１００の動作状態を検知し、検知結果を動作状態情報として出力する。動作状態は、本体部１１０に含まれる機械部品の位置、角度、軌道もしくは移動速度、又は、機械部品がユーザから受けている力もしくは圧力などを含む。また、動作状態は、駆動部１４０の状態を含んでもよい。具体的には、動作状態は、駆動部１４０の出力（動力の大きさ）などを含んでもよい。 The sensor 120 detects the operation state of the operation support apparatus 100 and outputs the detection result as operation state information. The operation state includes a position, an angle, a trajectory or a moving speed of a machine part included in the main body 110, or a force or pressure received from the user by the machine part. Further, the operation state may include the state of the driving unit 140. Specifically, the operation state may include an output (power level) of the drive unit 140 and the like.
　具体的には、センサ１２０は、例えばロータリエンコーダである。この場合、センサ１２０は、駆動部１４０の回転数（回転角度）を検出する。 Specifically, the sensor 120 is, for example, a rotary encoder. In this case, the sensor 120 detects the rotation speed (rotation angle) of the drive unit 140.
　制御部１３０は、制御内容決定装置２００によって決定された制御内容に基づいて駆動部１４０を制御する。例えば、駆動部１４０が電動モータである場合に、制御部１３０は、制御内容に基づいて電動モータの回転速度に対応するパルス信号を駆動部１４０に送信する。 The control unit 130 controls the drive unit 140 based on the control content determined by the control content determination device 200. For example, when the drive unit 140 is an electric motor, the control unit 130 transmits a pulse signal corresponding to the rotation speed of the electric motor to the drive unit 140 based on the control content.
　駆動部１４０は、本体部１１０を駆動する。例えば、駆動部１４０は、電動モータ、プーリー、ベルトなどを含む。駆動部１４０によって本体部１１０が駆動されることにより、ユーザの動作が機械的に支援される。 The driving unit 140 drives the main body unit 110. For example, the drive unit 140 includes an electric motor, a pulley, a belt, and the like. When the main body 110 is driven by the driving unit 140, the operation of the user is mechanically supported.
　［動作支援装置の本体部の具体例］
　次に、動作支援装置１００の本体部１１０の具体例について図２を参照しながら説明する。図２は、実施の形態１に係る動作支援装置１００の本体部１１０の斜視図である。各図において、Ｘ軸方向は、動作支援装置１００の進行方向と逆向きを表す。また、Ｚ軸方向は、鉛直方向の上向きであり、Ｘ軸と直交する方向を表す。また、Ｙ軸方向は、Ｘ軸及びＺ軸の各々と直交する方向を表す。 [Specific example of main part of motion support device]
Next, a specific example of the main body 110 of the operation support apparatus 100 will be described with reference to FIG. FIG. 2 is a perspective view of the main body 110 of the motion support apparatus 100 according to the first embodiment. In each figure, the X-axis direction represents the direction opposite to the traveling direction of the motion support apparatus 100. Further, the Z-axis direction is upward in the vertical direction and represents a direction orthogonal to the X-axis. The Y axis direction represents a direction orthogonal to each of the X axis and the Z axis.
　動作支援装置１００の本体部１１０は、座面に着座しているユーザが起立してから歩行するまでの動作を支援するために、図２に示すように、保持部１１１と、アーム部１１２と、基部１１３と、を備える。 As shown in FIG. 2, the main body 110 of the motion support device 100 is configured to support a motion of the user sitting on the seat surface from standing up to walking. And a base 113.
　保持部１１１は、ユーザに装着されることでユーザの上半身を保持する。保持部１１１は、アーム部１１２の一端に着脱可能に接続される。 The holding unit 111 holds the upper body of the user by being attached to the user. The holding part 111 is detachably connected to one end of the arm part 112.
　アーム部１１２は、２自由度のロボットアームである。アーム部１１２は、第１関節１１２ａと、第１アーム１１２ｂと、第２関節１１２ｃと、第２アーム１１２ｄと、接続部１１２ｅと、を備える。 The arm unit 112 is a robot arm with two degrees of freedom. The arm part 112 includes a first joint 112a, a first arm 112b, a second joint 112c, a second arm 112d, and a connection part 112e.
　第１アーム１１２ｂの一端は、第１関節１１２ａを介して回動可能に基部１１３に連結され、第１アーム１１２ｂの他端は、第２関節１１２ｃを介して回動可能に第２アーム１１２ｄに連結される。第１アーム１１２ｂは、駆動部１４０によって駆動され、第１関節１１２ａを中心に回動する。 One end of the first arm 112b is rotatably connected to the base 113 via the first joint 112a, and the other end of the first arm 112b is rotatably connected to the second arm 112d via the second joint 112c. Connected. The first arm 112b is driven by the driving unit 140 and rotates around the first joint 112a.
　第２アーム１１２ｄの一端は第２関節１１２ｃを介して回動可能に第１アーム１１２ｂに連結され、第２アーム１１２ｄの他端は、接続部１１２ｅに連結される。第２アーム１１２ｄは、駆動部１４０によって第１アーム１１２ｂとは独立して駆動され、第２関節１１２ｃを中心に回動する。 One end of the second arm 112d is rotatably connected to the first arm 112b via the second joint 112c, and the other end of the second arm 112d is connected to the connecting portion 112e. The second arm 112d is driven independently of the first arm 112b by the driving unit 140, and rotates about the second joint 112c.
　基部１１３は、アーム部１１２を支持する。ここでは、基部１１３は、床面を移動するための車輪を有し、駆動部１４０によって車輪の回転が制御されることにより床面をＸ軸の負の向きに移動する。この基部１１３の移動によってユーザの歩行動作を支援することができる。また、基部１１３には、アーム部１１２が接続される。 The base portion 113 supports the arm portion 112. Here, the base 113 has a wheel for moving the floor surface, and the rotation of the wheel is controlled by the drive unit 140 to move the floor surface in the negative direction of the X axis. The movement of the base 113 can support the user's walking motion. In addition, the arm portion 112 is connected to the base portion 113.
　ここで、ユーザの起立動作を支援する際の本体部１１０の動作について図３Ａ～図３Ｃを参照しながら説明する。また、図３Ａ～図３Ｃは、実施の形態１に係る動作支援装置１００の本体部１１０の動作を説明するための図である。 Here, the operation of the main body 110 when supporting the user's standing operation will be described with reference to FIGS. 3A to 3C. 3A to 3C are diagrams for explaining the operation of the main body 110 of the operation support apparatus 100 according to Embodiment 1. FIG.
　図３Ａにおいて、ユーザは、保持部１１１によって保持された状態で座面（例えば、ベッド、椅子、又は、トイレの便座など）に座っている。ここで、ユーザから起立動作の支援開始の指示入力を受けると、駆動部１４０は、図３Ｂに示すように、アーム部１１２を駆動して座位姿勢から前傾姿勢までの動作を支援する。さらに、駆動部１４０は、図３Ｃに示すように、アーム部１１２を駆動して前傾姿勢から起立姿勢までの動作を支援する。 3A, the user is sitting on a seat surface (for example, a bed, a chair, or a toilet seat) while being held by the holding unit 111. Here, when receiving an instruction input for starting support for the standing motion from the user, the driving unit 140 supports the motion from the sitting posture to the forward leaning posture by driving the arm unit 112 as shown in FIG. 3B. Furthermore, as shown in FIG. 3C, the driving unit 140 drives the arm unit 112 to support the operation from the forward leaning posture to the standing posture.
　［制御内容決定装置の構成］
　次に、制御内容決定装置２００について、図１を参照しながら具体的に説明する。制御内容決定装置２００は、動作支援装置１００のための制御内容を決定し、その制御内容を動作支援装置１００に出力する。図１に示すように、制御内容決定装置２００は、第１取得部２１０と、決定部２２０と、出力部２３０と、第２取得部２４０と、更新部２５０と、を備える。 [Configuration of control content determination device]
Next, the control content determination apparatus 200 will be specifically described with reference to FIG. The control content determination device 200 determines the control content for the operation support device 100 and outputs the control content to the operation support device 100. As illustrated in FIG. 1, the control content determination device 200 includes a first acquisition unit 210, a determination unit 220, an output unit 230, a second acquisition unit 240, and an update unit 250.
　制御内容決定装置２００は、例えば、プロセッサ及びメモリにより実現される。例えば、メモリに格納されたソフトウェアプログラムをプロセッサが実行したときに、プロセッサが、第１取得部２１０、決定部２２０、出力部２３０、第２取得部２４０、及び更新部２５０として機能する。また、制御内容決定装置２００は、第１取得部２１０、決定部２２０、出力部２３０、第２取得部２４０、及び更新部２５０に対応する専用の１以上の電子回路として実現されてもよい。 The control content determination device 200 is realized by, for example, a processor and a memory. For example, when the processor executes a software program stored in the memory, the processor functions as the first acquisition unit 210, the determination unit 220, the output unit 230, the second acquisition unit 240, and the update unit 250. Further, the control content determination apparatus 200 may be realized as one or more dedicated electronic circuits corresponding to the first acquisition unit 210, the determination unit 220, the output unit 230, the second acquisition unit 240, and the update unit 250.
　第１取得部２１０は、動作支援装置１００のセンサ１２０から動作状態情報を取得する。例えば、第１取得部２１０は、センサ１２０の出力信号を処理することにより、接続部１１２ｅの基部１１３に対する相対的な位置情報を動作状態情報として取得する。 The first acquisition unit 210 acquires operation state information from the sensor 120 of the operation support apparatus 100. For example, the 1st acquisition part 210 acquires relative position information to base 113 of connection part 112e as operation state information by processing an output signal of sensor 120.
　決定部２２０は、（i）制御内容決定ルールに従って、動作状態情報から動作支援装置１００の制御内容を決定する、又は、（ii）ランダムに動作支援装置１００の制御内容を決定する。つまり、決定部２２０は、（i）の決定及び（ii）の決定を含む複数の決定の中から１つの決定を選択的に実行する。このとき、決定部２２０は、確率εで（ii）の決定を選択する。εは、０より大きく１より小さい予め定められた値である。例えば、決定部２２０は、１－εの確率で（i）の決定を選択し、εの確率で（ii）の決定を選択する。 The determining unit 220 (i) determines the control content of the operation support apparatus 100 from the operation state information according to the control content determination rule, or (ii) determines the control content of the operation support apparatus 100 at random. That is, the determination unit 220 selectively executes one determination from among a plurality of determinations including the determination of (i) and the determination of (ii). At this time, the determination unit 220 selects the determination of (ii) with the probability ε. ε is a predetermined value larger than 0 and smaller than 1. For example, the determination unit 220 selects the determination of (i) with a probability of 1−ε, and selects the determination of (ii) with a probability of ε.
　制御内容決定ルールは、例えば、動作状態情報から複数の制御内容の各々の価値を推定するためのニューラルネットワークで表される。制御内容決定ルールは、図示しない記憶部に記憶されている。ニューラルネットワークについては図４を用いて後述する。 The control content determination rule is expressed by, for example, a neural network for estimating each value of a plurality of control content from the operation state information. The control content determination rule is stored in a storage unit (not shown). The neural network will be described later with reference to FIG.
　出力部２３０は、決定部２２０によって決定された制御内容を出力する。ここでは、出力部２３０は、動作支援装置１００に制御内容を出力する。 The output unit 230 outputs the control content determined by the determination unit 220. Here, the output unit 230 outputs the control content to the operation support apparatus 100.
　第２取得部２４０は、動作支援装置１００の動作支援によるユーザの快適さを示す快適性情報を取得する。この快適性情報は、入力装置３００を介してユーザから入力された情報を含む。例えば、第２取得部２４０は、ユーザによって入力された動作支援の快適性を示す値を入力装置３００から取得する。 The second acquisition unit 240 acquires comfort information indicating the comfort of the user by the operation support of the operation support apparatus 100. This comfort information includes information input from the user via the input device 300. For example, the second acquisition unit 240 acquires a value indicating comfort of operation support input by the user from the input device 300.
　また例えば、第２取得部２４０は、入力装置３００から音声信号を受信し、音声認識により所定のキーワードの発言を検出することにより快適性情報を取得してもよい。所定のキーワードは、ユーザの快適性を示す予め定められたキーワードである。例えば、所定のキーワードは、「痛い」あるいは「遅い」などである。 Further, for example, the second acquisition unit 240 may acquire the comfort information by receiving a voice signal from the input device 300 and detecting a speech of a predetermined keyword by voice recognition. The predetermined keyword is a predetermined keyword indicating the comfort of the user. For example, the predetermined keyword is “painful” or “slow”.
　更新部２５０は、第１取得部２１０によって取得された動作状態情報と、第２取得部２４０によって取得された快適性情報とに基づいて、決定部２２０で用いられる制御内容決定ルールを更新する。具体的には、更新部２５０は、快適性情報に基づく値を報酬として用いて複数の制御内容の価値を更新する。そして、更新部２５０は、更新された価値に基づいてニューラルネットワークのパラメータ（例えば重みｗ）を更新する。つまり、更新部２５０は、複数の制御内容の価値に基づいた強化学習により、ユーザに適応した制御内容の決定を学習する。 The update unit 250 updates the control content determination rule used by the determination unit 220 based on the operation state information acquired by the first acquisition unit 210 and the comfort information acquired by the second acquisition unit 240. Specifically, the updating unit 250 updates the values of the plurality of control contents using a value based on the comfort information as a reward. Then, the updating unit 250 updates the neural network parameters (for example, the weight w) based on the updated value. That is, the update unit 250 learns the determination of the control content adapted to the user by reinforcement learning based on the values of the plurality of control content.
　［ニューラルネットワークの説明］
　ここで、本実施の形態におけるニューラルネットワークについて図４を参照しながら説明する。図４は、実施の形態１に係る制御内容決定装置２００におけるニューラルネットワークの一例を示す概念図である。このニューラルネットワークは、多階層の人工ニューラルネットワークであり、動作状態情報に基づく環境ｓにおける複数の制御内容ａｉ（ｉ＝１～ｎ）の価値Ｑａｉを推定するための数学モデルである。 [Description of neural network]
Here, the neural network in the present embodiment will be described with reference to FIG. FIG. 4 is a conceptual diagram showing an example of a neural network in the control content determination apparatus 200 according to the first embodiment. This neural network is a multi-layered artificial neural network and is a mathematical model for estimating the value Qai of a plurality of control contents ai (i = 1 to n) in the environment s based on operation state information.
　［制御内容の具体例］
　図５は、実施の形態１における制御内容の一例を示す図である。 [Specific examples of control contents]
FIG. 5 is a diagram illustrating an example of control contents in the first embodiment.
　制御内容は、接続部１１２ｅの位置の時間変化を示す情報である。例えば、制御内容は、ＸＺ平面上の接続部１１２ｅの移動軌跡を示す。具体的には、制御内容は、各時間ｔｉ（ｉ＝０～ｍ）における、基部１１３に対する接続部１１２ｅの相対的な位置を示す。このような制御内容を含む複数の制御内容ａ１～ａｎの各々の環境ｓ（動作状態）における価値がニューラルネットワークによって推定される。 The control content is information indicating the time change of the position of the connection unit 112e. For example, the control content indicates the movement trajectory of the connecting portion 112e on the XZ plane. Specifically, the control content indicates the relative position of the connection part 112e with respect to the base part 113 at each time ti (i = 0 to m). The value in each environment s (operation state) of the plurality of control contents a1 to an including such control contents is estimated by the neural network.
　入力装置３００は、ユーザから動作支援の快適さを示す快適性情報の入力を受ける。例えば、入力装置３００は、動作支援装置１００に設けられ、タッチディスプレイ、機械式プッシュボタンなどを介して快適性情報の入力を受ける。また例えば、入力装置３００は、マイクロフォンであってもよい。この場合、入力装置３００は、ユーザから音声入力を受ける。 The input device 300 receives comfort information indicating the comfort of operation support from the user. For example, the input device 300 is provided in the operation support device 100 and receives comfort information input via a touch display, a mechanical push button, or the like. For example, the input device 300 may be a microphone. In this case, the input device 300 receives voice input from the user.
　［制御内容決定装置の動作］
　次に、以上のように構成された制御内容決定装置２００の動作について図６及び図７を参照しながら説明する。 [Operation of control content determination device]
Next, the operation of the control content determination apparatus 200 configured as described above will be described with reference to FIGS.
　図６は、実施の形態１に係る制御内容決定装置２００の処理を示すフローチャートである。この処理は、例えば、ユーザからの指示に基づいて実行されればよい。 FIG. 6 is a flowchart showing processing of the control content determination apparatus 200 according to Embodiment 1. This process may be executed based on an instruction from the user, for example.
　まず、第１取得部２１０は、動作状態情報を取得する（Ｓ１１０）。決定部２２０は、ニューラルネットワークに基づいて、動作状態情報から各制御内容の価値を推定する（Ｓ１２０）。 First, the first acquisition unit 210 acquires operation state information (S110). The determination unit 220 estimates the value of each control content from the operation state information based on the neural network (S120).
　続いて、決定部２２０は、確率εを用いて分岐処理を行う（Ｓ１３０）。ここでは、決定部２２０は、１－εの確率で（i）の決定を選択し、εの確率で（ii）の決定を選択する。 Subsequently, the determination unit 220 performs branch processing using the probability ε (S130). Here, the determination unit 220 selects the determination of (i) with a probability of 1−ε, and selects the determination of (ii) with a probability of ε.
　ここで、（ii）の決定が選択された場合（Ｓ１３０のε）、決定部２２０は、ランダムに制御内容を決定する（Ｓ１４０）。つまり、決定部２２０は、複数の制御内容の中からランダムに制御内容を選択する。言い換えると、決定部２２０は、ニューラルネットワークに基づいて推定される価値に依存せずに制御内容を決定する。 Here, when the determination of (ii) is selected (ε in S130), the determination unit 220 determines the control content at random (S140). That is, the determination unit 220 randomly selects a control content from a plurality of control details. In other words, the determination unit 220 determines the control content without depending on the value estimated based on the neural network.
　一方、（i）の決定が選択された場合（Ｓ１３０の１－ε）、決定部２２０は、推定された価値に基づいて制御内容を決定する（Ｓ１５０）。例えば、決定部２２０は、複数の制御内容の中から最も高い価値を有する制御内容を選択する。 On the other hand, when the determination of (i) is selected (1-ε of S130), the determination unit 220 determines the control content based on the estimated value (S150). For example, the determination unit 220 selects the control content having the highest value from the plurality of control contents.
　出力部２３０は、ステップＳ１４０又はステップＳ１５０で決定された制御内容を出力する（Ｓ１６０）。これにより、決定された制御内容に基づいて動作支援装置１００が制御される。 The output unit 230 outputs the control content determined in step S140 or step S150 (S160). Thereby, the motion support apparatus 100 is controlled based on the determined control content.
　その後、第２取得部２４０は、快適性情報を取得する（Ｓ１７０）。例えば、入力装置３００がディスプレイを有する場合、第２取得部２４０は、図７に示すようなグラフィカルユーザーインターフェース（ＧＵＩ）を介して、屋内におけるユーザの快適性を示す値を取得する。なお、図７のＧＵＩでは、スライダーを用いて快適性の値が入力されるが、これに限定される必要はない。ＧＵＩは、数値が直接入力されるテキストボックスを含んでもよいし、数値増加／減少ボタンを含んでもよいし、これらの組合せを含んでもよい。 Thereafter, the second acquisition unit 240 acquires comfort information (S170). For example, when the input device 300 includes a display, the second acquisition unit 240 acquires a value indicating the comfort of the user indoors via a graphical user interface (GUI) as illustrated in FIG. In the GUI of FIG. 7, the comfort value is input using a slider, but it is not necessary to be limited to this. The GUI may include a text box in which a numerical value is directly input, a numerical value increase / decrease button, or a combination thereof.
　続いて、更新部２５０は、動作状態情報及び快適性情報に基づいて、複数の制御内容の価値を更新する（Ｓ１８０）。このとき、快適性情報に基づく値が強化学習における報酬として用いられる。快適性情報に基づく値とは、快適性を示す値であり、例えば、快適性が高いほど増加する値である。 Subsequently, the updating unit 250 updates the values of the plurality of control contents based on the operation state information and the comfort information (S180). At this time, a value based on the comfort information is used as a reward in reinforcement learning. The value based on the comfort information is a value indicating comfort, for example, a value that increases as the comfort increases.
　さらに、更新部２５０は、更新された価値に基づいてニューラルネットワークのパラメータを更新する（Ｓ１９０）。つまり、更新部２５０は、更新された各制御内容の価値を教師信号として入力することにより、複数階層のニューラルネットワークのパラメータを学習する。 Furthermore, the update unit 250 updates the parameters of the neural network based on the updated value (S190). That is, the update unit 250 learns the parameters of a neural network having a plurality of layers by inputting the value of each updated control content as a teacher signal.
　このようなステップＳ１８０及びステップＳ１９０の処理が内部的に繰り返されることにより、いわゆる深層強化学習が行われる。なお、深層強化学習については、特に限定される必要はなく、従来技術が用いられてもよい。したがって、深層強化学習の詳細な説明については省略する。 The so-called deep reinforcement learning is performed by internally repeating the processes in steps S180 and S190. The deep reinforcement learning is not particularly limited, and a conventional technique may be used. Therefore, detailed description of the deep reinforcement learning is omitted.
　なお、快適性情報の取得は、制御内容の決定のたびに行われなくてもよい。つまり、ステップＳ１７０はスキップされてもよい。この場合、更新部２５０は、予め定められた値（例えば０）を報酬として用いて各制御内容の価値を学習してもよい。 In addition, acquisition of comfort information does not need to be performed every time the content of control is determined. That is, step S170 may be skipped. In this case, the update unit 250 may learn the value of each control content using a predetermined value (for example, 0) as a reward.
　［効果］
　以上のように、本実施の形態に係る制御内容決定装置２００は、動作支援装置１００の動作状態を示す動作状態情報を取得する第１取得部２１０と、（i）制御内容決定ルールに従って、動作支援装置１００の制御内容を動作状態情報から決定する、又は（ii）ランダムに動作支援装置１００の制御内容を決定する決定部２２０と、決定された制御内容を出力する出力部２３０と、動作支援装置１００による動作支援の快適さを示す快適性情報を取得する第２取得部２４０と、動作状態情報及び快適性情報に基づいて、制御内容決定ルールを更新する更新部２５０と、を備え、決定部２２０は、確率εで（ii）の決定を選択する。 [effect]
As described above, the control content determination device 200 according to the present embodiment operates according to the first acquisition unit 210 that acquires the operation state information indicating the operation state of the operation support device 100, and (i) the control content determination rule. The control content of the support device 100 is determined from the operation state information, or (ii) the control content of the motion support device 100 is determined at random, the output unit 230 that outputs the determined control content, and the motion support A second acquisition unit 240 that acquires comfort information indicating comfort of operation support by the apparatus 100; and an update unit 250 that updates a control content determination rule based on the operation state information and comfort information. The unit 220 selects the determination of (ii) with the probability ε.
　この構成により、更新部２５０は、快適性情報に基づいて制御内容決定ルールを更新することができる。したがって、制御内容決定装置２００は、ユーザの快適性の向上に適した制御内容決定ルールを学習することができ、個々のユーザに適応した動作支援を実現することができる。さらに、決定部２２０は、確率εでランダムな決定を選択するので、現在の制御内容決定ルールに縛られることなく、最適な制御内容を探査することができる。つまり、制御内容決定装置２００は、探査と学習結果の利用とのバランスを図ることができ、制御内容決定ルールを効果的に更新することができる。 With this configuration, the update unit 250 can update the control content determination rule based on the comfort information. Therefore, the control content determination apparatus 200 can learn a control content determination rule suitable for improving the comfort of the user, and can realize operation support adapted to each user. Furthermore, since the determination unit 220 selects a random determination with the probability ε, the optimal control content can be searched without being bound by the current control content determination rule. That is, the control content determination device 200 can balance the search and use of the learning result, and can effectively update the control content determination rule.
　また、本実施の形態に係る制御内容決定装置２００において、制御内容決定ルールは、動作状態情報から複数の制御内容の各々の価値を推定するためのニューラルネットワークで表され、更新部２５０は、快適性情報に基づく値を報酬として用いて複数の制御内容の価値を更新し、更新された価値に基づいてニューラルネットワークのパラメータを更新する。 Moreover, in the control content determination device 200 according to the present embodiment, the control content determination rule is represented by a neural network for estimating the value of each of the plurality of control content from the operation state information, and the updating unit 250 is comfortable A value based on sex information is used as a reward to update the value of a plurality of control contents, and a parameter of the neural network is updated based on the updated value.
　この構成により、いわゆる深層強化学習を制御内容決定装置２００に適用することができ、制御内容決定装置２００は、よりユーザに適した制御内容決定ルールを構築することができる。その結果、制御内容決定装置２００は、個々のユーザに適した動作支援を実現することができる。 With this configuration, so-called deep reinforcement learning can be applied to the control content determination device 200, and the control content determination device 200 can construct a control content determination rule more suitable for the user. As a result, the control content determination apparatus 200 can realize operation support suitable for each user.
　また、本実施の形態に係る制御内容決定装置２００において、第２取得部２４０は、音声認識により所定のキーワードの発言を検出することにより快適性情報を取得してもよい。 Moreover, in the control content determination apparatus 200 according to the present embodiment, the second acquisition unit 240 may acquire the comfort information by detecting an utterance of a predetermined keyword by voice recognition.
　この構成により、制御内容決定装置２００は、ユーザの快適性情報の入力の負荷を軽減することができ、ユーザの利便性を向上させることができる。 With this configuration, the control content determination apparatus 200 can reduce the burden of input of user comfort information, and can improve user convenience.
　（実施の形態２）
　次に、実施の形態２について説明する。実施の形態２では、動作支援が行われたときのユーザの安全性を示す安全度に基づいて複数の制御内容の中から１以上の制御内容を抽出し、抽出された１以上の制御内容の中からランダムに制御内容が決定される点が上記実施の形態１と主として異なる。以下に、実施の形態１と異なる点を中心に実施の形態２について説明する。 (Embodiment 2)
Next, a second embodiment will be described. In the second embodiment, one or more control contents are extracted from a plurality of control contents based on the safety level indicating the safety of the user when the operation support is performed, and the extracted one or more control contents The main difference from Embodiment 1 is that the contents of control are determined at random from the inside. The second embodiment will be described below with a focus on differences from the first embodiment.
　［制御内容決定装置の構成］
　実施の形態２に係る制御内容決定装置の詳細な構成について説明する。図８は、実施の形態２に係る制御内容決定装置２００Ａの機能構成を示すブロック図である。図８に示すように、制御内容決定装置２００Ａは、第１取得部２１０と、決定部２２０Ａと、出力部２３０と、第２取得部２４０と、更新部２５０Ａと、検知部２６０Ａと、を備える。 [Configuration of control content determination device]
A detailed configuration of the control content determination apparatus according to Embodiment 2 will be described. FIG. 8 is a block diagram illustrating a functional configuration of the control content determination apparatus 200A according to the second embodiment. As shown in FIG. 8, the control content determination device 200A includes a first acquisition unit 210, a determination unit 220A, an output unit 230, a second acquisition unit 240, an update unit 250A, and a detection unit 260A. .
　決定部２２０Ａは、（i）制御内容決定ルールに従って、動作支援装置１００の制御内容を動作状態情報から決定する、又は（ii）ランダムに制御内容を決定する。ここで、（ii）の場合に、決定部２２０Ａは、安全度情報を参照して、安全度が所定の条件を満たす１以上の制御内容の中からランダムに制御内容を決定する。 The determining unit 220A (i) determines the control content of the operation support apparatus 100 from the operation state information according to the control content determination rule, or (ii) determines the control content at random. Here, in the case of (ii), the determination unit 220A refers to the safety level information, and randomly determines the control content from among one or more control content in which the safety level satisfies a predetermined condition.
　安全度情報とは、複数の制御内容の各々に対して安全度が対応付けられた情報である。安全度は、動作支援が行われたときのユーザの安全性を示す値である。例えば、安全度情報は、複数の制御内容の各々に対して安全さを表す値が対応付けられたテーブルである。安全度情報は、図示しない記憶部に記憶されている。 Safety level information is information in which a safety level is associated with each of a plurality of control contents. The safety level is a value indicating the safety of the user when the operation support is performed. For example, the safety level information is a table in which a value representing safety is associated with each of a plurality of control contents. The safety level information is stored in a storage unit (not shown).
　所定の条件は、安全性が高い制御内容を決定するための条件である。例えば、所定の条件は、安全度の値が予め定められた閾値より大きいことである。 The predetermined condition is a condition for determining the control content with high safety. For example, the predetermined condition is that the safety value is larger than a predetermined threshold value.
　例えば、決定部２２０Ａは、安全度情報を参照して、複数の制御内容の中から、閾値より大きい安全度の値を有する１以上の制御内容を抽出する。そして、決定部２２０Ａは、抽出された１以上の制御内容からランダムに制御内容を決定する。 For example, the determination unit 220A refers to the safety degree information and extracts one or more control contents having a safety degree value larger than the threshold value from a plurality of control contents. Then, the determination unit 220A randomly determines the control contents from the extracted one or more control contents.
　検知部２６０Ａは、ユーザが安全かどうかを検知する。つまり、検知部２６０Ａは、ランダムに制御内容が決定された場合に、その決定された制御内容に基づく動作支援が安全か否かを決定する。例えば、ユーザが転倒した場合に、検知部２６０Ａは、ユーザが安全でない（つまり危険）と検知する。 The detection unit 260A detects whether the user is safe. That is, when the control content is randomly determined, the detection unit 260A determines whether the operation support based on the determined control content is safe. For example, when the user falls, the detection unit 260A detects that the user is not safe (that is, dangerous).
　更新部２５０Ａは、実施の形態１と同様に、動作状態情報と快適性情報とに基づいて、制御内容決定ルールを更新する。本実施の形態に係る更新部２５０Ａは、さらに、ランダムに制御内容が決定された場合に、検知部２６０Ａによる検知結果に基づいて、安全度情報を更新する。例えば、ユーザが安全でないと検知された場合に、更新部２５０Ａは、決定された制御内容の安全度の値を減少させる。逆に、例えばユーザが安全であると検知された場合に、更新部２５０Ａは、決定された制御内容の安全度の値を増加させる。 The update unit 250A updates the control content determination rule based on the operation state information and the comfort information as in the first embodiment. The update unit 250A according to the present embodiment further updates the safety level information based on the detection result by the detection unit 260A when the control content is randomly determined. For example, when it is detected that the user is not safe, the updating unit 250A decreases the value of the safety level of the determined control content. Conversely, for example, when it is detected that the user is safe, the update unit 250A increases the value of the safety level of the determined control content.
　［制御内容決定装置の動作］
　次に、以上のように構成された制御内容決定装置２００Ａの動作について図９を参照しながら説明する。図９は、実施の形態２に係る制御内容決定装置２００Ａの処理を示すフローチャートである。 [Operation of control content determination device]
Next, the operation of the control content determination apparatus 200A configured as described above will be described with reference to FIG. FIG. 9 is a flowchart showing processing of the control content determination apparatus 200A according to the second embodiment.
　ステップＳ１３０において（ii）の決定が選択された場合（Ｓ１３０のε）、決定部２２０Ａは、安全度情報に基づいて、複数の制御内容の中から１以上の制御内容を抽出する（Ｓ１３２Ａ）。例えば、決定部２２０Ａは、安全度情報を参照して、複数の制御内容ａ１～ａｎの中から、予め定められた閾値（例えば５０）より大きい安全度の値を有する制御内容を抽出する。 When the determination of (ii) is selected in step S130 (ε of S130), the determination unit 220A extracts one or more control contents from a plurality of control contents based on the safety degree information (S132A). For example, the determination unit 220A refers to the safety level information, and extracts control content having a safety level value greater than a predetermined threshold value (for example, 50) from the plurality of control content levels a1 to an.
　そして、決定部２２０Ａは、抽出された制御内容の中からランダムに制御内容を決定する（Ｓ１４０Ａ）。 Then, the determination unit 220A randomly determines the control content from the extracted control content (S140A).
　その後、ステップＳ１６０～Ｓ１９０が実行され、ランダムに制御内容が決定されていない場合は（Ｓ１９２ＡのＮｏ）、そのまま処理を終了する。一方、ランダムに制御内容が決定されていた場合は（Ｓ１９２ＡのＹｅｓ）、検知部２６０Ａは、動作支援が安全であったかどうかを検知する（Ｓ１９４Ａ）。更新部２５０Ａは、検知部２６０Ａによる検知結果に基づいて安全度情報を更新する（Ｓ１９６Ａ）。 Thereafter, Steps S160 to S190 are executed, and when the control content is not determined at random (No in S192A), the process is terminated as it is. On the other hand, when the control content is determined at random (Yes in S192A), the detection unit 260A detects whether the operation support is safe (S194A). The update unit 250A updates the safety level information based on the detection result by the detection unit 260A (S196A).
　［効果］
　以上のように、本実施の形態に係る制御内容決定装置２００Ａにおいて、決定部２２０Ａは、ランダムに制御内容が決定される場合に、複数の制御内容の各々について当該制御内容に基づいて動作支援が行われたときのユーザの安全性を示す安全度が対応付けられた安全度情報を参照して、安全度が所定の条件を満たす１以上の制御内容の中からランダムに制御内容を決定する。 [effect]
As described above, in the control content determination device 200A according to the present embodiment, when the control content is randomly determined, the determination unit 220A provides operation support for each of the plurality of control content based on the control content. With reference to the safety level information associated with the safety level indicating the safety level of the user when it is performed, the control level is randomly determined from one or more control levels that satisfy the predetermined level.
　この構成により、決定部２２０Ａは、ランダムに制御内容を決定したときに、ユーザに危険が生じる可能性を低減させることができる。つまり、決定部２２０Ａは、ランダムな決定において、ユーザに危険が生じる制御内容が決定されることを抑制することができる。 With this configuration, the determination unit 220A can reduce the possibility of danger to the user when determining the control content at random. That is, the determination unit 220 </ b> A can suppress the determination of the control content causing danger to the user in the random determination.
　また、本実施の形態に係る制御内容決定装置２００Ａは、さらに、動作支援が危険であったかどうかを検知する検知部２６０Ａを備え、更新部２５０Ａは、ランダムに制御内容が決定された場合に、さらに、検知部２６０Ａによる検知結果に基づいて安全度情報を更新する。 In addition, the control content determination device 200A according to the present embodiment further includes a detection unit 260A that detects whether the operation support is dangerous, and the update unit 250A further includes a control content when the control content is randomly determined. The safety degree information is updated based on the detection result by the detection unit 260A.
　この構成により、更新部２５０Ａは、動作支援によってユーザに危険が生じたかどうかの検知結果に基づいて危険度情報を更新することができ、危険度情報の改良を図ることができる。したがって、決定部２２０Ａは、ランダムな決定において、ユーザに危険を生じさせる制御内容が決定されることを抑制することができる。 With this configuration, the updating unit 250A can update the risk level information based on the detection result as to whether or not the user is at risk due to the operation support, and can improve the risk level information. Therefore, the determination unit 220 </ b> A can suppress the determination of the control content causing danger to the user in the random determination.
　（他の実施の形態）
　以上、本発明の１つまたは複数の態様に係る制御内容決定装置について、実施の形態に基づいて説明したが、本発明は、この実施の形態に限定されるものではない。本発明の趣旨を逸脱しない限り、当業者が思いつく各種変形を本実施の形態に施したものや、異なる実施の形態における構成要素を組み合わせて構築される形態も、本発明の１つまたは複数の態様の範囲内に含まれてもよい。 (Other embodiments)
The control content determination device according to one or more aspects of the present invention has been described based on the embodiment, but the present invention is not limited to this embodiment. Unless it deviates from the gist of the present invention, one or more of the present invention may be applied to various modifications that can be conceived by those skilled in the art, or forms constructed by combining components in different embodiments. It may be included within the scope of the embodiments.
　なお、上記各実施の形態では、第２取得部２４０は、入力装置３００から受信した情報に基づいて快適性情報を取得していたが、入力装置３００だけではなくセンサ１２０から受信した情報にも基づいて、快適性情報を取得してもよい。例えば、第２取得部２４０は、センサ１２０から受信した情報を用いて、入力装置３００から受信した情報を修正することにより快適性情報を取得してもよい。具体的には、第２取得部２４０は、ユーザの表情、脳波又は心拍数に基づいて、入力装置３００から受信した情報を修正してもよい。この場合、センサ１２０は、画像センサ、脳波センサ、又は、心拍センサを含めばよい。 In each of the above embodiments, the second acquisition unit 240 acquires the comfort information based on the information received from the input device 300. However, the second acquisition unit 240 acquires not only the input device 300 but also the information received from the sensor 120. Based on this, comfort information may be acquired. For example, the second acquisition unit 240 may acquire the comfort information by correcting the information received from the input device 300 using the information received from the sensor 120. Specifically, the second acquisition unit 240 may correct the information received from the input device 300 based on the user's facial expression, brain wave, or heart rate. In this case, the sensor 120 may include an image sensor, an electroencephalogram sensor, or a heart rate sensor.
　なお、上記各実施の形態では、深層強化学習を利用して、ユーザに適応した制御内容の決定を学習していたが、深層強化学習に限定されなくてもよい。例えば、制御内容決定ルールは、多階層のニューラルネットワークではなく、単階層のニューラルネットワークで表されてもよい。また、制御内容決定ルールは、ニューラルネットワークではなく、他の数学モデル（例えば、線形回帰、サポートベクタマシンなど）で表されてもよい。 In each of the above embodiments, the determination of the control content adapted to the user is learned using the deep reinforcement learning. However, the embodiment is not limited to the deep reinforcement learning. For example, the control content determination rule may be represented not by a multi-layer neural network but by a single-layer neural network. Further, the control content determination rule may be expressed not by a neural network but by another mathematical model (for example, linear regression, support vector machine, etc.).
　なお、上記各実施の形態では、主として２つの決定（（i）制御内容決定ルールに従って、動作状態情報から動作支援装置１００の制御内容を決定する、又は、（ii）ランダムに動作支援装置１００の制御内容を決定する）について説明したが、必ずしも２つの決定に限定される必要はない。例えば、３以上の決定の中から１つの決定が選択されてもよい。つまり、決定部は、（i）の決定及び（ii）の決定を含む複数の決定のうちのいずれかを選択的に実行すればよく、このとき、（ii）の決定が確率εで選択されればよい。 In each of the above embodiments, the control content of the motion support apparatus 100 is determined from the motion state information mainly according to two determinations ((i) control content determination rules, or (ii) the motion support apparatus 100 is randomly selected. Although the control content is determined), it is not necessarily limited to two determinations. For example, one decision may be selected from three or more decisions. In other words, the determination unit only needs to selectively execute one of a plurality of determinations including the determination of (i) and the determination of (ii). At this time, the determination of (ii) is selected with the probability ε. Just do it.
　なお、上記各実施の形態では、制御内容決定装置は、単一の装置で実現されていたが、互いに接続された複数の装置で実現されてもよい。例えば、制御内容決定装置は、クラウドコンピューティングによって実現されてもよい。 In addition, in each said embodiment, although the control content determination apparatus was implement | achieved by the single apparatus, you may implement | achieve by the several apparatus connected mutually. For example, the control content determination device may be realized by cloud computing.
　なお、上記実施の形態２では、安全度情報が更新されていたが、必ずしも安全度情報は更新されなくてもよい。この場合、制御内容決定装置２００Ａは検知部２６０Ａを備えなくてもよい。 In the second embodiment, the safety level information is updated. However, the safety level information is not necessarily updated. In this case, the control content determination device 200A may not include the detection unit 260A.
　また、上記各実施の形態における制御内容決定装置が備える構成要素の一部又は全部は、１個のシステムＬＳＩ（Ｌａｒｇｅ　Ｓｃａｌｅ　Ｉｎｔｅｇｒａｔｉｏｎ：大規模集積回路）から構成されているとしてもよい。例えば、制御内容決定装置２００は、第１取得部２１０と、決定部２２０と、出力部２３０と、第２取得部２４０と、更新部２５０とを有するシステムＬＳＩから構成されてもよい。 In addition, some or all of the components included in the control content determination device in each of the above embodiments may be configured by a single system LSI (Large Scale Integration). For example, the control content determination apparatus 200 may be configured by a system LSI having a first acquisition unit 210, a determination unit 220, an output unit 230, a second acquisition unit 240, and an update unit 250.
　システムＬＳＩは、複数の構成部を１個のチップ上に集積して製造された超多機能ＬＳＩであり、具体的には、マイクロプロセッサ、ＲＯＭ（Ｒｅａｄ　Ｏｎｌｙ　Ｍｅｍｏｒｙ）、ＲＡＭ（Ｒａｎｄｏｍ　Ａｃｃｅｓｓ　Ｍｅｍｏｒｙ）などを含んで構成されるコンピュータシステムである。前記ＲＯＭには、コンピュータプログラムが記憶されている。前記マイクロプロセッサが、前記コンピュータプログラムに従って動作することにより、システムＬＳＩは、その機能を達成する。 The system LSI is an ultra-multifunctional LSI manufactured by integrating a plurality of components on one chip. Specifically, a microprocessor, a ROM (Read Only Memory), a RAM (Random Access Memory), etc. It is a computer system comprised including. A computer program is stored in the ROM. The system LSI achieves its functions by the microprocessor operating according to the computer program.
　なお、ここでは、システムＬＳＩとしたが、集積度の違いにより、ＩＣ、ＬＳＩ、スーパーＬＳＩ、ウルトラＬＳＩと呼称されることもある。また、集積回路化の手法はＬＳＩに限るものではなく、専用回路または汎用プロセッサで実現してもよい。ＬＳＩ製造後に、プログラムすることが可能なＦＰＧＡ（Ｆｉｅｌｄ　Ｐｒｏｇｒａｍｍａｂｌｅ　Ｇａｔｅ　Ａｒｒａｙ）、あるいはＬＳＩ内部の回路セルの接続や設定を再構成可能なリコンフィギュラブル・プロセッサを利用してもよい。 Note that although the system LSI is used here, it may be called IC, LSI, super LSI, or ultra LSI depending on the degree of integration. Further, the method of circuit integration is not limited to LSI's, and implementation using dedicated circuitry or general purpose processors is also possible. An FPGA (Field Programmable Gate Array) that can be programmed after manufacturing the LSI or a reconfigurable processor that can reconfigure the connection and setting of the circuit cells inside the LSI may be used.
　さらには、半導体技術の進歩または派生する別技術によりＬＳＩに置き換わる集積回路化の技術が登場すれば、当然、その技術を用いて機能ブロックの集積化を行ってもよい。バイオ技術の適用等が可能性としてありえる。 Furthermore, if integrated circuit technology that replaces LSI emerges as a result of advances in semiconductor technology or other derived technology, it is naturally also possible to integrate functional blocks using this technology. Biotechnology can be applied.
　また、本発明の一態様は、このような制御内容決定装置だけではなく、制御内容決定装置に含まれる特徴的な構成部をステップとする制御内容決定方法であってもよい。また、本発明の一態様は、制御内容決定方法に含まれる特徴的な各ステップをコンピュータに実行させるコンピュータプログラムであってもよい。また、本発明の一態様は、そのようなコンピュータプログラムが記録された、コンピュータ読み取り可能な非一時的な記録媒体であってもよい。 Further, one aspect of the present invention may be a control content determination method that uses not only such a control content determination device but also a characteristic component included in the control content determination device as a step. Further, one aspect of the present invention may be a computer program that causes a computer to execute each characteristic step included in the control content determination method. One embodiment of the present invention may be a computer-readable non-transitory recording medium in which such a computer program is recorded.
　なお、上記各実施の形態において、各構成要素は、専用のハードウェアで構成されるか、各構成要素に適したソフトウェアプログラムを実行することによって実現されてもよい。各構成要素は、ＣＰＵまたはプロセッサなどのプログラム実行部が、ハードディスクまたは半導体メモリなどの記録媒体に記録されたソフトウェアプログラムを読み出して実行することによって実現されてもよい。ここで、上記各実施の形態の制御内容決定装置などを実現するソフトウェアは、次のようなプログラムである。 In each of the above embodiments, each component may be configured by dedicated hardware or may be realized by executing a software program suitable for each component. Each component may be realized by a program execution unit such as a CPU or a processor reading and executing a software program recorded on a recording medium such as a hard disk or a semiconductor memory. Here, the software that realizes the control content determination device of each of the above embodiments is a program as follows.
　すなわち、このプログラムは、コンピュータに、ユーザの動作を機械的に支援する動作支援装置の動作状態を示す動作状態情報を取得する第１取得ステップと、（i）制御内容決定ルールに従って、前記動作支援装置の制御内容を前記動作状態情報から決定する、又は（ii）ランダムに前記動作支援装置の制御内容を決定する決定ステップと、決定された前記制御内容を出力する出力ステップと、前記動作支援装置による動作支援の快適さを示す快適性情報を取得する第２取得ステップと、前記動作状態情報及び前記快適性情報に基づいて、前記制御内容決定ルールを更新する更新ステップと、を含む制御内容決定方法を実行させ、前記決定ステップでは、確率εで前記（ii）の決定を選択する。 That is, the program includes a first acquisition step of acquiring operation state information indicating an operation state of an operation support apparatus that mechanically supports a user operation, and (i) the operation support according to a control content determination rule. Determining a control content of a device from the operation state information; or (ii) determining a control content of the motion support device at random; an output step of outputting the determined control content; and the motion support device Control content determination including a second acquisition step of acquiring comfort information indicating the comfort of the operation support by, and an update step of updating the control content determination rule based on the operation state information and the comfort information In the determination step, the determination of (ii) is selected with probability ε.
　１００　動作支援装置
　２００、２００Ａ　制御内容決定装置
　２１０　第１取得部
　２２０、２２０Ａ　決定部
　２３０　出力部
　２４０　第２取得部
　２５０、２５０Ａ　更新部
　２６０Ａ　検知部 DESCRIPTION OF SYMBOLS 100 Operation support apparatus 200,200A Control content determination apparatus 210 1st acquisition part 220,220A determination part 230 Output part 240 2nd acquisition part 250,250A Update part 260A Detection part

Claims (7)

  1. 　ユーザの動作を機械的に支援する動作支援装置の動作状態を示す動作状態情報を取得する第１取得部と、
    　（i）制御内容決定ルールに従って、前記動作支援装置の制御内容を前記動作状態情報から決定する、又は（ii）ランダムに前記動作支援装置の制御内容を決定する決定部と、
    　決定された前記制御内容を出力する出力部と、
    　前記動作支援装置による動作支援の快適さを示す快適性情報を取得する第２取得部と、
    　前記動作状態情報及び前記快適性情報に基づいて、前記制御内容決定ルールを更新する更新部と、を備え、
    　前記決定部は、確率εで前記（ii）の決定を選択する、
    　制御内容決定装置。     A first acquisition unit that acquires operation state information indicating an operation state of an operation support device that mechanically supports a user operation;
    (I) According to the control content determination rule, the control content of the operation support device is determined from the operation state information, or (ii) the determination unit that randomly determines the control content of the operation support device;
    An output unit for outputting the determined control content;
    A second acquisition unit that acquires comfort information indicating comfort of operation support by the operation support device;
    An update unit that updates the control content determination rule based on the operation state information and the comfort information,
    The determination unit selects the determination of (ii) with a probability ε.
    Control content determination device.
  2. 　前記制御内容決定ルールは、動作情報から複数の制御内容の各々の価値を推定するためのニューラルネットワークで表され、
    　前記更新部は、前記動作情報に基づく値を報酬として用いて前記複数の制御内容の価値を更新し、更新された前記価値に基づいて前記ニューラルネットワークのパラメータを更新する、
    　請求項１に記載の制御内容決定装置。     The control content determination rule is represented by a neural network for estimating the value of each of a plurality of control content from operation information,
    The update unit updates a value of the plurality of control contents using a value based on the operation information as a reward, and updates a parameter of the neural network based on the updated value.
    The control content determination apparatus according to claim 1.
  3. 　前記第２取得部は、音声認識により所定のキーワードの発言を検出することにより前記快適性情報を取得する、
    　請求項１又は２に記載の制御内容決定装置。     The second acquisition unit acquires the comfort information by detecting an utterance of a predetermined keyword by voice recognition.
    The control content determination apparatus according to claim 1 or 2.
  4. 　前記決定部は、前記（ii）の決定において、複数の制御内容の各々について当該制御内容に基づいて動作支援が行われたときのユーザの安全性を示す安全度が対応付けられた安全度情報を参照して、前記安全度が所定の条件を満たす１以上の制御内容の中からランダムに前記制御内容を決定する、
    　請求項１～３のいずれか１項に記載の制御内容決定装置。     In the determination of (ii), the determination unit includes safety level information associated with a safety level indicating safety of the user when operation support is performed based on the control content for each of the plurality of control content. Referring to the above, the control content is determined at random from one or more control content satisfying a predetermined degree of safety,
    The control content determination device according to any one of claims 1 to 3.
  5. 　前記制御内容決定装置は、さらに、動作支援が安全であったかどうかを検知する検知部を備え、
    　前記更新部は、前記（ii）の決定が選択された場合に、さらに、前記検知部による検知結果に基づいて前記安全度情報を更新する、
    　請求項４に記載の制御内容決定装置。     The control content determination device further includes a detection unit that detects whether the operation support is safe,
    The update unit further updates the safety degree information based on a detection result by the detection unit when the determination of (ii) is selected.
    The control content determination apparatus according to claim 4.
  6. 　ユーザの動作を機械的に支援する動作支援装置の動作状態を示す動作状態情報を取得する第１取得ステップと、
    　（i）制御内容決定ルールに従って、前記動作支援装置の制御内容を前記動作状態情報から決定する、又は（ii）ランダムに前記動作支援装置の制御内容を決定する決定ステップと、
    　決定された前記制御内容を出力する出力ステップと、
    　前記動作支援装置による動作支援の快適さを示す快適性情報を取得する第２取得ステップと、
    　前記動作状態情報及び前記快適性情報に基づいて、前記制御内容決定ルールを更新する更新ステップと、を含み、
    　前記決定ステップでは、確率εで前記（ii）の決定を選択する、
    　制御内容決定方法。     A first acquisition step of acquiring operation state information indicating an operation state of an operation support device that mechanically supports a user's operation;
    (I) According to the control content determination rule, the control content of the operation support device is determined from the operation state information, or (ii) the control content of the operation support device is determined at random.
    An output step for outputting the determined control content;
    A second acquisition step of acquiring comfort information indicating comfort of operation support by the operation support device;
    Updating the control content determination rule based on the operating state information and the comfort information,
    In the determination step, the determination of (ii) is selected with probability ε.
    Control content determination method.
  7. 　請求項６に記載の制御内容決定方法をコンピュータに実行させるためのプログラム。 A program for causing a computer to execute the control content determination method according to claim 6.
PCT/JP2018/001123 2017-01-30 2018-01-17 Control content determination device and control content determination method WO2018139292A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2017-014847 2017-01-30
JP2017014847A JP2020054408A (en) 2017-01-30 2017-01-30 Control contents decision device and control contents decision method

Publications (1)

Publication Number Publication Date
WO2018139292A1 true WO2018139292A1 (en) 2018-08-02

Family

ID=62978884

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2018/001123 WO2018139292A1 (en) 2017-01-30 2018-01-17 Control content determination device and control content determination method

Country Status (2)

Country Link
JP (1) JP2020054408A (en)
WO (1) WO2018139292A1 (en)

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2002013271A (en) * 2000-04-25 2002-01-18 Matsushita Electric Works Ltd Handrail-position proposal system, method of proposing position of handrail, and recording medium on which handrail-position proposed program is recorded
JP2009288934A (en) * 2008-05-28 2009-12-10 Sony Corp Data processing apparatus, data processing method, and computer program
JP2011505030A (en) * 2007-10-31 2011-02-17 シーメンス アクチエンゲゼルシヤフト A method for exploring the state of technical systems with computer assistance.
WO2012048123A1 (en) * 2010-10-06 2012-04-12 Berkeley Bionics Human machine interfaces for lower extremity orthotics
JP2013162896A (en) * 2012-02-10 2013-08-22 Fuji Mach Mfg Co Ltd Action setting method
JP2016517104A (en) * 2013-03-26 2016-06-09 シーメンス アクチエンゲゼルシヤフトＳｉｅｍｅｎｓ Ａｋｔｉｅｎｇｅｓｅｌｌｓｃｈａｆｔ Method for open-loop control and / or closed-loop control of a technical system with computer assistance

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2002013271A (en) * 2000-04-25 2002-01-18 Matsushita Electric Works Ltd Handrail-position proposal system, method of proposing position of handrail, and recording medium on which handrail-position proposed program is recorded
JP2011505030A (en) * 2007-10-31 2011-02-17 シーメンス アクチエンゲゼルシヤフト A method for exploring the state of technical systems with computer assistance.
JP2009288934A (en) * 2008-05-28 2009-12-10 Sony Corp Data processing apparatus, data processing method, and computer program
WO2012048123A1 (en) * 2010-10-06 2012-04-12 Berkeley Bionics Human machine interfaces for lower extremity orthotics
JP2013162896A (en) * 2012-02-10 2013-08-22 Fuji Mach Mfg Co Ltd Action setting method
JP2016517104A (en) * 2013-03-26 2016-06-09 シーメンス アクチエンゲゼルシヤフトＳｉｅｍｅｎｓ Ａｋｔｉｅｎｇｅｓｅｌｌｓｃｈａｆｔ Method for open-loop control and / or closed-loop control of a technical system with computer assistance

Also Published As

Publication number Publication date
JP2020054408A (en) 2020-04-09

Similar Documents

Publication Publication Date Title
US20220249311A1 (en) Semi-supervised intent recognition system and method
US9962305B2 (en) Living support system and living support method
JP6967717B2 (en) Assist device and how to operate the assist device
JP6194971B2 (en) Providing personalized nursing based on user health records
JP6884526B2 (en) Standing assistance method and equipment
JP6945145B2 (en) Assist device and how to operate the assist device
JP7054801B2 (en) Assist device and assist method
WO2018012219A1 (en) Autonomous-behavior-type robot
Tomari et al. Development of smart wheelchair system for a user with severe motor impairment
JP6941817B2 (en) Assist device and how to operate the assist device
US20210259374A1 (en) Walking assistance method and apparatuses
JPWO2010050113A1 (en) Moving body control device and moving body control method
US10426682B2 (en) Robot, robot control method, method, and recording medium
JP2011000924A (en) Moving body, system including the same, method of operating the moving body, and program
JPWO2019003550A1 (en) Assist device and assist method
CN109662868A (en) Auxiliary device, householder method and program
WO2018139292A1 (en) Control content determination device and control content determination method
JPWO2018181640A1 (en) Joint structure suitable for robot joints
JP2018121697A (en) Adaptation information generation device and adaptation information generation method for movement assisting device
JP2018134724A (en) Operation estimation device and operation auxiliary device
US20220000701A1 (en) Assist device
Dometios et al. Towards ICT-supported bath robots: Control architecture description and localized perception of user for robot motion planning
JP7349038B1 (en) Care robot control device and care robot system
JP2019201761A (en) Massage data update device and massage machine
WO2022249523A1 (en) Assisting device

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 18744337

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 18744337

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: JP