WO2019189671A1 - Force-tactile transmitting system, force-tactile transmitting device, force-tactile transmitting method and program - Google Patents

Abstract

[Problem] To provide technology that can realize an experience with enhanced reality via a robot. [Solution] Provided is a force-tactile transmitting system that includes: an angling robot capable of performing an angling motion and a fishing-rod-shaped manipulator spaced from the angling robot; and a control device configured to be communicable with the angling robot and the fishing-rod-shaped manipulator, wherein, on the basis of information indicative of a current or prior external force inputted to a fishing rod mounted to the angling robot, the control device causes the fishing-rod-shaped manipulator to output a force-tactile sense generated by an external force inputted to the fishing rod.

Description

Force-tactile transmission system, force-tactile transmission device, force-tactile transmission method, and program

The present invention relates to a force / haptic transmission system, a force / haptic transmission device, a force / haptic transmission method, and a program.

In recent years, robots called avatars acting on behalf of users have been developed.
By using an avatar, even if the user is remote, the user can perform various experiences via the avatar by feeding back to the user the visual information or auditory information collected by the avatar.
In addition, the technique regarding an avatar is described in patent document 1, for example.

JP 2017-169839

However, what a user can experience using an avatar is limited to contents centered on visual information or auditory information, and reality is not sufficiently high. In addition, although proposals have been made to implement a technology for transmitting tactile sensation in an avatar, it has not yet been possible to realize a high-reality experience using an avatar.
That is, in the conventional technology, a technology capable of realizing a high reality experience via a robot such as an avatar has not been provided.
An object of the present invention is to provide a technique that enables a more realistic experience to be realized via a robot.

In order to solve the above problems, a force / tactile sensation transmission system according to one aspect of the present invention provides:
A force / tactile sensation transmission system including a fishing robot capable of performing a fishing operation, a fishing rod type manipulator installed apart from the fishing robot, and a control device configured to be able to communicate with the fishing robot and the fishing rod type manipulator. And
The controller is
Based on information indicating a current or past external force input to a fishing rod provided in the fishing robot, the fishing rod-type manipulator outputs a force / tactile force due to the external force input to the fishing rod.

According to the present invention, it is possible to provide a technology capable of realizing a more realistic experience via a robot.

It is a schematic diagram which shows the concept of the basic principle which concerns on this invention. It is a schematic diagram which shows the concept of control when the force / haptic transmission function is defined in the force / speed allocation conversion block FT by function. 1 is a schematic diagram showing a system configuration of a force / tactile sensation transmission system according to the present invention. It is a block diagram which shows the structure of a master side control apparatus. It is a schematic diagram which shows the structural example of the actuator used in a force-tactile transmission system. It is a flowchart which shows the flow of the force haptic transmission process which a force haptic transmission system performs. It is a flowchart which shows the flow of the remote fishing experience control process which a force-tactile transmission system performs. It is a schematic diagram which shows the other structural example of the actuator used in a force-tactile transmission system. It is a schematic diagram which shows the other structural example of the actuator used in a force-tactile transmission system. It is a schematic diagram which shows the other structural example of the actuator used in a force-tactile transmission system. It is a schematic diagram which shows the structural example of the fishing rod type manipulator comprised as a multi-axis mechanism. It is a schematic diagram which shows the structural example of the fishing robot comprised as a multi-axis mechanism.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.
First, basic principles applied to the force / tactile sensation transmission system, the force / tactile sensation transmission device, the force / tactile sensation transmission method, and the program according to the present invention will be described.

It should be noted that a human physical action is configured by individual “functions” such as one joint alone or in combination.
Therefore, hereinafter, in the present embodiment, “act” represents an integrated function realized by using individual “functions” of parts in the human body as components. For example, an action involving bending and stretching of the middle finger (an action of turning a screw or the like) is an integrated function having the function of each joint of the middle finger as a component.

(Basic principle)
The basic principle of the present invention is that any action can be mathematically expressed by three elements of force source, velocity (position) source, and transformation representing the action. Therefore, for a variable group defined by transformation and inverse transformation, By supplying control energy from the ideal force source and ideal velocity (position) source in a dual relationship to the controlled system, the extracted physical action is structured, reconstructed or expanded, and the physical action is reversible. Automatic realization (reproduction).

FIG. 1 is a schematic diagram showing the concept of the basic principle according to the present invention.
The basic principle shown in FIG. 1 represents a control law of an actuator that can be used to realize a human physical action. The current position of the actuator is used as an input, and at least one of position (or velocity) or force is used. The operation of the actuator is determined by performing the calculation in the region.
That is, the basic principle of the present invention is that the control target system S, the function-specific force / speed allocation conversion block FT, the ideal force source block FC or the ideal speed (position) source block PC, and the inverse conversion block IFT. It is expressed as a control law including

The control target system S is a robot operated by an actuator (for example, an avatar or the like installed at a location remote from the user), and controls the actuator based on acceleration or the like. Here, the control target system S realizes the function of one or a plurality of parts in the human body. If a control law for realizing the function is applied, the specific configuration is as follows. It is not necessarily a form that imitates the human body. For example, the control target system S can be a robot that controls an instrument (such as a fishing rod) used by a user with an actuator.

The function-specific force / speed allocation conversion block FT is a block that defines conversion of control energy into a speed (position) and force area set according to the function of the control target system S. Specifically, in the function-specific force / speed allocation conversion block FT, coordinate conversion is defined in which the function reference value (reference value) of the control target system S and the current position of the actuator are input. In this coordinate conversion, generally, an input vector whose elements are a reference value and a current speed (position) is converted into an output vector composed of a speed (position) for calculating a control target value of the speed (position), and a reference value In addition, an input vector having the current force as an element is converted into an output vector composed of a force for calculating a force control target value. Specifically, the coordinate conversion in the function-specific force / speed allocation conversion block FT is generalized as in the following equations (1) and (2).

In Equation (1), x ′ ₁ to x ′ _n (n is an integer of 1 or more) are velocity vectors for deriving the speed state value, and x ′ _a to x ′ _m (m is 1 or more). Is a vector whose elements are the reference value and the speed based on the action of the actuator (the speed of the actuator mover or the speed of the object moved by the actuator), and h _1a to h _nm are elements of the transformation matrix representing the function It is. In Expression (2), f ″ ₁ to f ″ _n (n is an integer of 1 or more) is a force vector for deriving force state values, and f ″ _a to f ″ _m ( m is an integer of 1 or more) is a vector whose elements are a force based on the reference value and the action of the actuator (the force of the moving element of the actuator or the force of the object moved by the actuator).
By setting the coordinate conversion in the function-specific force / speed allocation conversion block FT according to the function to be realized, various actions can be realized or an action accompanied by scaling can be reproduced.
That is, according to the basic principle of the present invention, in the function-specific force / velocity allocation conversion block FT, the variables of the actuator alone (variables in the real space) are converted into variable groups (virtual space in the virtual space) representing the functions to be realized. Variable) and assigns control energy to velocity (position) control energy and force control energy. For this reason, it is possible to independently give control energy of speed (position) and control energy of force as compared with a case where control is performed with a variable of the actuator alone (variable in real space).

The ideal force source block FC is a block that performs calculations in the force region in accordance with the coordinate conversion defined by the function-specific force / speed allocation conversion block FT. In the ideal force source block FC, a target value related to a force for performing a calculation based on the coordinate transformation defined by the function-specific force / speed allocation transformation block FT is set. This target value is set as a fixed value or a variable value depending on the function to be realized. For example, when realizing a function similar to the function indicated by the reference value, zero is set as the target value, and when scaling is performed, a value obtained by enlarging or reducing the information indicating the function to be reproduced is set. it can.

The ideal speed (position) source block PC is a block that performs calculations in the speed (position) area according to the coordinate conversion defined by the function-specific force / speed allocation conversion block FT. In the ideal speed (position) source block PC, a target value related to the speed (position) when performing the calculation based on the coordinate transformation defined by the function-specific force / speed assignment transformation block FT is set. This target value is set as a fixed value or a variable value depending on the function to be realized. For example, when realizing a function similar to the function indicated by the reference value, zero is set as the target value, and when scaling is performed, a value obtained by enlarging or reducing the information indicating the function to be reproduced is set. it can.

The inverse conversion block IFT is a block that converts the value of the velocity (position) and force region into the value (for example, voltage value or current value) of the input region to the control target system S.
Based on such a basic principle, when position information in the actuator of the control target system S is input to the function-specific force / speed allocation conversion block FT, speed (position) and force information obtained based on the position information. In the function-specific force / speed allocation conversion block FT, the control laws of the position and force regions corresponding to the functions are applied. Then, in the ideal force source block FC, a force according to the function is calculated, and in the ideal speed (position) source block PC, a speed (position) according to the function is calculated, and the force and speed (position). Control energy is distributed to each.

The calculation results in the ideal force source block FC and the ideal speed (position) source block PC are information indicating the control target of the control target system S, and these calculation results are used as the input values of the actuator in the inverse conversion block IFT, and control is performed. Input to the target system S.
As a result, the actuator of the control target system S executes the operation according to the function defined by the function-specific force / speed allocation conversion block FT, and the target robot operation is realized.
That is, in the present invention, it becomes possible to more appropriately realize the human physical action by the robot.

(Example of functions defined)
Next, a specific example of the function defined by the function-specific force / speed allocation conversion block FT will be described.
In the function-specific force / speed allocation conversion block FT, coordinate conversion for the speed (position) and force obtained based on the input current position of the actuator (conversion from real space to virtual space corresponding to the function to be realized) ) Is defined.
In the function-specific force / speed allocation conversion block FT, the speed (position) and force from the current position and the speed (position) and force as the reference value of the function are input, and the speed (position) and force are respectively input. The following control law is applied in the acceleration dimension.
That is, the force in the actuator is represented by the product of mass and acceleration, and the velocity (position) in the actuator is represented by the integral of acceleration. Therefore, by controlling the speed (position) and force through the acceleration region, the current position of the actuator can be acquired and the intended function can be realized.

Hereinafter, specific examples of functions used in the present invention will be described.
(Force / tactile transmission function)
FIG. 2 is a schematic diagram showing the concept of control when the force / tactile transmission function is defined in the function-specific force / speed allocation conversion block FT.
As shown in FIG. 2, as a function defined by the function-specific force / speed allocation conversion block FT, the operation of the master device is transmitted to the slave device, and the reaction force input from the object to the slave device is fed back to the master device. Function (bilateral control function) can be realized.
In this case, the coordinate conversion in the function-specific force / speed allocation conversion block FT is expressed as the following equations (3) and (4).

In Equation (3), x ′ _p is a speed for deriving the state value of speed (position), and x ′ _f is a speed related to the force state value. Further, x ′ _m is the speed (differential value of the current position of the master device) of the reference value (input from the master device), and x ′ _s is the current speed (differential value of the current position) of the slave device. Further, in the equation (4), f _p is the force on the status value of the speed (position), the f _f is the force for deriving the state value of the force. Further, f _m is a reference value (input from the master device), and f _s is a current force of the slave device.

(System configuration)
FIG. 3 is a schematic diagram showing a system configuration of the force-tactile transmission system 1 according to the present invention.
As shown in FIG. 3, the force / tactile sensation transmission system 1 includes a fishing rod type manipulator 10 as a master device, a head mounted display 20, a master side control device 30, a slave side control device 40, and a fishing robot as a slave device. The master side control device 30 and the slave side control device 40 are connected via a network 60 such as the Internet. Further, the fishing rod type manipulator 10 and the head mounted display 20 are configured to be able to communicate with the master-side control device 30 through a wireless communication network or a communication cable, and the slave-side control device 40 is connected with the fishing robot 50 through a wireless communication network or a communication cable. It is configured to be able to communicate. It is also possible to configure the force / tactile sensation transmission system 1 as a local system by connecting the master side control device 30 and the slave side control device 40 by wired communication such as a communication cable.

The force / tactile sensation transmission system 1 shown in FIG. 3 is a system that allows a remote user to experience fishing by operating a fishing robot 50 as an avatar with a fishing rod type manipulator 10. Further, in the force / tactile sensation transmission system 1, a force / tactile sensation due to an external force input to a fishing rod included in the fishing robot 50 is transmitted to the fishing rod type manipulator 10, and a force / tactile sensation by an operation performed by the user on the fishing rod type manipulator 10 is transmitted. 50.

Therefore, the user can experience fishing remotely with the feeling of actually touching the fishing rod.
As described above, according to the force / tactile sensation transmission system 1, it is possible to provide a technology capable of realizing a more realistic experience via a robot.
Further, in the force / tactile sensation transmission system 1, in addition to simply transmitting the force / tactile sense, the force / tactile sense is enlarged or reduced according to the setting, or only the force / haptic sense of a specific frequency is enlarged / reduced. It is possible to communicate or add virtual force and tactile sensation.
In the force / tactile sensation transmission system 1 of the present embodiment, a real haptics mode and a pseudo experience mode can be set as modes for the user to experience fishing. The real haptics mode is a mode in which a force / tactile sensation is transmitted between the fishing rod type manipulator 10 and the fishing robot 50 in real time, and the fishing robot 50 becomes an avatar of a user who uses the fishing rod type manipulator 10. The pseudo-experience mode is a mode that allows the user to experience the tactile sensation at the time of fishing by replaying data when the fish has been caught in the past.
Hereinafter, the configuration of the force / tactile sensation transmission system 1 will be described in detail.

The fishing rod type manipulator 10 is an operating device that imitates the whole fishing rod or the hand side, and is connected to a support base through a hinge 11 so as to be rotatable in the vertical direction. The hinge 11 is provided with an actuator 12 that applies rotational torque and an encoder 13 that detects the rotational angle of the hinge 11. The detection signal of the encoder 13 is output to the master-side control device 30, and the rotational torque output by the actuator 12 and the rotational position of the actuator are controlled by the master-side control device 30.

Further, the fishing rod type manipulator 10 includes a reel 14 at a predetermined distance from the end portion on the hand side (that is, the position of the hinge 11). In the present embodiment, the reel 14 is installed at a position corresponding to the distance between the human elbow and the hand (hereinafter referred to as “position R”) from the proximal end of the fishing rod type manipulator 10. Normally, when handling a fishing rod when fishing, the end of the fishing rod is adjusted to the position of the elbow, and the posture of holding the reel installation portion by hand is taken. Therefore, by installing the reel 14 of the fishing rod type manipulator 10 at the position R, the user operating the fishing rod type manipulator 10 operates the fishing rod type manipulator 10 with the same feeling as holding a fishing rod during actual fishing. can do. As will be described later, also in the fishing robot 50, the end of the fishing rod on the hand side is connected to the hinge, and the movement of the fishing rod around the end of the fishing rod on the hand side is detected. In addition, since there is a form of fishing in which no reel is used, the fishing rod type manipulator 10 may be configured not to include the reel 14. Also, the reel 14 may be provided with an actuator that applies rotational torque and an encoder that detects a rotational angle, like the hinge 11. In this case, the fishing rod 50 held by the fishing robot 50 is also provided with an actuator and an encoder, so that a force / tactile sensation can be transmitted between the reel 14 and the fishing rod reel.

The head mounted display 20 is a display that is used by being mounted on the head so as to cover the user's field of view, and displays an image taken by a camera installed on the fishing robot 50. In addition, the image displayed on the head mounted display 20 can also display the image etc. which were image | photographed in the past other than the image which the fishing robot 50 is image | photographing now. For example, it is possible to display an image taken at that time on the head-mounted display 20 in accordance with the fishing rod 50 manipulator 10 playing back the force and tactile sensation when a fish was caught in the fishing robot 50 in the past. Further, various display devices such as a stationary display, a projector and a screen can be used instead of the head mounted display 20.

The master-side control device 30 controls the entire force / tactile sensation transmission system 1 and is constituted by an information processing device such as a PC (Personal Computer) or an embedded microcomputer.
FIG. 4 is a block diagram illustrating a configuration of the master side control device 30.
In the present embodiment, the master-side control device 30 is constituted by a PC, and as shown in FIG. 4, a CPU (Central Processing Unit) 31, a RAM (Random Access Memory) 32, a storage device 33, and a communication interface ( Communication I / F) 34. Note that the master-side control device 30 includes a driver for driving the actuator 12, a battery for supplying power, an input device (for example, a keyboard and a mouse) for inputting various information, and the like. Various devices necessary for the above are mounted as appropriate.
In one area of the storage device 33, a parameter storage unit 33a for storing time-series parameters handled in the force / tactile sense transmission process is formed. By reading and reproducing the parameters stored in the parameter storage unit 33a in time series, it is possible to reproduce the force and haptic sense transmitted in the past by the fishing rod manipulator 10 and the fishing robot 50. The storage device 33 appropriately stores various data (such as fish images and audio data) used in the force / tactile transmission system 1.

The CPU 31 executes various programs for controlling the master-side control device 30, thereby realizing various target functions such as a basic force / haptic transmission function and additional functions.
Specifically, when the CPU 31 executes various programs, the CPU 31 implements, as a functional configuration, a position acquisition unit 31a, a force / tactile transmission unit 31b, a management unit 31c, and a data recording unit 31d. The
The position acquisition unit 31 a acquires a time-series detection value detected by the encoder 13 of the fishing rod manipulator 10 and a time-series detection value detected by the encoder 53 of the fishing robot 50.

The force / tactile sensation transmission unit 31b has functions of a function-specific force / speed allocation conversion block FT, an ideal force source block FC, an ideal speed (position) source block PC, and an inverse conversion block IFT in FIG. In the force / tactile sensation transmission unit 31b, coordinate transformation for realizing the force / tactile sense transmission function is defined.
That is, the force / tactile sensation transmission unit 31b is detected by the time detection value detected by the encoder 13 of the fishing rod type manipulator 10 and the encoder 53 of the fishing robot 50 from the position detection unit 31a in the real haptics mode. A time-series detected value (reference value) is input. Further, in the force / tactile sensation transmission unit 31b, in the pseudo experience mode, the time-series detection values detected by the encoder 13 of the fishing rod type manipulator 10 from the position detection unit 31a and the past read from the parameter storage unit 33a. (A time-series detection value detected in the past by the encoder 53 of the fishing robot 50) (reference value) is input. These time-series detection values represent the user's operation on the fishing rod manipulator 10 (manipulation of the fishing rod when fishing) and the current or past external force on the fishing robot 50 (external force on the fishing rod by a fish or the like). In addition, the force / tactile sensation transmission unit 31b applies coordinate transformation for realizing the force / tactile sensation transmission function to the velocity (position) and force information derived from the input detection value (position).

Then, the force / tactile sensation transmission unit 31b performs an operation in the velocity (position) region on the velocity (position) for deriving the state value of the velocity (position) obtained by the coordinate transformation. Similarly, the force / tactile sensation transmission unit 31b performs a calculation in the force region on the force for deriving the force state value obtained by the coordinate transformation. Further, the force / tactile sensation transmission unit 31b performs a uniform dimension processing on acceleration and the like on the calculation result in the calculated velocity (position) region and the calculation result in the force region, and realizes a force / tactile transmission function. Apply the inverse transformation of coordinate transformation to As a result, the force / tactile sensation transmission unit 31b calculates the calculation result in the calculated velocity (position) region and the calculation result in the force region as the value (current value) of the input region to the actuator 12 and the actuator 52 of the fishing robot 50 described later. Etc.). Then, the force / tactile sensation transmission unit 31 b outputs a value input to the actuator 12 to the actuator 12. Further, the force / tactile sensation transmission unit 31 b transmits the value of the input area to the actuator 52 of the fishing robot 50 to the slave-side control device 40 via the network 60. In addition, the value of the area | region of the input to the actuator 52 of the fishing robot 50 is transmitted to the slave side control apparatus 40 in real haptics mode, and is not transmitted in pseudo experience mode.

The management unit 31c performs various settings in the remote fishing experience control process and manages the progress of the process. Specifically, in the remote fishing experience control process, the management unit 31c sets to either the real haptics mode or the simulated experience mode according to a preset condition, or whether or not the fish has been successfully caught. Or performing an effect display and sound output when a fish hits. In this embodiment, when the remote fishing experience control process is performed, the real haptics mode is set immediately after the start of the process, and when the preset condition is satisfied, the pseudo experience mode is set. However, in the remote fishing experience control process, only the real haptics mode or the simulated experience mode may be set.
The data recording unit 31d records parameters when the force / tactile sensation transmission processing is performed in the real haptics mode in the parameter storage unit 33a in time series.

Returning to FIG. 3, the slave-side control device 40 is configured by an information processing device such as a PC or an embedded microcomputer. In the present embodiment, the slave-side control device 40 is configured as an embedded microcomputer and is configured as a device built in the fishing robot 50. The slave-side control device 40 transmits the time-series detection values detected by the encoder 53 of the fishing robot 50 to the master-side control device 30 via the network 60. In addition, the slave-side control device 40 outputs the input value to the actuator 52 transmitted from the master-side control device 30 to the actuator 52 of the fishing robot 50. The slave-side control device 40 is appropriately equipped with various devices necessary for the force / tactile sensation transmission system 1, such as a driver for driving the actuator 52, a battery for supplying power, and a speaker for outputting sound. Is done.

The fishing robot 50 is a robot that can hold a fishing rod and operates as an avatar of a user who operates the fishing rod-type manipulator 10. Therefore, the fishing robot 50 is installed at a place where fishing is performed, such as a fishing pond, a coast such as a rod and a quay, a fishing boat, a river shore, and a lake.
Specifically, the fishing robot 50 includes an arm 50A that holds a fishing rod having a fishhook and fishing line, and a portion corresponding to an elbow of the arm 50A is connected to a support base via a hinge 51 so as to be rotatable in the vertical direction. ing. The hinge 51 is provided with an actuator 52 that applies rotational torque and an encoder 53 that detects the rotational angle of the hinge 51. The detection signal of the encoder 53 is output to the slave-side control device 40, and the rotational torque output by the actuator 52 and the rotational position of the actuator are controlled by the slave-side control device 40. In addition, the reel provided in the fishing rod is installed at a position corresponding to the distance between the human elbow and the hand from the end on the hand side of the fishing rod. That is, the fishing rod reel is against the end of the fishing rod. It is installed at the same position as the reel 14 of the fishing rod type manipulator 10. The fishing robot 50 is provided with a camera for photographing a fishing spot (such as a water surface near a fishing line). Further, as described above, the reel of the fishing rod may be provided with an actuator for applying a rotational torque and an encoder for detecting a rotational angle, similarly to the hinge 51. In this case, by providing the reel 14 of the fishing rod type manipulator 10 also with an actuator and an encoder, force / tactile transmission can be performed between the reel 14 and the reel of the fishing rod.

[Example of actuator configuration]
FIG. 5 is a schematic diagram illustrating a configuration example of the actuator 100 used in the force-tactile transmission system 1.
The actuator 100 shown in FIG. 5 can be used as the actuator 12 or the actuator 52, and a fishing rod-like member of the fishing rod type manipulator 10 or a fishing rod held by the fishing robot 50 can be installed.
Specifically, the actuator 100 includes a motor 101, a gear 102, and an attachment member 103.

The motor 101 is configured by an electric motor such as a DC motor, for example, and outputs rotational torque to the gear 102.
The gear 102 amplifies the rotational torque input from the motor 101 and rotates the support member 103 with the amplified rotational torque.
The support member 103 is fixed to the output shaft of the gear 102 and rotates around the output shaft of the gear 102 by the rotational torque output from the gear 102. Further, the external force input to the support member 103 is input to the gear 102. When the actuator 100 is used as the actuator 12 of the fishing rod-type manipulator 10, the fishing rod-shaped member of the fishing rod-type manipulator 10 is fixed to the support member 103, and is attached to the rear end portion of the support member 103 (near the rotating shaft of the gear 102). Is touched by the user's elbow. When the actuator 100 is used as the actuator 52 of the fishing robot 50, a fishing rod held by the fishing robot 50 is fixed to the support member 103.

(Example of functions to be implemented)
In the force / tactile sensation transmission system 1, the following functions can be implemented.
(1) The elbow position of the user who uses the fishing rod type manipulator 10 is configured to match the rotation center of the actuator 12.
(2) The force and position data obtained when the fish has been caught in the past are stored and reproduced at an arbitrary timing (the actuator 12 of the fishing rod type manipulator 10 is driven based on the data) to simulate the feeling of fishing. Experience.

(3) The type of fish or the size of the fish is determined from the direction of pulling the fish and the strength. The direction and force of pulling the fish can be detected by providing a sensor for detecting the direction and magnitude of the force applied to the fishing line at the tip of the fishing rod. Note that the direction of the force may be detected by a camera installed in the fishing robot 50.
(4) The fish is discriminated by (3) and the vision system or the audio system is integrated to display the type of fish or the size of the fish on the display, and the type of fish or the size of the fish is also expressed by sound.
(5) From the length of the fishing line fed out from the reel 14 and the angle of the rod, the depth of the fishing hook (or bait) and the depth of the fish that is caught are identified.
(6) Provided with a live streaming system capable of handling quick movements of the kite.

(7) A pattern of force to be input to the rod is extracted from data when the fish has been caught in the past, and the force / tactile sensation is reproduced for each pattern in the fishing rod type manipulator 10. The pattern to be extracted can be, for example, recorded data for each section divided by the water depth at which the fish is located and divided at a timing with small movement. In the pseudo-experience mode for experiencing a force tactile sense at the time of fishing in the past, this pattern is selected and the force tactile sense is reproduced in the fishing rod type manipulator 10. In addition, when extracting a pattern, the water depth in which a fish exists can be identified from the length of the fishing line drawn out from the reel 14, and the angle of a rod.

(8) In the force / tactile sensation transmission system 1, the fishing rod type manipulator 10 can be installed, for example, in an airport lobby or the like, and the fishing robot 50 can be installed in a famous fishing spot. In this case, the user is assumed to be a boarding person who waits for the departure of the airplane. Therefore, after the user starts the remote fishing experience via the fishing robot 50, when the predetermined timing is reached, the user can reproduce the data when the fish has been caught in the past, thereby It can be controlled so that the force tactile sense at the time of fishing is simulated. Thereby, the user can actually experience remote fishing via the fishing robot 50 and can surely experience the force tactile sense when the fish is caught even if the fish does not hit. . The predetermined timing includes a timing that is a predetermined time before the user's boarding time (for example, one hour before), or after the user starts a remote fishing experience in the force-tactile transmission system 1. The timing at which a predetermined time (for example, 30 minutes) elapses can be set. Either remote fishing experience via the fishing robot 50 (real haptics mode) or pseudo fishing experience using data when fish were fished in the past (pseudo experience mode) is selected in advance. Then, the force / tactile sensation transmission system 1 may be operated.

(9) In the force / tactile sensation transmission system 1, the fishing rod type manipulator 10 has a fishing experience such as an airport, a transportation facility, a game center, an amusement facility such as an aquarium, and a training facility for learning fishing techniques. It can be installed in various places where it can be used.

(Operation)
Next, processing executed by the force / tactile sensation transmission system 1 will be described.
The force / tactile sensation transmission system 1 having the above-described function can perform various operations. As a process for realizing a remote fishing experience using the fishing robot 50, a force / tactile sensation transmission process, The remote fishing experience control process is executed.

(Force / tactile transmission processing)
FIG. 6 is a flowchart showing a flow of force / tactile sensation transmission processing executed by the force / haptic transmission system 1.
The force / tactile sensation transmission process is started in response to activation of a function for transmitting force / tactile sensation in the master-side control device 30.
In step S <b> 1, the position acquisition unit 31 a of the master-side control device 30 acquires information indicating the positions (rotation angles) of the hinges 11 and 51 from the encoder 13 of the fishing rod type manipulator 10 and the encoder 53 of the fishing robot 50. At this time, the position acquisition unit 31 a of the master-side control device 30 acquires the position (rotation angle) of the hinge 51 detected by the encoder 53 of the fishing robot 50 via the slave-side control device 40.
In step S2, the force / tactile sensation transmission unit 31b of the master-side control device 30 performs coordinate transformation (see FIG. 2) for force / haptic transmission. At this time, the force / tactile sensation transmission unit 31b of the master-side control device 30 acquires information indicating acceleration from information indicating the position (rotation angle) of the hinges 11 and 51, and appropriately converts the information into dimensions such as speed or force. Used for conversion.

In step S3, the force / tactile sensation transmission unit 31b of the master-side control device 30 calculates values of inputs to the actuators 12 and 52 from the control amount of speed (position) and the control amount of force based on the result of coordinate conversion. .
In step S4, the force / tactile sensation transmission unit 31b of the master-side control device 30 transmits values of inputs to the actuators 12 and 52 to the fishing rod manipulator 10 and the fishing robot 50, respectively.
In step S5, the fishing rod type manipulator 10 and the fishing robot 50 (slave-side control device 40) drive the actuators 12, 52 based on the input values transmitted from the master-side control device 30, and the encoders 13, 53 The position (rotation angle) of the hinges 11 and 51 is detected.

In step S6, the management unit 31c of the master-side control device 30 determines whether or not the end of the force / tactile sensation transmission process is instructed. The termination of the force / tactile sensation process is automatically instructed in response to the fact that all functions for transmitting force / tactile sensations are stopped in the master-side control device 30. Note that an operator who manages the master-side control device 30 can instruct the end of the force / tactile sensation transmission process.
If the end of the force-tactile transmission process is not instructed, it is determined as NO in Step S6, and the process proceeds to Step S1.
On the other hand, when the termination of the force / tactile sensation transmission process is instructed, YES is determined in step S6, and the force / tactile transmission process is terminated.

(Remote fishing experience control process)
Next, the remote fishing experience control process will be described.
FIG. 7 is a flowchart showing the flow of remote fishing experience control processing executed by the force / tactile sensation transmission system 1.
The remote fishing experience control process is started in response to the master side control device 30 instructing execution of the remote fishing experience control process.

In step S21, the management unit 31c of the master-side control device 30 sets the force / haptic transmission system 1 to the real haptics mode. In the real haptics mode, as described above, a force / tactile sensation is transmitted in real time between the fishing rod manipulator 10 and the fishing robot 50, and the fishing robot 50 becomes an avatar of the user who uses the fishing rod manipulator 10. .
In step S22, the management unit 31c of the master-side control device 30 starts a force / tactile sensation transmission process.

In step S <b> 23, the management unit 31 c of the master-side control device 30 determines whether or not a fish is hit in the fishing robot 50. Whether or not a fish has been hit is determined by using, for example, information such as the magnitude, acceleration, and frequency of an external force input to the fishing robot 50 as a reference threshold set based on experimental values or experience values. It can detect by comparing.
If the fish is not hit in the fishing robot 50, NO is determined in step S23, and the process proceeds to step S24.
On the other hand, when a fish is hit in the fishing robot 50, it is determined as YES in Step S23, and the process proceeds to Step S32.

In step S24, the management unit 31c of the master-side control device 30 determines whether or not it is a predetermined timing after the start of the remote fishing experience control process. Note that, as described above, as described above, as described above, a timing that is a predetermined time before the user's boarding time (for example, one hour before), or a remote fishing experience by the user in the force-tactile transmission system 1 The timing when a predetermined time (for example, 30 minutes) has passed can be set.
If the predetermined timing is not reached after the remote fishing experience control process is started, NO is determined in step S24, and the process proceeds to step S23.
On the other hand, if it is a predetermined timing after the start of the remote fishing experience control process, YES is determined in step S24, and the process proceeds to step S25.
In step S25, the management unit 31c of the master-side control device 30 sets the force / tactile sensation transmission system 1 to the pseudo experience mode. In the pseudo-experience mode, as described above, the data when the fish has been caught in the past is reproduced, so that the user can experience the tactile sensation at the time of fishing.

In step S26, the management unit 31c of the master-side control device 30 selects a force / tactile sensation transmission pattern. The force-tactile sensation transmission pattern is, for example, recorded data for each section obtained by classifying data when a fish has been caught in the past according to the depth of water in which the fish is located and at a timing when the movement is small.
In step S27, the management unit 31c of the master-side control device 30 reads the selected pattern data from the storage device.
In step S28, the force / tactile sensation transmission unit 31b of the master-side control device 30 reproduces the force / tactile sensation in the fishing rod manipulator 10 by performing coordinate conversion using the read pattern data as a reference value (see FIG. 1).

In step S <b> 29, the management unit 31 c of the master-side control device 30 displays the type of fish or the size of the fish on the head mounted display 20, and outputs the type of fish or the size of the fish from the speaker provided in the head mounted display 20. To do.
In step S30, the management unit 31c of the master-side control device 30 determines whether or not the fish has been pulled up to a depth of 50 cm when the user operates the fishing rod manipulator 10. Here, as a reference whether or not the fish has been caught, it is determined whether or not the fish has been raised within a depth of 50 cm. If a fish is pulled up within 50 cm of water, it can usually be caught by scooping it with “Tama”. “Depth of 50 cm” is an example, and can be appropriately changed according to the fishing situation (fishing place, type of fish to be aimed at, etc.).
If the fish has not been pulled up to a depth of 50 cm, NO is determined in step S30, and the process proceeds to step S26.
On the other hand, if the fish has been pulled up to a depth of 50 cm, YES is determined in step S30, and the process proceeds to step S31.

In step S <b> 31, the management unit 31 c of the master-side control device 30 displays on the head-mounted display 20 that the fishing has been successful, assuming that the fish that has been hit has been successfully caught.
After step S31, the remote fishing experience control process ends.
In step S <b> 32, the data recording unit 31 d of the master control device 30 starts recording external force data input to the fishing robot 50. If the data recorded at this time can reproduce the force-tactile state, such as the position (rotation angle) data detected by the encoder 53, or a control parameter calculated as a result of coordinate conversion, Various formats can be used.
In step S <b> 33, the management unit 31 c of the master-side control device 30 displays the type of fish or the size of the fish on the head mounted display 20, and outputs the type of fish or the size of the fish from the speaker provided in the head mounted display 20. To do.

In step S <b> 34, the management unit 31 c of the master-side control device 30 determines whether or not the fish has been pulled up to a depth of 50 cm by operating the fishing rod type manipulator 10.
If the fish has not been pulled up to a depth of 50 cm, NO is determined in step S34, and the process proceeds to step S26.
On the other hand, if the fish has been pulled up to a depth of 50 cm, YES is determined in step S34, and the process proceeds to step S33.

In step S <b> 35, the management unit 31 c of the master-side control device 30 displays on the head-mounted display 20 that the fishing has been successful, assuming that the fish that has been hit has been successfully caught.
In step S <b> 36, the data recording unit 31 d of the master control device 30 ends the recording of the external force data input to the fishing robot 50.
After step S36, the remote fishing control experience control process ends.

As described above, according to the force / tactile sensation transmission system 1 according to the present embodiment, the user can experience fishing remotely with a sense of actually touching a fishing rod.
As described above, according to the force / tactile sensation transmission system 1, it is possible to provide a technology capable of realizing a more realistic experience via a robot.

[Modification 1]
In the above-described embodiment, the configuration example illustrated in FIG. 5 is shown as the actuator 100 used as the actuator 12 or the actuator 52, but the configuration is not limited thereto.
For example, the output of the actuator 100 can be increased.
FIG. 8 is a schematic diagram illustrating another configuration example of the actuator 100 used in the force-tactile transmission system 1.
As shown in FIG. 8, the actuator 100 of this modification has a configuration in which two motors 101 are connected to the configuration example shown in FIG. 5.
Thereby, the output torque of the actuator 100 can be increased twice as compared with the configuration example shown in FIG.
Note that the output torque can be doubled by doubling the reduction ratio of the gear 102, but the reduction ratio of the gear 102 is doubled. As the influence of friction increases, the inertia that humans feel when inputting an external force to the actuator 100 increases with the square of the reduction ratio. Therefore, it is more advantageous from the viewpoint of inertia to connect two motors 101 than to double the reduction ratio of the gear 102.

[Modification 2]
In the embodiment and the first modification described above, the motor 101 can perform control to compensate for friction and gravity.
FIG. 9 is a schematic diagram illustrating another configuration example of the actuator 100 used in the force-tactile transmission system 1.
Since the actuator 100 includes the gear 102, the generation of friction is inevitable. Therefore, as shown in FIG. 9, it is possible to input a current for canceling the frictional force to the motor 101 and perform control for compensating the frictional force (applying compensation torque). At this time, a current is evenly input to the two motors 101 connected. As a result, it is possible to avoid a situation in which the final output torque decreases due to the current being biased to one of the motors 101.
Similarly, a current for compensating the weight of a reel or the like installed on the fishing rod held by the fishing rod type manipulator 10 or the fishing robot 50 can be input to the motor 101 to control the gravity. Also in this case, a current is equally input to the two motors 101 connected.

[Modification 3]
In the above-described embodiment and Modifications 1 and 2, the output shaft of the actuator 100 can be provided with a torque sensor.
FIG. 10 is a schematic diagram illustrating another configuration example of the actuator 100 used in the force-tactile transmission system 1.
As shown in FIG. 10, a torque sensor 104 can be provided on the output shaft of the actuator 100, whereby an external force input to the actuator 100 can be measured.
For example, when the actuator 100 is used as the actuator 52 of the fishing robot 50, it is possible to directly measure the torque when fishing a fish. Since the difference between the torque output by the two connected motors 101 multiplied by the gear ratio of the gear 102 and the value measured by the torque sensor 104 corresponds to friction, the reaction force (torque) can be calculated with higher accuracy. It becomes possible to measure, and a sharp force-tactile transmission becomes possible.

[Modification 4]
In the above-described embodiment, the example in which the fishing rod type manipulator 10 and the fishing robot 50 are configured as a uniaxial mechanism including the hinge 11 or the hinge 51 has been described.
On the other hand, the fishing rod type manipulator 10 and the fishing robot 50 can be configured as a multi-axis mechanism having a higher degree of freedom of movement.
FIG. 11 is a schematic diagram illustrating a configuration example of a fishing rod type manipulator 10 configured as a multi-axis mechanism.
FIG. 12 is a schematic diagram illustrating a configuration example of a fishing robot 50 configured as a multi-axis mechanism.

The fishing rod type manipulator 10 shown in FIG. 11 and the fishing robot 50 shown in FIG. 12 have the same joint mechanism, and the force / tactile sensation transmission unit 31b of the master-side control device 30 applies force between the actuators 100 provided in the corresponding joints. Tactile transmission is performed.
Specifically, as shown in FIG. 11, the fishing rod type manipulator 10 rotates around the rotation axis R1, the first joint J1 rotating around the rotation axis R1, the second joint J2 rotating around the rotation axis R2, and the rotation axis R3. And a third joint J3. The rotation axes R1 and R3 are set in a substantially vertical direction, and the rotation axis R2 is set in a substantially horizontal direction. Each of the first joint J1 to the third joint J3 is provided with an actuator 100 for driving the joint.
In the present embodiment, the first joint J1 is installed as a joint corresponding to a human shoulder, and the third joint J3 is installed as a joint corresponding to a human elbow.

Further, as shown in FIG. 12, the fishing robot 50 includes the first joint J1 to the third joint J3, similar to the fishing rod type manipulator 10, and actuators 100 are provided in these joints. Further, the fishing robot 50 is provided with a camera C for photographing the fishing spot. The camera C is installed at a position corresponding to the human viewpoint with respect to the first joint.

By using the fishing rod type manipulator 10 and the fishing robot 50 having such a configuration, in the force / tactile sensation transmission system 1, the real haptics mode is used to photograph the fishing ground from the position corresponding to the human viewpoint, A fishing robot 50 that performs fishing by a movement corresponding to the movement of the elbow is used as an avatar, and external force input to the fishing robot 50 is transmitted to the fishing rod type manipulator 10 as well as an operation input by a human to the fishing rod type manipulator 10. A remote fishing experience can be performed while accurately reproducing at 10.
Further, in the force / tactile sensation transmission system 1, by reproducing the data when the fish has been caught in the past in the pseudo experience mode, it is possible to cause the user to experience the force / tactile sense when fishing.
At this time, by using the fishing rod type manipulator 10 and the fishing robot 50, it is possible to perform fishing by movement corresponding to the movement of the human shoulder and elbow while photographing the fishing ground from the position corresponding to the human viewpoint. The user can perform a remote fishing experience or a simulated fishing experience with a more natural sense.

The force / tactile sensation transmission system 1 configured as described above includes a fishing rod type manipulator 10, a master side control device 30, and a fishing robot 50.
The fishing robot 50 can execute a fishing operation, and the fishing rod type manipulator 10 is installed apart from the fishing robot 50. Further, the master side control device 30 is configured to be able to communicate with the fishing robot 50 and the fishing rod type manipulator 10.
The master-side control device 30 causes the fishing rod type manipulator 10 to output a force / tactile sensation due to the external force input to the fishing rod based on information indicating the current or past external force input to the fishing rod provided in the fishing robot 50.
Thereby, the external force input to the fishing rod provided in the fishing robot 50 can be transmitted to the fishing rod manipulator 10.
Therefore, it is possible to provide a technology capable of realizing a more realistic experience via a robot.

The master-side control device 30 causes the fishing robot 50 to output a force haptic by the external force input to the fishing rod type manipulator 10 based on information indicating the external force input by operating the fishing rod type manipulator 10.
Thereby, the operation which the user performed with respect to the fishing rod type manipulator 10 can be accurately reproduced in the fishing robot 50, and fishing can be experienced in the position away from the fishing spot.

The master control device 30 obtains the operation in the first mode (real haptics mode) for transmitting the force / tactile sensation between the fishing rod type manipulator 10 and the fishing robot 50 in real time and the data when the fish has been caught in the past. By reproducing, the fishing rod type manipulator 10 can execute the operation in the second mode (pseudo-experience mode) for artificially outputting the force / tactile sense at the time of fishing.
As a result, it is possible to experience fishing in real time at a position away from the fishing ground, and to reproduce the data when the fish has been caught in the past and experience fishing in a pseudo manner.

The master-side control device 30 executes switching between the first mode and the second mode in accordance with preset conditions.
Thereby, depending on the situation, you can choose to experience fishing in real time at a position away from the fishing ground, and to reproduce the data when you caught fish in the past and experience fishing in a pseudo manner It becomes possible.

The preset condition includes the time until the departure time of the transportation facility.
Thereby, it is possible to provide the user with a fishing experience with appropriate contents in accordance with the waiting time of the transportation facility.

Note that the present invention can be appropriately modified and improved within the scope of the effects of the present invention, and is not limited to the above-described embodiments and modifications.
For example, the present invention is implemented in the force / haptic transmission system 1 in addition to being realized as the force / haptic transmission system 1, the master side control device 30, or the slave side control device 40 (force / haptic transmission device) in the above-described embodiment. It can be realized as a force / tactile sensation transmission method constituted by each step or a program executed by a processor to realize the function of the force / tactile sensation transmission system 1.
Further, in the above-described embodiment, the example in which the master-side control device 30 and the slave-side control device 40 are configured as different devices has been described, but it is also possible to configure them as an integrated control device.

In the above-described embodiment, when determining the water depth of the fish, the water depth of the fish is determined by the amount of fishing line wound on the reel of the fishing rod held by the fishing robot 50, or the water depth of the fish is determined by a camera installed in the fishing robot. You can do it. Further, when it is determined whether or not the fish has been caught from the water, it can be determined by the fact that when the fish is caught from the water, the buoyancy is lost and the weight applied to the fishing line increases rapidly.

In the above-described embodiment, the case where the force / tactile sensation is bidirectionally transmitted between the fishing rod manipulator 10 and the fishing robot 50 has been described as an example, but the present invention is not limited thereto. For example, if the user holding the fishing rod type manipulator 10 is to experience the feeling of a fishing rod when a fish is caught, the force sense of the external force input to the fishing robot 50 is output from the fishing rod type manipulator 10, and the fishing rod type manipulator The force / tactile sensation by the operation input to 10 may not be transmitted to the fishing robot 50 (or the operation may be transmitted in a rounded manner). When the operation by the operation input to the fishing rod manipulator 10 is rounded and transmitted to the fishing robot 50, for example, only a large operation such as lifting the fishing rod or winding up the reel 14 is transmitted to the fishing robot 50, and an accurate force / tactile sense is transmitted. Can be omitted. In this case, the fishing robot 50 may automatically perform control for fishing the fish.

The processing in the above-described embodiment or the like can be executed by either hardware or software.
That is, it is only necessary that the force / tactile sensation transmission system 1 has a function capable of executing the above-described processing, and what functional configuration and hardware configuration are used to realize this function is not limited to the above-described example.
When the above-described processing is executed by software, a program constituting the software is installed on a computer from a network or a storage medium.

The storage medium for storing the program includes a removable medium distributed separately from the apparatus main body, or a storage medium incorporated in the apparatus main body in advance. The removable medium is composed of, for example, a semiconductor memory (flash memory or the like), a magnetic disk, an optical disk, a magneto-optical disk, or the like. The optical disc is composed of, for example, a CD-ROM (Compact Disc-Read Only Memory), a DVD (Digital Versatile Disc), a Blu-ray Disc (registered trademark), and the like. The magneto-optical disk is configured by MD (Mini-Disc) or the like. Further, the storage medium incorporated in advance in the apparatus main body is constituted by, for example, a ROM or a hard disk in which a program is stored.

In addition, the said embodiment has shown an example to which this invention is applied, and does not limit the technical scope of this invention. That is, the present invention can be modified in various ways such as omission and replacement without departing from the gist of the present invention, and can take various embodiments other than the above-described embodiments. Various embodiments that the present invention can take and modifications thereof are included in the invention described in the scope of claims and the equivalents thereof.

S Control target system, FT function-specific force / speed allocation conversion block, FC ideal force source block, PC ideal speed (position) source block, IFT inverse conversion block, 1 force tactile transmission system, 10 fishing rod type manipulator, 11, 51 hinge 12, 52, 100 actuator, 13, 53 encoder, 14 reel, 20 head mounted display, 30 master side control device, 31 CPU, 31a position acquisition unit, 31b force / tactile transmission unit, 31c management unit, 31d data recording unit, 32 RAM, 33 storage device, 33a parameter storage unit, 34 communication interface, 40 slave side control device, 50 fishing robot, 50A arm, 60 network, 101 motor, 102 gear, 103 mounting member, 10 Torque sensor, R1-R3 rotation axis, J1-J3 first joint to third joint, C camera

Claims (9)

1. A force / tactile sensation transmission system including a fishing robot capable of performing a fishing operation, a fishing rod type manipulator installed apart from the fishing robot, and a control device configured to be able to communicate with the fishing robot and the fishing rod type manipulator. And
   The controller is
   A force / tactile sensation transmission system that causes the fishing rod type manipulator to output a force / tactile sensation caused by an external force input to the fishing rod based on information indicating a current or past external force input to a fishing rod provided in the fishing robot.
2. The control device causes the fishing robot to output a force / tactile force due to an external force input to the fishing rod type manipulator based on information indicating an external force input by operating the fishing rod type manipulator. 2. The force / tactile sensation transmission system according to 1.
3. The control device reproduces data when a fish has been caught in the past by reproducing an operation in a first mode in which a force / tactile sensation is transmitted between the fishing rod manipulator and the fishing robot in real time. 3. The force / tactile sensation transmission system according to claim 1, wherein the type manipulator can execute an operation in a second mode in which a force / tactile sense at the time of fishing is pseudo-outputted.
4. The force / tactile sensation transmission system according to claim 3, wherein the control device executes the first mode and the second mode by switching according to a preset condition.
5. The force-tactile transmission system according to claim 4, wherein the preset condition includes a time until a departure time of the transportation facility.
6. The fishing rod type manipulator and the fishing robot include the same joint mechanism and an actuator installed in the joint,
   The force / tactile sensation transmission system according to claim 1, wherein the control device transmits force / tactile sensation between the actuators provided in the corresponding joints.
7. A force-tactile sensation transmission device configured to be able to communicate with a fishing robot capable of performing a fishing operation and a fishing rod type manipulator installed away from the fishing robot,
   A force / tactile sensation transmission device that causes the fishing rod type manipulator to output a force / tactile sensation caused by an external force input to the fishing rod based on information indicating a current or past external force input to a fishing rod provided in the fishing robot.
8. A force / tactile transmission method executed by a force / tactile transmission device configured to be communicable with a fishing robot capable of performing a fishing operation and a fishing rod type manipulator installed away from the fishing robot,
   A force tactile sensation including the step of outputting a force tactile sensation due to the external force input to the fishing rod to the fishing rod type manipulator based on information indicating a current or past external force input to the fishing rod provided in the fishing robot. Transmission method.
9. A computer that controls a fishing robot capable of performing a fishing operation and a force / tactile sensation transmission device configured to be communicable with a fishing rod type manipulator installed away from the fishing robot.
   A program for realizing a function of outputting a force / tactile force by an external force input to the fishing rod to the fishing rod-type manipulator based on information indicating a current or past external force input to a fishing rod provided in the fishing robot. .

Images