JP7260502B2 - Information presentation device, remote control system and program - Google Patents

Description

The present invention relates to an information presentation device, a remote control system and a program.

As the background art of this technical field, the following patent document 1 describes "enables an operator to suitably detect a change in inclination of an unmanned mobile body" and "a remote control system 1 includes an unmanned mobile body 2, a remote control device 3 which is operated by an operator and performs wireless communication with the unmanned mobile body 2 to control the operation of the unmanned mobile body 2; 19, an output amount calculation unit 33 that calculates an output amount based on information related to the inclination angle of the unmanned mobile body from the inclination angle detection unit 19, and the above calculated by the output amount calculation unit 33 for the operator. and an operation unit 31 as a haptic sense presenting unit that presents a haptic sense corresponding to the amount of output” (see abstract).

JP 2018-142123 A

By the way, the technique described above has a problem that the types of information items that can be presented by tactile stimulation are small.
SUMMARY OF THE INVENTION It is an object of the present invention to provide an information presentation device, a remote control system, and a program capable of presenting a wide variety of information items by means of tactile stimulation.

In order to solve the above-described problems, an information presentation apparatus of the present invention includes: operation information representing an operation state of an operation device by a user; device operation information representing an operation state of an operation target device remotely operated according to the operation information; a device status estimating unit for estimating the information item corresponding to one or more occurring device statuses from among a plurality of information items whose priority is determined based on; A notification item setting unit that selects the information item with the highest priority from among them as a notification item to be notified to the user; a notification item notification unit that outputs notification item information representing the notification item; and a tactile sense presentation information output unit that outputs tactile sense presentation information representing the content of the tactile sense stimulus given to the user.

According to the present invention, various types of information items can be presented by tactile stimulation.

1 is a block diagram of a remote control system according to a preferred first embodiment; FIG. It is a figure which shows the example of the information item in 1st Embodiment. 1 is a schematic diagram of an operation device and a tactile presentation device; FIG. 10 is a flow chart of an information presentation routine;

[Premise of the embodiment]
2. Description of the Related Art In recent years, there is a serious labor shortage in operating construction machines such as excavators, and there is a demand for remote control of construction machines from a remote location. However, remote control of construction machines increases the possibility of poor workability due to the lack of feeling of riding. This is because, in a remote control room or the like, various bodily perception information such as a sense of tilt and centrifugal force, which one feels with one's body at a construction site, is missing. Also, for the same reason, a remote-operating user is more likely to perform inappropriate operations due to a lack of riding sensation, which increases the possibility of overturning the construction machine or damaging the body.

As a countermeasure, it is conceivable to visually notify the user of the missing bodily perception information using a meter, a numerical value, or the like. However, this increases the amount of visual information, increases the cognitive cost of reading, and may cause user fatigue and reduced work efficiency. Therefore, the preferred embodiment described later aims to make the information intuitively perceivable and reduce the visual cognitive load by using tactile stimulation to notify the information missing due to remote control.

In particular, in remote operation, it is effective to give the user tactile feedback in order to enhance the sense of boarding, which is lacking in actual boarding, and it is believed that this can improve the user's sense of self-esteem. And even with the same tactile stimulus, it is thought that various types of information can be presented by performing different sensory recall (assignment of meaning) depending on the situation.

Here, if various pieces of information are concurrently transmitted to the user by tactile feedback, there is a high possibility that the user will be confused. Therefore, it is considered preferable to switch information to be transmitted by haptic feedback while reducing the number of pieces of information to be transmitted concurrently. As a result, it is possible to present a wide variety of information to the user while reducing the user's visual cognitive load, thereby improving workability and safety.

[First embodiment]
<Configuration of Embodiment>
FIG. 1 is a block diagram of a remote control system 10 according to the first preferred embodiment.
1, the remote control system 10 includes a control device 16 (information presentation device, computer), a user state detection unit 22, an information output unit 24, an operation device 26, a tactile presentation device 28, and an operation target device 30. , a device information detection unit 42 , and an environmental state detection unit 44 . The controller 16 and the other components then communicate bi-directionally via the network 18 .

The device 30 to be operated is a device arranged at the construction site 14 and remotely controlled, and is a power shovel in the illustrated example. In the illustrated example, the operation target device 30 includes a bucket 32 , an arm 34 , a vehicle body 36 and crawler belts 38 . Bucket 32 and arm 34 are hydraulically driven to excavate earth and sand at construction site 14 . Further, the operation target device 30 is self-propelled by driving the crawler belt 38 . The vehicle body 36 can turn with respect to the crawler belt 38 together with the arm 34 and the bucket 32 .

The device 30 to be operated receives operation information D30 from the control device 16, and drives each part based on this operation information D30. The device information detection unit 42 detects various types of information in the operation target device 30 and outputs the detection result to the control device 16 as the device operation information D42. Further, the environmental state detection unit 44 measures various environments at the construction site 14 and outputs the results to the control device 16 as environmental information D44.

More specifically, the device information detection unit 42 includes a tilt sensor that measures the tilt angle of the crawler belt 38, a load sensor that measures the load of the bucket 32, and the hydraulic pressure of each part of the bucket 32 and the arm 34. , a vibration sensor for measuring the amount of vibration of the operation target device 30, a bucket tip distance sensor, a center-of-gravity position sensor for measuring the center-of-gravity position of the operation target device 30, a traveling speed sensor for the track 38, and a turning speed of the vehicle body 36. A sensor and the like are provided (both are not shown). The bucket tip distance sensor measures the "bucket tip distance", which means "the distance from the tip position of the bucket 32 to an object such as the ground".

The environmental state detection unit 44 also includes a human sensor that detects people around, an ambient sound sensor that collects ambient sounds, a driver's seat camera that captures an image of the direction of the bucket 32 from the driver's seat, and a peripheral image of the operation target device 30. It is equipped with a peripheral camera that captures images, a soil sensor that measures the state of the ground (unevenness of the ground, muddy state of the ground, soil hardness of the ground material, etc.), an anemometer, a rain gauge, etc. (all not shown).

Also, the user state detection section 22 , the information output section 24 , the operation device 26 and the tactile sense presentation device 28 are provided in the remote control room 12 . The operating device 26 is a device for remotely operating the device 30 to be operated, and is operated by the user 52 . The details of the operation performed on the operation device 26 are supplied to the control device 16 as operation information D26. The control device 16 generates the above-described operation information D30 based on this operation information D26, and supplies it to the operation target device 30. FIG. However, the control device 16 may supply the operation information D26 as it is to the operation target device 30 as the operation information D30.

The haptic presentation device 28 provides haptic feedback to the user based on the haptic presentation information D28 received from the control device 16. FIG. For example, the tactile presentation device 28 transmits tactile stimulation to the user by vibrating the operation device 26 . In this embodiment, the tactile presentation device 28 notifies the user of the data value for any one of the plurality of information items by tactile feedback. Here, the amount related to tactile stimulation is called "stimulus amount". For example, the stimulus amount can be the vibration amplitude of the tactile presentation device 28 . Moreover, when the tactile sense presentation device 28 repeats on/off of the vibration at a predetermined cycle, the “length of the period during which the vibration is on” can be used as the stimulus amount.

FIG. 2 is a diagram showing an example of information items in this embodiment.
In the illustrated example, information items to be notified to the user (hereinafter referred to as notification items) are selected from a total of nine information items A1 to A3, B4 to B7, C8 and C9. Further, each information item has a corresponding "equipment status (status of operation target equipment 30, etc.)". Then, when the device status occurs, there is a possibility that the information item is selected as the notification item. For example, the information item A1 (peripheral object distance) corresponds to the device status of "peripheral object approaching" (some object is approaching the operation target device 30). can be selected as Further, the information item A3 (centrifugal force) can be selected as a notification item when a device condition of "large centrifugal force" (large centrifugal force generated in bucket 32) occurs.

These information items have a priority to be notified to the user 52. In FIG. 2, the higher the column, the higher the priority. That is, the information items A1 to A3 are mainly related to ensuring safety, and therefore have the highest priority. Information items B4 to B7 are mainly items related to work efficiency and machine maintenance, and therefore have the next highest priority. Information items C8 and C9 are mainly related to the prevention of operation errors, and have the lowest priority. Details of each information item and device status shown in FIG. 2 will be described later.

Returning to FIG. 1, the information output unit 24 includes a display 24a and a speaker 24b. The display 24a may be a flat panel display or a head mounted display. Then, the information output unit 24 receives notification item information D24 and general information D25 from the control device 16, and notifies the user 52 of these contents by video and audio.

Here, the notification item information D24 notifies the user 52 of the above notification items. The general information D25 is various information other than the notification item information D24. For example, the general information D25 includes a moving image captured by a driver's seat camera (not shown) provided in the driver's seat of the operation target device 30, peripheral images of the operation target device 30, peripheral audio of the operation target device 30, operation target device 30 includes data indicating various states.

The user state detection unit 22 measures the state of the user 52 other than the operation information D26, and outputs the result as the user state information D22. The user state detection unit 22 includes a user-captured camera (not shown) that captures the user 52, a line-of-sight detection device that detects the line of sight of the user 52 based on the image captured by the user-captured camera, and a thermometer that detects the body temperature of the user 52. A heart rate meter or the like for detecting the heart rate of the user 52 is provided.

The control device 16 includes general computer hardware such as a CPU (Central Processing Unit), RAM (Random Access Memory), ROM (Read Only Memory), and SSD (Solid State Drive). , an OS (Operating System), application programs, various data, and the like are stored. The OS and application programs are developed in RAM and executed by the CPU. In FIG. 1, the inside of the control device 16 shows functions implemented by application programs and the like as blocks.

That is, the control device 16 includes an input/output section 60 , a database section 70 and a calculation section 80 . The input/output unit 60 transmits/receives the above-described various information D22 to D44 to/from each device in the remote control room 12 and the construction site 14 via the network 18. FIG. The database unit 70 includes a relational database 72, a work-related database 74, and an individual worker database 76. The relationship database 72 defines the relationship between the information D22, D26, D42, D44 input to the control device 16 and the estimated "equipment status" (see FIG. 2).

The work-related database 74 also stores work processes at the construction site 14, construction drawing data, and the like. Also, the worker personal database 76 stores various personal information of the user 52 . In particular, the operator's personal database 76 stores personal correction information for correcting the stimulus amount according to user's 52 preference. For example, if the user 52 is a veteran, it is conceivable to weaken the amount of stimulation.

The computing unit 80 also includes a device status estimating unit 82, a notification item setting unit 84 (notification item setting means), a notification item notification unit 86 (notification item notification means), a video delay measurement unit 88, and tactile presentation information. and an output unit 90 (tactile presentation information output means). The tactile sensation presentation information output section 90 includes a data value calculation section 92 and a stimulus amount calculation section 94 . Based on the information D22, D26, D42, and D44 input to the control device 16 and the contents of the relational database 72, the device status estimation unit 82 estimates one or more device statuses (see FIG. 2). and the corresponding information item. Furthermore, the device status estimation unit 82 identifies the highest priority item, which is the information item with the highest priority (the information item in the uppermost column in FIG. 2), from among the estimated information items.

The notification item setting unit 84 determines whether or not to set the highest priority item specified by the device status estimation unit 82 as a notification item to be notified of data to the user. The notification item notification unit 86 also outputs the set notification item to the information output unit 24 via the input/output unit 60 and the network 18 as notification item information D24.

The data value calculator 92 in the tactile sense presentation information output unit 90 calculates the data value for the highest priority item. Further, the stimulus amount calculation unit 94 in the tactile sense presentation information output unit 90 calculates the stimulus amount corresponding to the user 52 based on the data values corresponding to the notification items and the operator's personal database 76 . Then, the tactile sense presentation information output section 90 outputs the tactile sense presentation information D28 designating the calculated stimulus amount to the tactile sense presentation device 28 via the input/output section 60 and the network 18 .

FIG. 3 is a schematic diagram of the operation device 26 and the tactile presentation device 28. As shown in FIG.
The operating device 26 includes a left working lever 26WL, a right working lever 26WR, a left travel lever 26DL, and a right travel lever 26DR. The left and right work levers 26WL and 26WR are used for operating the bucket 32 and the arm 34 and turning the vehicle body 36. As shown in FIG. The left and right running levers 26DL and 26DR are used to operate the crawler belt 38 to run. Therefore, the operation information D26 (see FIG. 1) includes information regarding the operation of the bucket 32, the operation of the arm 34, the turning operation of the vehicle body 36, and the traveling operation of the crawler belt 38.

The tactile sense presentation device 28 also includes four vibrating units 28WR, 28WL, 28DR, and 28DL. The vibrating units 28WR, 28WL, 28DR, and 28DL are attached to the lower portions of the levers 26WR, 26WL, 26DR, and 26DL, respectively, and vibrate based on the tactile sense presentation information D28 to vibrate the corresponding levers. As a result, when the user 52 grips the vibrated lever, a tactile stimulus due to the vibration is transmitted to the hand of the user 52 . When the tactile sense presentation information D28 is supplied to the tactile sense presentation device 28, all the vibrating units 28WR, 28WL, 28DR, and 28DL may be vibrated, or some vibrating units corresponding to the notification item information D24 may be vibrated. good.

<Details of information items and equipment status>
Here, with reference to FIG. 2 again, details of each information item and device status will be described.
・A1 (peripheral object distance):
The device status corresponding to the information item A1 (peripheral object distance) is "peripheral object approaching". This means that a peripheral object such as a "person" or an "object" exists in the vicinity of the operation target device 30 (for example, within a predetermined distance from the operation target device 30). The meaning of the data value in the information item A1 is the "distance" from the operation target device 30 to the surrounding objects. If the user does not notice the surrounding objects of the operation target device 30, there is a possibility that a contact accident may occur. Therefore, it is preferable that the user 52 grasps the distance between the operation target device 30 and the surrounding objects. The peripheral objects and the distance from the operation target device 30 to the peripheral objects may be detected using the human sensor value or peripheral camera image included in the environment information D44.

Further, it is possible to determine whether or not the user 52 is gazing at a peripheral object (more precisely, the image of the peripheral object displayed on the display 24a) based on the line-of-sight information of the user 52 included in the user state information D22. Therefore, this information item A1 may be enabled when the user 52 is not gazing at the surrounding objects. Then, the stimulus amount designated by the tactile sensation presentation information D28 should be decreased as the "distance" increases. For example, the stimulus amount may be inversely proportional to the "distance".

・A2 (Aircraft Inclination Index):
The "body tilt index" of the information item A2 is, for example, the body tilt angle. The device status corresponding to this is "vehicle tilt index large". This means that the operation target device 30 continues to travel in a situation where the body tilt index is greater than the predetermined value. The meaning of the data value in the information item A2 is the body tilt index of the device 30 to be operated. When the operation target device 30 is running, the machine body tilt index changes. Therefore, if the user 52 overlooks the change, the operation target device 30 may overturn. Therefore, it is preferable that the user 52 can continuously grasp the change in the body tilt index while the operation target device 30 is running.

It is conceivable to directly apply the aircraft inclination angle included in the equipment operation information D42 as the aircraft inclination index. However, a value obtained by slightly correcting the aircraft inclination angle may be used as the aircraft inclination index. For example, if the ground at the construction site 14 is uneven or muddy, or if the wind speed or amount of rainfall is high, the aircraft inclination angle may change rapidly. Therefore, if the unevenness of the ground, the state of muddy ground, the wind speed, the amount of rainfall, etc. included in the environment information D44 are greater than predetermined values, the aircraft tilt index may be corrected to a value greater than the aircraft tilt angle. .

Further, when it is detected that the user is gazing at the travel levers 26DL, 26DR, etc., based on the line-of-sight information of the user 52 included in the user state information D22, the body tilt index is set larger than the body tilt angle. good too. Further, when the user 52 is performing a travel operation (operating the travel levers 26DL and 26DR), the information item A2 may be validated according to the operation information D26. The stimulus amount specified by the tactile sense presentation information D28 is preferably increased as the body tilt index increases. For example, the stimulus amount may be proportional to the body tilt index.

・A3 (centrifugal force):
The device status corresponding to the information item A3 (centrifugal force) is "large centrifugal force". This means that the vehicle body 36 is turning in a situation where the centrifugal force applied to the device 30 to be operated is greater than a predetermined value. The meaning of the data value in the information item A3 is "centrifugal force applied to the operation target device 30". If the centrifugal force becomes excessive, the device 30 to be operated may be overturned. Therefore, when the centrifugal force is large, it is preferable that the user 52 can continuously grasp the centrifugal force.

The centrifugal force can be obtained from the "bucket load sensor value" included in the equipment operation information D42. Further, when the user 52 is performing the turning operation of the bucket 32, the information item A3 may be validated according to the operation information D26. Further, when the user is gazing at the movement destination of the bucket 32 based on the line-of-sight information of the user 52 included in the user state information D22, the information item A3 may be activated. The stimulus amount designated by the tactile sensation presentation information D28 is preferably increased as the centrifugal force increases. For example, the stimulus amount may be proportional to the centrifugal force.

・B4 (bucket loading capacity):
The equipment status corresponding to the information item B4 (bucket loading capacity) is "large bucket loading capacity". This means that the load capacity of the bucket 32 is greater than the predetermined weight. The data value in the information item B4 is the "loading amount of the bucket 32". When the soil in the bucket 32 contains a lot of water, the load capacity of the bucket 32 becomes heavier than it looks. Therefore, when the load capacity of the bucket 32 is large, it is preferable for the user 52 to grasp the size. The bucket loading amount can be obtained according to the "oil pressure change amount in the bucket", the "bucket loading amount sensor value", the "ground measurement result" included in the environment information D44, and the like. The amount of stimulus designated by the tactile presentation information D28 should be increased as the load increases. For example, the stimulus amount may be proportional to the payload.

・ B5 (soil hardness):
The device status corresponding to the information item B5 is "soil hardness is high". This means that the soil hardness of the ground to be worked on is greater than a predetermined value, and more specifically, the load applied to the bucket 32 when the bucket 32 is inserted into the ground is greater than a predetermined value. means The data value in the information item B5 is "soil hardness of ground to be worked". If the soil hardness of the work target is high, when the bucket 32 is inserted into the ground, a part of the operation target device 30 is lifted from the ground, and the operation target device 30 is tilted. If this inclination is large, the operation target device 30 will fall over, so it is preferable for the user to grasp the soil hardness of the work target.

The soil hardness, that is, the load applied to the bucket 32 can be obtained from the amount of change in the "oil pressure in the bucket 32" included in the equipment operation information D42. Also, the amount of vibration in the device 30 to be operated when the bucket 32 is inserted into the ground (this is included in the device operation information D42), the ambient sound, that is, the magnitude of the impact sound (this is included in the environment information D44) Soil hardness can also be determined by Furthermore, the soil hardness can also be obtained indirectly from the "ground material" (for example, rock composition, water content, etc.) in the environment information D44. Then, the stimulus amount specified by the tactile presentation information D28 is preferably increased as the soil hardness increases. For example, the amount of stimulation may be proportional to soil hardness.

・B6 (bucket tip distance):
As described above, the bucket tip distance is the distance from the tip position of the bucket 32 to an object such as the ground. The device status corresponding to this is "small bucket tip distance". This means that the bucket tip distance is less than or equal to the predetermined value. Here, the "object" is not limited to the ground, and may be the bottom of a puddle or a buried object in the ground. Then, the data value in the information item B6 becomes "bucket tip distance".

When the bucket 32 collides with an object such as the ground at high speed, excessive force is applied to the operation target device 30, which may cause damage to the machine body. Therefore, it is preferable for the user 52 to grasp the bucket tip distance when the bucket 32 is close to the object. The bucket tip distance can be obtained from the "bucket tip distance sensor value" included in the equipment operation information D42 and the "picture information around the bucket" included in the environment information D44. Then, the stimulus amount designated by the tactile sensation presentation information D28 is preferably decreased as the bucket tip distance is increased. For example, the amount of stimulation may be inversely proportional to bucket tip distance.

・B7 (remaining relief distance):
A state in which the arm 34 (see FIG. 1) of the device 30 to be operated is fully extended is called a “relief state”. The "remaining relief distance" is the result of subtracting the current length of the arm 34 from the length of the arm 34 in the relief state. The device status corresponding to the information item B7 is "remaining relief distance is small". This means that the remaining relief distance is equal to or less than the predetermined value. The data value in the information item B7 is "remaining relief distance".

If a strong force is applied to the bucket 32 or the arm 34 while the device 30 to be operated is in the relief state, the machine body may be damaged. Therefore, it is preferable for the user 52 to know the remaining relief distance when the arm 34 is close to the relief state. The remaining relief distance can be obtained from the amount of change in the "oil pressure of arm 34" included in the equipment operation information D42. Here, the remaining relief distance becomes a problem especially when excavating operation is being performed. Therefore, in the operation information D26 or D30, the information item B7 may be validated when the "excavation operation" is detected. Then, the amount of stimulation designated by the tactile sense presentation information D28 is preferably decreased as the remaining relief distance becomes longer. For example, the stimulus amount may be inversely proportional to the remaining relief distance.

・C8 (travel speed change amount):
The running speed change amount of the information item C8 is, for example, the absolute value of the acceleration with respect to the running speed of the operation target device 30 . Further, the equipment status corresponding to this means that the amount of change in running speed is large, that is, the amount of change in running speed is greater than or equal to a predetermined value. The data value in the information item C8 is the running speed change amount described above. When the operation target device 30 is in a running state, the running speed may change depending on the state of the ground of the construction site 14 .

At that time, if the user 52 in the remote control room 12 does not notice the change in the traveling speed, it may lead to an accident. Therefore, it is preferable to inform the user 52 of the large change in running speed. The travel speed change amount can be obtained based on the travel speed included in the device operation information D42. The stimulus amount designated by the tactile sensation presentation information D28 is preferably increased as the running speed change amount increases. For example, the amount of stimulus may be proportional to the amount of change in running speed.

・C9 (video delay time):
As described above, the general information D25 includes video signals. An information item C9 is the delay time of the video signal in the network 18. FIG. The corresponding device status (video delay time/length) means that the video delay time is equal to or longer than a predetermined time. The data value in the information item C9 is the video delay time described above. If the user 52 is unaware of the video signal delay, it may lead to an accident. Therefore, when the delay time of the video signal is large, it is preferable to inform the user 52 of the delay time. The video delay time is measured by the video delay measuring unit 88 (see FIG. 1). Then, the amount of stimulation specified by the tactile sense presentation information D28 should be increased as the video delay time becomes longer. For example, it is preferable to make the amount of stimulation proportional to the delay time.

<Operation of the first embodiment>
FIG. 4 is a flow chart of an information presentation routine executed by the control device 16. As shown in FIG.
When the process proceeds to step S2 in FIG. 4, the input/output unit 60 (see FIG. 1) receives information D22, D26, D42 and D44 from each device in the remote control room 12 and the construction site 14. FIG. Next, when the process proceeds to step S4, the device status estimating unit 82 (see FIG. 1) generates a Estimates one or more device statuses and corresponding information items (see FIG. 2) that are currently in use. Also, when a plurality of information items are estimated in step S4, the device status estimation unit 82 compares the priorities of these information items and specifies the highest priority item, which is the information item with the highest priority.

Next, when the process proceeds to step S6, the data value calculator 92 (see FIG. 1) calculates the data value for the specified highest priority item. Next, when the process proceeds to step S8, the notification item setting unit 84 determines whether or not the highest priority item satisfies the setting condition to be set for the notification item based on the calculated data value. . This setting condition is, for example, a condition such as "the data value of the highest priority item is greater than a predetermined threshold value corresponding to the highest priority item".

If "No" is determined in step S8, the process returns to step S2. After that, the operations of steps S2 to S8 are repeated until "Yes" is determined in step S8. Then, if it is determined as "Yes" in step S8, the process proceeds to step S10, and the notification item setting unit 84 sets the highest priority item as the notification item.

Next, when the process proceeds to step S12, the notification item setting unit 84 determines whether or not the notification item has been changed. That is, the notification item setting unit 84 determines whether or not the notification item set when step S10 is executed has changed from the previously set notification item. If "Yes" is determined in step S12, the process proceeds to step S14, and if "No" is determined, the process proceeds to step S16.

In step S14, the notification item notification unit 86 (see FIG. 1) transmits, via the input/output unit 60, the notification item information D24 notifying that the notification item has been changed to the information output unit 24. FIG. The information output unit 24 notifies the user 52 that the notification item has been changed. For example, if the notification item after change is "A1: Peripheral object distance" (see FIG. 2), it is conceivable to display a blinking character string "Peripheral object distance" on the display 24a (see FIG. 1). . Also, instead of (or in addition to) this, the speaker 24b may emit a sound saying "The notification item has been changed to the distance to surrounding objects."

Returning to FIG. 4, when the process proceeds to step S16, the tactile presentation information output unit 90 (see FIG. 1) outputs the data value calculated in step S6 and the contents of the worker's personal database 76. Based on this, the amount of stimulation to be given to the user is calculated. Next, when the process proceeds to step S18, the tactile sense presentation information output unit 90 outputs tactile sense presentation information D28 for notifying the tactile sense presentation device 28 (see FIG. 1) of the calculated stimulus amount. Thereby, the tactile sense presentation device 28 transmits to the user 52 the tactile sense stimulation of the stimulus amount according to the tactile sense presentation information D28. When the process of step S18 is completed, the process returns to step S2, and the operations described above are repeated.

<Effect of the first embodiment>
As described above, according to the preferred embodiment, the information presentation device (16) includes the operation information D26 representing the operation state of the operation device 26 by the user 52, and the operation target device 30 remotely operated according to the operation information D26. a notification item setting unit 84 that selects a notification item that is an information item to be notified to the user 52 from among a plurality of information items based on the device operation information D42 that indicates the operating state of the device; A notification item notification unit 86 that outputs item information D24, and a tactile sense presentation information output unit 90 that outputs tactile sense presentation information D28 representing the content of the tactile sense stimulus given to the user 52 corresponding to the notification item. Thereby, the information presentation device (16) can present various kinds of meanings according to the notification item in response to the tactile stimulus.

More preferably, the notification item setting unit 84 selects the notification item based on the environment information D44, which is information about the surrounding environment of the operation target device 30, in addition to the operation information D26 and the device operation information D42. Thereby, the meaning of the tactile sense stimulation can be switched according to the surrounding environment.

In addition to the operation information D26, the device operation information D42, and the environment information D44, the notification item setting unit 84 also operates based on the user state information D22, which is information representing the state of the user 52 other than the operation information D26. It is more preferable to select a notification item. As a result, the notification item can be selected according to, for example, the line of sight and physical condition of the user 52 .

In addition, the tactile presentation information output unit 90 outputs the operation information D26, the device operation information D42, the environment information D44 that is information about the surrounding environment of the operation target device 30, and the user information other than the operation information D26 that indicates the state of the user 52. More preferably, the tactile sense presentation information D28 is calculated based on one or more information selected from the group consisting of the state information D22. Accordingly, an appropriate tactile sense stimulation can be given to the user 52 based on the operation information D26, the device operation information D42, the environment information D44, or the tactile sense presentation information D28.

[Modification]
The present invention is not limited to the embodiments described above, and various modifications are possible. The above-described embodiments are illustrated for easy understanding of the present invention, and are not necessarily limited to those having all the described configurations. Further, other configurations may be added to the configurations of the above embodiments, and part of the configurations may be replaced with other configurations. Also, the control lines and information lines shown in the drawings are those considered to be necessary for explanation, and do not necessarily show all the control lines and information lines necessary on the product. In practice, it may be considered that almost all configurations are interconnected. Possible modifications to the above embodiment are, for example, the following.

(1) In the above-described embodiment, only one notification item is selected from among a plurality of information items to notify the user 52 of a data value by tactile stimulation. However, the notification item is not limited to only one item. For example, by selecting a plurality of notification items and assigning the four vibration units 28WR, 28WL, 28DR, and 28DL (see FIG. 3) to these plurality of notification items, data values for the plurality of notification items can be sent to the user 52 at the same time. can be notified.

(2) Since the hardware of the control device 16 in the above embodiment can be realized by a general computer, the flowchart shown in FIG. may be distributed via

(3) In the above embodiment, the information item selection unit 84 selects notification items based on the line-of-sight information included in the user state information D22. The physical condition of the user 52 may be determined using the physical condition of the user 52 and the notification item may be selected based on the physical condition of the user 52 .

(4) The processing shown in FIG. 4 and the other processing described above have been described as software processing using a program in the above embodiment, but some or all of them are ASIC (Application Specific Integrated Circuit; It may be replaced with hardware processing using an FPGA (Field Programmable Gate Array) or the like. Also, the database unit 70 shown in FIG. 1 may be provided in a cloud (not shown) connected to the network 18 .

(5) In the above embodiment, a power shovel was illustrated as the device 30 to be operated, but the device 30 to be operated is not limited to a power shovel. It may be a vehicle, a ship, or the like.

10 remote control system 16 control device (information presentation device, computer)
24 information output unit 26 operation device 28 tactile presentation device 30 operation target device 52 user 84 notification item setting unit (notification item setting means)
86 notification item notification unit (notification item notification means)
90 tactile presentation information output unit (tactile presentation information output means)
D22 User status information D24 Notification item information D26 Operation information D28 Tactile sense presentation information D42 Device operation information D44 Environment information

Claims (6)

A plurality of pieces of information whose priority is determined based on operation information indicating the operation state of an operation device by a user and device operation information indicating the operation state of an operation target device remotely operated according to the operation information. a device status estimating unit for estimating, from among the items, the information item corresponding to one or more occurring device statuses;
a notification item setting unit that selects the information item with the highest priority from the estimated information items as a notification item to be notified to the user;
a notification item notification unit that outputs notification item information representing the notification item;
and a tactile sense presentation information output unit that outputs tactile sense presentation information representing the content of the tactile sense stimulus given to the user, corresponding to the notification item. The notification item setting unit selects the notification item based on the operation information, the device operation information, and environmental information that is information about the surrounding environment of the operation target device,
The device to be operated includes a bucket for excavating earth and sand, and the device operation information includes the load capacity of the bucket and the tilt angle of the device to be operated.
2. The information presentation device according to claim 1, characterized by: The notification item setting unit sets the notification items based on user state information, which is information representing the state of the user other than the operation information, in addition to the operation information, the device operation information, and the environment information. is to select
The user state information includes gaze, body temperature or heart rate of the user
3. The information presentation device according to claim 2, characterized by: The tactile sense presentation information output unit outputs information from the operation information, the device operation information, environment information that is information about the surrounding environment of the operation target device, and user state information that is information representing the state of the user other than the operation information. 2. The information presentation device according to claim 1, wherein the tactile sense presentation information is calculated based on one or a plurality of information selected from the group consisting of. A plurality of pieces of information whose priority is determined based on operation information indicating the operation state of an operation device by a user and device operation information indicating the operation state of an operation target device remotely operated according to the operation information. a device status estimating unit for estimating, from among the items, the information item corresponding to one or more occurring device statuses;
a notification item setting unit that selects the information item with the highest priority from the estimated information items as a notification item to be notified to the user;
a notification item notification unit that outputs notification item information representing the notification item;
a tactile sense presentation information output unit that outputs tactile sense presentation information representing the content of the tactile sense stimulus given to the user corresponding to the notification item;
the operation target device;
a tactile presentation device that provides tactile stimulation to the user based on the tactile presentation information;
and an information output unit that notifies the user of the notification item based on the notification item information. the computer,
A plurality of pieces of information whose priority is determined based on operation information indicating the operation state of an operation device by a user and device operation information indicating the operation state of an operation target device remotely operated according to the operation information. equipment status estimating means for estimating, from among the items, the information item corresponding to one or more occurring equipment statuses;
notification item setting means for selecting the information item with the highest priority from the estimated information items as the notification item to be notified to the user;
notification item notification means for outputting notification item information representing the notification item;
tactile sense presentation information output means for outputting tactile sense presentation information representing the content of the tactile sense stimulus given to the user, corresponding to the notification item;
A program to function as

Images