JP2020131768A - Maneuvering system, maneuvering device, maneuvering control method, and program - Google Patents

Abstract

To solve the problem that a maneuverer is unable to perform maneuvering on grasping a circumferential situation of an object to be maneuvered.SOLUTION: The present invention relates to a maneuvering system having a moving body and a maneuvering device for remotely maneuvering the moving body. The moving body has a circumference information acquisition part which measures information on a circumference of the moving body, and a transmission part which transmits the circumference information measured by the circumference information acquisition part to the maneuvering device. The maneuvering device has a reception part which received the circumference information, an operation part which receives an operation for maneuvering the moving body, and a force sense imparting part which imparts force sense to the operation based upon the received circumference information.SELECTED DRAWING: Figure 6

Description

The present disclosure relates to a maneuvering system, a maneuvering device, a maneuvering control method, and a program.

An inspection system is known that inspects a structure such as a high place, a narrow place, or a place where a harmful substance is generated by bringing a remote-controlled reading value close to the structure.

For example, Patent Document 1 includes an unmanned floating machine provided with a floating means for levitating in the air by remote operation, and a control device for operating the unmanned floating machine. The unmanned floating machine provides information to be inspected for an object to be inspected. The flight state detecting means for detecting the flight state of the unmanned levitation aircraft, and the transmitting means for transmitting the detected flight state to the control device, the control device includes the unmanned control device. Having a transmission means for notifying the operator of the levitation machine by force or hearing, and a control means for operating the transmission means in an operation mode according to a flight state received from the unmanned levitation machine. The featured inspection system is described.

However, the inspection system described in Patent Document 1 detects the flight state of the unmanned float, but does not detect the state around the unmanned float.

Therefore, there is a problem that the operator cannot grasp the surrounding conditions of the object to be operated and operate the aircraft.

In order to solve the above-mentioned problems, the invention according to claim 1 is a control system including a moving body and a control device for remotely controlling the moving body, wherein the moving body is around the moving body. The control device has a surrounding information acquisition unit that measures the information of the surroundings and a transmission unit that transmits the surrounding information measured by the surrounding information acquisition unit to the control device, and the control device receives the surrounding information. The control system includes an operation unit that receives an operation for manipulating the moving body, and a force sense imparting unit that gives a force sense to the operation based on the received surrounding information.

According to the present invention, there is an effect that the operator of the moving body can more accurately grasp and operate the surrounding conditions of the object to be operated.

It is explanatory drawing of the whole structure of the control system which concerns on 1st Embodiment. It is an external view of the moving body which concerns on 1st Embodiment. It is a hardware block diagram of the moving body which concerns on 1st Embodiment. It is an external view of the control device which concerns on 1st Embodiment. It is a hardware block diagram of the control device which concerns on 1st Embodiment. It is a functional block diagram of the control system which concerns on 1st Embodiment. It is a flowchart which shows the maneuvering process. It is a flowchart which shows the ambient information transmission processing. It is explanatory drawing of the distance information as an example of the surrounding information. It is a flowchart which shows the force sense giving process. It is an external view of the control device which concerns on 2nd Embodiment. It is a flowchart which shows the shape image display processing. It is an external view of the bridge K as a shape image generation target. It is a figure explaining the display example of a shape image and a position image. This is a display example of the input assistance screen. It is a flowchart which shows the input auxiliary processing. It is a flowchart which shows the input auxiliary image display processing.

Hereinafter, modes for carrying out the invention will be described with reference to the drawings. In the description of the drawings, the same elements are designated by the same reference numerals, and duplicate description will be omitted.

FIG. 1 is a schematic view of a maneuvering system according to the present embodiment.

As shown in FIG. 1, the maneuvering system S1 constructs the inspection system S3 together with the diagnostic system S2. The control system S1 of the present embodiment is constructed by the moving body 1 and the control device 2.

The moving body 1 and the control device 2 constituting the control system S1 can communicate with each other via radio, and the operator O can control the moving body 1 by operating the control device 2.

In the present embodiment, the moving body 1 is a flying body such as a drone, a multicopter, an unmanned aerial vehicle, etc., and is used for inspection of a large structure such as a bridge K, which cannot be reached by a person without scaffolding. It is possible. For example, the bridge K shown in FIG. 1 is a relatively large structure having a pier portion K1 and a main body portion K2. The operator O can operate the moving body 1 by the control device 2 to move the moving body 1, and bring the moving body 1 equipped with the camera and the sensor closer to a desired portion of the bridge K to perform the inspection.

More specifically, the moving body 1 equipped with the camera flies near the wall surface of the bridge K and images the wall surface of the bridge K, so that the data for the investigation is a deformed part such as a crack H on the wall surface. You can get a close-up image of. Furthermore, by continuously imaging the wall surface of the bridge K while flying near the wall surface of the bridge K, a close-up image of the bridge K that is continuously positioned can be obtained. By separately combining these close-up images, it is possible to obtain an image (deformed development view) showing the position of the deformed portion in the wall surface of the bridge K, a deformed image, or the like. In addition to the camera, various sensors for moving the moving body 1 and for detecting the state of the bridge K may be mounted.

The inspector, and if the inspector has an assistant, the assistant may write images / sounds showing the state of the deformation and comments on the deformation in field notes, etc., and in some cases, the entire bridge K and its surroundings. Or take an image of. As the field note, a tablet-type PC or the like may be used.

Next, the diagnostic system S2 will be described. The diagnostic system S2 is constructed by the diagnostic processing device 3 and the diagnostic management server 4. Further, the diagnostic processing device 3 and the diagnostic management server 4 constituting the diagnostic system S2 can communicate with each other via the communication network N. The communication network N is constructed by the Internet, a mobile communication network, a LAN (Local Area Network), and the like. Communication network N includes not only wired communication, but also 3G (3rd Generation), 4G (4th Generation), 5G (5th Generation), WiMAX (Worldwide Interoperability for Microwave Access), LTE (Long Radio), etc. The network may be included. Further, the diagnostic processing device 3 can communicate by a short-range communication technique such as NFC (Near Field Communication) (registered trademark).

The image data of the bridge K obtained by the moving body 1 and the detected data obtained from each sensor are transmitted to the diagnostic management server 4 via the diagnostic processing device 3 and can be managed by the diagnostic management server 4. Is.

Since not only bridges but also structures such as dams and buildings are covered with concrete, deformation such as cracks occurs due to aging. If left as it is, the concrete pieces may come off and damage vehicles and people in transit. Therefore, the inspector conducts regular inspections and reports the inspection results to the national or prefectural organizations (hereinafter referred to as "countries, etc."). In order to make this report, the inspector must submit the documents required by the national and local governments.

The inspector obtains from a field note such as comments written at the time of inspection, a photograph of the deformation obtained by photographing the deformed part of the building, and a government office that describes information identifying the building part. Prepare documents to be submitted to the national government based on the ledger.

Further, the diagnostic processing device 3 is a computer (information processing device) for inputting various data such as diagnostic information including a determination result of deformation, a diagnosis target image, a diagnostic area, and the like, and a dedicated application program for drawing is provided. It is installed. The user of the diagnostic processing device 3 inputs comments such as field notes, image data of the bridge K as an example of the structure, and detection data of each sensor into the diagnostic processing device 3. After inputting to another device, data may be transferred from the other device to the diagnostic processing device 3. Further, the user of the diagnostic processing device 3 inputs the data of the ledger for identifying the bridge or the like obtained from the national government or the like into the diagnostic processing device 3. The length and height of bridges, etc. are recorded in the ledger. In this case, the diagnostic processing device 3 is an input device.

Further, the diagnostic processing device 3 is equipped with a browser, and the diagnostic processing device 3 can display the expanded image sent from the diagnostic management server 4. In addition, the user of the diagnostic processing device 3 draws a line or the like from above the deformed portion such as a crack shown on the developed image to quantify the position of the deformed portion which is an image. Do.

The user of the diagnostic processing device 3 downloads the data of the submitted document including the deformed development drawing created by drawing the deformed part from the diagnostic management server 4, and prints the document or the data in the non-printed state. To the government, etc.

The diagnostic management server 4 manages data such as expanded images, comments, and ledgers. In addition, the diagnostic management server 4 manages data in which deformed portions such as cracks drawn by the diagnostic processing device 3 are quantified, and manages coordinate data in images.

FIG. 2 describes an example of the appearance of the moving body 1 according to the first embodiment. FIG. 2 is a schematic configuration diagram of the moving body 1 according to the first embodiment. (A) is a plan view of the moving body 1, (B) is a left side view of (A), and (C) is a front view of (A). The appearance configuration shown in FIG. 2 may have the same configuration in each embodiment, and components may be added or deleted as needed.

The moving body 1 is a remote-controlled flying body having a frame 101, a driving device 102, a rotary wing 103, and a leg portion 104.

The moving body 1 is a movable imaging device having an imaging device 105 that images a subject and acquires an captured image. The image pickup device 105 may be a monocular camera that captures a part of the periphery of the moving body 1 to acquire a plane image, or captures the periphery of the moving body 1 to obtain a spherical (360 °) panoramic image. It may be a special photographing device to be acquired. Further, the acquired image may be a moving image or a still image, and may be both a moving image and a still image. Further, the acquired image may be an image including sound data together with the image. The image pickup apparatus 105 is an example of an image pickup unit.

The processing or operation of the moving body 1 such as flight and imaging is controlled by the control device 100.

As shown in FIG. 3A, the frame 101 is arranged so that the web portion 101a and the flange portions 101b arranged at both ends of the web portion 101a have an H-shape in a plan view. However, the frame 101 is not limited to this, and may have a configuration having four arms extending radially from the center of the moving body 1 and provided in an X shape so as to form a right angle to each other. Further, the arm may extend radially from the center by the number of the drive device 102 and the rotary blade 103.

The drive device 102 is a motor that rotates the rotary blade 103, and a brushless motor is preferable. The drive device 102 is attached to each of the tip portions of the two flange portions 101b and is connected to the rotation shaft of the rotary blade 103. Each drive device 102 is controlled independently of each other by the control device 100. The control device 100 is an example of a control unit.

The rotary blade 103 is attached to each of the drive devices 102, and is rotated by the drive of the drive device 102. The rotation speed of the rotary blade 103 is adjusted by independently controlling the drive of each drive device 102. The number of rotors 103 is not limited to four in the illustrated example, but if it is 3 or more, it functions so that stable flight can be performed as an air vehicle, and even numbers such as 4, 6 and 8 are used. It is desirable to have. Further, as shown in FIG. 3A, the rotary blade 103 is not limited to a form in which a plurality of rotary blades 103 are arranged on substantially the same plane, and a plurality of rotary blades 103 arranged on substantially the same plane. The rotary blades 103 may be stacked in multiple stages.

The leg portion 104 is connected to the lower surface of the two flange portions 101b at a substantially central position. The lower portion of the leg portion 104 is formed in an inverted V shape to protect the moving body 1 from impact when the moving body 1 lands on the ground.

The control device 100 is attached to the lower surface side of the web portion 101a of the frame 101. The control device 100 controls the drive of the drive device 102 according to signals from various sensors installed according to the purpose and control signals transmitted from the outside.

Further, the control device 100 independently controls the rotation speed of each rotor 103 by independently controlling the drive of each drive device 102, and moves the moving body 1 up, down, forward, backward, and horizontally. Move in each direction and move into the air to hover (stop flight).

The image pickup device 105 is a camera that acquires an image by a signal from the control device 100. The image pickup apparatus 105 is fixed to the web portion 101a of the frame 101.

The image pickup device 105 of the present embodiment is installed so that the first image pickup device 105a is installed so that the optical axis is substantially horizontal when the moving body 1 is flying, and the optical axis is substantially vertically upward when the moving body 1 is flying. It also has a second imaging device 105b.

With the above configuration, a vertical portion such as the side surface of the bridge K can be imaged by the first imaging device 105a, and the back side of the bridge K can be imaged by the second imaging device 105b.

The distance measuring sensor 106 of the present embodiment is a sensor that detects the distance between the moving body 1 and the surrounding structure. The distance measuring sensor 106 is installed at substantially the center of the moving body 1 in the horizontal direction. The range finder is, for example, a TOF (Time Of Flight) sensor, a LIDAR (Light Detection and Ranking) sensor, and the like, but is not limited thereto.

Next, the hardware configuration of the moving body 1 in the embodiment will be described with reference to FIG. The hardware configuration shown in FIG. 3 may have the same configuration in each embodiment, and components may be added or deleted as needed.

The moving body 1 includes a control device 100 that controls the processing or operation of the moving body 1. As described above, the control device 100 is provided inside the moving body 1. The control device 100 may be provided outside the moving body 1, or may be provided as a device separate from the moving body 1. The control device 100 is an example of an information processing device.

The control device 100 includes a CPU (Central Processing Unit) 11, a ROM (Read Only Memory) 12, a RAM (Random Access Memory) 13, an HDD (Hard Disk Drive) 14, a wireless I / F (Inter Face) 15, and a media I / F. It has an F16, a network I / F17, an extended I / F18 and a bus line 19. In addition, it may have a sound input / output I / F and the like.

The CPU 11 controls the entire moving body 1. The CPU 11 is an arithmetic unit that realizes each function of the moving body 1 by reading a program or data stored in a ROM 12 or an HD (Hard Disk) 14a or the like on a RAM 13 and executing a process. The control system S2 realizes an information processing method such as a steering control method according to the present invention by, for example, executing a part or all of the program according to the present invention on the CPU 11.

The RAM 13 is a volatile memory used as a work area or the like of the CPU 11. The ROM 12 is a non-volatile memory capable of holding a program or data even when the power is turned off. The HDD 14 controls reading or writing of various data to the HD 14a according to the control of the CPU 11. The HD14a stores various data such as programs.

The wireless I / F15 is a communication interface for wireless communication with the control device 2.

The media I / F16 controls reading or writing (storage) of data to a recording medium 17a such as a USB (Universal Serial Bus) memory, a memory card, an optical disk, or a flash memory.

The network I / F17 is, for example, a communication interface such as a wired or wireless LAN (Local Area Network). In addition, network I / F17 includes 3G (3rd Generation), LTE (Long Term Evolution), 4G (4th Generation), 5G (5th Generation), Zigbee (registered trademark), BLE (Bluetooth (registered trademark) Low Energy), and so on. A communication interface for millimeter-wave wireless communication may be provided.

Further, the drive device 102, the photographing device 105, and the distance measuring sensor 106 described above are connected to the control device 100 via the extended I / F 18. Further, various sensors 107 other than the distance measuring sensor, a GPS (Global Positioning System) receiving unit 108, and a timer 109 are connected.

Based on the command from the CPU 11, the drive device 102 rotationally drives the rotary blade 103 to fly and move the moving body 1. The various sensors 107 are, for example, sensors such as an electronic magnetic compass, a gyro compass, and an acceleration sensor that detect the geomagnetism used for the flight function. In addition, a barometer, a thermometer, a photometer, a motion sensor, an illuminance meter, or the like may be provided. The GPS receiving unit 108 receives GPS signals from GPS satellites. The timer 109 is a measuring device having a time measuring function. The timer 109 may be a computer soft timer.

The bus line 19 is an address bus, a data bus, or the like for electrically connecting each of the above components, and transmits an address signal, a data signal, various control signals, and the like. The CPU 11, ROM 12, RAM 13, HDD 14, wireless I / F15, media I / F16, network I / F17, and expansion I / F18 are connected to each other via the bus line 19.

Next, the appearance of the control device 2 in the embodiment will be described with reference to FIG.

In FIG. 4, the configuration of the operation unit 201 of the control device 2 will be described. The operation unit 201 of the present embodiment is an operation unit capable of being input by a user such as the operator O by a force sense and giving a force sense to the user's operation. It can also be said that the device can constrain the user's operation by a reaction force. It is also called a force feedback device, a haptic interface, or the like.

The operation unit 201 has a handle portion 202, joint portions 203a, 203b, 203c, connection portions 204a, 204b, rotary shafts 205a, 205b, 205c, 205d, encoders 206a, 206b, 206c, 206d, and a pedestal 207.

As shown in FIG. 4, the handle portion 202 and the joint portions 203a, 203b, 203c are connected by the connecting portions 204a, 204b. The joint portion 203c is installed on the pedestal 207.

Each of the joint portions 203a, 203b, 203c has a rotation shaft 205a, 205b, 205c, respectively, the handle portion 202 and the connection portion 204a rotate with respect to the joint portion 203a, and the connection portion 204a and the connection portion 204b are joint portions. It rotates with respect to 203b, and the connection portion 204b rotates with respect to the joint portion 203c. The pedestal 207 has a rotation shaft 205d, and the joint portion 203c rotates with respect to the pedestal 207.

With the above configuration, the user can perform an operation for maneuvering the moving body 1 by moving the handle portion 202 to obtain position (X, Y, Z) / posture (roll, pitch, yaw) information and their respective amounts. You can enter it.

For example, when the moving body 1 is a flying body, the throttle (up and down), the rolling (left and right), the pitch (front and back), and the yaw (turning) operations of the flying body can be controlled. The handle portion 202 is not limited to the lever type as shown in FIG. 4, and may be a stick type, a pen type, or the like.

Then, each encoder 206a, 206b, 206c, 206d measures the rotation angles of the rotation shafts 205a, 205b, 205c, 205d, respectively, and calculates the position / attitude information of the handle portion 202.

The encoder and motor are not limited to the above-mentioned rotary type, and may be linear type.

Next, the hardware configuration of the control device 2 in the embodiment will be described with reference to FIG. The hardware configuration shown in FIG. 5 may have the same configuration in each embodiment, and components may be added or deleted as needed.

The control device 2 includes a control device 200 that controls the processing or operation of the control device 2. The control device 200 is provided inside the housing of the control device 2. The control device 200 may be provided outside the housing of the control device 2, or may be provided as a device separate from the control device 2. The control device 200 is an example of a control unit.

The control device 200 includes a CPU (Central Processing Unit) 21, a ROM (Read Only Memory) 22, a RAM (Random Access Memory) 23, an HDD (Hard Disk Drive) 24, a wireless I / F25, an operation unit I / F26, and a media I. It includes / F27, input / output I / F28, extended I / F29, and bus line 30. In addition, it may have sound input / output I / F and the like.

The CPU 21 controls the entire control device 2. The CPU 21 is an arithmetic unit that realizes each function of the control device 2 by reading a program or data stored in a ROM 22 or an HD (Hard Disk) 24a or the like onto a RAM 23 and executing processing. The control system S2 realizes an information processing method such as a steering control method according to the present invention by, for example, executing a part or all of the program according to the present invention on the CPU 21.

The RAM 23 is a volatile memory used as a work area or the like of the CPU 21. The ROM 22 is a non-volatile memory capable of holding a program or data even when the power is turned off. The HDD 24 controls reading or writing of various data to the HD 24a according to the control of the CPU 21. The HD24a stores various data such as programs.

The wireless I / F 25 is an interface for wireless communication with the mobile body 1.

The operation unit I / F 26 is connected to the encoder 206 and the motor 208, and is an interface for controlling the operation unit 201.

The media I / F 27 controls reading or writing (storage) of data to a recording medium 27a such as a USB (Universal Serial Bus) memory, a memory card, an optical disk, or a flash memory.

The input / output I / F 28 is an interface for inputting / outputting characters, numerical values, various instructions, etc. to / from various external devices. The input / output I / F28 controls the display of various information such as characters or images on a display which is a display device such as an LCD (Liquid Crystal Display). Further, in addition to the display, the input / output I / F 28 may be connected to input means such as a mouse and a keyboard to control them.

Further, a timer 209 is connected to the control device 200 via the expansion I / F29. The timer 209 is a measuring device having a time measuring function. The timer 209 may be a soft timer by a computer.

The bus line 30 is an address bus, a data bus, or the like for electrically connecting each of the above components, and transmits an address signal, a data signal, various control signals, and the like. The CPU 21, ROM 22, RAM 23, HDD 24, wireless I / F 25, operation unit I / F 26, media I / F 27, input / output I / F 28, and expansion I / F 29 are connected to each other via a bus line 30.

In addition, the control device 2 processes sound input / output I / F that processes sound signal input / output with a microphone or speaker under the control of the CPU 21, and processes communication with other devices via a communication network under the control of the CPU 21. It may have a network I / F or the like.

First, the functional configuration of the control device 100 that controls the processing or operation of the moving body 1 will be described with reference to FIG. The functions realized by the control device 100 include the transmission / reception unit 110, the operation input reception unit 120, the movement control unit 130, the image pickup control unit 140, the surrounding information acquisition unit 150, the judgment unit 160, the storage / reading unit 170, and the storage unit 1000. Including.

The transmission / reception unit 110 is a function of transmitting / receiving various data or information to / from the control device 2 by wireless communication. The transmission / reception unit 110 transmits, for example, the ambient information acquired by the ambient information acquisition unit 150 to the control device 2. Further, the transmission / reception unit 110 receives, for example, a control instruction transmitted from the control device 2. The transmission / reception unit 110 is mainly realized by the processing of the CPU 11 shown in FIG. 5 and the wireless I / F15. The transmission / reception unit 110 is an example of a transmission unit.

The operation input receiving unit 120 is a function of receiving an operation on the moving body 1. For example, it accepts an operation or the like according to a control instruction received from the control device 2. The operation input receiving unit 120 is mainly realized by the processing of the CPU 11 shown in FIG.

The movement control unit 130 is a function of controlling the movement of the moving body 1 by driving the driving device 102. The movement control unit 130 moves the moving body 1 by controlling the driving of the driving device 102 in response to a steering instruction transmitted from the control device 2, for example. The movement control unit 130 is mainly realized by the processing of the CPU 11 shown in FIG. 5 and the extended I / F18. The movement control unit 130 is an example of the movement control unit.

The image pickup control unit 140 is a function of instructing the image pickup device 105 to perform an image pickup process. The image pickup control unit 140 transmits, for example, instruction information for instructing the image pickup by the image pickup device 105 based on the control instruction of the control device 2 to the image pickup device 105. The image pickup control unit 140 is mainly realized by the processing of the CPU 11 shown in FIG. 5 and the extended I / F18.

The surrounding information acquisition unit 150 is a function of acquiring information around the moving body 1 such as the distance between the moving body 1 and the surrounding structure measured by the distance measuring sensor 106. The surrounding information acquisition unit 150 measures and acquires surrounding information, and can also be called a surrounding information measuring unit. The surrounding information may be image data acquired by the image pickup apparatus 105 as long as it is the surrounding information of the moving body 1, or may be various detection results detected by various sensors. The surrounding information acquisition unit 150 is mainly realized by the processing of the CPU 11 shown in FIG. 5 and the extended I / F18.

The determination unit 160 is a function for making various determinations, which will be described later. The determination unit 160 is mainly realized by the processing of the CPU 11 shown in FIG.

The storage / reading unit 170 is a function of storing various data in the storage unit 1000 or reading various data from the storage unit 1000. Storage / Reading Unit 180 This is mainly realized by the processing of the CPU 11 shown in FIG. The storage unit 1000 is mainly realized by the ROM 12, HD 14a and the recording medium 17a shown in FIG. 5, but may be realized by the temporary storage function of the RAM 13.

The storage unit 1000 stores various settings required for remote control in advance. In addition, various information acquired when the moving body 1 operates, for example, image data captured by the image pickup control unit 140, detection results of various sensors, and the like can be stored. It is also possible to temporarily store the control command received from the control device 2. The image data, the detection result, etc. stored in the storage unit 1000 may be deleted when a predetermined time has elapsed since they were acquired, or the data transmitted to the control device 2 may be deleted. It may be configured to be deleted, or it may be deleted based on the pilot instruction from the control device 2.

Next, the functional configuration of the control device 200 of the control device 2 will be described with reference to FIG. The functions realized by the control device 200 are the transmission / reception unit 210, the operation input reception unit 220, the operation instruction generation unit 230, the force sense imparting unit 240, the display control unit 250, the shape image generation unit 260, the position image generation unit 270, and the auxiliary. It includes an image generation unit 280, a determination unit 290, a storage / reading unit 300, and a storage unit 2000. A dedicated application program for remotely controlling the moving body 1 is pre-installed in the control device 2. The control device 2 realizes each function by, for example, the CPU 21 executing the installed application program. The control device 200 is an example of a control unit.

The transmission / reception unit 210 is a function of transmitting / receiving various data or information to / from the mobile body 1 via wireless communication. The transmission / reception unit 210 receives, for example, the surrounding information acquired by the surrounding information acquisition unit 150 included in the moving body 1 from the moving body 1. Further, the image data acquired by the image pickup device 105 of the moving body 1 is received from the moving body 1. The transmission / reception unit 210 is mainly realized by the processing of the CPU 21 shown in FIG. 5 and the wireless I / F 25. The transmission / reception unit 210 is an example of a reception unit.

The operation input receiving unit 220 is a function of receiving various selections or operation inputs to the control device 2. The operation input receiving unit 220 is mainly realized by the processing of the CPU 21 and the operation unit I / F 26. The operation input receiving unit 220 may be realized by an input means other than the operation unit 201, for example, input by voice, operation input by a keyboard or a mouse.

The maneuvering instruction generation unit 230 is a function of generating a maneuvering instruction to the moving body 1. The maneuvering instruction generation unit 230 is mainly realized by the processing of the CPU 21.

The force sense applying unit 240 is a function of performing various processes for giving a force sense to the operation by the operation unit 201. The force sense imparting unit 240 is mainly realized by the processing of the CPU 21 or the processing of the operation unit I / F 26.

The display control unit 250 is a function of displaying various images on the display device. The display control unit 250 is mainly realized by the processing of the CPU 21 shown in FIG. 5 and the input / output I / F 28. The display control unit 250 is an example of the display control unit.

The shape image generation unit 260 generates an image showing the shape of the surrounding structure of the moving body 1, for example, a three-dimensional model image. The shape image generation unit 260 is mainly realized by the processing of the CPU 21 shown in FIG.

The position image generation unit 270 is a function of generating an image showing the position of the moving body 1 with respect to the surrounding structure, for example, a cursor image in the three-dimensional model. The position image generation unit 270 is mainly realized by the processing of the CPU 21 shown in FIG.

The auxiliary image generation unit 280 is a function of generating an auxiliary image which is an image assisting the maneuvering of the moving body 1 using the shape image. The auxiliary image generation unit 280 is mainly realized by the processing of the CPU 21 shown in FIG.

The determination unit 290 is a function for executing various determinations described later. The determination unit 290 is mainly realized by the processing of the CPU 21 shown in FIG.

The storage / reading unit 300 is a function of storing various data in the storage unit 2000 or reading various data from the storage unit 2000. The storage / reading unit 300 is mainly realized by the processing of the CPU 21 shown in FIG. The storage unit 2000 is mainly realized by the ROM 22, the RAM 23, and the recording medium 27a shown in FIG.

Further, the storage unit 2000 stores various settings required for remote control in advance. In addition, various information received from the moving body 1, for example, image data captured by the image pickup control unit 140, detection results of various sensors, and the like can be stored. The image data, the detection result, and the like stored in the storage unit 1000 may be deleted when a predetermined time has elapsed since they were acquired.

FIG. 7 is a flowchart showing the maneuvering process. This is a flow that is executed when the operator inputs an operation related to movement or imaging to the control device.

First, the operation input receiving unit 220 receives the operation of the operator (S11). Then, the maneuvering instruction generation unit 230 generates a maneuvering instruction for the moving body 1 based on the received operation (S12). Then, the transmission / reception unit 210 transmits the generated maneuvering instruction to the moving body 1.

In this way, the moving body 1 that has received the control instruction transmitted from the control device 2 is remotely controlled by operating according to the received control instruction.

FIG. 8 is a flowchart showing the surrounding information acquisition process. This is a flow executed in the moving body 1 when the movement is controlled by the operation of the operating unit 201 of the control device 2 with respect to the moving body 1.

First, the surrounding information acquisition unit 150 acquires surrounding information, which is information around the moving body 1, as a peripheral information acquisition step (S21). Then, the determination unit 160 determines whether the surrounding information of the moving body 1 is updated (S22). If it has not been updated (No), the process returns to step S21 and the surrounding information is acquired again.

If it has been updated (yes), the transmission / reception unit 110 transmits the acquired surrounding information to the control unit 1 as a peripheral information transmission step (S23). Then, the determination unit 160 determines whether or not the operation end instruction has been received from the control device 2 (S24), and if it receives it (in the case of Yes), ends this flow.

On the other hand, if the end instruction has not been received in step S24 (No), the process returns to step S21, and the present processing flow is continued while the control by the control device 2 continues.

In this way, while the moving body 1 is being steered by the control device 2, the moving body 1 continuously acquires surrounding information, which is information on the surroundings of the moving body 1, and transmits the surrounding information to the control device 2.

Here, an example of surrounding information will be described with reference to FIG.

FIG. 9 is a conceptual diagram of the moving body 1 and the surrounding structure F, which is a structure around the moving body 1. The distance between the moving body 1 and the surrounding structure F differs depending on the location of the surrounding structure F. Therefore, the distance information, which is the detection result of the distance measuring sensor 106, may acquire a plurality of different values. In the present embodiment, the shortest distance d between the distance measuring sensor 106 and the surrounding structure F is transmitted to the control device 2 as surrounding information.

FIG. 10 is a flow executed in the control device 2 when the movement is controlled by the operation of the operation unit 201 of the control device 2 with respect to the moving body 1.

First, the force sense imparting unit 240 sets a threshold value d0 of the distance between the moving body 1 and the surrounding structure as a threshold value setting step (S31). The setting in this step S31 may be set by acquiring the input of the threshold value d0 executed by the user such as the operator O at the timing of this step S31, or the threshold value d0 stored in the storage unit 2000 in advance may be set. It may be acquired from the storage / reading unit 300 and set. The threshold value of the shortest distance between the moving body 1 and the surrounding structure, which is the surrounding information, is an example of a predetermined value.

Here, the determination unit 290 determines whether or not the operation input reception unit 220 accepts the update of the position / attitude information from the operation unit 201 as the operation reception step (S32). If there is no update (No), step S32 is repeated. When there is an update in step S32 (in the case of Yes), the force sense imparting unit 240 calculates the magnitude of the force sense as the force sense calculation step (step S33).

Here, the calculation of force sense will be described. The force sense imparting unit 240 gives the force sense to the operation based on the shortest distance d from the moving body 1 to the surrounding structure which is the surrounding information received by the transmitting / receiving unit 210 and the predetermined value d0 which is the threshold value of the surrounding information. Calculate the size F. That is, the settings are as follows (Equation 1) and (Equation 2).

When d> d0 F = 0 ... (Equation 1)
When d ≦ d0 F = k (d0−d); k is a constant ... (Equation 2)

As described above, the strength of the force sense is changed according to the distance indicated by the surrounding information. More specifically, when the shortest distance d to the surrounding structure among the distances indicated by the surrounding information becomes a predetermined value d0 or less or less, the strength of the force sense is increased. Here, as an example, d0 is set to the length from the distance measuring sensor 106 to the farthest point of the structure 1 when the distance measuring sensor 106 is provided substantially in the center of the moving body 1. That is, when trying to approach the surrounding structure closer than d0, a reaction force acts on the handle portion 202 of the operation portion 201 in the direction of separation. Therefore, it is possible to intuitively move the structure 1 along the surrounding structure without contacting the surrounding structure.

Returning to the explanation of the flow, the force sense applying unit 240 gives the force sense of the magnitude F calculated in step S33 to the operation unit 201 (S34) as the force sense applying step.

Then, when there is an update in step S32 (in the case of Yes), the maneuvering instruction generation unit 230 generates a maneuvering instruction based on the position / attitude information as a maneuvering instruction generation step (S35), and further, as a transmission / reception instruction transmission step. The generated maneuvering instruction is transmitted to the moving body 1 (S36). The moving body 1 that has received the maneuvering instruction executes an operation according to the maneuvering instruction.

Then, the determination unit 290 determines whether or not there is an instruction to end the main flow from the pilot O (S37). If there is an end instruction (yes), this flow ends. On the other hand, if there is no end instruction (No), the process returns to step S32.

As described above, according to the present embodiment, there is an effect that the operator of the moving body 1 can more accurately grasp and control the surrounding situation of the moving body 1.

That is, for example, even if the operator can grasp the moving direction of the moving body by the control device, he / she cannot maneuver without visually confirming the surrounding information of the moving body. On the other hand, according to the present invention, in a control system that moves a moving body close to a structure by remote control, when the moving body is operated in a place where it is difficult to see directly from the operator, or when the operator operates while visually observing. Even if you do not have the advanced technology for this, you can inspect the moving body by approaching it to the structure or invading or moving it into a narrow space.

In addition, by giving force feedback to the operator's operation input from the information on the position / attitude relationship between the moving body and nearby structures, the operator can grasp the surrounding information of the aircraft in real time. .. Therefore, the operator can easily determine how to maneuver the flying object with respect to the structure to be inspected, and can intuitively maneuver in close proximity to the structure.

As a method of controlling the flying object without visually observing, for example, there are the following (1) to (3).
(1) Using a device that senses the position information of the flying object, such as GPS or a total station, the flight is performed while comparing the flight routes set on the map obtained in advance.
(2) The pilot on the ground operates the aircraft while watching the camera image mounted on the aircraft.
(3) Information according to the speed, attitude, altitude, etc. of the aircraft is transmitted to the operator by force sense or hearing.

Of these, in the case of (1), there is a shield between the GPS satellite or the total station installed on the ground and the flying object, or the structure itself becomes a shield, especially in the vicinity of the structure to be inspected. In many cases, the fundamental problem is that the position information of the air vehicle cannot be obtained.

In the case of (2), there is a system used in drone racing and the like as a drone remote control by FPV (First Person View, first person view) image. This is a system that wirelessly transmits the camera image mounted on the aircraft and displays it in real time on the head-mounted display worn by the pilot, and the pilot flies the set course while watching this image and sets the time. It is a competition. However, in inspection applications where the vehicle moves close to the structure at a relatively low speed, it is difficult to operate while paying attention to the inspection target while paying attention to the surrounding conditions when maneuvering by relying only on the image.

In the case of (3), it is possible to grasp the state of the flying object and its surroundings without taking one's eyes off the flying object, but the transmitted information is not linked with the operation input, and intuitive operation is difficult.

On the other hand, in the present embodiment, a sense of force is given to the operation by the operator, and the operator of the moving body 1 intuitively grasps the surrounding situation of the moving body 1 in conjunction with his / her own operation. It has the effect of being able to steer. By using the distance information with the surrounding structure as the surrounding information, it is possible to prevent, for example, the mechanism for the moving body 1 to come into contact with the surrounding structure and move. In addition, when conducting a survey of surrounding structures, it is possible to facilitate keeping the distance from the surrounding structures constant, and to assist in obtaining appropriate survey results.

In this embodiment, the shortest distance to the surrounding structure is used as the surrounding information, but the present invention is not limited to this. For example, by using ambient temperature information and the concentration of harmful substances as ambient information and giving a force sensation to restrict movement in a direction higher than a predetermined value, heat and harmful substances may adversely affect the moving body 1. Can be prevented.

Next, the second embodiment will be described with reference to FIGS. 11 to 17. The system configuration, hardware configuration, and functional configuration of this embodiment can be the same as those of the first embodiment. Therefore, the system configuration, the hardware configuration, and the functional configuration of the present embodiment will be described in the same manner as those described in FIGS. 1 to 10.

FIG. 11 is an explanatory diagram of an example in which the control device 2 further includes a display device 5. The display device 5 is connected to the control device 2 via the input / output I / F 28. The display device 5 displays, for example, an image taken by the photographing device 105 transmitted from the moving body 1, detection information of various sensors of the moving body 1, information on the operation status, and the like. The operator of the control device 2 can operate the moving body 1 while visually grasping the situation of the remote moving body 1 by looking at the display image displayed on the display device 5. The display device 5 is an example of a display unit.

The display device 5 is a head-mounted display having a fixed portion 51, a direction sensor 52, and a display screen 53. The fixing portion 51 fixes the display device 5 to the head of the operator O. Since the display device 5 is fixed to the head of the operator O, the direction sensor 52 moves together with the head of the operator O and detects the direction in which the head of the operator O moves. Examples of the direction sensor 52 include, but are not limited to, an acceleration sensor and a gyro sensor. Various display images are displayed on the display screen 53. The display screen 53 displays an image so as to face the eyes of the pilot O. In FIG. 11, the display image side of the display screen 53 is enlarged and shown. The display device 5 can change the display position of the display image in conjunction with the detection result of the direction sensor 52. The display device 5 is not limited to the head-mounted display, and may be a desktop display in which the display position is changed by using a mouse or a keyboard.

In the present embodiment, as an example, a three-dimensional model (display model such as a point cloud or mesh) D and a 3D cursor C, which are images showing the three-dimensional shape of the surrounding structure of the moving body 1, are displayed on the display screen 53. An example will be described. The 3D cursor C is displayed at a position synchronized with the position / posture input by the operator O by the handle portion 202 of the operation unit 201. Then, the display position of the three-dimensional model is changed in conjunction with the movement of the head of the pilot O who is equipped with the display device 5 which is a head-mounted display. As a result, the operator O can control the surrounding structure and the moving body 1 while changing the position and angle in the three-dimensional model. The positional relationship between the surrounding structure and the moving body 1 in the three-dimensional model may be calculated or changed based on the operation information of the operation unit 201, or based on the image transmitted from the moving body 1 or the information of various sensors. It may be calculated / changed, or it may be calculated / changed by combining both information.

FIG. 12 is a flowchart showing the shape image display process. This is a flow that starts when a maneuvering mode using a shape image is specified by a user such as pilot O.

First, the shape image generation unit 260 acquires surrounding structure information for generating a three-dimensional model (S41). The surrounding structure information is, for example, an image of the surrounding structure for generating a shape image captured by the image pickup apparatus 105. Then, the shape image generation unit 260 generates a three-dimensional model of the surrounding structure as a shape image generation step (S42). Then, the display control unit 250 displays the generated three-dimensional model as a shape image display step (S43).

Next, the position image generation unit 270 acquires the position / posture information of the moving body 1 based on the input of the operation unit 201 (S44). Then, as a position image position calculation step, the position of the 3D cursor C with respect to the three-dimensional model of the surrounding structure is calculated (S45). Then, the display control unit 250 displays the 3D cursor C, which is a position image, in the three-dimensional model image M as the position image display step (S46).

Here, the determination unit 290 determines whether or not the position / attitude information is updated by the operation unit 201 (S47). When there is an update (in the case of Yes), the process returns to step S44, and the position image generation unit 270 generates and displays the 3D cursor C based on the updated new position / attitude information.

On the other hand, if there is no update in step S47 (No), the determination unit 290 further determines whether or not there is an instruction to end the main flow from the control device 2 (S48). If there is an end instruction (yes), this flow ends. On the other hand, if there is no end instruction (No), the process returns to step S47. In this way, according to this processing flow, the position of the 3D cursor C, which is the position image, is changed as the position / posture information of the moving body 1 is updated by the operation with respect to the operation unit 201.

In this way, the display control unit 250 displays the shape image showing the shape of the surrounding structure, and displays the moving body of the position with respect to the surrounding structure as the position image showing the position with respect to the shape image. Therefore, on the display screen 53, a shape image showing the shape of the surrounding structure of the moving body 1 and a position image showing the position of the moving body with respect to the surrounding structure and the position with respect to the shape image are displayed. Therefore, the operator O can operate the control device 2 while checking the positional relationship between the surrounding structure and the moving body 1 by looking at the display on the display screen 53.

In this flow, the three-dimensional model is generated in steps S41 and S42, but it may be generated in advance and stored in the storage unit 2000. In that case, when the control mode using the shape image is specified by the user such as the operator O and the present processing flow is started, the three-dimensional model may be acquired from the storage unit 2000 and displayed.

Here, an example of a three-dimensional model, which is an example of a shape image, will be described with reference to FIGS. 13 and 14.

First, FIG. 13 shows the appearance of the bridge K as an example of the object for which the three-dimensional model is generated. The bridge K has a pier portion K1 and a main body portion K2 as described above. As shown in FIG. 13, the main body portion K2 has a main girder K21, a floor plate K22, a side surface portion K23, and a road surface K24. An overhanging portion K22a is formed on the outside of the main girder K21 of the floor plate K22.

Of these, the parts of the bridge K for which an investigation and inspection request using the moving body 1 is required are the parts below the road surface K24 through which people and vehicles pass, the lower surface and side surfaces of the main girder K21, and the back surface (overhanging portion) of the floor plate K22. K22a) and the like. Hereinafter, the floor board K22 will be described.

FIG. 14 is a diagram illustrating a display example of a shape image and a position image. FIG. 14 (A) is a photograph of an example of the bridge K, and FIG. 14 (B) is a three-dimensional model M which is a shape image of the bridge shown in FIG. 14 (A). In the three-dimensional model M, the portion corresponding to the main girder K21 is indicated as K21M, and the portion corresponding to the floor plate K22 is indicated as K22M.

As shown in FIG. 14B, a three-dimensional model (three-dimensional mesh model) M of a structure in the vicinity of the machine body constructed by using data such as an image of a camera mounted on the moving body 1 and the above-mentioned The 3D cursor C is displayed. The operating space of the operation unit 201 and the three-dimensional model space have an appropriate coordinate system correspondence including scaling so that the moving body 1 whose movable area of the operation unit 201 is the real space can cover the investigation target range of the bridge K. Is related by.

The operator O visually receives the reaction force received from the bridge K of the mesh model M as the peripheral structure of the moving body 1 from the handle portion 202, and makes the 3D cursor C come into contact with the bridge K of the mesh model M. You can trace the surface while checking. In this way, the pilot O also receives the guide by force so as not to bring the moving body 1 too close to or too far from the bridge K, and visually confirms the inside of the three-dimensional model space. Can be steered and moved.

Here, consider a case where the floor plate K22 in the dotted line region R in the photograph of FIG. 14A is imaged by the image pickup apparatus 105b using FIG. FIG. 15 is a diagram illustrating a display example of a shape image and a position image. 15 (A) is a simplified view of only the vicinity of the region R in the three-dimensional model M shown in FIG. 14 (B) for the sake of explanation. In FIG. 15A, the moving body 1 is located at the shortest distance d from the main girder K21M in FIG. 15A. Further, the region A in FIG. 15A shows the imaging range of the imaging device 105b.

As shown in FIG. 14, in the floor plate K22, the region R is surrounded by the main girder K21 and the girder orthogonal to the main girder K21. The orthogonal digits are shown as image P in FIG. Therefore, as shown in FIG. 15A, the operator O looks at the display image of the display screen 53 while being given a constant force sense by the operation unit 201, and moves the operation unit 201 along the image P. By moving the moving body 1 can be moved along the main girder K21 or the like along the moving locus G1 while maintaining the shortest distance d with respect to the girder. In order to investigate the deformation of the bridge K without omission, there is a request to comprehensively image the surface of the bridge K, but by imaging while moving like the moving locus G1 in this way, FIG. 15 ( The region R1M as a predetermined region represented by the alternate long and short dash line in A) can be comprehensively imaged.

By using the distance information as the surrounding information in this way, it is possible to intuitively move while maintaining a predetermined distance from the surrounding structure. As for the timing of imaging by the image pickup device 105, the control device 2 may transmit a control instruction to the moving body 1, or the image pickup may be performed at a predetermined timing measured by the timer 109 of the moving body 1.

Next, imaging of a region other than R1M in the region R will be described. FIG. 15B shows how the imaging device 105b is imaging the other region R2M indicated by the alternate long and short dash line. In order to comprehensively photograph the inside of the area R, there is a request to image an area other than R1M. In FIG. 15B, the threshold value d0 of the shortest distance is changed to be offset by h in order to comprehensively move the region other than the region R1M.

In this case as well, as in the case of FIG. 15A, the operator O intuitively touches the girder by moving the operation unit 201 along the girder while being given a certain force sense by the operation unit 201. The moving body 1 can be moved while maintaining a certain distance. The difference from the case of FIG. 15A is that in FIG. 15B, the movement locus G2 is offset inward by h as compared with the movement locus G1 of FIG. 15A. is there.

As a result of changing from the movement locus G1 to the movement locus G2, it is possible to comprehensively image another region R2M in the region R different from the predetermined region R1M. By further increasing the predetermined value, it is possible to move a movement locus different from the movement locus G2 and comprehensively image the entire region R.

Next, the input auxiliary image P'displayed in FIG. 15B will be described. The input auxiliary image P'is an image that is not displayed in FIG. 15 (A), and shows an image of digits offset by h with respect to the image P. By presenting the input auxiliary image P'to the operator O, the operator OH can more easily visually grasp the movement along the girder. That is, when the threshold value d0 is offset, the same magnitude of force is given even though the digit and the 3D cursor C are visually separated from each other as compared with the case of FIG. 15A. In that case, the pilot O may feel uncomfortable. By showing the input auxiliary image P', the operator O can visually move the 3D cursor C along the digits with the same feeling as when operating the operation unit 201 in FIG. 15 (A). it can.

A part of the predetermined region R1M and the other region R2M may overlap. When joining the images later, it is preferable that there is some overlap. The offset distance h may be appropriately set in consideration of the photographing distance of the image pickup apparatus 105b, the angle of view, the overlap rate between the predetermined region R1 and the other region R2, and the like.

FIG. 16 is a flowchart showing the input assist processing. FIG. 16 shows, as an example, a flow in which the pilot O and the like move the moving body 1 and select and start the image when comprehensively imaging a certain area as described above.

First, the force sense imparting unit 240 sets a threshold value d0 of the distance between the moving body 1 and the surrounding structure (S51). The pilot O may input in this processing flow, or a value stored in the storage unit 2000 in advance may be used. The threshold value of the shortest distance between the moving body 1 and the surrounding structure, which is the surrounding information, is an example of a predetermined value.

Here, the determination unit 290 determines whether or not the operation input reception unit 220 has received the update reception of the position / attitude information from the operation unit 201 as the operation reception step (S52). If there is no update (No), step S52 is repeated. When there is an update in step S52 (in the case of Yes), the force sensation imparting unit 240 calculates the magnitude of the force sensation (step S53). Here, since the calculation of the force sense is the same as in step S33 of FIG. 10, the description thereof is omitted here.

Next, the force sense imparting unit 240 gives the force sense of the magnitude F calculated in step S53 to the operation unit 201 (S54).

Then, when there is an update in step S52 (in the case of Yes), the maneuvering instruction generation unit 230 generates a maneuvering instruction based on the position / attitude information (S55), and transmits the generated maneuvering instruction to the moving body 1 (S56). ). As described above, the moving body 1 that has received the maneuvering instruction executes the operation according to the maneuvering instruction.

Next, the determination unit 290 determines whether or not there is an auxiliary instruction input for changing the imaging area from the operator O or the like (S57). When instructed in step S57 (in the case of Yes), the force sense applying unit 240 changes the threshold value of the surrounding information as a predetermined value changing step (S58). That is, the threshold value of the shortest distance between the moving body 1 and the surrounding structure is changed from d0 before the change to d0 + h, and the process returns to step S52. By doing so, the pilot O is subsequently provided with haptic feedback from the operating unit 201 to assist in imaging the region shown in R2M of FIG. 16 (B). When the predetermined value before the change is, for example, 50 cm, the predetermined value after the change is larger, for example, 100 cm.

If there is no instruction in step S57 (No), it is determined whether or not there is an instruction to end the flow (S59). If there is an end instruction (yes), this flow ends. On the other hand, if there is no end instruction (No), the process returns to step S52. After step S52, the shortest distance threshold remains d0. In the case of this flow, the threshold value d0 of the shortest distance increases each time the step S58 is passed.

FIG. 17 is a flowchart showing an input auxiliary image display process. As an example, this processing flow is a flow that starts when an input auxiliary image display instruction is given by the operator O or the like.

First, the auxiliary image generation unit 280 acquires the immediately preceding movement locus information, for example, the movement locus G1 in FIG. 15A (S61) as the movement locus acquisition step. Next, the auxiliary image generation unit 280 extracts a reference image from the acquired movement locus information as a reference image extraction step (S62).

Explaining the reference image, it can be assumed that, for example, from the movement locus G1 of FIG. 15 (A), the force sensation feedback by the shortest distance between the moving body 1 and the surrounding structure is given by using the image P as the surrounding structure. Therefore, the image P is extracted as a reference image for generating the input auxiliary image P'. Further, the auxiliary image generation unit 280 acquires the offset value h of the threshold value d0 of the shortest distance from the surrounding image (S63).

Then, the auxiliary image generation unit 280 generates the input auxiliary image P'from the reference image, the movement locus information, and the offset value (S64). Specifically, the input auxiliary image P'is generated by offsetting the image P by h on the side where the movement locus G1 of the image P which is the reference image exists. Then, the display control unit 250 displays the input auxiliary image P'in the 3D model space (S65), and this processing flow ends.

● Supplement ●
The functions of each embodiment can be realized by a computer-executable program described in a legacy programming language such as an assembler, C, C ++, C #, or Java (registered trademark), an object-oriented programming language, or the like, and ROM, EPROM. (Electrically Erasable Read-Only Memory), EPROM (Erasable Program Read-Only Memory), Flash Memory, Flexible Disc, CD-ROM, CD-RW, DVD-ROM, DVD-RAM, DVD-ROM, DVD-ROM It can be stored in a device-readable recording medium such as a card or MO (Magnet-Optical computer), or distributed through a telecommunications line.

In addition, some or all of the functions of each embodiment can be implemented on a programmable device (PD) such as FPGA (Field Programmable Gate Array), or can be implemented as an ASIC, and each embodiment can be implemented. Circuit configuration data (bit stream data) to be downloaded to PD to realize the function on PD, HDL (Hardware Description Language) for generating circuit configuration data, VHDL (Very High Speed Integrated Circuits Technology) -It can be distributed on a recording medium as data described in HDL or the like.

Although the control device, the moving body, the control system, the control method and the program according to one embodiment of the present invention have been described so far, the present invention is not limited to the above-described embodiment, and other embodiments Changes such as additions, changes or deletions can be made within the range that can be conceived by those skilled in the art.

S1 Maneuvering system S2 Diagnostic system S3 Inspection system 1 Mobile body 100 Control device (example of control unit)
110 Transmitter / receiver (example of transmitter)
120 Operation input reception unit 130 Movement control unit 140 Imaging control unit 150 Surrounding information acquisition unit 2 Control device 200 Control device (example of control unit)
210 Transmitter / receiver (example of receiver)
220 Operation input reception unit 230 Maneuvering instruction generation unit 240 Power sense application unit 250 Display control unit 260 Shape image generation unit 270 Position image generation unit 280 Auxiliary image generation unit 5 Display unit

JP-A-2018-116443

Claims (14)

With a moving body
A control device that remotely controls the moving body,
It is a maneuvering system that has
The moving body is
A surrounding information acquisition unit that measures information around the moving body, and
A transmission unit that transmits the surrounding information measured by the surrounding information acquisition unit to the control device, and a transmission unit.
Have,
The control device
A receiver that receives the surrounding information and
An operation unit that accepts operations for maneuvering the moving body,
Based on the received surrounding information, a force sense applying unit that gives a force sense to the operation, and
Maneuvering system with. The maneuvering system according to claim 1, wherein the surrounding information acquisition unit measures the distance from the moving body to the surrounding structure, which is a structure around the moving body, as the surrounding information. The maneuvering system according to claim 2, wherein the force sense applying unit changes the strength of the force sense according to a distance indicated by the surrounding information. The maneuvering system according to claim 3, wherein the force sense applying unit increases the strength of the force sense when the shortest distance to the surrounding structure is less than or equal to a predetermined value among the distances indicated by the surrounding information. .. The moving body has an imaging unit that captures a subject and acquires an captured image.
When the force sense imparting unit images a predetermined region in the surrounding structure by the imaging unit and then images another region of the surrounding structure, the force sense applying unit changes the predetermined value and uses the surrounding information. The maneuvering system according to claim 4, wherein when the shortest distance to the surrounding structure among the indicated distances becomes less than or less than the predetermined value after the change, the strength of the force sense is increased. The maneuvering system
It has a display control unit that displays an image on the display unit.
The display control unit displays a shape image showing the shape of the surrounding structure, and displays the moving body at a position with respect to the surrounding structure as a position image showing the position with respect to the shape image. Any one of the maneuvering systems. The maneuvering system according to claim 6, wherein the shape image is a three-dimensional model image of the surrounding structure, and the position image is a 3D cursor. The display unit is a head-mounted display.
The control system according to claim 6 or 7, wherein the display control unit changes the display of the shape image based on the detection result of the sensor that detects the movement of the head-mounted display. The maneuvering system according to any one of claims 1 to 8, wherein the moving body is a drone, a multicopter, or an unmanned aerial vehicle. The operation unit is a control system according to any one of claims 1 to 9, which has a haptic interface. The maneuvering system according to any one of claims 2 to 10, wherein the peripheral structure is a bridge. A control device that remotely controls a moving body
A receiving unit that receives the surrounding information measured by the surrounding information acquisition unit that measures the information around the moving body, which is acquired by the moving body and transmitted to the control device.
An operation unit that accepts operations for maneuvering the moving body,
A control device having a force sense applying unit that gives a force sense to the operation based on the received surrounding information. It is a control method that controls a moving body remotely.
An operation reception step that accepts an operation for manipulating the moving body, and
A force sense imparting step that gives a force sense to the operation based on the surrounding information measured by the surrounding information acquisition unit that acquires the information around the moving body and transmits it to the control device. And, including maneuvering control methods. A program that causes a computer to perform the method according to claim 13.