JP2018008574A - Display device, search system, display method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a display device, a search system, a display method, and a program which can efficiently notify a user of a relationship between information acquired by an unmanned aircraft and a target detection object.SOLUTION: A display device comprises a first sensor information acquisition unit, a first position information acquisition unit, a second sensor information acquisition unit, a second position information acquisition unit, and an integration display processing unit. The first sensor information acquisition unit acquires first sensor information detected by a first sensor mounted on an unmanned aircraft. The first position information acquisition unit acquires first position information of the unmanned aircraft detected by a first position detection unit mounted on the unmanned aircraft. The second sensor information acquisition unit acquires second sensor information detected by a second sensor mounted on a navigation apparatus. The second position information acquisition unit acquires second position information of the navigation apparatus detected by a second position detection unit mounted on the navigation apparatus. The integration display processing unit integrates and displays the first sensor information and the second sensor information based on the first position information and the second position information.SELECTED DRAWING: Figure 1

Description

  One embodiment of the present invention relates to a display device, a search system, a display method, and a program for displaying acquired information.

  In recent years, for example, in the fishery for catching bonito, it has been performed to search around the own ship using a manned aircraft (for example, a manned helicopter) and to search for a fish school of bonito.

  On the other hand, in recent years, sensors such as cameras are mounted on unmanned aircraft, and information is collected by taking images from the sky. For example, Patent Document 1 discloses a system for photographing a predetermined water area from above using an unmanned aerial vehicle equipped with a thermo camera and observing a water flow in the predetermined water area based on a water surface temperature distribution image obtained by the photographing. Proposed. Patent Document 2 describes a configuration in which video captured by an unmanned aerial vehicle is transmitted / received via satellite communication.

JP2015-127669A Japanese Patent Publication No. 5776731

  When using an unmanned aerial vehicle, it is important to inform the user of the relationship between the information acquired by the unmanned aerial vehicle and the target detection target object (for example, a school of fish).

  One embodiment of the present invention provides a display device, a search system, a display method, and a program capable of efficiently notifying a user of the relationship between information acquired by an unmanned aerial vehicle and a target detection object. With the goal.

  The display device includes a first sensor information acquisition unit, a first position information acquisition unit, a second sensor information acquisition unit, a second position information acquisition unit, and an integrated display processing unit. . The first sensor information acquisition unit acquires first sensor information detected by a first sensor mounted on the unmanned aircraft. The first position information acquisition unit acquires first position information of the unmanned aircraft detected by a first position detection unit mounted on the unmanned aircraft. The second sensor information acquisition unit acquires second sensor information detected by a second sensor mounted on the navigation device. The second position information acquisition unit acquires second position information of the navigation device detected by a second position detection unit mounted on the navigation device. The integrated display processing unit displays the first sensor information and the second sensor information in an integrated manner based on the first position information and the second position information.

  In this way, the display device displays the first sensor information and the second sensor information in an integrated manner, thereby efficiently providing the user with the relationship between the information detected by the unmanned aircraft and the target detection target object. I can inform you. For example, the display device integrally displays the sea surface image captured by the camera as the first sensor on the screen on which the echo image of the bird flock obtained by the radar is displayed as the second sensor. Thereby, the user can see the image of the sea surface at the position of the echo image of the bird swarm, and can confirm whether the fish swarm exists at the position of the echo image of the bird swarm.

  According to one embodiment of the present invention, it is possible to efficiently notify a user of the relationship between information acquired by an unmanned aerial vehicle and a target detection target object.

It is a block diagram which shows the whole structure of a search system. It is a block diagram which shows the structure of a display apparatus. It is a functional block diagram of a control part. It is a functional block diagram of a control part. It is a functional block diagram of a control part. 6A and 6B are diagrams illustrating an example of a screen displayed on the display unit. It is a figure which shows the example which superimposes the information which concerns on an unmanned aircraft on a camera image. It is a flowchart which shows operation | movement of a display apparatus. It is a block diagram which shows the whole structure of a search system. It is a block diagram which shows the whole structure of a search system. It is a figure which shows the communication aspect in the case of operating a some unmanned aircraft. It is a block diagram which shows the whole structure of a search system. FIG. 13A and FIG. 13B are diagrams illustrating an example of a screen displayed on the display unit. It is a functional block diagram of a control part. FIG. 15A and FIG. 15B are diagrams showing position information of each device. FIG. 16A is a diagram illustrating a calculation method of the third position information of the detection target, and FIG. 16B is a diagram illustrating a relationship between the third position information and the unmanned aircraft. It is the figure which showed the example which detects a school of birds with a radar, and image | photographs a school of fish with the camera of an unmanned aircraft. It is the figure which showed the example which displays the image of an unmanned aircraft outside the range of a radar screen.

  FIG. 1 is a block diagram showing the overall configuration of the search system. The search system includes an unmanned aircraft 1, a satellite communication device 3A, a satellite communication device 3B, a satellite ground station 5A, a satellite ground station 5B, the Internet 7, and a ship 2.

  The unmanned aerial vehicle 1 is an aircraft that performs autonomous flight, and is, for example, a multi-rotor helicopter. The unmanned aerial vehicle includes an aircraft having a function of landing and moving on the water. The unmanned aerial vehicle 1 includes a position detection unit 11, a body control unit 12, a camera 13, and a satellite communication unit 14. Each component is connected by a network (LAN) or signal lines of various standards.

  The position detection unit 11 is an example of a first position detection unit, and acquires first position information indicating the position (latitude, longitude, and altitude) of the device itself, for example, by GPS or the like.

  Airframe control unit 12 controls each component of unmanned aerial vehicle 1. Airframe control unit 12 reads an operation program from a memory (not shown) and controls unmanned aerial vehicle 1. For example, the airframe control unit 12 controls and sets the rotor based on the position of the own device acquired by the position detection unit and the information about the route and the target position received via the satellite communication unit 14. Autonomous control is performed to move the device to a target position along the route.

  The camera 13 is an example of a first sensor mounted on the unmanned aircraft 1. The camera 13 photographs the surroundings of the own device. The captured image data is output to the outside via the satellite communication unit 14.

  The unmanned aerial vehicle 1 and the ship 2 transmit and receive various types of information via satellite communication. That is, the unmanned aircraft 1 and the ship 2 transmit / receive various information to / from each other via the satellite communication device 3A, the satellite ground station 5A, the Internet 7, the satellite ground station 5B, and the satellite communication device 3B. However, as shown in FIG. 9 or FIG. 10, the unmanned aircraft 1 and the ship 2 can transmit and receive various types of information via wireless communication in other embodiments. The unmanned aerial vehicle 1 and the ship 2 can also transmit and receive various types of information via both wireless communication and satellite communication.

  The ship 2 is equipped with various nautical equipment. In this example, the ship 2 includes a position detection unit 21, an AIS receiver 22, a display device 23, and a satellite communication unit 24.

  The position detection unit 21 is an example of a second position detection unit, and acquires the position (latitude and longitude) of the ship by, for example, GPS. The second position information acquired by the position detector 21 is input to the display device 23, for example, and displayed as the position of the ship on the chart.

  The display device 23 is a multi-function display that displays information acquired from various devices. FIG. 2 is a block diagram illustrating a configuration of the display device 23. The display device 23 includes an operation unit 210, an input / output interface (I / F) 211, a control unit 212, a display unit 213, and a storage unit 214.

  The control unit 212 reads a program stored in the storage unit 214 and performs various operations. For example, the control unit 212 reads the chart data stored in the storage unit 214 and displays the chart on the display unit 213. Further, the control unit 212 acquires the second position information from the position detection unit 21 via the input / output I / F 211. As described above, the control unit 212 displays the position of the ship on the chart. Thereby, the display device 23 functions as a plotter.

  As shown in FIG. 12, when the ship 2 includes the radar 27, the control unit 212 can acquire the detection result from the radar 27 via the input / output I / F 211 and display the radar screen. . Thereby, the display device 23 also functions as a display unit of the radar device. Furthermore, the control unit 212 can also superimpose and display the detection result of the radar 27 on the chart.

  The control unit 212 performs various operations in accordance with instructions received from the user via the operation unit 210. For example, the control unit 212 accepts an instruction for an autonomous flight flight path (including a target position) of the unmanned aircraft 1. The control unit 212 outputs information related to the accepted flight path for autonomous flight from the input / output I / F 211. Information related to the flight path for autonomous flight is transmitted to the unmanned aircraft 1. The body control unit 12 of the unmanned aerial vehicle 1 performs autonomous control of the unmanned aircraft 1 based on the received information relating to the flight path for autonomous flight. Thereby, the display device 23 also functions as a navigation device that controls the unmanned aircraft 1. However, the navigation device of the unmanned aerial vehicle 1 may be realized by another device (for example, a personal computer).

  Next, FIGS. 3, 4, 5, and 14 are functional block diagrams of the control unit 212. The control unit 212 reads a program stored in the storage unit 214, and implements each configuration illustrated in FIG. 3, FIG. 4, FIG. 5, or FIG.

  As shown in FIGS. 3, 4, 5, and 14, the control unit 212, as one embodiment, includes a first position information acquisition unit 250, a first sensor information acquisition unit 251, and a second sensor information acquisition unit 252. , A second position information acquisition unit 257, and an integrated display processing unit 253. In another embodiment, the control unit 212 further includes an analysis unit 254 as shown in FIGS. 4 and 5. In the embodiment of FIG. 5, the control unit 212 further includes a target position calculation unit 255. In the embodiment of FIG. 14, the control unit 212 includes a position calculation unit 259.

  The first position information acquisition unit 250 acquires first position information of the unmanned aircraft 1. The first sensor information acquisition unit 251 acquires information from the unmanned aerial vehicle 1 via the input / output I / F 211. Here, since the first sensor is the camera 13 as an example, the first sensor information acquisition unit 251 acquires image data captured by the camera 13 as the first sensor information.

  The second sensor information acquisition unit 252 acquires information from various nautical devices. The second sensor is an AIS receiver 22 in this example. The second sensor information acquisition unit 252 acquires AIS information as the second sensor information from the AIS receiver 22. In addition, the second position information acquisition unit 257 acquires second position information. The AIS information includes, for example, identification information of each ship, position, course, moving speed, or target position. The second sensor also corresponds to, for example, an antenna in the radar 27 and an electromagnetic wave transmission / reception unit. Alternatively, a scanning sonar (not shown) corresponds to the second sensor.

  The integrated display processing unit 253 integrates the information acquired by the first sensor information acquisition unit 251 and the information acquired by the second sensor information acquisition unit 252 based on the first position information and the second position information. Then, a process of displaying on the display unit 213 is performed.

  FIG. 6A is a diagram illustrating an example of a screen displayed on the display unit 213. The display unit 213 displays the own ship image 401, the unmanned aircraft image 402, the other ship image 403, and the camera image 404. That is, in the example of FIG. 6A, the display device 23 functions as a plotter.

  FIG. 8 is a flowchart showing the operation of the display device 23. When the integrated screen shown in FIG. 6A is displayed, first, the first position information acquisition unit 250 acquires first position information, and the first sensor information acquisition unit 251 captures image data captured by the camera 13. Is obtained (s11). Moreover, the 2nd sensor information acquisition part 252 acquires various information from a navigation apparatus, and the 2nd position information acquisition part 257 acquires 2nd position information (s12). Here, the second sensor information acquisition unit 252 acquires AIS information.

  Then, the integrated display processing unit 253 determines the positions of the unmanned aircraft 1 and the own ship (s13), and performs integrated display (s14). That is, the integrated display processing unit 253 reads the chart data stored in the storage unit 214 and displays the chart on the display unit 213. Further, the integrated display processing unit 253 displays the own ship image 401 on the chart based on the second position information. In the example of FIG. 6A, the integrated display processing unit 253 displays the ship's own ship on the chart so that the position of the ship's own device is the center and the traveling direction of the ship matches the screen direction (heads up). Display position. 6A, the integrated display processing unit 253 displays the other ship image 403 on the chart based on the AIS information. Further, the integrated display processing unit 253 displays the course and the moving speed of the other ship with, for example, a solid line vector.

  Then, the integrated display processing unit 253 displays the unmanned aircraft image 402 on the chart based on the first position information of the unmanned aircraft 1. In addition, the integrated display processing unit 253 displays the camera image 404 acquired by the first sensor information acquisition unit 251 in the vicinity of the unmanned aircraft image 402. However, the camera image 404 need not always be displayed. For example, when the user operates the operation unit 210 to perform an operation of selecting the unmanned aircraft image 402, the integrated display processing unit 253 displays the camera image 404 in the vicinity of the unmanned aircraft image 402.

  The camera image 404 may display the image data captured by the camera 13 as it is, but for example, as shown in FIG. 7, it is possible to superimpose information related to the unmanned aircraft 1. In this case, the integrated display processing unit 253 superimposes the first position information of the unmanned aircraft 1 and text information indicating the timing (time) at which the first position information is received on the image data captured by the camera. Displayed as an image 404.

  In addition, the integrated display process part 253 displays the advancing direction and moving speed by a solid line vector, when the information which shows the advancing direction and moving speed of the unmanned aircraft 1 is received as 1st sensor information further. The traveling direction and the moving speed can be estimated by the display device 23 based on the position change of the unmanned aircraft 1.

  Moreover, the integrated display process part 253 displays the imaging direction of the camera 13 with a broken line, when the information which shows the imaging direction of the camera 13 is received.

  Furthermore, in this example, the unmanned aerial vehicle image 402 showing the current position of the unmanned aerial vehicle 1 is displayed. However, the integrated display processing unit 253 displays a wake image based on the past position information of the unmanned aircraft 1. May be.

  As described above, the display device 23 integrally displays the information obtained by the first sensor mounted on the unmanned aircraft 1 and the information obtained by the second sensor mounted on the ship's equipment. For example, when the user uses a school of fish as the detection target, the user knows the sea surface image (bird group image) obtained by the camera 13 of the unmanned aircraft 1, the position of the ship, and the position of the unmanned aircraft 1. Therefore, it is possible to efficiently grasp the positional relationship between the school of fish that is the object to be detected and the image captured by the unmanned aerial vehicle 1.

  FIG. 6B is a diagram illustrating a second example of a screen displayed on the display unit 213. The display unit 213 displays a ship image 401, an unmanned aircraft image 402, an echo image 503 of another ship, a camera image 404, an echo image 504 on land, and an echo image 505 of a bird group. In the example of FIG. 6B, the display device 23 functions as a display unit for radar.

  In this case, the ship 2 includes a radar 27 as in the example shown in FIG. Based on the detection result of the radar 27, the integrated display processing unit 253 displays a radar screen centered on the position of the own apparatus as shown in FIG. Then, the integrated display processing unit 253 displays the unmanned aircraft image 402 on the radar screen based on the first position information of the unmanned aircraft 1.

  In this case, as shown in FIG. 15A, the integrated display processing unit 253 first displays a screen of an orthogonal coordinate system with the position of the own ship as the origin, and displays the first position information (absolute position) of the own ship. On the other hand, the position of each pixel in the screen is determined. Then, the integrated display processing unit 253 displays the unmanned aircraft image 402 in an integrated manner at the pixel position corresponding to the second position information (absolute position).

  Thereby, the user can check the position of the unmanned aircraft and the photographed sea surface image on the screen on which the echo image 505 of the bird flock is displayed. Therefore, the user can confirm the actual sea surface condition (such as the size of the bird swarm) at the position of the echo image 505 of the bird swarm. Can be confirmed. In the example of FIG. 6B, the integrated display processing unit 253 displays the first position of the unmanned aircraft 1 in the radar screen (the area where the relative positional relationship between the navigation equipment and the second sensor information is illustrated). An example is shown in which a camera image 404 is displayed in an area corresponding to information. However, the integrated display processing unit 253 may display the camera image 404 outside the radar screen as shown in FIG.

  Note that the integrated display processing unit 253 can also integrally display the chart shown in FIG. 6A and the radar screen shown in FIG.

  When the ship 2 includes the radar 27, the control unit 212 includes a position calculation unit 259 as illustrated in FIG. 14, thereby calculating the position of the detection target (third position information). be able to.

  As illustrated in FIG. 16A, the position calculation unit 259 detects bird group echo data (bird group echo image 505) from the echo data detected by the radar 27. Then, the position calculation unit 259 calculates the direction and distance of the birds from the first position information (absolute position) of the ship based on the echo data. Finally, the position calculation unit 259 calculates the position of the bird group (third position information) based on the calculated direction and distance of the bird group.

  The third position information is used for setting the target position of the unmanned aerial vehicle 1 or setting the shooting direction of the camera 13 as shown in FIG. For example, the user sets the target position of the unmanned aerial vehicle 1 to the third position information via the operation unit 210 or instructs the camera 13 to point the shooting direction toward the third position information. In this case, the camera 13 of the unmanned aerial vehicle 1 acquires an image (bird group image) captured at a position corresponding to the third position information.

  The image captured by the camera 13 may be a bird group image. For example, as shown in FIG. 17, the radar 27 detects the echo data of the bird group as the second detection target, and the camera 13 The aspect which image | photographs the image of the fish school image | photographed in the water surface direction as a 1st detection target object may be sufficient.

  Next, processing of the analysis unit 254 illustrated in FIG. 4 will be described. The analysis unit 254 analyzes various information acquired from the unmanned aircraft 1. The analysis unit 254 performs image analysis of image data captured by the camera 13, for example. For example, the analysis unit 254 extracts a bird image from the image data using template matching or the like. And the analysis part 254 estimates the distance with a bird group from the magnitude | size of a bird image. The analysis unit 254 estimates third position information indicating the position of the bird group based on the estimated distance, the shooting direction of the camera 13, and the first position information.

  The integrated display processing unit 253 displays bird group information based on the third position information estimated by the analysis unit 254. For example, the analysis unit 254 displays a bird group image 701 indicating the third position of the bird group as illustrated in FIG. In addition, when the unmanned aircraft 1 includes a radar as the first sensor, the analysis unit 254 can also estimate the third position information based on the radar detection result.

  Further, the analysis unit 254 may perform processing for counting the number of extracted bird images and estimating the size of the bird group. Information relating to the estimated size of the bird flock is input to the integrated display processing unit 253 and is integratedly displayed on the display unit 213. For example, as shown in FIG. 13B, the integrated display processing unit 253 changes the size of the bird group image 701 according to the estimated size of the bird group, so that information related to the size of the bird group is obtained. It may be expressed.

  The analysis unit 254 can also estimate the moving direction or moving speed of the bird group based on the information acquired from the unmanned aircraft 1. For example, when the unmanned aerial vehicle 1 is stationary, the analysis unit 254 detects the coordinates of the birds in the image data, and calculates changes in the coordinates of the birds or the size of the birds at a plurality of timings. Thus, the moving direction or moving speed can be estimated. Further, when the unmanned aerial vehicle 1 is moving, the moving direction or moving speed of the bird group is obtained by adding the moving amount and moving direction of the unmanned aircraft 1 to the change in the coordinates of the bird group in the image data. Can be calculated. For example, as shown in FIG. 13A or 13B, the integrated display processing unit 253 displays the moving direction or moving speed estimated by the analyzing unit 254 as a solid line vector.

  In the example of FIGS. 13A and 13B, the image 701 of the school of birds that is the second detection target is displayed, but the image of the school of fish that is the first detection target is displayed. Also good.

  The analysis unit 254 can also extract the sea surface image and estimate the water temperature based on the color of the extracted sea surface image. Information related to the estimated water temperature is input to the integrated display processing unit 253 and is integratedly displayed on the display unit 213.

  Next, processing of the target position calculation unit 255 shown in FIG. 5 will be described. The target position calculation unit 255 sets a flight path for autonomous flight of the unmanned aircraft 1. In the above example, the flight path for autonomous flight including the target position of the unmanned aircraft 1 is determined only in accordance with an instruction from the user, but in the example of FIG. 5, the information from the first sensor or the second Based on information from the sensor, the target position calculation unit 255 calculates the target position of the unmanned aircraft 1. The calculated target position of the unmanned aerial vehicle 1 is output to the unmanned aerial vehicle 1 via the input / output I / F 211.

  For example, in the example of FIG. 6A, when the user selects the camera image 404 using the operation unit 210 and instructs to track the bird group of the camera image 404, the target position calculation unit 255 Then, a position corresponding to the bird group in the camera image 404 or a position away from the bird group by a predetermined distance is calculated as the target position. However, in this case, the analysis unit 254 recognizes an image of the detection target object (for example, a bird group) from the camera image.

  The target position calculation unit 255 can cause the unmanned aircraft 1 to track the detection target corresponding to the instructed image by repeating such calculation and output of the target position. It is also possible to set a flight path for autonomous flight.

  Further, for example, in the example of FIG. 6B, when the user selects the echo image 505 of the bird flock using the operation unit 210 and instructs to track the echo image 505, the target position calculation unit 255 sets the position (latitude and longitude) corresponding to the echo image 505 as the target position.

  Next, FIG. 9 and FIG. 10 are block diagrams showing the overall configuration of the search system. As shown in FIG. 9, the search system may be an embodiment in which the unmanned aircraft 1 further includes a wireless communication unit 19 and the ship 2 further includes a wireless communication unit 29. As shown in FIG. 9, the search system is an embodiment in which the unmanned aircraft 1 includes a wireless communication unit 19 instead of the satellite communication unit 14, and the ship 2 includes a wireless communication unit 29 instead of the satellite communication unit 24. Is also possible.

  When the unmanned aircraft 1 further includes the wireless communication unit 19 and the ship 2 further includes the wireless communication unit 29, the unmanned aircraft 1 and the ship 2 can communicate with each other by the wireless communication unit 19 and the wireless communication unit 29. In addition, various information is transmitted and received through the wireless communication unit 19 and the wireless communication unit 29. The unmanned aircraft 1 and the ship 2 transmit and receive various types of information via the satellite communication unit 14 and the satellite communication unit 24 when the wireless communication unit 19 and the wireless communication unit 29 cannot communicate.

  As a result, the search system performs communication without using satellite communication, which is relatively expensive, but uses satellite communication to prevent an increase in cost when communication by the wireless communication unit is not possible. Can be operated flexibly.

  Further, when the unmanned aircraft 1 is an embodiment in which the wireless communication unit 19 is provided instead of the satellite communication unit 14 and the ship 2 is provided with the wireless communication unit 29 instead of the satellite communication unit 24, a satellite having a relatively high cost. Communication is not used. Moreover, in this embodiment, since the unmanned aircraft does not include the satellite communication unit, the unmanned aircraft can be reduced in weight, and the moving speed can be improved and the cruising distance can be increased.

  In this embodiment, as shown in FIG. 11, the search system preferably includes a plurality of unmanned aircraft (in FIG. 11, unmanned aircraft 1A, unmanned aircraft 1B, and unmanned aircraft 1C). Each unmanned aircraft performs wireless communication with each other and functions as a relay device. Thereby, the search system constructs an ad hoc network system. In this case, the first sensor information acquisition unit 251 acquires information on the first sensor from any one unmanned aircraft 1 via the ad hoc network system.

  In this case, the unmanned aerial vehicle that captures and recognizes the image related to the detection target object tracks the detection target object, and the other unmanned aircraft functions as a relay device. Therefore, in the example of FIG. 11, even if the configuration of the satellite communication unit is not provided, the first unmanned aircraft 1 at a distant position exceeds the area where the ship 2 and one unmanned aircraft 1 can communicate. Various information acquired by one sensor is transmitted to the ship 2.

(Modification 1)
As a first modification, the first sensor mounted on the unmanned aircraft 1 may be a thermo camera. In this case, information regarding the water temperature is acquired as information acquired by the first sensor. Further, for example, a heat source of another ship can be detected. For example, when the object to be detected is a suspicious ship, information on the heat source acquired by the unmanned aerial vehicle 1 becomes important information for tracking the suspicious ship. Therefore, in the ship 2, the user can confirm the suspicious ship using the thermo camera as the first sensor while grasping the position of the suspicious ship using the second sensor such as a radar.

(Modification 2)
As a second modification, the first sensor mounted on the unmanned aerial vehicle 1 may be an AIS receiver. In this case, the unmanned aerial vehicle 1 can receive AIS information of other ships. Therefore, the AIS information of other distant ships that cannot be received by the ship 2 can be acquired by the unmanned aircraft 1 and the AIS information can be confirmed by the ship 2.

(Modification 3)
As a third modification, the first sensor mounted on the unmanned aircraft 1 may be an aviation laser detector. The aviation laser detector irradiates a near infrared laser and a green laser at the same time, A device for observing echoes from the seabed, or from a school of fish. Thereby, the unmanned aerial vehicle 1 functions as a fish finder for detecting fish in the water from the air.

(Modification 4)
Modification 4 is a mode in which the unmanned aerial vehicle 1 lands. In this case, the unmanned aircraft 1 includes, as the first sensor, a thermometer, an underwater camera, a fish finder, or a sonar (an ultrasonic measuring device that detects other than directly under the ship). In this case, the unmanned aerial vehicle 1 can directly observe underwater. When the unmanned aerial vehicle 1 includes a fish finder or sonar, the analysis unit 254 estimates the position information (third position information) of the fish school based on the detection result of the fish finder or sonar. Can do. In this case, the integrated display processing unit 253 displays an image (fish group image) indicating the position of the fish school on the display unit 213.

  In the present embodiment, an example in which the navigation device and the display device are mounted on a ship is shown. However, the nautical equipment is not limited to those mounted on the ship. For example, nautical equipment may be installed in ground facilities. For example, in a ground radar base station, information acquired by an unmanned aerial vehicle may be integrated and displayed on a radar screen. That is, the display device only needs to acquire various types of information from the navigation equipment installed in the ground facilities and display the information acquired by the unmanned aircraft in an integrated manner.

  In the present embodiment, the display device 23 is an example of a multi-function display. For example, in a radar device including an antenna, a signal processing function, and a display device, an echo is displayed on the display device of the radar device. Information obtained from the first sensor may be integrated and displayed on an echo image obtained from the data. In this case, an apparatus comprising a configuration for acquiring various types of information from an unmanned aerial vehicle, a configuration for acquiring information such as an echo image, and a processing unit that performs a process of displaying an image on a display of the radar device is provided by the present invention. This corresponds to the display device.

  The description of this embodiment is illustrative in all respects and not restrictive. The scope of the present invention is shown not by the above embodiments but by the claims. Furthermore, the scope of the present invention is intended to include all modifications within the meaning and scope equivalent to the scope of the claims.

  Search systems can be used in various industries.

  For example, when a fire occurs on another ship at sea, the display device displays the approximate position of the fire ship as information acquired by a second sensor (for example, radar) in the ship or a ground base station. The user grasps the approximate position of the fire boat and dispatches a fire boat. At this time, the search system sets the route of the unmanned aircraft 1 based on the information (fire ship position) obtained by the second sensor. Alternatively, the user sets the route of the unmanned aircraft 1. The unmanned aerial vehicle 1 acquires images of a fire ship and a fireboat using a camera. The display device integrates and displays the information acquired by the radar in the ship or the ground base station and the image acquired by the unmanned aircraft 1. Thereby, for example, it is possible to display whether or not the fireboat is appropriately discharging water to the fireboat.

  Further, for example, when heavy rain occurs, the display device displays the area where the heavy rain occurs by displaying information acquired by the second sensor (for example, weather radar) in the disaster prevention center. The user specifies the heavy rain occurrence area and sets the route of the unmanned aircraft 1. The unmanned aerial vehicle 1 uses the first sensor to acquire various types of information (for example, an image showing a damage situation, etc.) of a heavy rain occurrence location. The display device displays the information acquired by the weather radar in the disaster prevention center and the image of the heavy rain occurrence location acquired by the unmanned aircraft 1 in an integrated manner. Thereby, the user can grasp the damage situation of the heavy rain occurrence place and can consider disaster countermeasures.

DESCRIPTION OF SYMBOLS 1, 1A, 1B, 1C ... Unmanned aerial vehicle 2 ... Ship 3A, 3B ... Satellite communication apparatus 5A, 5B ... Satellite ground station 7 ... Internet 11 ... Position detection part 12 ... Airframe control part 13 ... Camera 14 ... Satellite communication part 19 ... Wireless communication section 21 ... Position detection section 22 ... AIS receiver 23 ... Display device 24 ... Satellite communication section 27 ... Radar 29 ... Wireless communication section 210 ... Operation section 211 ... Input / output I / F
212 ... Control unit 213 ... Display unit 214 ... Storage unit 251 ... First information acquisition unit 252 ... Second sensor information acquisition unit 253 ... Integrated display processing unit 254 ... Analysis unit 255 ... Target position calculation unit 401 ... Own ship image 402 ... Unmanned aircraft image 403 ... Other ship image 404 ... Camera images 503, 504, 505 ... Echo images

Claims (15)

A first sensor information acquisition unit for acquiring first sensor information detected by a first sensor mounted on an unmanned aerial vehicle;
A first position information acquisition unit for acquiring first position information of the unmanned aircraft detected by a first position detection unit mounted on the unmanned aircraft;
A second sensor information acquisition unit for acquiring second sensor information detected by a second sensor mounted on the navigation device;
A second position information acquisition unit for acquiring second position information of the navigation device detected by a second position detection unit mounted on the navigation device;
An integrated display processing unit that displays the first sensor information and the second sensor information in an integrated manner based on the first position information and the second position information;
A display device comprising: The display device according to claim 1,
The second detection object is extracted from the second sensor information, the azimuth and distance of the second detection object are calculated from the second position information, and the second position information and the azimuth of the second detection object are calculated. And a position calculating unit that calculates third position information of the second detection target based on the distance,
The first sensor information acquisition unit acquires first sensor information of a first detection target detected at a position corresponding to the third position information.
Display device. The display device according to claim 1 or 2,
The integrated display processing unit displays the first sensor information in a region corresponding to the first position information in a region where a relative positional relationship between the navigation device and the second sensor information is illustrated.
Display device. The display device according to claim 1 or 2,
The integrated display processing unit displays the first sensor information outside a region in which a relative positional relationship between the navigation device and the second sensor information is illustrated;
Display device. The display device according to any one of claims 1 to 4,
The first sensor information acquisition unit and the first position information acquisition unit respectively acquire the first sensor information and the first position information via satellite communication.
Display device. The display device according to any one of claims 1 to 4,
The first sensor information acquisition unit and the first location information acquisition unit respectively acquire the first sensor information and the first location information via an ad hoc network system.
Display device. The display device according to any one of claims 1 to 6,
The second sensor is a radar;
The second sensor information includes echo data of the radar.
Display device. The display device according to claim 7,
Based on echo data acquired by the radar, further comprising a target position calculation unit for calculating a target position of the unmanned aircraft,
Display device. A display device according to any one of claims 1 to 8,
The first sensor is a camera;
The first sensor information includes an image taken by the camera.
Display device. The display device according to claim 2,
The first sensor is a camera;
The first sensor information includes an image taken by the camera,
The first detection object is photographed in the water surface direction by the camera,
The second sensor is a radar;
The second sensor information includes echo data of the radar,
The second detection object is detected by echo data of the radar.
Display device. The display device according to claim 9 or 10,
By analyzing the image, the analysis unit for estimating the size of the detection target object,
The integrated display processing unit further displays information on the scale of the detection target;
Display device. The display device according to claim 9 or 10,
By analyzing the image, the analysis unit for estimating the moving speed of the detection target object,
The integrated display processing unit further displays information on a moving speed of the detection object;
Display device. A display device according to any one of claims 1 to 12,
The unmanned aerial vehicle;
A navigation device for controlling the unmanned aerial vehicle;
Search system with A first sensor information acquisition process for acquiring first sensor information detected by a first sensor mounted on an unmanned aerial vehicle;
A first position information acquisition process for acquiring first position information of the unmanned aerial vehicle detected by a first position detection unit mounted on the unmanned aircraft;
A second sensor information acquisition process for acquiring second sensor information detected by a second sensor mounted on the navigation device;
A second position information acquisition process for acquiring second position information of the navigation device detected by a second position detection unit mounted on the navigation device;
An integrated display process for integrating and displaying the first sensor information and the second sensor information based on the first position information and the second position information;
Display method to do. On the computer,
A first sensor information acquisition process for acquiring first sensor information detected by a first sensor mounted on an unmanned aerial vehicle;
A first position information acquisition process for acquiring first position information of the unmanned aerial vehicle detected by a first position detection unit mounted on the unmanned aircraft;
A second sensor information acquisition process for acquiring second sensor information detected by a second sensor mounted on the navigation device;
A second position information acquisition process for acquiring second position information of the navigation device detected by a second position detection unit mounted on the navigation device;
An integrated display process for integrating and displaying the first sensor information and the second sensor information based on the first position information and the second position information;
A program that executes

Images