TWM583606U - Marine mammal tracking system and carrier thereof - Google Patents

Abstract

A marine mammal tracking system includes an optical capture array, an acoustic capture array, a memory and a processor. The optical capture array is configured to capture images in an optical detection area around a carrier. The acoustic capture array is configured to capture sounds in an acoustic detection area around the carrier. The memory stores with instructions. The processor is configured to access the instructions to perform following steps: receiving the images and sounds; determining presences of visual characteristics of a marine mammal from the images or presences of audible characteristics of the marine mammal from the sounds; recording the visual characteristics and the audible characteristics of the marine mammal; determine a direction orienting to the marine mammal; guiding the carrier to the direction; and re-determining the direction according to specific time periods in order to track the marine mammal.

Description

Marine mammal tracking system and vehicle

This case relates to a tracking system and a vehicle, particularly a tracking system and a vehicle applied to marine mammals.

In recent years, countries around the world have paid increasing attention to marine ecological conservation. Among them, the number of marine mammals (for example, whales and dolphins) is a representative indicator of the richness of the marine food chain.

However, with the needs of human development, many economic facilities must also be installed in the ocean, such as oil drilling platforms, offshore wind turbines, and so on. However, when these economic facilities are installed, seismic survey operations and piling operations may be required in the sea area. The noise generated by these operations will affect or harm marine mammals in the sea area.

In order to establish a balance in marine ecology and economic development, many advanced countries have established marine mammal observer systems (Marine Mammal Observer, MMO). When working in the sea, marine mammal observers can observe the appearance of marine mammals (such as cetaceans) in the sea with the naked eye or sonar device. If they are found in the sea, they can request the operation to be suspended and try to record Track whale dolphin for continuous assessment of feasibility in the area industry.

However, with the rapid development of the marine economy, the number of marine mammal observers cannot meet the demand. Moreover, since marine mammal observers observe marine mammals with the naked eye, or read sonar devices through experience to confirm marine mammals, they will be prone to omissions. In addition, marine mammal observers cannot observe marine mammals with the naked eye at night. Obviously, the current system of marine mammal observers has many difficulties.

An implementation aspect of the present case relates to a marine mammal tracking system. The marine mammal tracking system includes an optical acquisition array, an acoustic acquisition array, a memory, and a processor. The processor is coupled to the memory, the acoustic capture array and the image capture array. The optical capture array is used to capture an image around a vehicle in an optical detection range. The acoustic capture array is used to capture a sound around the vehicle in an acoustic detection range. The memory stores at least one instruction. The processor is used for accessing the at least one instruction to: receive the image and the sound; determine whether the image has at least one visual feature of the marine mammal or whether the sound has at least one acoustic feature of the marine mammal; if the image Confirming at least one visual feature of the marine mammal or confirming the at least one acoustic feature of the marine mammal in the sound, recording the at least one visual feature of the marine mammal and the at least one sound feature; judging based on the image or the sound A direction of the marine mammal; guiding the vehicle to travel in that direction; confirming at least one visual feature of the marine mammal in the image or the at least one acoustic feature of the marine mammal in the sound according to a time interval again Judge this direction twice to track the marine mammal.

Another aspect of the present invention relates to a marine mammal tracking vehicle. The marine mammal tracking vehicle includes a carrier, a power unit, an optical acquisition array, an acoustic acquisition array, a memory, and a processor. The processor is coupled to the power unit, the memory, the acoustic capture array and the image capture array. The power device is used to provide power to the load. The optical capture array is used to capture an image of the specific surroundings of the payload in an optical detection range. The acoustic capture array is used to capture a sound around the carrier in an acoustic detection range. The memory stores at least one instruction. The processor is used to access the at least one instruction to perform the following steps: receiving the image and the sound; determining whether the image has at least one visual feature of the marine mammal or whether the sound has at least one acoustic feature of the marine mammal; if Confirming at least one visual feature of the marine mammal in the image or confirming at least one acoustic feature of the marine mammal in the sound, recording the at least one visual feature of the marine mammal and the at least one acoustic feature; based on the image or The sound judges a direction of the marine mammal; controls the power unit to make the load move in that direction; confirms at least one visual feature of the marine mammal in the image or the marine mammal in the sound according to a time interval The at least one sound feature is used to determine the direction again to track the marine mammal.

In one embodiment, the processor controls the power unit according to a safety distance to make the load travel in the direction.

In one embodiment, the optical capture array has a plurality of optical capture devices with multiple orientations, so that the optical detection range covers the specific surroundings of the carrier omnidirectionally.

In one embodiment, the optical capture devices in the optical capture array have a night vision mode.

In one embodiment, the at least one visual feature of the marine mammal includes at least one of a dorsal fin shape and a tail fin shape of the marine mammal.

Therefore, according to the technical content of this case, the embodiment of this case can be operated in a sea area by providing a marine mammal tracking system and a marine mammal tracking vehicle to automatically identify, record and track the sea area during the day and night. Marine mammals can avoid the omissions caused by human operations, thereby improving the efficiency of marine mammal ecological observation.

100‧‧‧Marine mammal tracking system

110‧‧‧ Optical Capture Array

120‧‧‧ Acoustic Acquisition Array

130‧‧‧Memory

140‧‧‧Processor

200‧‧‧Marine mammal tracking vehicle

210‧‧‧ contains specific

220‧‧‧ Power Unit

230‧‧‧ Optical Capture Array

240‧‧‧ Acoustic Acquisition Array

250‧‧‧Memory

260‧‧‧Processor

300‧‧‧ Marine mammal tracking method

S310 ~ S370‧‧‧step flow

100A‧‧‧ Big Data Monitoring Center

100B‧‧‧Unmanned Ship Monitoring Center

D1‧‧‧ Information

Figure 1 is a schematic diagram of a marine mammal tracking system based on an embodiment of the present case; Figure 2 is a schematic diagram of a marine mammal tracking vehicle based on an embodiment of the present case; and Figure 3 is an embodiment based on the present case The schematic diagram of the marine mammal tracking method shown in FIG. 4A is a schematic diagram of the visual characteristics of marine mammals based on an embodiment of the present invention; and FIG. 4B is the visual diagram of marine mammals based on an embodiment of the present invention Schematic illustration of features.

The following will clearly illustrate the spirit of this case with diagrams and detailed descriptions. Any person with ordinary knowledge in the technical field who understands the embodiments of this case can be changed and modified by the techniques taught in this case without departing from the scope of this case. Spirit and scope.

The terminology used herein is for describing particular embodiments only and is not intended as a limitation of the present case. Singular forms such as "a", "the", "the", "the" and "the", as used herein, also include the plural.

Regarding the "first", "second", ..., etc. used in this document, they do not specifically refer to the order or order, nor are they used to limit the case. They are only used to distinguish elements or operations described in the same technical terms. .

As used herein, "coupled" or "connected" can refer to two or more components or devices directly making physical contact with each other, or indirectly making physical contact with each other, or two or more components or devices to each other. Operation or action.

The terms "including", "including", "having", "containing" and the like used in this article are all open-ended terms, which means including but not limited to.

As used herein, "and / or" includes any and all combinations of the things described.

Regarding the directional terms used in this article, such as: up, down, left, right, front or back, etc., are only directions referring to the attached drawings. Therefore, the directional terms used are used to illustrate and not to limit the case.

Regarding the terms used herein, unless otherwise specified, each term usually has the ordinary meaning of being used in this field, in the content of this case, and in special content. Certain terms used to describe this case will It is discussed elsewhere in this specification to provide additional guidance to those skilled in the art on the description of this case.

FIG. 1 is a schematic diagram of a marine mammal tracking system based on an embodiment of the present invention. As shown in FIG. 1, in this embodiment, a marine mammal tracking system 100 may include an optical acquisition array 110, an acoustic acquisition array 120, a memory 130, and a processor 140. The processor 140 and the optical acquisition array 110. Acoustic capture array 120 and memory 130 are communicatively coupled or electrically coupled. In some embodiments, the marine mammal tracking system 100 is installed on a ship. In some embodiments, the processor 140 of the marine mammal tracking system may include but is not limited to the integration of a single processing unit or a plurality of microprocessors, and the integration of the single processing unit or the microprocessor may be electrically coupled to the memory 130, The memory 130 may be an internal or external memory, including a volatile or non-volatile memory.

In some embodiments, the processor 140 may access the at least one instruction from the memory 130 and execute the at least one instruction to further implement an application program defined by the at least one instruction (for example, the method 300 of FIG. 3 described later). ). To facilitate understanding, the procedures defined by the at least one instruction will be detailed in the following paragraphs. In some embodiments, the processor 140 may be implemented by an application-specific integrated circuit. The foregoing implementation of the processor 140 is used as an example, and various hardware components such as circuits or modules that can implement the processor 140 are within the scope of the present application.

In various embodiments, the processor 140 is configured to execute an artificial intelligence algorithm to operate in cooperation with the above application program. In some embodiments, the memory 130 stores a preset data, which corresponds to a virtual model of an artificial intelligence algorithm (for example, it may be a deep learning model, but this case does not use this as Limit), the virtual model has one or more weights or parameters associated with artificial intelligence algorithms. By executing the above-mentioned application program, the processor 140 can train and modify the one or more weights or parameters according to the information collected by the optical capture array 110 and / or the acoustic capture array 120 to improve the artificial intelligence algorithm. Accuracy. As such, in these embodiments, the processor 140 may read the virtual model from the memory 130 to execute artificial intelligence algorithms to efficiently track marine mammals.

In some embodiments, in addition to storing the at least one instruction, the memory 130 of the marine mammal tracking system may further store (or temporarily store) data required by the processor 140 to execute the application program, and store (or temporarily store) the processing. Data generated by the processor 140 after executing the application program.

For example, in some embodiments, the marine mammal tracking system 100 may be further connected to at least one server of a large data monitoring center 100A and / or an unmanned ship monitoring center 100B. When performing the method 300 mentioned later, the marine mammal tracking system 100 may return the collected related data D1 (such as the visual characteristics, sound characteristics, and / or the direction of the guidance vehicle, etc. mentioned later) to the Big data monitoring center 100A and / or unmanned ship monitoring center 100B. As such, in some embodiments, the big data monitoring center 100A may store relevant data D1 to modify or train the aforementioned virtual model of the artificial intelligence algorithm according to these data to further improve the accuracy of automatic navigation. Furthermore, in some embodiments, the unmanned ship monitoring center 100B may provide information about navigation directions and the like in related data D1 to an navigator with driving related experience through an output interface to determine whether the navigation direction is correct, thereby improving Precision of automatic navigation.

In some embodiments, the optical capture array 110 in the marine mammal tracking system 100 includes a plurality of optical capture devices (not shown). The optical capture device can correspond to multiple orientations of the ship. The optical capture devices can be general digital cameras, high-definition cameras, etc., or low-light cameras, infrared cameras, laser cameras, etc. with night vision mode. The optical capturing devices each have a viewing angle, and the optical capturing devices can be disposed on the ship in a distributed or centralized manner. By integrating the viewing angles of the optical capturing devices, the optical capturing array 110 can have The optical detection range of the angle around the ship is generally omnidirectional, and the optical capture array 110 can obtain images in the optical detection range. For example, in some embodiments, the optical capture devices may be dispersedly disposed at the bow, the ship's side, and the stern of the ship, so that the optical capture array 110 can capture the image of the surroundings of the ship as a whole. In some embodiments, the optical capture devices can be centrally and circumferentially disposed at a relatively high point on the ship, so that the optical capture array 110 can capture the image of the surroundings of the ship as a whole. It should be understood that, in some embodiments, the viewing angles and focal lengths of the optical capture devices in the optical capture array 110 are adjustable, and the processor 140 may control the optical capture devices to rotate or move to make the optical The capture array 110 can still capture the entire image of the ship's surroundings.

In some embodiments, the acoustic capture array 120 in the marine mammal tracking system 100 includes a plurality of sonar devices (not shown), and the sonar devices may respectively correspond to multiple orientations of the ship. The sonar devices can be arranged on the ship in a decentralized or centralized manner, and can emit sound waves of different frequencies and receive corresponding echoes into the water body of the sea area where the ship travels according to specific intervals, and can also receive other sounds in the water body. The acoustic capture array 120 may have an acoustic detection range covering a water body around the ship. In some embodiments, the sonar devices are centrally disposed on the bow or stern of the ship, so that the optical acquisition array 110 can capture the sound around the ship as a whole.

As mentioned above, in some embodiments, the marine mammal tracking system 100 is installed on the ship, and the ship may be a large ship such as a cruise ship or a ferry, or a small ship such as a fishing boat or a speedboat, or an unmanned observation ship or unmanned patrol. Unmanned ships such as boats. In some embodiments, the optical acquisition array 110 and the acoustic acquisition array 120 in the marine mammal tracking system 100 are disposed on the ship, and the memory 130 and the processor 140 are not disposed on the ship. The processor 140 is communicatively coupled to the electronic device on the ship, and can perform unidirectional or bidirectional information exchange with the optical capture array 110 and the acoustic capture array 120 on the ship. Therefore, in these embodiments, the marine mammal tracking system 100 can be used to calculate and guide the ship to travel in a sea area, and then track the marine mammal in the sea area.

It should be understood that, in some embodiments, marine mammals (Marine Mammals) refer to mammals that live in the sea for a long time or need to depend on resources in the ocean, such as sea otters, polar bears, manatees, seals, sea lions, walruses, whales, Dolphins and more. In some embodiments, the marine mammal tracking system 100 of the present invention can be used to track cetacean marine mammals that move relatively long distances in specific sea areas, such as cetaceans.

It should be understood that the implementation of this case is only described by taking marine mammals as an example, but this case is not limited to this. It should be understood that the tracking vehicle, system or method provided in this case can also be used to track other types of marine life in different applications.

FIG. 2 is a schematic diagram of a marine mammal tracking vehicle based on an embodiment of the present invention. As shown in Figure 2, in some embodiments, an ocean The mammalian tracking vehicle 200 may include a carrier 210, a power unit 220, an optical acquisition array 230, an acoustic acquisition array 240, a memory 250, and a processor 260. In this embodiment, the power device 220, the optical capture array 230, the acoustic capture array 240, the memory 250, and the processor 260 are all disposed on the carrier 210. Similar to the embodiment in FIG. 1, the processor 260 is communicatively coupled or electrically coupled with the power device 220, the optical capture array 230, the acoustic capture array 240, and the memory 250. Similar to the embodiment shown in FIG. 1, the processor 260 can access at least one instruction from the memory 250 and execute the at least one instruction to further implement the application program defined by the at least one instruction. Similarly, for ease of understanding, the procedures defined by the at least one instruction will be detailed in the following paragraphs.

It should be understood that the implementation and functions of the optical capture array 230, the acoustic capture array 240, the memory 250, and the processor 260 of the marine mammal tracking vehicle 200 shown in FIG. 2 are substantially the same as those shown in FIG. The optical acquisition array 110, the acoustic acquisition array 120, the memory 130, and the processor 140 in the marine mammal tracking system 100 of FIG. 2 can refer to the embodiment in FIG. 1 together.

In some embodiments, the carrier 210 of the marine mammal tracking vehicle 200 may be a hull for carrying persons or an unmanned hull for setting up a device. Correspondingly, the carrier 210 may have a plurality of cabins for accommodating persons or setting up devices. It should be understood that, in some embodiments, the specific 210 further includes a communication device (not shown in the figure), a positioning device (not shown in the figure), and a warning device (not shown in the figure). For example, in some embodiments, the communication device carrying the specific 210 further includes a satellite network communication unit, a wireless network (Wi-Fi) communication unit, and an automatic ship identification system (AIS). The system communication unit, etc., whereby the processor 260 on the specific 210 can perform information transmission through the above-mentioned channels. For example, in some embodiments, the positioning device containing the specific 210 includes a Global Positioning System (GPS) unit, a navigation unit, an electronic compass unit, etc., whereby the processor 260 on the specific 210 may operate in cooperation with the positioning device. To control the movement of specific 210. For example, in some embodiments, the warning device carrying the specific 210 further includes a warning light, a buzzer, etc., so that other ships in the sea can be notified of the location of the specific 210.

In some embodiments, the power unit 220 of the marine mammal tracking vehicle 200 may include an energy supply unit (not shown) and a drive unit (not shown), wherein the energy supply unit is used to provide the drive unit The energy source causes the drive unit to propel the specific 210 to travel in the body of water. It should be understood that the power device 220 of the marine mammal tracking vehicle 200 may be an electric power device or an internal combustion power device. For example, in some embodiments, the power supply unit of the power unit 220 may include a solar panel and a lithium battery, and the drive unit of the power unit 220 may be an electric motor, whereby the lithium battery can supply power to the electric motor to drive Traveling in water, solar panels can reserve power for lithium batteries during the voyage of marine mammal tracking vehicle 200. In some embodiments, the power supply unit of the power unit 220 may include a fuel reserve tank, and the drive unit of the power unit 220 may be an internal combustion engine, whereby the fuel reserve tank can deliver fuel to the internal combustion engine, and the internal combustion engine burns fuel. In order to push the specific 210 to travel in the body of water.

In some embodiments, the processor 260 in the marine mammal tracking vehicle 200 may perform unidirectional or bidirectional information exchange with the power device 220, the optical acquisition array 230, the acoustic acquisition array 240, and the memory 250. Accordingly, in these embodiments, the processor 260 in the marine mammal tracking vehicle 200 may control The power unit 220 is used to drive the carrier 210 to travel in a sea area, and then track marine mammals in the sea area.

FIG. 3 is a schematic diagram of a marine mammal tracking method based on an embodiment of the present invention. In this embodiment, the marine mammal tracking method 300 may be executed by the ship navigation system 100 shown in the embodiment in FIG. 1, or may be executed by the marine mammal tracking vehicle 200 shown in the embodiment in FIG. 2. Refer to Figure 1 or Figure 2 together. In detail, in the embodiment of FIGS. 1 and 2, the at least one instruction is stored in the memory 120 (or the memory 250), and the marine mammal tracking method 00 shown in FIG. 3 is associated with the at least one instruction. . The processor 140 (or the processor 260) can access and execute the at least one instruction from the memory 120 (or the memory 250) to implement an application program defined by the at least one instruction, and the application program corresponds to the ocean Mammal Tracking Method 300. The marine mammal tracking method 300 is used to control a ship (including the specific 210 shown in FIG. 2) to track marine mammals in a sea area. Furthermore, as described previously, in various embodiments, the process of the marine mammal tracking method 300 may be implemented with the processor 140 or 260 executing an artificial intelligence algorithm. In this embodiment, the detailed steps of the marine mammal tracking method 300 will be described in the following paragraphs.

Step S310: Receive an image around a vehicle captured by an optical capture array in an optical detection range.

In some embodiments, as shown in FIG. 1, the optical capture array 110 in the marine mammal tracking system 100 includes the optical capture devices, and the optical capture devices may be disposed on the ship in a distributed or centralized manner. Thus, the optical capture array 110 can cover the optical detection range around the ship approximately omnidirectionally, and the optical capture array 110 can continuously capture the optical detection in the sea area. Images within range. It should be noted that, at night or when visibility is poor, the optical capture devices can be switched to night vision mode, so that the optical capture array 110 can still capture images in the optical detection range in the sea area. In some embodiments, after the optical capture array 110 captures an image in the optical detection range, the optical capture array 110 may perform appropriate pre-processing or format processing on the image to convert it into an electrical signal, and then transmit the electrical signal. Go to processor 140 for processing. In some embodiments, after the optical capture array 110 captures an image in the optical detection range, the optical capture array 110 can directly convert the image into an electrical signal, and then transmit the electrical signal to the processor 140 for processing.

Similarly, as shown in FIG. 2, the optical acquisition array 230 of the marine mammal tracking carrier 200 can also cover an optical detection range around the carrier 210 in an approximately omnidirectional manner. When the carrier 210 travels in a sea area, the optical capture array 230 can continuously capture an image within the optical detection range in the sea area, and transmit the electric signal corresponding to the image to the processor 260 for processing. .

Step S320: Receive a sound around the vehicle captured by an acoustic capture array in an acoustic detection range.

In some embodiments, as shown in FIG. 1, the acoustic capture array 120 in the marine mammal tracking system 100 has the acoustic detection range covering the water body around the ship, and the acoustic capture array 120 can continuously emit sound waves into the water body. , And receive the corresponding echo and other sounds in the body of water. In some embodiments, after the acoustic capture array 120 captures the sound in the acoustic detection range, the acoustic capture array 120 may perform appropriate pre-processing or format processing on the sound to convert it into electrical signals, and then transmit the electrical signals. Go to processor 140 for processing. In some embodiments, the acoustic capture array 120 captures sounds within the acoustic detection range. Afterwards, the acoustic capture array 120 can directly convert the sound into an electrical signal, and then transmit the electrical signal to the processor 140 for processing.

Similarly, as shown in FIG. 2, the acoustic acquisition array 240 of the marine mammal tracking vehicle 200 can also cover an acoustic detection range around the carrier 210 substantially omnidirectionally. When the specific 210 travels in the sea area, the acoustic capture array 240 can continuously capture the sound in the acoustic detection range in the sea area, and transmit the electric signal corresponding to the sound to the processor 260 for processing. deal with.

Step S330: Determine whether there is at least one visual feature of marine mammals in the image or whether there is at least one acoustic feature of marine mammals in the sound.

In some embodiments, as shown in FIG. 1, after the processor 140 in the marine mammal tracking system 100 receives an electrical signal corresponding to an image from the optical capture array 110, the processor 140 may perform at least one image analysis on the image. A program to identify whether there is at least one visual feature of a marine mammal in the image. It should be understood that, because the images captured by the optical capture array 110 mainly correspond to parts of the body of water in the sea, marine mammals (for example, whales and dolphins) are more likely to appear on the body as the back and tail. Therefore, in one or more embodiments, the visual characteristics mainly recognized by the processor 140 may include at least one of a back color, a back pattern, a dorsal fin shape, a tail color, a tail pattern, or a tail fin shape of marine mammals. For better understanding, please refer to FIG. 4A and FIG. 4B together. The dorsal fin shape and the caudal fin shape will be described as examples below.

FIG. 4A is a schematic diagram of the visual characteristics of marine mammals based on an embodiment of the present invention. As shown in FIG. 4A, in some embodiments, the optical capture array 110 in the marine mammal tracking system 100 may be used in the optical detection system. A first image is obtained in the measurement range (looking at the light when the image was acquired, roughly speaking, the first image is only visible on the water surface), and the first image includes a part of a whale dolphin image. In some embodiments, the parts of the whale dolphin that are more likely to be exposed to the water are the back and tail. The processor 140 in the marine mammal tracking system 100 may identify the visual features in the image including the dorsal fin shape and the tail fin Shape (as shown in the box in Figure 4A). By analyzing whether there are objects similar to the dorsal fin shape and the tail fin shape of the whale dolphin in the first image, the processor 140 may determine whether the whale dolphin appears in the first image.

FIG. 4B is a schematic diagram of the visual characteristics of marine mammals based on an embodiment of the present invention. As shown in FIG. 4B, in some embodiments, the optical capture array 110 in the marine mammal tracking system 100 can acquire a second image in the optical detection range (roughly, the visibility of the second image Only part of the water surface), the second image includes a part of an image of a whale. In some embodiments, the parts of the cetacean that are more likely to be exposed to the water are the back and the tail. The processor 140 in the marine mammal tracking system 100 can identify the visual features in the image that include the shape of the cetacean's tail (such as Figure 4B box selection). By analyzing whether there is an object similar to the cetacean's tail fin shape in the second image, the processor 140 can determine whether the cetacean appears in the second image.

Similarly, as shown in FIG. 1, after the processor 140 in the marine mammal tracking system 100 receives an electrical signal corresponding to the sound from the acoustic acquisition array 120, the processor 140 may perform at least one acoustic analysis program on the sound. In order to identify whether there is at least one sound feature of the marine mammal in the sound. It should be understood that, because the sound captured by the acoustic capture array 120 mainly corresponds to a part of the body of water in the sea area, marine mammals (for example, cetaceans) are more likely to be in the body of water. The identifiable feature is its call. For this reason, the sound features mainly recognized by the processor 140 may include marine mammal calls. For example, in some embodiments, the processor 140 may set an adaptive filter according to the calling frequency of a known marine mammal, and use the adaptive filter to execute the at least one acoustic analysis program, thereby improving the at least one acoustic Analyze the efficiency of the process.

As mentioned above, in some embodiments, as shown in FIG. 1, the memory 130 may store visual characteristic information and sound characteristic information of a variety of marine mammals, such as the color, pattern, dorsal fin shape, Tail color, pattern, tail fin shape, call, etc. Thereby, when the processor 140 executes the image analysis program and the acoustic analysis program, the processor 140 can access the visual feature information and the sound feature information of the marine mammals from the memory 130, so as to obtain the information obtained by the optical capture array 110. The image and the sound acquired by the acoustic capture array 120 are compared.

Similarly, in the embodiment of FIG. 2, after the processor 260 of the marine mammal tracking vehicle 200 receives the electric signals corresponding to the image and the sound, the processor 260 may perform an image analysis program on the image and perform the sound analysis on the image. An acoustic analysis program is performed to identify whether there are visual characteristics of the marine mammal in the image and whether there are acoustic characteristics of the marine mammal in the sound.

Step S340: if at least one visual feature of the marine mammal is confirmed in the image or the at least one acoustic feature of the marine mammal is confirmed in the sound, the at least one visual feature of the marine mammal and the at least one sound feature are recorded.

In some embodiments, as shown in FIG. 1, when the processor 140 in the marine mammal tracking system 100 determines that the marine mammal is present in the image, When there is at least one visual feature, or when the processor 140 determines that there is the at least one sound feature of the marine mammal in the sound, the processor 140 may execute a recording program to include relevant information of the marine mammal (including the at least one visual feature). And the at least one sound feature) are stored in the memory 130. For example, in some embodiments, when the processor 140 determines that there is a visual feature (eg, dorsal fin shape) of a whale in the image, the processor 140 may access historical data from the memory 130. If there is no record about the visual feature in the historical data, the processor 140 may generate a first record about the whale and dolphin according to the visual feature, and store the first record in the memory 130. In some embodiments, if there is a second record of the visual feature in the historical data of the memory 130, the processor 140 may increase the discovery time of the cetacean in the second record. In this way, the processor 140 can continuously track the existing marine mammals in the historical data and record the newly discovered marine mammals.

Similarly, in the embodiment of FIG. 2, when the processor 260 of the marine mammal tracking vehicle 200 recognizes that there are visual features of the marine mammal in the image, or recognizes that there are sound features of the marine mammal in the sound, the processor 260 The recording process may be performed to store the information related to the marine mammals in the memory 250.

Step S350: Determine a direction of the marine mammal according to the image or the sound.

In some embodiments, as shown in FIG. 1, when the processor 140 in the marine mammal tracking system 100 determines that the at least one visual feature of the marine mammal is present in the image, the processor 140 may also determine the marine mammal's One direction. For example, as mentioned earlier, the optical capture array 110 includes these optical captures. The image capturing device acquires the image by at least one optical capturing device, and the processor 140 can initially determine the direction in which the marine mammal is located according to the angle pointed by the at least one optical capturing device. Further, the processor 140 can accurately determine the direction in which the marine mammal is located according to the position where the visual characteristics of the marine mammal are captured in the image. Alternatively, in some embodiments, the images of the marine mammals may be captured by different optical capture devices at the same time, and the processor 140 may determine the direction in which the marine mammals are located according to the relative positions set by the optical capture devices.

In addition, in some embodiments, the processor 140 may further determine the distance between the marine mammal and the ship. For example, if the optical capture array 110 includes an infrared camera, the image obtained by the optical capture array 110 may include depth information, whereby the processor 140 may determine the distance between the marine mammal and the infrared camera based on the depth information. Further judge the distance between the marine mammal and the ship.

In some embodiments, as shown in FIG. 1, when the processor 140 in the marine mammal tracking system 100 determines that there is the at least one sound feature of the marine mammal in the sound, the processor 140 may determine the direction of the marine mammal . For example, according to the foregoing description, the acoustic capture array 120 includes the sonar devices, and the sound is acquired by at least one of the sonar devices. The processor 140 may determine the direction in which the marine mammal is located according to the array beamforming method. In addition, in some embodiments, the processor 140 may further determine the distance between the marine mammal and the ship by using the acoustic capture array 120. For example, the processor 140 may control the acoustic capture array 120 to emit sound waves in the direction in which the marine mammal is located, and then determine the distance from the marine mammal to the ship according to the corresponding echo.

Similarly, in the embodiment of FIG. 2, when the processor 260 of the marine mammal tracking vehicle 200 recognizes that there are visual features of the marine mammal in the image, or recognizes that there are sound features of the marine mammal in the sound, the processor 260 The distance between the marine mammal and the specific 210 can also be determined based on the image or the sound. Further, by using the positioning device on the concrete 210, the processor 260 can obtain a relative position or an absolute position of the concrete 210 in the sea area. Correspondingly, based on the distance and direction of the marine mammal, the processor 260 can calculate the relative position or absolute position of the marine mammal in the sea area. In some embodiments, the processor 260 may record the relative position or absolute position of the marine mammal in the memory 250 to update historical data corresponding to the marine mammal.

In some embodiments, the processor 260 may execute an artificial intelligence algorithm to analyze at least one sound feature and / or at least one visual feature in steps S330 and S340 and output the data as the data D1 in FIG. 1. By repeatedly performing steps S310 to S340 repeatedly, the processor 260 can obtain multiple pieces of data D1 and modify the relevant parameters in the virtual model corresponding to the artificial intelligence algorithm according to the multiple pieces of data D1. In this way, after the relevant parameters in the virtual model have been modified (or trained) according to a sufficient amount of data D1, the processor 260 can obtain a more accurate virtual model and execute artificial intelligence algorithms based on this virtual model to automatically Track marine mammals.

In addition, as described previously, in some embodiments, at least one sound feature and / or at least one visual feature in steps S330 and S340 may be used as the data D1 in FIG. 1 and uploaded to the big data monitoring center 100A. In this way, the big data monitoring center 100A can collect a large number of visual features and / or sound features And other information to effectively train the virtual model corresponding to the artificial intelligence algorithm. In this way, the trained virtual model can be used by multiple marine mammal tracking vehicles 200.

Step S360: Guide the vehicle to travel in the direction.

In some embodiments, as shown in FIG. 1, the processor 140 in the marine mammal tracking system 100 may determine the direction and distance of the marine mammal from the ship according to the image or the sound. The processor 140 may send information about the direction and The telegraph signal of the distance to the navigation device of the ship, thereby guiding the ship to maintain a safe distance to travel in that direction. It should be understood that the safety distance is adjustable, and its purpose is to properly maintain the relative distance between the ship and the marine mammal, which can avoid disturbing the marine mammal, and can also better respond when the marine mammal turns.

Similarly, in some embodiments, as shown in FIG. 2, the processor 260 of the marine mammal tracking vehicle 200 may judge the direction and distance of the marine mammal from the specific 210 based on the image or the sound, and the processor 260 may directly The power device 220 on the carrier 210 is controlled to guide the carrier 210 to keep the safety distance in the direction. In some embodiments, by including the positioning device on the specific 210, the processor 260 can obtain information about obstacles (for example, islands or reefs, etc.) in the sea. In this way, the processor 260 can dodge or bypass the obstacle when the guidance carrier 210 is traveling in that direction, which can prevent the carrier 210 from being damaged.

Step S370: Confirm the at least one visual feature of the marine mammals in the image or the at least one acoustic feature of the marine mammals in the sound according to a time interval to determine the direction again to track the marine mammal class.

In some embodiments, as shown in FIG. 1, the processor 140 in the marine mammal tracking system 100 may guide the ship to maintain the safe distance in the direction to track the marine mammal. Since the marine mammal may change the direction of travel, the processor 140 may re-confirm the characteristics of the marine mammal in the image or sound according to a time interval during the progress of the ship, so as to continuously confirm the direction of the marine mammal to avoid The marine mammal was lost in the sea. For example, according to the possible habits of the marine mammal (for example, whale-dolphin ventilation time), the processor 140 may set the time interval accordingly, and the processor 140 confirms the characteristics of the marine mammal in the image according to the time interval, thereby judging Whether the marine mammal's direction of travel has changed.

Similarly, in some embodiments, as shown in FIG. 2, the processor 260 of the marine mammal tracking vehicle 200 may re-confirm the characteristics of the marine mammal in the image or sound according to the time interval, so as to continuously confirm the marine In the traveling direction of the mammal, the processor 260 may control the power unit 220 on the carrier 210 to guide the carrier 210 to maintain the safe distance to travel in the direction, and then continuously track the marine mammal.

It should be noted that, in some embodiments, the processor 260 may simultaneously monitor the visual characteristics of the marine mammal in the image and the acoustic characteristics of the marine mammal in the sound to track the marine mammal. For example, in some embodiments, at a first point in time, the processor 260 recognizes a tail fin feature of a whale dolphin in the image, and the processor 260 judges the distance and direction of the whale dolphin according to the above manner, and controls the power device 220 Travel in that direction. At the second point in time, the processor 260 recognizes that the sound obtained from this direction is similar to the whale dolphin. The processor 260 determines that the call may come from the whale, and continuously controls the power unit 220 to cause the carrier 210 to travel in that direction. At a third point in time, the processor 260 may simultaneously determine whether the whale dolphin turns according to the tail fin characteristics of the whale dolphin and the call of the whale dolphin, so as to continuously track the whale dolphin.

It should be noted that, in some embodiments, the processor 260 of the marine mammal tracking vehicle 200 may continuously track marine mammals according to the above manner, and update the historical records in the memory 250. Further, by using a specific 210 communication device, the processor 260 can transmit the historical records in the memory 250 to a remote server at a specific time (for example, the big data monitoring center 100A in FIG. 1 and / or Unmanned ship monitoring center 100B), the remote server can store the historical record. Conversely, the processor 260 can also access the information from the remote server at a specific time to update the historical records in the memory 250 by using the communication device carrying the specific 210. In the above manner, the marine mammal tracking vehicle 200 can continuously obtain and update the latest information on marine mammals.

In some embodiments, the processor 260 may execute an artificial intelligence algorithm to analyze whether the directions determined in steps S350 and / or S370 are valid. In the initial stage, by repeatedly performing steps S350 and / or S370 repeatedly, the processor 260 can obtain multiple pieces of data D1 (including multiple pieces of direction information), and modify the virtual model corresponding to the artificial intelligence algorithm according to the multiple pieces of data D1. Related parameters. Similarly, after the relevant parameters in the virtual model have been modified (or trained) according to a sufficient amount of data D1, the processor 260 can obtain a more accurate virtual model, and perform artificial intelligence algorithms to determine the correctness based on this virtual model. direction. For example, in some embodiments, the direction determined in step S350 or S370 may be used as the data D1 in FIG. 1 and uploaded to the unmanned ship monitoring center 100B. In this way, unmanned ships The monitoring center 100B can provide these directions to the navigator to determine whether it is valid. In this way, after collecting a large amount of direction information in the unmanned ship monitoring center 100B, the virtual model corresponding to the artificial intelligence algorithm can be effectively trained. In this way, the trained virtual model can be used by multiple marine mammal tracking vehicles 200 to provide more accurate tracking directions.

It should be understood that the multiple steps of the above-mentioned marine mammal tracking method 300 are merely examples, and are not limited to be performed in the order in the above examples. Without departing from the operation mode and scope of the embodiments of the present disclosure, various operations under the marine mammal tracking method 300 can be appropriately added, replaced, omitted, or performed in a different order. For example, in some embodiments, the order of related operations may be adjusted according to the training process of artificial intelligence algorithms.

According to the above embodiments, it should be understood that the present case provides a marine mammal tracking system 100, a marine mammal tracking vehicle 200, and a marine mammal tracking method 300, which can identify, record and track marine mammals in the sea at all times, avoiding the need for pure personnel Omissions caused by operations. If the marine mammal tracking technology of this case is applied to the fleet, and real-time data collection and transmission in the fleet is realized through a remote server, this case can establish an effective marine mammal tracking network. This marine mammal tracking network can be used in cooperation with existing marine mammal observer associations and potential customers, and is widely used in marine environmental exploration operations and marine ecological conservation operations.

It should be understood that although the present example is disclosed above by way of example, it is not intended to limit the present case. Any person skilled in the art can make various modifications and retouches without departing from the spirit and scope of the case. Therefore, the scope of protection of this case should be Subject to the scope of the attached patent application.

Claims (7)

A marine mammal tracking vehicle includes: a carrier specific; a power device for providing power to the carrier specific; and an optical capture array for capturing an image of the carrier specific surroundings in an optical detection range ; An acoustic capture array for capturing a sound around the carrier in an acoustic detection range; a memory storing at least one command; and a processor coupled to the power device and the memory The acoustic capture array and the image capture array are used to access the at least one instruction to: receive the image and the sound; determine whether the image has at least one visual feature of a marine mammal or whether the sound has an ocean At least one acoustic feature of a mammal; if at least one visual feature of the marine mammal is confirmed in the image or the at least one acoustic feature of the marine mammal is confirmed in the sound, the at least one visual feature of the marine mammal and the at least one A sound feature; judging a direction of the marine mammal based on the image or the sound; controlling the power device so that the load is specifically toward the party Travel; and according to a time interval to confirm that the marine mammals or at least one visual feature of the sound of the marine mammals in the image of the at least one sound feature to determine the direction again, so that the tracking marine mammals. The marine mammal tracking vehicle according to claim 1, wherein the processor controls the power unit according to a safe distance to make the vehicle travel in that direction. The marine mammal tracking vehicle according to claim 1, wherein the optical capture array has a plurality of optical capture devices with a plurality of orientations, so that the optical detection range covers the specific surroundings of the load omnidirectionally. The marine mammal tracking vehicle according to claim 3, wherein the optical capture devices in the optical capture array have a night vision mode. The marine mammal tracking vehicle according to claim 1, wherein the at least one visual feature of the marine mammal includes at least one of a dorsal fin shape and a tail fin shape of the marine mammal. A marine mammal tracking system includes: an optical capture array for capturing an image of a vehicle around in an optical detection range; an acoustic capture array for capturing in an acoustic detection range A sound around the vehicle; a memory storing at least one command; and a processor coupled to the memory, the acoustic capture array, and the image capture array to access the at least one command to : Receiving the image and the sound; determining whether there is at least one visual feature of the marine mammal in the image or whether there is at least one acoustic feature of the marine mammal in the sound; if at least one visual feature of the marine mammal is confirmed in the image or Confirming the at least one sound feature of the marine mammal in the sound, recording the at least one visual feature and the at least one sound feature of the marine mammal; judging a direction of the marine mammal based on the image or the sound; guiding the carrier Having to travel in that direction; and confirming the at least one visual feature or the sound of the marine mammal in the image according to a time interval Characterized in that the at least one sound in the marine mammals to determine whether the re-direction, so that the tracking marine mammals. The marine mammal tracking system according to claim 6, wherein the optical capture array has a plurality of optical capture devices with a plurality of orientations, so that the optical detection range covers the periphery of the vehicle omnidirectionally.