TWI686778B - Ship collision avoidance device with computer vision - Google Patents

Abstract

This disclosure is related to a ship collision avoidance device with computer vision. The ship collision avoidance device is installed on a ship hull. The ship collision avoidance device comprises an image capturing unit, a spatial image recognition unit, a database module, an engine control unit, a ship control unit, a human machine interface unit and an alarm unit. After the surrounding image is captured, the spatial image recognition unit and the module related to image processing are used to identify and analyze related objects in the image. Furthermore, after judging the distance and calculating the relative speed, the ship collision avoidance device figure out the distance to CPA (Closest Point of Approach) of the ship and compare it with a preset safety value. If the pre-set safety value has been reached, an alarm is issued to inform the ship's related driver to actively or passively enter the collision avoidance driving mode to achieve the safe navigation of the ship.

Description

Computer vision ship collision avoidance device

The invention relates to a computer vision ship collision avoidance device, in particular to a device mounted on a general ship or a ship, and actively or passively avoids the collision of the ship by visual image recognition.

In the prior art, regarding the technology to avoid collisions between ships or ships sailing at sea, most of them are currently used: 1. Radar, 2. AIS (Automatic Identification System) system, 3. Satellite positioning communication transmission technology, etc. To achieve the purpose of avoiding collisions of ships with high-cost sensors or complex induction systems or advanced communication transmission and radar equipment. However, such prior art often cannot avoid improper adjustment by the operator, or if the AIS equipment is not installed on other ships, or the operator fails to shut down due to active shutdown. It needs to be improved.

The invention discloses a computer vision ship collision avoidance device, which can judge and deal with the image recognition of the ship and the surrounding environment or objects, that is, at a safe distance, it can avoid the possibility of collision, or make the ship avoid meetings The place where the collision occurred; either the acceleration or deceleration of the ship or the braking mode, so that the ship arrives at the expected collision site earlier or later, which can effectively avoid the collision of the ship. In addition, the invention can be easily installed on a ship or a sailing vessel, and the function and effect of an autonomous ship collision avoidance can be achieved simply.

A computer vision ship collision avoidance device of the present invention is installed on a hull, including: at least one image capturing unit, the image capturing unit is installed somewhere or at the top of the hull for capturing Image information of the surrounding environment and objects of the hull; a spatial image recognition unit, electrically coupled to the at least one image capturing unit, for recognizing the captured images of the surrounding environment and objects, and outputting after the recognition There are a plurality of control signals; a database module, coupled to the spatial image recognition unit, is used to provide the spatial image recognition unit needs to compare with the corresponding reference data when performing the recognition; an engine control unit is coupled to The spatial image recognition unit receives one of the plurality of control signals to control the engine operation of the hull; a ship engine control unit, coupled to the spatial image recognition unit, receives the plurality of control signals A control signal for controlling the traveling direction of the hull; and an alarm unit coupled to the spatial image recognition unit, when the spatial image recognition unit recognizes the surrounding environment or object of the hull, a collision is about to occur , An alarm is issued.

1‧‧‧ hull

2‧‧‧shore

3‧‧‧ Breakwater

4‧‧‧ First ship

5‧‧‧ second ship

6‧‧‧ Lighthouse

10‧‧‧Spatial image recognition unit

11‧‧‧ First image capture unit

11a‧‧‧Video camera lens

11b‧‧‧Thermal image sensor module

11c‧‧‧Infrared image sensor module

11d‧‧‧Long distance induction die

12: Second image capture unit

13: Third image capture unit

14: Fourth image capture unit

15: Fifth image capture unit

18: control signal

20: Database module

22: Spatial data module

24: Distance data module

26: Aircraft data comparison judgment module

30: Engine control unit

32: throttle control unit

34: clutch control unit

40: Marine engine control unit

42: Main rudder control unit

44: auxiliary rudder control unit

60: Human-machine interface unit

70: Alarm unit

FIG. 1 is a schematic side view of an installation position according to an embodiment of the invention; FIG. 2 is a schematic top view of an installation position according to an embodiment of the invention; FIG. 3 is a schematic diagram of a circuit block connection according to an embodiment of the invention; and FIG. 4 is an image capture according to an embodiment of the invention A schematic diagram of the internal composition of the fetch unit; FIG. 5 is a schematic diagram of an image capture implementation in an embodiment of the present invention.

The invention discloses a computer vision ship collision avoidance device, which can perform the identification and analysis of the distance between related objects in the image through the related device of spatial image recognition and the technology related to image processing, and then further find out the relationship between the ship and surrounding objects Closest Point of Approach (CPA: Closest Point of Approach), to determine whether the CPA has approached a preset distance value, if the proximity has reached the preset value, an alarm will be issued to inform the ship driver to take collision avoidance measures; or take the initiative Start the collision avoidance driving mode to achieve the purpose and effectiveness of the ship’s safe navigation.

In the following, reference will be made to the accompanying drawings in order to more fully describe the various exemplary embodiments, and some exemplary embodiments are shown in the accompanying drawings. However, the concepts of the present invention may be embodied in many different forms and should not be interpreted as being limited to the exemplary embodiments set forth herein. Rather, providing these exemplary embodiments will make the invention detailed and complete, and will fully convey the scope of the inventive concept to those skilled in the art. In the drawings, the size and relative size of the ship, image capturing unit and navigation environment can be exaggerated for clarity. Similar numbers always indicate similar components.

It should be understood that although the terms front end, back end, left end, right end, or top end, etc. may be used herein to describe the location of various elements, such elements should not be limited by these terms. These terms are used to clearly distinguish one component from another and their different locations. Therefore, the left-end (or front-end) element discussed below may be referred to as the right-end (or back-end) element without departing from the teachings of the inventive concept; the same "top" used herein is merely to illustrate the relative positional relationship of the elements, not It must be the correspondence between the top and bottom of each element. If the terms "first" or "second" are used in this article, they are only used to clearly distinguish one component from another component, and there is not necessarily a certain order relationship between the two. As used herein, the term "and/or" includes any and all combinations of one or more of the associated listed items. Also, this article may use the terms "at least one," "plurality," or "plurality" to describe multiple elements with more than a single element, but these multiple elements are not limited to the implementation of one, two, three, or four The number of elements above indicates the technology implemented.

1 and 2 are schematic diagrams of the implementation of the present invention, showing that the computer vision ship collision avoidance device of the present invention is installed on a hull 1; and the present invention includes at least one image capturing unit, for example, a The first image capturing unit 11 is used to capture relevant image information of the environment and objects around the hull 1. However, when the present invention is actually implemented, it may include, but is not limited to, a second image capturing unit 12, a third image capturing unit 13, a fourth image capturing unit 14, and a fifth image capturing unit 15. 1 and 2 show that the first image capturing unit 11 is provided in the The front end of the hull, the second image capture unit 12 is provided at the rear end of the hull 1, the third image capture unit 13 is provided at the right end of the hull 1, and the fourth image capture unit 14 is provided at the hull 1 The left end of the; and the fifth image capture unit is provided on the top of the hull 1. That is to say, in practical application of the computer vision ship collision avoidance device of the present invention, only a single first image capturing unit 11 can be provided on the hull 1, or five image capturing units 11 can be provided at a time. ~15, of course, it can be installed on the hull 1 in two, three or four installations. Therefore, the above description of the installation position of the image capturing unit is only one of the implementation descriptions of the embodiments of the present invention, and in actual implementation, such installation position relationship should not be limited.

In the embodiment of the computer vision ship collision avoidance device of the present invention shown in FIG. 3, in addition to the first image capturing unit 11 in the at least one image capturing unit, it includes: a spatial image recognition unit 10, a The database module 20, an engine control unit 30, a marine control unit 40, a human-machine interface unit 60, and an alarm unit 70. The spatial image recognition unit 10 is electrically coupled to the first image capture unit 11 for recognizing images of the surrounding environment and objects captured, and outputs a plurality of control signals 18 after the recognition; The plurality of control signals 18 are respectively coupled to the alarm unit 70, the engine control unit 30 and the marine control unit 40; the database module 20 is coupled to the spatial image recognition unit 10 for providing the spatial image recognition unit 10 When performing identification, comparison and corresponding reference data are required; the engine control unit 30 is coupled to the spatial image identification unit 10 and receives one of the control signals 18 to control the engine operation of the hull 1 For example, to control the acceleration, deceleration or braking of the engine of the hull 1; the ship control unit 40 is coupled to the spatial image recognition unit 10 for receiving one control signal 18 of the plurality of control signals 18, mainly Used to control the direction of travel of the hull 1.

The spatial image recognition unit 10 according to the present invention is actually used, and the image processing framework of its constituent content includes: A. Image pre-processing: execution includes image filtering and enhancement processing; B. Image cutting: execution includes Sea and land separation and sea target cutting; C. Image recognition: Perform target identification, including ships, shores, breakwaters, lighthouses, or islands and reefs; D. Perform target distance and azimuth measurements.

The alarm unit 70 in the computer vision ship collision avoidance device of the present invention is also coupled to the space image recognition unit 10, and its main function is that when the space image recognition unit 10 recognizes the surrounding environment or object of the hull, it will soon happen to each other. In the event of a collision, the alarm unit 70 issues an alarm as a warning to warn the crew of the ship or the surrounding ship. The man-machine interface unit 60 is two-way electrically coupled to the spatial image recognition unit 10, and is mainly used to perform input-output control of the man-machine interface, that is, to provide the ship driver or the crew of the hull 1 with the ability to personally Control or operate the computer vision ship collision avoidance device to avoid collision.

It should be emphasized that the actual embodiment of the present invention mainly uses the active collision avoidance operation mode as the main operation method. When a ship collision situation may occur, the computer vision ship collision avoidance device will actively guide the ship The body 1 performs various collision avoidance treatments such as deceleration, steering, or stationary. The manual operation of the human-machine interface unit 60 is only an auxiliary operation mode, or is only an operation interface that is set in advance to perform setting operations or auxiliary operations when the situation is not critical. When the hull 1 is actually in danger, the computer vision ship collision avoidance device will still actively take measures to avoid it.

In an embodiment, the aforementioned spatial image recognition unit 10, database module 20, engine control unit 30, marine control unit 40, man-machine interface unit 60, alarm unit 70 and other components are actually implemented during manufacturing, The overall installation is a whole computer structure, that is, the above-mentioned spatial image recognition unit 10, database module 20, engine control unit 30, marine control unit 40, ship The occupant judgment unit 50, the man-machine interface unit 60, the alarm unit 70 and other components are formed to facilitate installation by the user or consumer on the hull 1 after purchase.

The database module 20 shown in FIG. 3 includes a spatial data module 22, a distance data module 24, and an aircraft data comparison judgment module 26. Spatial data module 22 is coupled to the spatial image recognition unit 10, and is used to store image data of the surrounding environment encountered by a general ship while sailing, and image data of objects encountered by a general ship while sailing. And the reference space value of the relative position of the surrounding environment and objects in space and the relative position of the hull 1 can be stored, for example, at 11 o'clock on the left front of the hull 1 or on the right front of the hull 1 Information about the spatial position of the clock direction or the 1 o'clock direction, etc.

The distance data module 24 is also coupled to the spatial image recognition unit 10 for storing the preset distance data of the hull 1 and the surrounding environment; it also stores the preset distance data between the hull 1 and the object . The stored preset distance data is mainly the straight-line distance between the hull 1 and the surrounding environment or the straight-line distance between the hull 1 and the object, etc., including the stored content The safety distance between the hull 1 and the surrounding environment is, for example, 1 km; or the safety distance between the hull 1 and the object, for example, 1.5 km, etc. The present invention is not limited in practical use .

The aircraft data comparison and judgment module 26 of FIG. 3 is also coupled to the spatial image recognition unit 10, and can be used to perform the distance measurement and storage of the hull 1 and the surrounding environment and the object, and to execute the judgment Whether the distance between the hull 1 and the surrounding environment or between the hull 1 and the object reaches the previously preset distance data. If the approached distance is less than the preset safety distance, an alarm will be issued immediately, or the engine of the hull 1 will be decelerated or stopped automatically, and the steering control of the rudder will be executed to avoid collision.

The engine control unit 30 shown in FIG. 3 includes a throttle control unit 32 and a clutch control unit 34. The throttle control unit 32 is provided in the engine control unit 30 to receive the control signal 18 of the plurality of control signals output from the spatial image recognition unit 10 after the recognition judgment has been performed, to execute the control of the hull 1 Control of engine throttle operation. The control of the throttle action includes control actions for deceleration, acceleration, reverse, braking, etc. of the hull 1. The clutch control unit 34 is also provided in the engine control unit 30 to receive the spatial image recognition unit 10 One of the control signals 18 output from the plurality of control signals outputted through the identification and judgment in the control system executes the action of controlling the clutch of the engine to control the operation of the engine of the hull 1.

The marine engine control unit 40 shown in FIG. 3 includes a main rudder control unit 42 and a secondary rudder control unit 44. The main rudder control unit 42 is provided in the ship engine control unit 40 for receiving one of the control signals 18 of the plurality of control signals output by the spatial image recognition unit 10 after the recognition judgment, and executes the control of the hull The control of the main rudder action in 1 can thus further control or change the direction of the hull 1 to avoid collision. The auxiliary rudder control unit 44 is also provided in the ship engine control unit 40 for receiving one of the control signals 18 of the plurality of control signals output by the spatial image recognition unit 10 after the recognition judgment, to execute the control of the hull 1 The control of the auxiliary rudder movement is used to assist the control of the hull 1 to change the direction of navigation.

FIG. 4 further discloses the internal components of the first image capture unit 11 in the present invention. The first image capture unit 11 includes an image capture lens 11a, a thermal image sensor module 11b, and an infrared image The sensing module 11c and a long-distance sensing module 11d. It should be emphasized that the internal composition of the first image capture unit 11 is not limited to the image camera lens 11a, the thermal image sensor module 11b, the infrared image sensor module 11c and the long distance. The sensor module 11d and other four components, but actually only have one, two or three camera lenses or related sensor modules, etc., can also be implemented separately to make an image capture module. This is based on the user's own usage, and the scope of the embodiments of the present invention is not limited.

The image capturing lens 11a in FIG. 4 is disposed in the first image capturing unit 11 and outputs an image signal to be coupled to the spatial image recognition unit 10. The thermal image sensing module 11b is set in the first image capturing unit 11, which is a thermal image capturing method, capturing the thermal image of the surrounding environment or surrounding objects of the hull 1 and outputting a thermal image signal And coupled to the spatial image recognition unit 10 for further Image recognition processing and judgment. The infrared image sensing module 11c is also set in the first image capturing unit 11 and captures infrared images of the surrounding environment of the hull 1 or the surrounding objects by the infrared image capturing method. And output an infrared image signal coupled to the spatial image recognition unit 10. The long-distance sensing module 11d is provided in the first image capturing unit 11 and adopts a long-distance sensing method to sense the surrounding environment of the hull 1 or detect the distance of surrounding objects, and output the information to be coupled and transmitted to In the spatial image recognition unit 10, further spatial recognition and judgment are performed later.

FIG. 5 is a schematic diagram of an image capturing implementation in an embodiment of the present invention. The implementation schematic can be obtained by the image capturing function of the aforementioned first to fifth image capturing units to obtain the hull 1 while sailing The image signals of surrounding environment and surrounding objects. The surrounding environment includes but is not limited to the shore 2, the breakwater 3, and the lighthouse 6. The surrounding objects on the hull 1 include the first ship 4 and the second ship 5. With the image data of FIG. 5, the computer vision ship collision avoidance device of the present invention can further determine whether the spatial direction and the distance between the shore 2 and the hull 1 fall within a safe distance range ; If not, the control signal 18 issued by the spatial image recognition unit 10 will control the alarm unit 70 to issue an alarm, or control the engine control unit 30 and the marine control unit 40 to perform deceleration or braking of the hull 1, In addition, the necessary reversal direction is executed, so that the hull 1 can actively carry out the control and operation to avoid collision, and can effectively avoid the occurrence of collision.

Similarly, the present invention can also perform the judgment of the space direction and the safety distance of the mutual distance between the hull 1 and the breakwater 3, the first ship 4, the second ship 5, the lighthouse 6 and the surrounding environment and surrounding objects to avoid The collision happened. It should be emphasized that the surrounding environment and surrounding objects described in the present invention are not limited to the above-mentioned shore 2, breakwater 3, first ship 4, second ship 5 and lighthouse 6, etc. The environment and objects encountered, such as docks, pilot boats, boats, canoes, large oil tankers, whales or large drifts, etc., can be identified by the computer vision ship collision avoidance device of the present invention to raise an alarm or Actively avoid occurrence with hull 1 collision.

In summary, the present invention discloses a computer vision ship collision avoidance device. Through the above-mentioned image recognition mode of collision avoidance, it is possible to avoid collision situations at a safe distance or avoid ship collisions. The location of the collision; either the acceleration or deceleration of the ship or the braking mode, so that the ship arrives at the expected collision location earlier or later, which can effectively avoid the collision of the ship.

With the design of the present invention, a ship or a sailing vessel can achieve the function and role of an autonomous ship collision avoidance after installing or installing the patent of the present invention, without costing high personnel costs, that is, those who have all-weather navigational observations, can Effectively improve the safety of ships or sailing ships, so that the safety of maritime navigation can be further guaranteed. Recently, the number of private yachts has increased rapidly, and the issue of yacht safety has become more and more important. The present invention can provide an autonomous ship collision avoidance system that is in line with economic benefits and performs well, and can effectively achieve the above-mentioned goals of safety and low cost. It is obvious that the present invention has extremely strong novelty, progress and practicality and other patent requirements.

The content described in the description of the present invention is only an example of the preferred embodiment, and when it cannot be used to limit the scope of protection of the present invention, any relevant local changes, minor modifications, corrections or additions of technology, or digital or position The above changes, etc., still do not deviate from the scope of the present invention.

1‧‧‧ hull

11‧‧‧ First image capture unit

12‧‧‧Second image capture unit

13‧‧‧ Third image capture unit

15‧‧‧ fifth image capture unit

14‧‧‧ Fourth image capture unit

Claims (8)

A computer vision ship collision avoidance device, which is installed on a hull, includes: at least one image capturing unit, the image capturing unit is installed on the top of the hull to capture the surrounding environment of the hull and Image information of the object; a spatial image recognition unit, electrically coupled to the at least one image capture unit, for recognizing the captured images of the surrounding environment and objects, and outputting a plurality of control signals after the recognition; A database module, coupled to the spatial image recognition unit, for providing comparison and corresponding reference data required by the spatial image recognition unit when performing recognition; an engine control unit, coupled to the spatial image recognition unit, Receiving a control signal from the plurality of control signals to control the engine operation of the hull; a ship engine control unit, coupled to the spatial image recognition unit, receives a control signal from the plurality of control signals to use To control the direction of travel of the hull; an alarm unit, coupled to the space image recognition unit, when the space image recognition unit recognizes the surrounding environment or object of the hull, an alarm will be issued when a collision is imminent; The database module includes: a spatial data module, coupled to the spatial image recognition unit, for storing image data of the surrounding environment encountered by a general ship while sailing, and storing the general ship's encounters while sailing Image data of the object; a distance data module, coupled to the spatial image recognition unit, for storing preset distance data of the hull and the surrounding environment; and storing the preset distance of the hull and the object Data; and an aircraft data comparison and judgment module, coupled to the spatial image recognition unit, for performing the distance between the hull and the surrounding environment and the object, and performing the judgment of the hull and the surrounding environment Or whether the distance between the hull and the object reaches the preset distance data. The computer vision ship collision avoidance device as described in item 1 of the request, which includes a man-machine interface unit, coupled to the spatial image recognition unit, for performing input and output control of the man-machine interface, providing personnel to control the computer vision Operation of ship collision avoidance devices. The computer vision ship collision avoidance device as described in claim 1, wherein the at least one image capturing unit includes a first image capturing unit, a second image capturing unit, and a third image capturing unit , A fourth image capturing unit and a fifth image capturing unit. The computer vision ship collision avoidance device as described in claim 3, wherein the first image capturing unit, a second image capturing unit, a third image capturing unit, a fourth image capturing unit and a The fifth image capturing unit is correspondingly arranged on the front end, rear end, left end, right end and top end of the hull. The computer vision ship collision avoidance device as described in item 1 of the claim, wherein the engine control unit includes: a throttle control unit in the engine control unit to receive the spatial image recognition unit after the recognition judgment One of the plurality of control signals is output to perform control of the throttle operation of the engine; and a clutch control unit is in the engine control unit to receive the output of the spatial image recognition unit after recognition judgment One of the plurality of control signals controls the clutch operation of the engine. The computer vision ship collision avoidance device as described in item 1 of the claim, wherein the ship machine control unit includes: a main rudder control unit, which is in the ship machine control unit to receive the spatial image recognition unit via recognition One of the plurality of control signals output after the judgment performs control of the main rudder action in the hull, thereby controlling the hull to change the direction of navigation; and a pair of rudder control units are located on the ship In the machine control unit, to receive the plurality of control signals output by the spatial image recognition unit after identification and judgment One of the control signals performs the control of the action of the secondary rudder in the hull, thereby assisting in controlling the hull to change the direction of navigation. The computer vision ship collision avoidance device as described in item 1 of the claim, wherein the at least one image capturing unit includes: an image photographing lens, which is disposed in the at least one image capturing unit and outputs an image signal coupling In the spatial image recognition unit; a thermal image sensing module, installed in the at least one image capture unit, outputting a thermal image signal coupled to the spatial image recognition unit; an infrared image sensing module, set In the at least one image capture unit, an infrared image signal is output coupled to the spatial image recognition unit; and a long-distance sensor module is provided in the at least one image capture unit to output a proximity signal coupling In the spatial image recognition unit. The computer vision ship collision avoidance device as described in item 1 of the claim, wherein the surrounding environment includes shores, breakwaters and lighthouses; the object includes ships of various types.

Images