JP2020161886A - System for confirming ship periphery - Google Patents

Abstract

To allow for intuition recognition of distance between a ship and a mooring facility when reaching the shore.SOLUTION: A system for confirming ship periphery comprises: a plurality of cameras (13f, 13b, 13l, and 13r) that photographs ship periphery; a control device (21) that generates a bird's eye view (301) expressing a ship (101) and ship periphery in a bird's eye on the basis of an image acquired by a photographing of the plurality of cameras; a monitor (15)displaying the bird's eye view generated by the control device; and an inclination sensor (40) that detects the inclination of the ship. The control device generates an image expressing the ship of the attitude in accordance with the inclination of the ship as a bird's eye view on the basis of the inclination of the ship detected by an inclination sensor, and makes a display device display the image.SELECTED DRAWING: Figure 5

Description

The present disclosure relates to a peripheral confirmation system used to assist maneuvering a ship.

Conventionally, in order to support the maneuvering of a ship, the surroundings of the ship are photographed by multiple cameras, and by combining the photographed peripheral images, a pseudo bird's-eye view image as if the ship and its surroundings were looked down from above was created. , It is known to display on a display device mounted on a ship. A ship maneuvering support system (maneuvering support device) that displays such a bird's-eye view image is disclosed in, for example, Patent Document 1.

In the ship maneuvering support system disclosed in Patent Document 1, a graphic image showing at least the extension of a ship (own ship) equipped with the system is superimposed and displayed at the center position of a bird's-eye view image. Then, in this ship maneuvering support system, the shape of the graphic image of the ship is corrected to a shape corresponding to the cross-sectional shape obtained by cutting the hull at the waterline in consideration of the shaking of the ship. The deviation between the cross-sectional shape of the hull and the cross-sectional shape of the hull on the waterline is suppressed.

JP-A-2010-09365

Even if the shape of the graphic image of the ship is modified so that the position of the waterline can be seen as in the ship maneuvering support system disclosed in Patent Document 1, the position of the mooring facility such as the pier or the quay is above the water surface. It is not enough to grasp the sense of distance from the mooring facility when berthing. In addition, it is difficult to grasp the sense of distance from the mooring facility because it is not possible to intuitively grasp the shaking of the ship. Therefore, if an attempt is made to berth by relying on such a ship maneuvering support system, there is a risk that the ship will hit the mooring facility.

The technique of the present disclosure has been made in view of these points, and an object thereof is to make it easier to recognize the distance between the ship and the mooring facility at the time of berthing.

In order to achieve the above object, in the technique of the present disclosure, a bird's-eye view image showing the ship in a posture according to the inclination is created and displayed.

Specifically, the technology of the present disclosure includes a camera that photographs the surroundings of the ship, a control device that creates a bird's-eye view image of the ship and the surroundings of the ship based on the images acquired by the camera, and The target is a ship peripheral confirmation system equipped with a display device that displays a bird's-eye view image created by a control device.

The ship peripheral confirmation system according to the technique of the present disclosure further includes a tilt detecting means for detecting the tilt of the ship. Then, the control device creates an image representing the ship in a posture corresponding to the inclination of the ship as a bird's-eye view image based on the inclination of the ship detected by the inclination detecting means, and displays the image on the display device.

According to this configuration, a bird's-eye view image showing the ship in a posture corresponding to the inclination of the ship is displayed, so that the sway of the ship can be intuitively grasped in the bird's-eye image, which is caused by the inclination of the ship. You can see the displacement. This makes it easier to recognize the distance between the ship and the mooring facility when berthing.

In the ship peripheral confirmation system, it is preferable that the control device creates a bird's-eye view image based on the inclination of the ship detected by the inclination detecting means so as to suppress the shaking of the water surface caused by the shaking of the ship.

According to this configuration, in the bird's-eye view image, the shaking of the water surface is suppressed, so that the shaking of the ship can be more easily recognized.

Further, the ship peripheral confirmation system may include a memory in which a plurality of ship images representing ships having different postures are stored. In this case, the control device selects a ship image having a posture corresponding to the tilt of the ship detected by the tilt detecting means from a plurality of ship images stored in the memory, and displays an image including the selected ship image as a bird's-eye view image. You may create it.

According to this configuration, a ship image having a posture corresponding to the inclination of the ship is selected from a plurality of ship images representing ships having different postures, and a bird's-eye view image including the selected ship image is created. It is possible to easily realize a ship peripheral confirmation system related to technology.

Further, the ship's peripheral confirmation system may include a memory in which three-dimensional data of the ship is stored. In this case, the control device may create an image representing the ship in a posture corresponding to the inclination of the ship detected by the inclination detecting means as a bird's-eye view image based on the three-dimensional data of the ship stored in the memory. ..

According to this configuration, a bird's-eye view image showing the ship in a posture corresponding to the inclination of the ship is created based on the three-dimensional data of the ship. Therefore, in the bird's-eye view image, the inclination direction and the degree of inclination of the ship are subdivided. be able to. As a result, it is possible to enhance the reality of the situation of the ship and its surroundings that can be recognized by the bird's-eye view image.

According to the ship peripheral confirmation system according to the technique of the present disclosure, it is possible to easily recognize the distance between the ship and the mooring facility at the time of berthing. As a result, the maneuvering of the ship can be more preferably supported by the visual information, and the safety of the maneuvering can be enhanced.

FIG. 1 is a plan view of a ship equipped with the peripheral confirmation system according to the embodiment. FIG. 2 is a side view of a ship equipped with the peripheral confirmation system according to the embodiment. FIG. 3 is a block diagram showing the configuration of the peripheral confirmation system according to the embodiment. FIG. 4 is a schematic diagram of an image acquired by taking a picture with a hemispherical camera. FIG. 5 is a schematic view of a bird's-eye view image displayed on the monitor when the ship is not tilted in the peripheral confirmation system according to the embodiment. FIG. 6 is a schematic view of a bird's-eye view image displayed on the monitor when the ship is tilted forward in the peripheral confirmation system according to the embodiment. FIG. 7 is a schematic view of a bird's-eye view image displayed on the monitor when the ship is tilted backward in the peripheral confirmation system according to the embodiment. FIG. 8 is a schematic view of a bird's-eye view image displayed on the monitor when the ship is tilted to the left in the peripheral confirmation system according to the embodiment. FIG. 9 is a schematic view of a bird's-eye view image displayed on the monitor when the ship is tilted to the right in the peripheral confirmation system according to the embodiment. FIG. 10 is a schematic view of a bird's-eye view image displayed on the monitor when the ship is tilted to the left front in the peripheral confirmation system according to the embodiment. FIG. 11 is a schematic view of a bird's-eye view image displayed on the monitor when the ship is tilted forward to the right in the peripheral confirmation system according to the embodiment. FIG. 12 is a schematic view of a bird's-eye view image displayed on the monitor when the ship is tilted to the left rear in the peripheral confirmation system according to the embodiment. FIG. 13 is a schematic view of a bird's-eye view image displayed on the monitor when the ship is tilted to the right rear in the peripheral confirmation system according to the embodiment.

Hereinafter, exemplary embodiments will be described in detail with reference to the drawings.

The ship peripheral confirmation system according to this embodiment is a system for confirming the situation of the ship and its surroundings by an image in a ship maneuvering operation such as when the ship berths.

The configuration of the ship's peripheral confirmation system will be described with reference to FIGS. 1 to 3. FIG. 1 is a plan view of a ship 101 equipped with a peripheral confirmation system 11. FIG. 2 is a side view of the ship 101 equipped with the peripheral confirmation system 11. Further, FIG. 3 is a block diagram showing the configuration of the peripheral confirmation system 11 of the ship 101. Note that FIG. 1 does not strictly show the postures of the plurality of cameras 13f, 13b, 13l, and 13r included in the peripheral confirmation system 11.

As shown in FIGS. 1 to 3, the peripheral confirmation system 11 of the ship 101 includes a plurality of cameras 13f, 13b, 13l, 13r installed on the ship 101 at predetermined intervals, and the inside of the cabin 103 and the flying deck. A monitor 15 as a display device installed in the 104, various sensors 17 and 19 for detecting the attitude of the ship, and a control device 21 for comprehensively controlling the system 11 are provided.

The plurality of cameras 13f, 13b, 13l, 13r are a front camera 13f installed on the bow deck 105 on the bow side, a back camera 13b installed on the stern deck 107 on the stern side, and a central portion in the length direction of the ship 101. The left camera 13l installed on the left side of the stern and the right camera 13r installed on the right side of the stern. These four cameras 13f, 13b, 13l, and 13r are installed so as to photograph the periphery of the ship 101 including the outer peripheral portion of the ship 101.

The front camera 13f is attached to a support post 109 erected at the center position in the left-right direction of the bow deck 105 with the optical axis of the lens facing the bow (outer peripheral portion of the ship 101) in a posture of facing diagonally forward and downward. The front camera 13f captures a landscape of a predetermined range Rf in front of the ship 101 as a moving image or a still image, and acquires a front image Pf as a peripheral image.

The back camera 13b is attached to the support column 110 erected at the center position in the left-right direction of the stern deck 107 with the optical axis of the lens facing the stern (outer peripheral portion of the ship 101) side of the ship 101 in a posture of facing diagonally downward rearward. Has been done. The back camera 13b captures a landscape of a predetermined range Rb behind the ship 101 as a moving image or a still image, and acquires a rear image Pb as a peripheral image.

The left camera 13l is attached to the left side surface of the cabin 103 with the optical axis of the lens facing the gunwale 111 (outer peripheral portion of the ship 101) on the port side in a posture of facing diagonally downward to the left. The left camera 13l captures a landscape of a predetermined range Rl on the left side of the ship 101 as a moving image or a still image, and acquires a left image Pl as a peripheral image.

The light camera 13r is attached to the right side surface of the cabin 103 with the optical axis of the lens facing the gunwale 112 (outer peripheral portion of the ship 101) on the starboard side in a posture of facing diagonally downward to the right. The light camera 13r captures a landscape of a predetermined range Rr on the right side of the ship 101 as a moving image or a still image, and acquires a right image Pr as a peripheral image.

The shooting range Rf of the front camera 13f and the shooting range Rl of the left camera 13l include a common shooting range RCfl (a range shaded in the upper left of the paper in FIG. 1) that overlaps with each other. The shooting range Rf of the front camera 13f and the shooting range Rr of the light camera 13r include a common shooting range RCfr (a range shaded in the upper right corner of the paper in FIG. 1) that overlaps with each other. The shooting range Rb of the back camera 13b and the shooting range Rl of the left camera 13l include a common shooting range RCbl (a range shaded in the lower left corner of the paper in FIG. 1) that overlaps with each other. The shooting range Rb of the back camera 13b and the shooting range Rr of the light camera 13r include a common shooting range RCbr (a range shaded in the lower right corner of the paper in FIG. 1) that overlaps with each other.

In this embodiment, the water height H, which is the distance from the installation position of the front camera 13f, the back camera 13b, the left camera 13l, and the right camera 13r to the waterline 113 of the ship 101, is set to the same height. The installation angle of the front camera 13f, the installation angle of the back camera 13b, and the installation angles of the left camera 13l and the right camera 13r are set to be different from each other. The installation angle of the left camera 13l and the installation angle of the right camera 13r are set to the same angle.

The installation angles of the cameras 13f, 13b, 13l, and 13r can be arbitrarily set. That is, the installation angle of the front camera 13f, the installation angle of the back camera 13b, and the installation angle of the left camera 13l and the right camera 13r may be the same angle, or the installation angle of the left camera 13l and the right camera 13r may be the same. The angles may be different from each other.

Further, the front camera 13f, the back camera 13b, the left camera 13l and the right camera 13r are all waterproof and dustproof specifications. Further, each of these cameras 13f, 13b, 13l, 13r is detachably attached to the ship 101. For example, the cameras are attached to the hull when the ship 101 is used, but can be removed after the ship 101 is used. , It can be replaced when a problem occurs.

The front camera 13f, the back camera 13b, the left camera 13l, and the right camera 13r are wide-angle cameras each having a wide-angle lens 23 having a wider angle of view than the standard lens. As the wide-angle lens 23, for example, a lens having an angle of view of 180 degrees or more is used. Specifically, the front camera 13f, the back camera 13b, the left camera 13l, and the right camera 13r in this embodiment are wide-angle photographing cameras, so-called hemispherical cameras, each having a wide-angle lens 23.

FIG. 4 shows a conceptual diagram of the image 201 acquired by shooting with a hemispherical camera. The front image Pf acquired by the front camera 13f, the rear image Pb acquired by the back camera 13b, the left image Pl acquired by the left camera 13l, and the right camera 13r. As shown in FIG. 4, the right image Pr is a circular image 201 that looks around 360 degrees around the optical axis A of the wide-angle lens 23. The circular image 201 corresponds to an image obtained by projecting and transforming a three-dimensional optical image projected on the outer surface of the wide-angle lens 23 into a planar image along the optical axis A.

The monitor 15 is arranged in front of the steering seat 115 in the cabin 103 and around the steering seat 115 of the flying deck 104 at a position that can be seen by the operator seated in the steering seat 115 during maneuvering. The monitor 15 has a ship created based on the front image Pf, the rear image Pb, the left image Pl, and the right image Pr acquired by the front camera 13f, the back camera 13b, the left camera 13l, and the right camera 13r. A pseudo bird's-eye view image 301 is displayed as if looking down on 101 and its surroundings from above.

5 to 13 show a schematic view of the bird's-eye view image 301 that can be displayed on the monitor 15. FIG. 5 is a schematic view of the bird's-eye view image 301 when the ship 101 is not tilted. FIG. 6 is a schematic view of the bird's-eye view image 301 when the ship 101 is tilted forward. FIG. 7 is a schematic view of the bird's-eye view image 301 when the ship 101 is tilted backward. FIG. 8 is a schematic view of the bird's-eye view image 301 when the ship 101 is tilted to the left. FIG. 9 is a schematic view of the bird's-eye view image 301 when the ship 101 is tilted to the right. FIG. 10 is a schematic view of the bird's-eye view image 301 when the ship 101 is tilted forward to the left. FIG. 11 is a schematic view of the bird's-eye view image 301 when the ship 101 is tilted forward to the right. FIG. 12 is a schematic view of the bird's-eye view image 301 when the ship 101 is tilted to the left rear. FIG. 13 is a schematic view of the bird's-eye view image 301 when the ship 101 is tilted to the right rear.

As shown in FIG. 5, the bird's-eye view image 301 is a composite image formed by combining four partial bird's-eye view images 301p: a front bird's-eye view image BEVf, a rear bird's-eye view image BEVb, a left bird's-eye view image BEVl, and a right bird's-eye view image BEVr. For example, the bird's-eye view image 301 is configured such that the four partial bird's-eye view images 301p are seamlessly connected and integrated in a common shooting range RCfl, RCfr, RCbl, and RCbr, or are displayed side by side as separate data. To. In this bird's-eye view image 301, the ship image 303 showing the upper surface of the ship 101 is shown at the center position. The ship image 303 may be, for example, an illustration or a photograph.

As the ship image 303, there are a plurality of patterns of images 303a, 303b, 303c, 303d, 303e, 303f, 303g, 303h, 303i representing ships 101 having different inclination postures (see FIGS. 5 to 13). The ship image 303 presented in the bird's-eye view image 301 is selectively selected from a plurality of ship images 303a, 303b, 303c, 303d, 303e, 303f, 303g, 303h, 303i according to the inclination of the ship 101. It is changed according to the shaking of the ship 101. As a result, the shaking of the ship 101 in the bird's-eye view image 301 is expressed.

The monitor 15 is composed of a liquid crystal display panel or the like having a touch panel function. The monitor 15 switches the range of the bird's-eye view image 301 to be displayed, and displays individual peripheral images (front image Pf, rear image Pb, left image Pl, right image Pr) instead of the bird's-eye view image 301. It also serves as an operation unit 25 for performing the operation.

As the various sensors 17 and 19, a gyro sensor 17 and an acceleration sensor 19 are provided. The gyro sensor 17 and the acceleration sensor 19 constitute a tilt sensor 40 that detects the tilt of the ship 101 itself, the front camera 13f, the back camera 13b, the left camera 13l, and the right camera 13r, which change according to the shaking of the ship 101. .. The tilt sensor 40 is an example of a tilt detecting means for detecting the tilt of the ship 101.

The gyro sensor 17 detects how fast the ship 101, the front camera 13f, the back camera 13b, the left camera 13l, and the right camera 13r are rotating when the ship 101 shakes. The acceleration sensor 19 detects in what direction the ship 101, the front camera 13f, the back camera 13b, the left camera 13l, and the right camera 13r are rotating when the ship 101 shakes.

The gyro sensor 17 and the acceleration sensor 19 are built in, for example, at least one of the front camera 13f, the back camera 13b, the left camera 13l, the right camera 13r, and the control device 21. The gyro sensor 17 and the acceleration sensor 19 may be installed on the ship 101 separately from the cameras 13f, 13b, 13l, 13r and the control device 21.

The control device 21 is a controller based on a well-known microcomputer. The control device 21 is installed in the cabin 103, for example, and receives power from the battery of the ship 101 to the front camera 13f, the back camera 13b, the left camera 13l, the right camera 13r, the monitor 15, and the like. In addition to distributing and supplying power, data is exchanged with each of the cameras 13f, 13b, 13l, 13r, each monitor 15, and various sensors 17, 19.

Specifically, as shown in FIG. 3, the control device 21 includes a signal processing unit 29 that performs predetermined processing on image data, a main memory 31 that temporarily stores programs and data, and a program for the main memory 31. A memory control unit 33 that controls writing and reading of data, a CPU (Central Processing Unit) 35 that executes a program read in the main memory 31, an auxiliary memory 37 that stores various programs and data, and a monitor 15. It is provided with a monitor driver 39 for operating the above.

The signal processing unit 29 and the CPU 35 typically include an IC (Integrated Circuit), an LSI (Large Scale Integration), or an ASIC (Application Specific Integration Circuit).
) And so on. The main memory 31 is typically realized by a volatile memory such as a DRAM (Dynamic Random Access Memory) or an SDRAM (Synchronous DRAM). The auxiliary memory 37 is typically realized by a non-volatile memory such as a flash memory.

The signal processing unit 29 outputs image data of each frame for the front image Pf, the rear image Pb, the left image Pl, and the right image Pr output from the front camera 13f, the back camera 13b, the left camera 13l, and the right camera 13r. Four partial bird's-eye view images 301p are created based on the image data of each peripheral image Pf, Pb, Pl, Pr acquired one after another, and these four partial bird's-eye view images 301p (BEVf, BEVb, BEVl, BEVr) are created. ), And the ship image 303 is combined so as to overlap with the center position to create one bird's-eye view image 301.

The auxiliary memory 37 stores the external parameters of the front camera 13f, the back camera 13b, the left camera 13l and the right camera 13r, and the internal parameters of the front camera 13f, the back camera 13b, the left camera 13l and the right camera 13r. There is. Here, the "external parameter" is a numerical value indicating an installation state such as an installation position and an installation angle of the camera. Further, the "internal parameter" is a numerical value representing an optical property peculiar to the camera, such as information on the optical axis position and focal length of the camera and distortion such as wide-angle distortion.

Further, in the auxiliary memory 37, a plurality of patterns of ship images 303 representing ships 101 having different tilted postures are stored.

In this embodiment, the ship image 303 includes a non-tilting ship image 303a (see FIG. 5) showing the upper surface of the ship 101 in a non-tilted state and a forward leaning ship showing the upper surface of the ship 101 in a forward tilted state. Image 303b (see FIG. 6), backward tilted vessel image 303c (see FIG. 7) showing the upper surface of the vessel 101 tilted backward, and left tilted vessel showing the upper surface of the vessel 101 tilted to the left. Image 303d (see FIG. 8), right-leaning ship image 303e (see FIG. 9) showing the upper surface of the ship 101 tilted to the right, and left forward tilting showing the upper surface of the ship 101 leaning to the left forward. The ship image 303f (see FIG. 10), the right forward leaning ship image 303g (see FIG. 11) showing the upper surface of the ship 101 leaning forward to the right, and the upper surface of the ship 101 leaning to the left rearward are shown. A total of nine images of the left backward tilted ship image 303h (see FIG. 12) and the right backward tilted ship image 303i (see FIG. 13) showing the upper surface of the ship 101 tilted to the right rear are stored in the auxiliary memory 37. ing.

Further, the auxiliary memory 37 contains a cutout range used for the bird's-eye view image 301 in the front image Pf, a cutout range used for the bird's-eye view image 301 in the rear image Pb, a cutout range used for the bird's-eye view image 301 in the left image Pl, and a right image Pr. Each set value for the cutout range used for the bird's-eye view image 301 in the above is stored.

The signal processing unit 29 includes a preprocessing unit 41, an image correction unit 43, a cutting processing unit 45, and an image synthesizing unit 47. The preprocessing unit 41, the image correction unit 43, the cropping processing unit 45, and the image synthesizing unit 47 have functions realized by collaboration between hardware such as a microcomputer and software such as a program executed on the hardware. It is a department.

The memory control unit 33 is a preprocessing unit 41, an image correction unit 43, a cutting processing unit 45, and an image synthesizing unit 47, and is predetermined to image data for a front image Pf, a rear image Pb, a left image Pl, and a right image Pr. After reading the image data from the main memory 31 and performing the predetermined processing on the image data for the front image Pf, the rear image Pb, the left image Pl, and the right image Pr when performing the processing of The data is written to the main memory 31.

The preprocessing unit 41 refers to the image data of the front image Pf, the rear image Pb, the left image Pl, and the right image Pr acquired from the front camera 13f, the back camera 13b, the left camera 13l, and the right camera 13r, respectively. As preprocessing, adjustment processing such as digital clamp, pixel defect correction, and gain control is performed.

The image correction unit 43 reads the internal parameters and external parameters of the front camera 13f, the back camera 13b, the left camera 13l, and the right camera 13r stored in the auxiliary memory 37 into the main memory 31 via the memory control unit 33.

Then, the image correction unit 43 performs distortion correction to convert the image data of the front image Pf into an image without distortion such as wide-angle distortion based on the internal parameters of the front camera 13f. The image correction unit 43 also performs the same distortion correction on the image data of the rear image Pb based on the internal parameters of the back camera 13b. The image correction unit 43 also performs the same distortion correction on the image data of the left image Pl based on the internal parameters of the left camera 13l. The image correction unit 43 also performs the same distortion correction on the image data for the right image Pr based on the internal parameters of the light camera 13r.

Further, the image correction unit 43 performs a viewpoint conversion process of converting the image data of the front image Pf into an image as if looking down from above based on the internal parameters and the external parameters of the front camera 13f. The image correction unit 43 also performs the same viewpoint conversion process on the image data of the rear image Pb based on the internal parameters and the external parameters of the back camera 13b. The image correction unit 43 also performs the same viewpoint conversion process on the image data of the left image Pl based on the internal parameters and the external parameters of the left camera 13l. The image correction unit 43 also performs the same viewpoint conversion process on the image data of the right image Pr based on the internal parameters and the external parameters of the light camera 13r.

The cutout processing unit 45 has a cutout range stored in the auxiliary memory 37 for the bird's-eye view image 301 of the front image Pf, a cutout range used for the bird's-eye view image 301 of the rear image Pb, and a cutout range used for the bird's-eye view image 301 of the left image Pl. , And each set value for the cutout range used for the bird's-eye view image 301 of the right image Pr is read out to the main memory 31 via the memory control unit 33.

Then, the cutout processing unit 45 sets the front image Pf, the rear image Pb, and the left side after the above-mentioned distortion correction and viewpoint conversion processing are performed according to the set values of the cutout range in these peripheral images Pf, Pb, Pl, and Pr. By partially cutting out the image data for the image Pl and the right image Pr, the partial bird's-eye view image data for the front bird's-eye view image BEVf, the rear bird's-eye view image BEVb, the left bird's-eye view image BEVl, and the right bird's-eye view image BEVr is created. ..

When the signal processing unit 29 cuts out such image data, the front image Pf, based on the tilt of the front camera 13f, the back camera 13b, the left camera 13l, and the right camera 13r detected by the tilt sensor 40, Tilt of image data for rear image Pb, left image Pl and right image Pr, and cutout of partial bird's-eye view image 301p (front bird's-eye view image BEVf, rear bird's-eye view image BEVb, left bird's-eye view image BEVl and right bird's-eye view image BEVr) Correct the position. Such correction is called an electronic gimbal, and is a technique for canceling the displacement of the captured image in the rotational direction due to the dynamic change in the posture of the camera.

The partial bird's-eye view image 301p is tilted as the directions of the cameras 13f, 13b, 13l, and 13r change due to the hull swaying in the waves. The inclination of the partial bird's-eye view image 301p looks larger as the distance from the hull increases in the partial bird's-eye view image 301p because the above-mentioned viewpoint conversion process is performed.

On the other hand, in the present embodiment, as described above, since the electronic gimbal is effective in creating the bird's-eye view image 301, even if the ship 101 shakes and tilts, the front camera 13f and the back camera that follow the posture of the ship 101 The inclination of each partial bird's-eye view image 301p is corrected according to the degree of inclination of 13b, the left camera 13l, and the right camera 13r. Thereby, the inclination of the partial bird's-eye view image 301p due to the hull swaying in the waves can be reduced. This is effective in reducing the shaking of the bird's-eye image 301 when it is displayed in real time. The partial bird's-eye view image data created in this way is written in the main memory 31 by the memory control unit 33.

The image synthesizing unit 47 controls the memory of four partial bird's-eye view image data of the front bird's-eye view image BEVf, the rear bird's-eye view image BEVb, the left bird's-eye view image BEVl, and the right bird-view image BEVr written in the main memory 31 for each frame. The bird's-eye view image data of the bird's-eye view image 301 is created by combining the partial bird's-eye view image data read by the unit 33 and synthesizing the image data of the ship image 303 so as to be located at the center of the image. The bird's-eye view image data thus created is written in the main memory 31 by the memory control unit 33.

At this time, the image synthesizing unit 47 has a plurality of ship images 303a, 303b, 303c, 303d, 303e, 303f, 303g, 303h, stored in the auxiliary memory 37 based on the inclination of the ship 101 detected by the inclination sensor 40. From the 303i, the ship image 303 having a posture corresponding to the inclination of the ship 101 is selected. Here, as the ship image 303 having a posture corresponding to the inclination of the ship 101, the non-tilting ship image 303a is selected when it is determined that the inclination of the ship 101 detected by the inclination sensor 40 is relatively small or not. To. In other cases, in the ship image, the tilt direction of the ship 101 detected by the tilt sensor 40 is any of forward, rear, left, right, left front, right front, left rear, and right rear. The ship image 303 with a tilted posture in the corresponding direction is selected after determining whether or which direction the ship is close to.

That is, when it is determined that the tilting direction of the ship 101 corresponds to the front, the forward leaning ship image 303b is selected. When it is determined that the tilting direction of the ship 101 corresponds to the rear, the backward leaning ship image 303c is selected. When it is determined that the tilting direction of the ship 101 corresponds to the left side, the left-leaning ship image 303d is selected. When it is determined that the tilting direction of the ship 101 corresponds to the right side, the right-leaning ship image 303e is selected. When it is determined that the tilting direction of the ship 101 corresponds to the left front, the left forward leaning ship image 303f is selected. When it is determined that the tilting direction of the ship 101 corresponds to the front right, the right forward leaning ship image 303g is selected. When it is determined that the tilting direction of the ship 101 corresponds to the left rear, the left backward leaning ship image 303h is selected. When it is determined that the tilting direction of the ship 101 corresponds to the rear right, the right backward tilted ship image 303i is selected.

Then, the image synthesizing unit 47 creates bird's-eye image data for the bird's-eye image including the selected ship image 303 (303a, 303b, 303c, 303d, 303e, 303f, 303g, 303h or 303i). The bird's-eye view image data created in this way is written to the main memory 31 by the memory control unit 33. The bird's-eye view image 301 shown by the bird's-eye view image data is created so as to suppress the shaking of the water surface caused by the shaking of the ship 101 by the correction using the electronic gimbal.

The bird's-eye view image data of each frame written in the main memory 31 is read by the memory control unit 33 in response to a command from the CPU 35, and is passed to the monitor driver 39. The monitor driver 39 drives the monitor 15 based on the bird's-eye view image data read from the main memory 31. As a result, the bird's-eye view image 301 based on the bird's-eye view image data is displayed on the monitor 15 as a real-time image (live view).

As the display of the monitor 15, when the ship 101 is not tilted, a bird's-eye view image 301 including the non-tilted ship image 303a is displayed as shown in FIG. When the ship 101 is tilted forward, a bird's-eye view image 301 including a forward leaning ship image 303b is displayed as shown in FIG. When the ship 101 is tilted backward, a bird's-eye view image 301 including the backward tilted ship image 303c is displayed as shown in FIG. When the ship 101 is tilted to the left, a bird's-eye view image 301 including a left-leaning ship image 303d is displayed as shown in FIG. When the ship 101 is tilted to the right, a bird's-eye view image 301 including the right-tilted ship image 303e is displayed as shown in FIG. When the ship 101 is tilted forward to the left, a bird's-eye view image 301 including the left forward leaning ship image 303f is displayed as shown in FIG. When the ship 101 is tilted forward to the right, a bird's-eye view image 301 including a right forward leaning ship image 303g is displayed as shown in FIG. When the ship 101 is tilted to the left rearward, a bird's-eye view image 301 including the left backward tilted ship image 303h is displayed as shown in FIG. When the ship 101 is tilted rearward to the right, a bird's-eye view image 301 including the right backward tilted ship image 303i is displayed as shown in FIG. By switching the ship image 303 according to the inclination of the ship 101 in this way, the sway of the ship 101 is expressed in the bird's-eye view image 301.

According to the ship peripheral confirmation system 11 according to this embodiment, the bird's-eye view image 301 including the ship image 303 in the posture corresponding to the inclination of the ship 101 is displayed. Therefore, in the bird's-eye view image 301, the shaking of the ship 101 is intuitive. It is possible to grasp the displacement caused by the inclination of the ship 101 due to the shaking. Moreover, since the bird's-eye view image 301 is designed to suppress the shaking of the water surface, it is possible to make it easier to recognize the shaking of the ship 101. As a result, it is possible to easily recognize the distance between the ship 101 and the mooring facility at the time of berthing. As a result, the maneuvering of the ship 101 can be more preferably supported by visual information, and the safety of maneuvering can be enhanced.

− Modified example of the embodiment −
In the ship peripheral confirmation system 11 according to this modification, the three-dimensional data of the ship 101 is stored in the auxiliary memory 37 instead of the image data of the nine ship images 303. The control device 21 in the ship's peripheral confirmation system 11 represents the ship 101 in a posture corresponding to the tilt of the ship 101 detected by the tilt sensor 40 based on the three-dimensional data of the ship 101 stored in the auxiliary memory 37. Image 303 is generated, and a bird's-eye view image 301 including the ship image 303 is created.

According to the ship peripheral confirmation system 11 according to this modification, the bird's-eye view image 301 representing the ship 101 in a posture corresponding to the inclination of the ship 101 is created based on the three-dimensional data of the ship 101. In, the inclination direction and the degree of inclination of the ship 101 can be subdivided and expressed. As a result, the reality of the situation of the ship 101 and its surroundings that can be recognized by the bird's-eye view image 301 can be enhanced.

As described above, preferred embodiments have been described as an example of the techniques disclosed herein. However, the technique disclosed herein is not limited to this, and can be applied to embodiments in which changes, replacements, additions, omissions, etc. are made as appropriate. In addition, the components described in the attached drawings and the detailed description may include components that are not essential for solving the problem. Therefore, it should not be immediately determined that those non-essential components are essential by the fact that those non-essential components are described in the accompanying drawings and detailed description.

For example, the above embodiment may have the following configuration.

In the above embodiment, one ship image 303 representing the upper surface of the ship 101 with respect to the posture in which the ship 101 is tilted forward, backward, left, right, left front, right front, left rear, and right rear is totaled. Nine images are stored in the auxiliary memory 37, and a ship image 303 having a posture corresponding to the inclination of the ship 101 is selected from the nine ship images 303 to create a bird's-eye view image 301. Technology is not limited to this. In the auxiliary memory 37, only four ship images 303 showing the upper surface of the ship in front, rear, left and right of the ship are stored, and among these four ship images, the inclination of the ship is adjusted. A ship image may be selected to create a bird's-eye view image containing the selected ship image.

Further, in the auxiliary memory 37, a ship image 303 showing the upper surface of the ship 101 with respect to the front, rear, left, right, left front, right front, left rear and right rear of the ship 101 is displayed on the inclination of the ship 101. The degree is divided into a plurality of stages and a plurality of sheets are stored, and a bird's-eye view image 301 including the ship image 303 of the corresponding tilting posture is created according to the tilting direction and the tilting degree of the ship 101 detected by the tilt sensor 40. It may be. Further, the inclination direction of the ship 101 in the ship image may be more subdivided than the above-mentioned eight directions. In short, as the bird's-eye view image 301, a ship image 303 having a posture corresponding to the inclination of the ship 101 may be created and displayed on the monitor 15.

In the above embodiment, the ship's peripheral confirmation system 11 includes four cameras, a front camera 13f, a back camera 13b, a left camera 13l, and a right camera 13r, but the technique of the present disclosure is not limited to this. The number of cameras included in the ship's peripheral confirmation system 11 may be 3 or less or 5 or more, and any number of cameras can be adopted as long as the bird's-eye view image 301 can be created.

In the above embodiment, the front camera 13f, the back camera 13b, the left camera 13l and the right camera 13r are hemispherical cameras, but the technique of the present disclosure is not limited to this. The hemispherical camera is only an example of a camera, and a camera equipped with a wide-angle lens or a camera equipped with a standard lens may be used, and any camera can be adopted.

In the above embodiment, the case where the bird's-eye view image 301 created by the ship's peripheral confirmation system 11 is an image showing the periphery of the ship 101 in all directions has been described as an example, but the technique of the present disclosure includes this. Not exclusively. For example, the bird's-eye view image 301 may be an image that partially shows the periphery of the ship 101, such as an image showing only the front side and the left and right sides of the ship 101 and an image showing only the rear side and the left and right sides of the ship 101.

In the above embodiment, the case where the bird's-eye view image 301 created by the ship's peripheral confirmation system 11 is displayed on the monitor 15 installed on the ship 101 has been described as an example, but the technique of the present disclosure is not limited to this. .. For example, the ship's peripheral confirmation system 11 can be used when performing remote control, and the created bird's-eye view image 301 is transmitted to the control center, and the bird's-eye view image 301 is displayed on the monitor installed in the control center. It may be designed to do.

In the above embodiment, in the ship peripheral confirmation system 11, the tilt sensor 40 including the gyro sensor 17 and the acceleration sensor 19 is used to detect the tilt of the ship 101 and the cameras 13f, 13b, 13l, 13r. , The technology of the present disclosure is not limited to this. The tilt sensor 40 is only an example of tilt detecting means for detecting the tilt of the ship 101 and each of the cameras 13f, 13b, 13l, 13r, and the tilt detecting means are the front camera 13f, the back camera 13b, the left camera 13l and the right camera. It may be composed of at least one camera of 13r and the control device 21, and the inclination of the ship 101 may be detected by analyzing the peripheral image of the ship 101 taken by the camera with the control device 21. ..

As described above, the technique of the present disclosure is useful for a peripheral confirmation system used for ship operation support.

A ... Optical axis BEVf ... Front bird's-eye view image BEVb ... Rear bird's-eye view image BEVl ... Left bird's-eye view image BEVr ... Right bird's-eye view image Pf ... Front image (peripheral image)
Pb ... Rear image (peripheral image)
Pl ... Left image (peripheral image)
Pr ... Right image (peripheral image)
Rf, Rb, Rl, Rr ... Shooting range 11 ... Ship peripheral confirmation system 13f ... Front camera 13b ... Back camera 13l ... Left camera 13r ... Right camera 15 ... Monitor 17 ... Gyro sensor 19 ... Acceleration sensor 21 ... Control device 23 ... Wide-angle lens 25 ... Operation unit 29 ... Signal processing unit 31 ... Main memory 33 ... Memory control unit 35 ... CPU
37 ... Auxiliary memory 39 ... Monitor driver 40 ... Tilt sensor (tilt detection means)
41 ... Pre-processing unit 43 ... Image correction unit 45 ... Cut-out processing unit 47 ... Image composition unit 101 ... Ship 103 ... Shelter 104 ... Flying deck 105 ... Bow deck 107 ... Stern deck 109, 110 ... Support 111 ... Left-side gunnel 112 ... Right-side Gunnel 113 ... Waterline 115 ... Steering seat 201 ... Image 301 ... Bird-view image 301p ... Partial bird-view image 303 ... Ship image 303a ... Non-tilting ship image 303b ... Forward-leaning ship image 303c ... Backward-leaning ship image 303d ... Left-leaning ship image 303e ... Right leaning ship image 303f ... Left forward leaning ship image 303g ... Right forward leaning ship image 303h ... Left backward leaning ship image 303i ... Right backward leaning ship image

Claims (4)

A camera that takes a picture of the area around the ship,
A control device that creates a bird's-eye view image of the ship and its surroundings based on the image acquired by the camera.
A ship peripheral confirmation system including a display device for displaying the bird's-eye view image created by the control device.
Further provided with an inclination detecting means for detecting the inclination of the ship,
Based on the inclination of the ship detected by the inclination detecting means, the control device creates an image representing the ship in a posture corresponding to the inclination of the ship as the bird's-eye view image and displays it on the display device. A ship peripheral confirmation system featuring. In the ship peripheral confirmation system according to claim 1,
The control device creates a bird's-eye view image of the ship based on the inclination of the ship detected by the inclination detecting means so as to suppress the shaking of the water surface caused by the shaking of the ship. system. In the ship peripheral confirmation system according to claim 1 or 2.
It has a memory in which a plurality of ship images representing the ships in different postures are stored.
The control device selects the ship image of the posture corresponding to the tilt of the ship detected by the tilt detecting means from the plurality of ship images stored in the memory, and selects the ship as the bird's-eye view image. A ship peripheral confirmation system characterized by creating an image including an image. In the ship peripheral confirmation system according to claim 1 or 2.
It has a memory in which the three-dimensional data of the ship is stored.
Based on the three-dimensional data of the ship stored in the memory, the control device creates, as the bird's-eye view image, an image representing the ship in a posture corresponding to the tilt of the ship detected by the tilt detecting means. A ship peripheral confirmation system characterized by this.

Images