CN116700275A - Unmanned operation method, unmanned operation device, unmanned operation equipment and unmanned operation storage medium for ship unloader - Google Patents
Unmanned operation method, unmanned operation device, unmanned operation equipment and unmanned operation storage medium for ship unloader Download PDFInfo
- Publication number
- CN116700275A CN116700275A CN202310783592.8A CN202310783592A CN116700275A CN 116700275 A CN116700275 A CN 116700275A CN 202310783592 A CN202310783592 A CN 202310783592A CN 116700275 A CN116700275 A CN 116700275A
- Authority
- CN
- China
- Prior art keywords
- ship
- cabin
- cargo ship
- cargo
- images
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 77
- 238000005516 engineering process Methods 0.000 claims description 53
- 238000007781 pre-processing Methods 0.000 claims description 22
- 230000004927 fusion Effects 0.000 claims description 18
- 238000004590 computer program Methods 0.000 claims description 15
- 238000001514 detection method Methods 0.000 claims description 15
- 230000000694 effects Effects 0.000 abstract description 2
- 238000010586 diagram Methods 0.000 description 8
- 230000006870 function Effects 0.000 description 7
- 230000009286 beneficial effect Effects 0.000 description 5
- 238000012545 processing Methods 0.000 description 5
- 238000001914 filtration Methods 0.000 description 3
- 230000008447 perception Effects 0.000 description 3
- 238000003384 imaging method Methods 0.000 description 2
- 230000004807 localization Effects 0.000 description 2
- 238000013507 mapping Methods 0.000 description 2
- 238000003062 neural network model Methods 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 230000001960 triggered effect Effects 0.000 description 2
- 238000013459 approach Methods 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 238000011217 control strategy Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 238000007599 discharging Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 230000011218 segmentation Effects 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 230000002194 synthesizing effect Effects 0.000 description 1
Landscapes
- Ship Loading And Unloading (AREA)
Abstract
The embodiment of the invention provides an unmanned operation method, device and equipment of a ship unloader and a storage medium, and relates to the technical field of ship unloader operation. The unmanned operation method of the ship unloader comprises the following steps: acquiring cargo ship body images acquired by all ship body sensing cameras on the ship unloader in real time, and acquiring all the cargo ship body images at the current moment; and according to the positions of all the cargo ship body images in the positioning operation area, generating a first planning route for controlling the ship unloader to enter and exit the operation area. The embodiment of the invention can realize unmanned control in the operation process of the ship unloader, and has the technical effect of improving the operation efficiency of the ship unloader.
Description
Technical Field
The invention relates to the technical field of ship unloader operation, in particular to a unmanned operation method, device and equipment of a ship unloader and a storage medium.
Background
Currently, most bucket chain continuous ship unloaders are operated by manual operation. After the cargo ship is on shore, a driver sequentially controls the ship unloader to approach the cargo ship, controls the material taking head to enter and exit the ship cabin and controls the material taking head to take materials in the cabin according to a material taking plan, the position of the cargo ship, the ship type and cabin section distribution, and the problems of high labor intensity, low operation efficiency and the like exist in the manual driving operation process.
In recent years, unmanned control machines have been applied to the floor in a plurality of industrial fields, and the application of unmanned technology to chain bucket type continuous ship unloader has become an industry development trend. How to perform unmanned control in the operation process of the ship unloader, such as approaching a cargo ship, taking a material head to enter and exit a ship cabin, taking a material head to take materials, and the like, improves the operation efficiency of the ship unloader, and becomes the current research direction.
Disclosure of Invention
The embodiment of the invention aims to provide an unmanned operation method, device and equipment of a ship unloader and a storage medium, which are used for realizing unmanned control in the operation process of the ship unloader and improving the technical effect of the operation efficiency of the ship unloader.
In a first aspect, an embodiment of the present invention provides an unmanned operation method for a ship unloader, which is characterized by including:
acquiring cargo ship body images acquired by all ship body sensing cameras on the ship unloader in real time, and acquiring all the cargo ship body images at the current moment;
and according to the positions of all the cargo ship body images in the positioning operation area, generating a first planning route for controlling the ship unloader to enter and exit the operation area.
In the implementation process, a group of cargo ship body images are acquired in real time at the beginning of the operation of the ship unloader to perform cargo ship approaching navigation, unmanned control can be performed in the operation process of approaching the ship unloader to the cargo ship, and the operation efficiency of the ship unloader is improved.
Further, the unmanned operation method of the ship unloader further comprises the following steps:
when the ship unloader enters the operation area, acquiring cargo ship cabin outer images acquired by all cabin outer sensing cameras on the ship unloader in real time, and acquiring all cargo ship cabin outer images at the current moment;
and according to the positions of all the cargo ship cabin outside image positioning operation hatches, generating a second planning route for controlling the material taking heads on the ship unloader to come in and go out of the operation hatches.
In the implementation process, a group of images outside the cargo ship cabin are acquired in real time at the stage that the ship unloader enters the operation area to carry out cabin entering and exiting navigation of the material taking head, unmanned control can be carried out in the cabin entering and exiting operation process of the material taking head, and the operation efficiency of the ship unloader is improved.
Further, the unmanned operation method of the ship unloader further comprises the following steps:
when the material taking head enters the operation hatch, acquiring cargo ship cabin images acquired by sensing cameras in each cabin on the ship unloader in real time, and obtaining all cargo ship cabin images at the current moment;
and generating a third planning route for controlling the material taking head to take materials in the operation cabin based on a predefined material taking strategy according to the positions of all the image positioning bulkheads in the cargo ship cabin.
In the implementation process, the unmanned control can be carried out in the material taking process of the material taking head by acquiring a group of images in the cargo ship cabin in real time at the stage of entering the operation hatch of the material taking head so as to carry out material taking navigation of the material taking head, and the operation efficiency of the ship unloader is improved.
Further, the positioning the position of the working area according to all the cargo ship body images specifically includes:
performing image preprocessing, image registration and image fusion on all the cargo ship body images to obtain target cargo ship body images;
performing cargo ship detection according to the target cargo ship body image by adopting a machine vision technology;
identifying a ship form, a bow and a stern of the cargo ship upon detection of the cargo ship to determine a hold area of the cargo ship;
generating a ship body three-dimensional point cloud according to the ship body image of the target cargo ship by adopting a binocular vision technology;
carrying out three-dimensional space positioning on the cargo ship based on the ship body three-dimensional point cloud to obtain the position of the cargo ship;
and selecting an area around the cargo ship, which is close to the cabin area, as the operation area based on the position of the cargo ship, and positioning the position of the operation area.
In the implementation process, the cargo ship approaching navigation is performed according to all cargo ship body images by combining two-dimensional image detection and three-dimensional point cloud positioning, so that the position of the cargo ship can be rapidly and accurately positioned, an operation area is reasonably selected, and the operation efficiency of the ship unloader is further improved.
Further, the positioning the position of the working area according to all the cargo ship body images specifically further includes:
reconstructing a three-dimensional grid model of the cargo ship based on the ship body three-dimensional point cloud;
determining the attitude of the cargo ship according to the three-dimensional grid model of the cargo ship;
and adjusting a predefined material taking strategy according to the attitude of the cargo ship.
In the implementation process, the ship attitude is obtained in real time, and the material taking strategy is adjusted according to the ship attitude, so that the influence of tides and surges on the ship body can be evaluated in an auxiliary manner, and the material taking strategy is adjusted in time, thereby being beneficial to further improving the operation efficiency of the ship unloader.
Further, the positioning the position of the operation hatch according to all the images outside the cargo ship cabin specifically includes:
performing image preprocessing, image registration and image fusion on all the cargo ship cabin images to obtain target cargo ship cabin images;
performing cabin section identification according to the images outside the target cargo ship cabin by adopting a machine vision technology to obtain each cabin section of the cargo ship;
generating a cabin section three-dimensional point cloud according to the target cargo ship cabin external image by adopting a multi-vision technology;
carrying out three-dimensional space positioning on the hatches of the cabin segments based on the three-dimensional point cloud of the cabin segments to obtain the hatch positions of the cabin segments;
And selecting a working cabin section from all the cabin sections based on a predefined working plan, and acquiring the hatch position of the working cabin section as the position of the working hatch.
In the implementation process, through combining two-dimensional image detection and three-dimensional point cloud positioning, the material taking head can enter and exit the cabin navigation according to all the images outside the cargo ship cabin, so that the position of the operation hatch can be rapidly and accurately positioned, and the operation efficiency of the ship unloader can be further improved.
Further, the step of generating a second planned route for controlling the material taking heads on the ship unloader to come in and go out of the operation hatch according to the positions of all the cargo ship cabin outer images, and the step of further comprises:
and when the second planning route is generated, adopting a multi-vision technology, positioning the position of the obstacle on the second planning route according to the image outside the target cargo ship cabin, and adjusting the second planning route according to the position of the obstacle on the second planning route.
In the implementation process, the obstacle on the second planning route is detected and positioned, and the second planning route is adjusted according to the position of the obstacle on the second planning route, so that the safety of the material taking head in and out of the operation hatch can be ensured.
Further, the positioning the position of the bulkhead according to the images in all the cargo ship cabins specifically comprises:
performing image preprocessing, image registration and image fusion on all the images in the cargo ship cabin to obtain an image in the target cargo ship cabin;
dividing the bulkhead and the material level according to the image in the cabin of the target cargo ship by adopting a machine vision technology;
generating a bulkhead three-dimensional point cloud according to the image in the target cargo ship cabin by adopting a multi-vision technology;
and carrying out three-dimensional space positioning on the bulkhead based on the bulkhead three-dimensional point cloud to obtain the position of the bulkhead.
In the implementation process, through combining two-dimensional image detection and three-dimensional point cloud positioning, the position of the bulkhead can be rapidly and accurately positioned by carrying out the material taking head material taking navigation according to images in all cargo ship cabins, and the operation efficiency of the ship unloader is further improved.
Further, the generating a third planned route for controlling the material taking head to take materials in the operation cabin based on a predefined material taking strategy according to the positions of all the image positioning bulkheads in the cargo ship cabin further comprises:
and when the third planning route is generated, adopting a multi-vision technology, positioning the position of the obstacle on the third planning route according to the image in the cabin of the target cargo ship, and adjusting the third planning route according to the position of the obstacle on the third planning route.
In the implementation process, the obstacle on the third planning route is detected and positioned, and the third planning route is adjusted according to the position of the obstacle on the third planning route, so that the safety of the material taking head can be ensured.
Further, the positioning the position of the bulkhead according to the images in all the cargo ship cabins specifically further comprises:
performing material type identification according to the images in the target cargo ship cabin by adopting a machine vision technology to obtain the material type of the cargo;
generating a material level three-dimensional point cloud according to the image in the target cargo ship cabin by adopting a multi-vision technology;
reconstructing a three-dimensional grid model of the cargo material based on the three-dimensional point cloud of the material surface;
and adjusting the material taking strategy by combining the material type of the cargo material and the three-dimensional grid model of the cargo material.
In the implementation process, the material taking strategy is adjusted by combining the material type of the material identified from the image and the three-dimensional grid of the material reconstructed by using the three-dimensional point cloud of the material surface, so that the material taking strategy can be further optimized, and the safety and the high efficiency of the material taking head can be ensured.
In a second aspect, an embodiment of the present invention provides an unmanned operation device for a ship unloader, including:
the image acquisition module is used for acquiring cargo ship body images acquired by all ship body sensing cameras on the ship unloader in real time to acquire all the cargo ship body images at the current moment;
And the navigation operation module is used for positioning the position of the operation area according to all the cargo ship body images and generating a first planning route for controlling the ship unloader to come in or go out of the operation area.
In a third aspect, embodiments of the present invention provide an electronic device comprising a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor; the memory is coupled to the processor and the processor when executing the computer program implements the unmanned operation method of the ship unloader as described above.
In a fourth aspect, embodiments of the present invention provide a computer-readable storage medium including a stored computer program; wherein the unmanned operation method of the ship unloader is controlled by the equipment where the computer readable storage medium is located when the computer program runs.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings that are needed in the embodiments of the present invention will be briefly described below, it should be understood that the following drawings only illustrate some embodiments of the present invention and should not be considered as limiting the scope, and other related drawings can be obtained according to these drawings without inventive effort for a person skilled in the art.
Fig. 1 is a schematic flow chart of an unmanned operation method of a ship unloader according to a first embodiment of the present invention;
fig. 2 is a schematic view showing the arrangement positions of a hull-aware camera, an out-of-cabin-aware camera, and an in-cabin-aware camera on a ship unloader according to an example of the first embodiment of the present invention;
fig. 3 is a schematic view of a hull-aware camera, an out-of-cabin aware camera, an in-cabin aware camera connection server, which are examples in the first embodiment of the present invention;
fig. 4 is a schematic flow chart of an unmanned operation method of a ship unloader according to an alternative embodiment of the present invention;
fig. 5 is a schematic flow chart of an unmanned operation method of a ship unloader according to another alternative embodiment of the present invention;
fig. 6 is a schematic structural view of an unmanned operation device of a ship unloader according to a second embodiment of the present invention;
fig. 7 is a schematic structural diagram of an electronic device according to a third embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be described below with reference to the accompanying drawings in the embodiments of the present invention.
It should be noted that: in the description of the present invention, the terms "first," "second," and the like are used merely to distinguish between descriptions and are not to be construed as indicating or implying relative importance. Meanwhile, step numbers herein are only for convenience of explanation of the embodiments of the present invention, and are not used as limiting the order of execution of the steps. The method provided by the embodiment of the invention can be executed by the related terminal equipment, and the following description takes the server as an execution body as an example.
Referring to fig. 1, fig. 1 is a schematic flow chart of an unmanned operation method of a ship unloader according to a first embodiment of the present invention. The first embodiment of the invention provides an unmanned operation method of a ship unloader, which comprises the following steps of S101 to S102:
s101, acquiring cargo ship body images acquired by all ship body sensing cameras on a ship unloader in real time, and acquiring all cargo ship body images at the current moment;
s102, positioning the position of the working area according to all cargo ship body images, and generating a first planning route for controlling the ship unloader to enter and exit the working area.
As an example, a set of ship body sensing cameras are provided on the ship unloader in advance so that ship body images at various angles are acquired by using the respective ship body sensing cameras to acquire all ship body images. For example, as shown in fig. 2, the arrangement positions of the ship sensing cameras on the ship unloader are shown in the state that at least two ship sensing cameras are arranged at one end, close to the sea side, of a cart platform of the ship unloader in advance, so that each ship sensing camera moves along with the cart platform in parallel with a reference cargo ship, each ship sensing camera is calibrated, after calibration, the viewing angles spliced by all ship sensing cameras can cover the side surfaces of the ship bodies of the reference cargo ship, and it is ensured that all ship body images of the cargo ship contain complete information of the side surfaces of the ship bodies of the cargo ship in a scene that the cargo ship is berthed at a wharf. After a set of hull-aware cameras is provided on the ship unloader, each hull-aware camera may be connected to a server, i.e., the integrated server in fig. 3, with reference to fig. 3.
When the ship unloader starts to operate, the ship unloader triggers each ship perception camera in real time to acquire ship body images of the cargo ships under each angle, all ship body images at the current moment are acquired, the position of an operation area is positioned according to all ship body images at the current moment, and a first planning route is generated by combining the position of the ship unloader at the current moment and the position of the operation area, so that the ship unloader is controlled to enter and exit the operation area according to the first planning route.
According to the embodiment of the invention, the ship approaching navigation is performed by acquiring the ship body images of the group of ships in real time at the beginning operation stage of the ship unloader, so that unmanned control can be performed in the operation process of approaching the ship of the ship unloader, and the operation efficiency of the ship unloader is improved.
Referring to fig. 4, fig. 4 is a flow chart of an unmanned operation method of a ship unloader according to an alternative embodiment of the present invention. In an alternative embodiment, the unmanned operation method of the ship unloader further comprises steps S103 to S104:
s103, after the ship unloader enters an operation area, acquiring cargo ship cabin outer images acquired by all cabin outer sensing cameras on the ship unloader in real time, and acquiring all cargo ship cabin outer images at the current moment;
s104, positioning the positions of the operation hatches according to all the images outside the cargo ship cabins, and generating a second planning route for controlling the material taking heads on the ship unloader to go in and out of the operation hatches.
As an example, a set of outside-cabin sensing cameras are provided on the ship unloader in advance so that outside-cabin images at various angles are acquired by using the respective outside-cabin sensing cameras, and all outside-cabin images are acquired. For example, as shown in fig. 2, the arrangement positions of all cabin-outside sensing cameras on the ship unloader are shown in the position, at least four cabin-outside sensing cameras are arranged around the top end of a lifting cylinder (BE cylinder) of the ship unloader in advance, so that all cabin-outside sensing cameras rotate along with the lifting cylinder, all cabin-outside sensing cameras are calibrated, through calibration, the spliced field angles of all cabin-outside sensing cameras can cover most of decks before a material taking head enters a cabin of a reference cargo ship, part of decks outside an operation hatch are covered after the material taking head enters the cabin of the reference cargo ship, and it is ensured that all cargo ship cabin-outside images contain complete information outside the cabin of the cargo ship under the scene that the ship unloader enters an operation area. After a set of outside-cabin sensing cameras are provided on the ship unloader, each outside-cabin sensing camera may be connected to a server, i.e., the integrated server in fig. 3, with reference to fig. 3.
When the ship unloader enters the operation area, all cabin outer sensing cameras are triggered in real time to acquire cargo ship cabin outer images under all angles, all cargo ship cabin outer images at the current moment are acquired, the position of an operation hatch is positioned according to all cargo ship cabin outer images at the current moment, a second planning route is generated by combining the position of a material taking head on the ship unloader at the current moment and the position of the operation hatch, and the material taking head is controlled to come in and go out of the operation hatch according to the second planning route.
According to the embodiment of the invention, a group of images outside the cargo ship cabin are acquired in real time at the stage that the ship unloader enters the operation area to carry out cabin entering and exiting navigation of the material taking head, unmanned control can be carried out in the cabin entering and exiting operation process of the material taking head, and the operation efficiency of the ship unloader is improved.
Referring to fig. 5, fig. 5 is a schematic flow chart of an unmanned operation method of a ship unloader according to an alternative embodiment of the present invention. In an alternative embodiment, the unmanned operation method of the ship unloader further comprises steps S105 to S106:
s105, after the material taking head enters the operation hatch, acquiring the images in the cargo ship cabins acquired by the sensing cameras in each cabin on the ship unloader in real time, and acquiring all the images in the cargo ship cabins at the current moment;
s106, according to the positions of the image positioning bulkheads in all cargo ship cabins, a third planning route for controlling the material taking heads to take materials in the operation cabins is generated based on a predefined material taking strategy.
As an example, a set of in-cabin sensing cameras is provided on the ship unloader in advance so that in-cabin images of the cargo ship at various angles are acquired by using the respective in-cabin sensing cameras, and all in-cabin images of the cargo ship are acquired. For example, as shown in fig. 2, the arrangement positions of the cabin sensing cameras on the ship unloader are at least four cabin sensing cameras are arranged around the bottom end of the lifting cylinder of the ship unloader in advance, so that the cabin sensing cameras rotate along with the lifting cylinder, the cabin sensing cameras are calibrated, and after calibration, the spliced field angles of all cabin sensing cameras can cover the bulkhead and the charge level around the pick-up head after the pick-up head enters the cabin of the reference cargo ship, so that the images in all cargo ship cabins contain complete information in the cargo ship cabin under the condition that the pick-up head enters the operation hatch. After a set of in-cabin perception cameras are provided on the ship unloader, each in-cabin perception camera may be connected to a server, i.e. the integrated server in fig. 3, with reference to fig. 3.
The ship body sensing cameras, the cabin sensing cameras and the cabin sensing cameras are calibrated by the same reference cargo ship, and a unified three-dimensional space coordinate system can be established.
Compared with most laser scanners or laser radars, the camera is more suitable for being installed and deployed on a ship unloader with strong vibration, particularly a lifting cylinder of the ship unloader because the internal structure does not contain a high-speed motor device. The camera can provide rich color and texture information required by ship character recognition, cabin material recognition, obstacle type recognition and the like, and is beneficial to optimizing a ship unloader control strategy and improving the operation efficiency of the ship unloader.
When the material taking heads enter the operation hatch, the in-cabin sensing cameras are triggered in real time to collect images in cargo ship cabins under various angles, all the images in the cargo ship cabins at the current moment are obtained, the positions of the bulkheads are positioned according to all the images in the cargo ship cabins at the current moment, a third planning route is generated based on a predefined material taking strategy and combined with the positions of the material taking heads and the bulkheads at the current moment, and the material taking heads are controlled to take materials in the operation cabin according to the third planning route.
According to the embodiment of the invention, the images in the cargo ship cabin are acquired in real time to carry out the material taking navigation of the material taking head in the stage that the material taking head enters the operation hatch, so that unmanned control can be carried out in the material taking operation process of the material taking head, and the operation efficiency of the ship unloader is improved.
In an alternative embodiment, the positioning the working area according to all the ship hull images specifically includes: performing image preprocessing, image registration and image fusion on all cargo ship body images to obtain target cargo ship body images; carrying out cargo ship detection according to the target cargo ship body image by adopting a machine vision technology; identifying the ship type, the bow and the stern of the cargo ship when the cargo ship is detected, so as to determine the cabin area of the cargo ship; generating a ship body three-dimensional point cloud according to the ship body image of the target cargo ship by adopting a binocular vision technology; carrying out three-dimensional space positioning on the cargo ship based on the ship body three-dimensional point cloud to obtain the position of the cargo ship; based on the position of the cargo ship, an area around the cargo ship, which is close to the cabin area, is selected as a working area, and the position of the working area is located.
As an example, at the stage of starting the operation of the ship unloader, when all the cargo ship hull images at the current moment are acquired, image preprocessing is performed on all the cargo ship hull images at the current moment, the image preprocessing includes image filtering, image denoising, image enhancement and the like, and all the cargo ship hull images after preprocessing are obtained.
And carrying out image registration and image fusion on all the preprocessed cargo ship body images, and splicing all the preprocessed cargo ship body images into one image to obtain the target cargo ship body image.
Wherein image registration (Image Registration) is a process of matching and overlaying at least two images acquired by different sensing cameras. Image Fusion (Image Fusion) refers to the process of processing the Image data about the same target acquired by two or more sensing cameras by Image processing, signal processing, computer vision, artificial intelligence and other technologies, extracting the beneficial information in the respective channels to the maximum extent, and finally synthesizing into a high-quality Image.
And carrying out cargo ship detection according to the target cargo ship hull image by adopting a machine vision technology, if the cargo ship is not detected, indicating that the wharf does not stop the cargo ship at the moment, continuously acquiring all cargo ship hull images at the next moment to carry out cargo ship detection, and if the cargo ship is detected, indicating that the wharf stops the cargo ship at the moment, continuously identifying the ship type, the bow and the stern of the cargo ship according to the target cargo ship hull image, so as to determine the cabin area of the cargo ship.
For target detection, the machine vision is to replace human eyes with a machine to measure and judge. The machine vision system converts the shot object into image signals through a machine vision product (namely an image shooting device, namely a CMOS and a CCD), the image signals are transmitted to a special image processing system to obtain the form information of the shot object, the image processing system converts the image signals into digital signals according to the information of pixel distribution, brightness, color and the like, various operations are carried out on the signals to extract the characteristics of the object, and then the on-site equipment action is controlled according to the judging result. For target identification, a pre-trained deep neural network model can be adopted to conduct classification identification on the ship body image of the target cargo ship, and the ship shape, the ship head and the ship tail of the cargo ship can be identified.
And generating a ship body three-dimensional point cloud according to the ship body image of the target cargo ship by adopting a binocular vision technology, and carrying out three-dimensional space positioning on the cargo ship based on the ship body three-dimensional point cloud to obtain the position of the cargo ship.
The binocular stereo vision (Binocular Stereo Vision) is an important form of machine vision, and is a method for acquiring three-dimensional geometric information of an object by acquiring two images of the object to be measured by using imaging equipment at different positions based on a parallax principle, calculating position deviation between corresponding points of the images, calculating depth of pixels according to limit constraint on the basis of calibrating acquisition camera parameters. The positioning target position based on the three-dimensional point cloud can adopt the existing three-dimensional point cloud positioning algorithm, and details are not repeated here.
In order to allow the pick heads to efficiently access the cabins of the cargo ships for picking, the area around the cargo ships, which is close to the cabin area, is selected as the working area based on the position of the cargo ships. The operation area is mainly determined by a predefined operation plan, the operation plan comprehensively considers the balance of the ship body and the arrangement sequence of all cabin sections, such as taking materials from 3 cabins, taking materials from 1 cabin and taking materials from 2 cabins, determining the current operation cabin section according to the operation plan, taking the hatch of the current operation cabin section as an operation hatch, combining the maximum operation distance of the material taking head, selecting an area which is close to the operation hatch around the cargo ship but has the distance between the material taking head and the operation hatch not exceeding the maximum operation distance of the material taking head as the operation area, and positioning the position of the operation area.
According to the embodiment of the invention, the two-dimensional image detection and the three-dimensional point cloud positioning are combined, and the approaching navigation of the cargo ship is carried out according to all the ship body images, so that the position of the cargo ship can be rapidly and accurately positioned, the operation area is reasonably selected, and the operation efficiency of the ship unloader is further improved.
In an alternative embodiment, the positioning the working area according to all the ship hull images specifically further includes: reconstructing a three-dimensional grid model of the cargo ship based on the three-dimensional point cloud of the ship body; determining the attitude of the cargo ship according to the three-dimensional grid model of the cargo ship; and adjusting a predefined material taking strategy according to the attitude of the cargo ship.
As an example, at the stage of starting operation of the ship unloader, when all cargo ship hull images at the current moment are subjected to image preprocessing, image registration and image fusion to obtain target cargo ship hull images, a binocular vision technology is adopted, when a ship three-dimensional point cloud is generated according to the target cargo ship hull images, a three-dimensional grid model of the cargo ship is reconstructed based on the ship three-dimensional point cloud, the posture of the cargo ship is determined according to the three-dimensional grid model of the cargo ship, a predefined taking strategy is adjusted according to the posture of the cargo ship, the influence of sea tides and surges on the cargo ship hull is conveniently evaluated according to the posture of the cargo ship in real time, the taking strategy is timely adjusted, and taking of the taking head is controlled based on the optimized taking strategy.
In order to avoid frequent adjustment of the material taking strategy, the ship body three-dimensional point cloud can be generated according to the ship body image of the target cargo ship obtained at the moment only after the ship unloader enters the operation area, the three-dimensional grid model of the cargo ship is rebuilt based on the ship body three-dimensional point cloud, the posture of the cargo ship is determined according to the three-dimensional grid model of the cargo ship, and the material taking strategy is adjusted according to the posture of the cargo ship.
According to the embodiment of the invention, the ship attitude is obtained in real time, and the material taking strategy is adjusted according to the ship attitude, so that the influence of tides and surges on the ship body can be evaluated in an auxiliary manner, and the material taking strategy is adjusted in time, thereby being beneficial to further improving the operation efficiency of the ship unloader.
In an alternative embodiment, the positioning the position of the operation hatch according to all the images outside the cargo ship cabin specifically includes: performing image preprocessing, image registration and image fusion on all the images outside the cargo hold to obtain a target image outside the cargo hold; performing cabin section identification according to the images outside the cabins of the target cargo ship by adopting a machine vision technology to obtain each cabin section of the cargo ship; generating a cabin section three-dimensional point cloud according to the external images of the target cargo ship cabin by adopting a multi-vision technology; carrying out three-dimensional space positioning on the hatches of each cabin section based on the cabin section three-dimensional point cloud to obtain the hatch positions of each cabin section; and selecting the operation cabin from all cabin sections based on a predefined operation plan, and acquiring the hatch position of the operation cabin as the position of the operation hatch.
As an example, at the stage of the ship unloader entering the working area, when all the cargo ship cabin images at the current moment are acquired, image preprocessing is performed on all the cargo ship cabin images at the current moment, wherein the image preprocessing comprises image filtering, image denoising, image enhancement and the like, so that all the cargo ship cabin images after preprocessing are obtained.
And carrying out image registration and image fusion on all the preprocessed cargo ship cabin outside images, and splicing all the preprocessed cargo ship cabin outside images into one image to obtain the target cargo ship cabin outside image.
And carrying out cabin section identification according to the images outside the cabins of the target cargo ship by adopting a machine vision technology, and determining the distribution condition of each cabin section of the cargo ship to obtain each cabin section of the cargo ship.
And generating a cabin three-dimensional point cloud according to the external images of the target cargo ship cabin by adopting a multi-vision technology, and respectively carrying out three-dimensional space positioning on the hatch of each cabin based on the cabin three-dimensional point cloud to obtain the hatch position of each cabin. Wherein the multi-vision system employs three or more imaging devices.
Based on a predefined operation plan, selecting an operation cabin from all cabins, for example, the operation plan is to take material from 3 cabins, then from 1 cabin and finally from 2 cabins, and the 3 cabins of the cargo ship are selected as the operation cabin, and the hatch position of the 3 cabins is obtained as the position of the operation hatch because the material from 1 cabin, 2 cabin and 3 cabin is not completely taken at the moment.
According to the embodiment of the invention, through combining two-dimensional image detection and three-dimensional point cloud positioning, the feeding and discharging navigation of the material taking head is performed according to all the images outside the cargo ship cabin, so that the position of the operation hatch can be rapidly and accurately positioned, and the operation efficiency of the ship unloader can be further improved.
In an alternative embodiment, the positioning the position of the operation hatch according to all the images outside the cargo ship cabin generates a second planned route for controlling the access of the pick-up head on the ship unloader to the operation hatch, and the method further includes: when the second planning route is generated, a multi-vision technology is adopted, the position of the obstacle on the second planning route is positioned according to the image outside the target cargo ship cabin, and the second planning route is adjusted according to the position of the obstacle on the second planning route.
As an example, in the stage that the ship unloader enters the working area, when the second planning route is generated, a multi-vision technology is adopted, the position of an obstacle on the second planning route is positioned according to the image outside the target cargo ship cabin, and the second planning route is adjusted according to the position of the obstacle on the second planning route, so that the problem that the material taking head cannot enter and exit the cabin accurately and safely due to the obstacle in the cabin entering and exiting process is effectively avoided.
According to the embodiment of the invention, the obstacle on the second planning route is detected and positioned, and the second planning route is adjusted according to the position of the obstacle on the second planning route, so that the safety of the material taking head in and out of the operation hatch can be ensured.
In an alternative embodiment, the positioning the position of the bulkhead according to the images in all cargo ship cabins specifically includes: performing image preprocessing, image registration and image fusion on all the images in the cargo ship cabins to obtain images in the target cargo ship cabins; dividing the bulkhead and the material level according to the images in the cabin of the target cargo ship by adopting a machine vision technology; generating a bulkhead three-dimensional point cloud according to an image in a target cargo ship cabin by adopting a multi-vision technology; and carrying out three-dimensional space positioning on the bulkhead based on the bulkhead three-dimensional point cloud to obtain the position of the bulkhead.
As an example, at the stage of the material taking head entering the operation hatch, when all the cargo ship cabin images at the current moment are acquired, image preprocessing is performed on all the cargo ship cabin images at the current moment, wherein the image preprocessing comprises image filtering, image denoising, image enhancement and the like, so as to obtain all the cargo ship cabin images after preprocessing.
And carrying out image registration and image fusion on all the preprocessed cargo ship cabin images, and splicing all the preprocessed cargo ship cabin images into one image to obtain the target cargo ship cabin image.
And (3) performing rapid segmentation and identification on the bulkhead and the material level according to the images in the cabin of the target cargo ship by adopting a machine vision technology.
The bulkhead three-dimensional point cloud is generated according to the images in the target cargo ship cabin by adopting a multi-vision technology, for example, the bulkhead three-dimensional point cloud is calculated by adopting an SLAM (Simultaneous Localization and Mapping) algorithm, and the bulkhead is positioned in three-dimensional space based on the bulkhead three-dimensional point cloud, so that the position of the bulkhead is obtained.
The main functions of the SLAM algorithm are as follows:
1. establishing a three-dimensional point cloud map of the cabin environment;
2. realizing the real-time positioning of the material taking head in the cabin environment;
3. the bulkhead, the material level and the obstacle can be positioned in the three-dimensional point cloud space by combining the image information of the bulkhead and the material level which are segmented in advance.
According to the embodiment of the invention, through combining two-dimensional image detection and three-dimensional point cloud positioning, the position of the bulkhead can be rapidly and accurately positioned by carrying out the material taking head and material taking navigation according to the images in all cargo ship cabins, and the operation efficiency of the ship unloader is further improved.
In an alternative embodiment, the generating a third planned route for controlling the pick head to pick materials in the operation cabin based on a predefined pick-up strategy according to the positions of the image positioning bulkheads in all cargo ship cabins further includes: and when the third planning route is generated, adopting a multi-vision technology, positioning the position of the obstacle on the third planning route according to the image in the cabin of the target cargo ship, and adjusting the third planning route according to the position of the obstacle on the third planning route.
As an example, in the stage that the material taking head enters the operation hatch, when the third planning route is generated, a multi-vision technology is adopted, an obstacle on the third planning route is positioned according to an image in the cabin of the target cargo ship, and the third planning route is adjusted according to the position of the obstacle on the third planning route, so that the problem that the material taking head cannot accurately and safely take materials due to the obstacle in the material taking process is effectively avoided.
According to the embodiment of the invention, the obstacle on the third planning route is detected and positioned, and the third planning route is adjusted according to the position of the obstacle on the third planning route, so that the safety material taking of the material taking head can be ensured.
In an alternative embodiment, the positioning the position of the bulkhead according to the images in all cargo ship cabins specifically further includes: performing material type identification according to the images in the cabin of the target cargo ship by adopting a machine vision technology to obtain the material type of the cargo; generating a material level three-dimensional point cloud according to an image in a target cargo ship cabin by adopting a multi-vision technology; reconstructing a three-dimensional grid model of the cargo material based on the three-dimensional point cloud of the material surface; and adjusting a material taking strategy by combining the material type of the cargo material and the three-dimensional grid model of the cargo material.
As an example, when the pick-up head enters the operation hatch, and the target cargo ship cabin image is obtained according to all cargo ship cabin images at the current moment, the machine vision technology is also adopted to perform material type identification according to the target cargo ship cabin image, so as to obtain the material type of the cargo. For target recognition, a pre-trained deep neural network model can be adopted to carry out classification recognition on images in a cabin of a target cargo ship, and the material type of cargo materials can be recognized.
The three-dimensional point cloud of the material level is generated according to the images in the target cargo ship cabin by adopting a multi-vision technology, for example, SLAM (Simultaneous Localization and Mapping) algorithm is adopted to calculate the three-dimensional point cloud of the material level, a three-dimensional grid model of the cargo is rebuilt based on the three-dimensional point cloud of the material level, the material taking strategy is adjusted by combining the material type of the cargo and the three-dimensional grid model of the cargo, the material taking strategy is conveniently adjusted in real time according to the material type of the cargo and the placing posture of the cargo, and the material taking head is controlled to take materials based on the optimized material taking strategy.
According to the embodiment of the invention, the material taking strategy is adjusted by combining the material type of the material identified from the image and the three-dimensional grid of the material reconstructed by using the three-dimensional point cloud of the material surface, so that the material taking strategy can be further optimized, and the safety and the high efficiency of the material taking head are ensured.
Referring to fig. 6, fig. 6 is a schematic structural diagram of an unmanned operation device of a ship unloader according to a second embodiment of the present invention. The second embodiment of the present invention provides an unmanned operation device for a ship unloader, comprising: the image acquisition module 201 is used for acquiring the cargo ship body images acquired by each ship body sensing camera on the ship unloader in real time to acquire all cargo ship body images at the current moment; the navigation operation module 202 is configured to generate a first planned route for controlling the ship unloader to enter and exit the operation area according to the positions of the operation area located by the ship body images of all the cargo ships.
In an alternative embodiment, the image acquisition module 201 is further configured to acquire, in real time, the images outside the cargo ship cabins acquired by the external sensing cameras on the ship unloader after the ship unloader enters the operation area, so as to obtain all the images outside the cargo ship cabins at the current moment; the navigation operation module 202 is further configured to locate the position of the operation hatch according to all the images outside the cargo ship cabin, and generate a second planned route for controlling the access of the pick-up head on the ship unloader to the operation hatch.
In an alternative embodiment, the image obtaining module 201 is further configured to obtain, in real time, the cargo ship cabin images collected by the sensing cameras in each cabin on the ship unloader after the material taking head enters the operation hatch, so as to obtain all cargo ship cabin images at the current moment; the navigation operation module 202 is further configured to generate a third planned route for controlling the pick head to pick materials in the operation cabin based on a predefined pick-up strategy according to the positions of the image positioning bulkheads in all cargo ship cabins.
In an alternative embodiment, the positioning the working area according to all the ship hull images specifically includes: performing image preprocessing, image registration and image fusion on all cargo ship body images to obtain target cargo ship body images; carrying out cargo ship detection according to the target cargo ship body image by adopting a machine vision technology; identifying the ship type, the bow and the stern of the cargo ship when the cargo ship is detected, so as to determine the cabin area of the cargo ship; generating a ship body three-dimensional point cloud according to the ship body image of the target cargo ship by adopting a binocular vision technology; carrying out three-dimensional space positioning on the cargo ship based on the ship body three-dimensional point cloud to obtain the position of the cargo ship; based on the position of the cargo ship, an area around the cargo ship, which is close to the cabin area, is selected as a working area, and the position of the working area is located.
In an alternative embodiment, the positioning the working area according to all the ship hull images specifically further includes: reconstructing a three-dimensional grid model of the cargo ship based on the three-dimensional point cloud of the ship body; determining the attitude of the cargo ship according to the three-dimensional grid model of the cargo ship; and adjusting a predefined material taking strategy according to the attitude of the cargo ship.
In an alternative embodiment, the positioning the position of the operation hatch according to all the images outside the cargo ship cabin specifically includes: performing image preprocessing, image registration and image fusion on all the images outside the cargo hold to obtain a target image outside the cargo hold; performing cabin section identification according to the images outside the cabins of the target cargo ship by adopting a machine vision technology to obtain each cabin section of the cargo ship; generating a cabin section three-dimensional point cloud according to the external images of the target cargo ship cabin by adopting a multi-vision technology; carrying out three-dimensional space positioning on the hatches of each cabin section based on the cabin section three-dimensional point cloud to obtain the hatch positions of each cabin section; and selecting the operation cabin from all cabin sections based on a predefined operation plan, and acquiring the hatch position of the operation cabin as the position of the operation hatch.
In an alternative embodiment, the positioning the position of the operation hatch according to all the images outside the cargo ship cabin generates a second planned route for controlling the access of the pick-up head on the ship unloader to the operation hatch, and the method further includes:
When the second planning route is generated, a multi-vision technology is adopted, the position of the obstacle on the second planning route is positioned according to the image outside the target cargo ship cabin, and the second planning route is adjusted according to the position of the obstacle on the second planning route.
In an alternative embodiment, the positioning the position of the bulkhead according to the images in all cargo ship cabins specifically includes: performing image preprocessing, image registration and image fusion on all the images in the cargo ship cabins to obtain images in the target cargo ship cabins; dividing the bulkhead and the material level according to the images in the cabin of the target cargo ship by adopting a machine vision technology; generating a bulkhead three-dimensional point cloud according to an image in a target cargo ship cabin by adopting a multi-vision technology; and carrying out three-dimensional space positioning on the bulkhead based on the bulkhead three-dimensional point cloud to obtain the position of the bulkhead.
In an alternative embodiment, the generating a third planned route for controlling the pick head to pick materials in the operation cabin based on a predefined pick-up strategy according to the positions of the image positioning bulkheads in all cargo ship cabins further includes: and when the third planning route is generated, adopting a multi-vision technology, positioning the position of the obstacle on the third planning route according to the image in the cabin of the target cargo ship, and adjusting the third planning route according to the position of the obstacle on the third planning route.
In an alternative embodiment, the positioning the position of the bulkhead according to the images in all cargo ship cabins specifically further includes: performing material type identification according to the images in the cabin of the target cargo ship by adopting a machine vision technology to obtain the material type of the cargo; generating a material level three-dimensional point cloud according to an image in a target cargo ship cabin by adopting a multi-vision technology; reconstructing a three-dimensional grid model of the cargo material based on the three-dimensional point cloud of the material surface; and adjusting a material taking strategy by combining the material type of the cargo material and the three-dimensional grid model of the cargo material.
The implementation process of the functions and roles of each module in the above device is specifically shown in the implementation process of the corresponding steps in the above method, and will not be described herein again.
Referring to fig. 7, fig. 7 is a schematic structural diagram of an electronic device according to a third embodiment of the present invention. A third embodiment of the invention provides an electronic device 30 comprising a processor 301, a memory 302 and a computer program stored in the memory 302 and configured to be executed by the processor 301; the memory 302 is coupled to the processor 301, and the processor 301 executes the computer program to implement the unmanned operation method of the ship unloader according to the first embodiment of the present invention, and achieve the same advantages as the unmanned operation method.
The processor 301 may implement the method according to any embodiment of the unmanned ship unloader operation method according to the first embodiment of the present invention when reading the computer program from the memory 302 via the bus 303 and executing the computer program.
The processor 301 may process digital signals and may include various computing structures. Such as a complex instruction set computer architecture, a reduced instruction set computer architecture, or an architecture that implements a combination of instruction sets. In some examples, processor 301 may be a microprocessor.
Memory 302 may be used for storing instructions to be executed by processor 301 or data relating to the execution of instructions. Such instructions and/or data may include code to implement some or all of the functions of one or more of the modules described in embodiments of the present invention. The processor 301 of the disclosed embodiment may be configured to execute instructions in the memory 302 to implement the unmanned operation method of the ship unloader according to the first embodiment of the present invention. Memory 302 includes dynamic random access memory, static random access memory, flash memory, optical memory, or other memory known to those skilled in the art.
A fourth embodiment of the present invention provides a computer-readable storage medium including a stored computer program; the unmanned operation method of the ship unloader, which is described in the first embodiment of the invention, is executed by the equipment where the computer readable storage medium is located when the computer program runs, and the same beneficial effects as the unmanned operation method of the ship unloader can be achieved.
In summary, the embodiment of the invention provides a method, a device, equipment and a storage medium for unmanned operation of a ship unloader, wherein the unmanned operation method of the ship unloader comprises the following steps: acquiring cargo ship body images acquired by all ship body sensing cameras on the ship unloader in real time to acquire all cargo ship body images at the current moment; and according to the positions of all the ship body images positioning operation areas, generating a first planning route for controlling the ship unloader to enter and exit the operation areas. According to the embodiment of the invention, the ship approaching navigation is performed by acquiring the ship body images of the group of ships in real time at the beginning operation stage of the ship unloader, so that unmanned control can be performed in the operation process of approaching the ship of the ship unloader, and the operation efficiency of the ship unloader is improved.
In the several embodiments provided in the present invention, it should be understood that the disclosed apparatus and method may be implemented in other manners. The apparatus embodiments described above are merely illustrative, for example, flow diagrams and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of apparatus, methods and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
In addition, functional modules in the embodiments of the present invention may be integrated together to form a single part, or each module may exist alone, or two or more modules may be integrated to form a single part.
The functions, if implemented in the form of software functional modules and sold or used as a stand-alone product, may be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention may be embodied essentially or in a part contributing to the prior art or in a part of the technical solution, in the form of a software product stored in a storage medium, comprising several instructions for causing a computer device (which may be a personal computer, a server, a network device, etc.) to perform all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a random access Memory (RAM, random Access Memory), a magnetic disk, or an optical disk, or other various media capable of storing program codes.
The foregoing is merely illustrative of the present invention, and the present invention is not limited thereto, and any person skilled in the art will readily appreciate variations or alternatives within the scope of the present invention. Therefore, the protection scope of the invention is subject to the protection scope of the claims.
Claims (13)
1. An unmanned operation method of a ship unloader is characterized by comprising the following steps:
acquiring cargo ship body images acquired by all ship body sensing cameras on the ship unloader in real time, and acquiring all the cargo ship body images at the current moment;
and according to the positions of all the cargo ship body images in the positioning operation area, generating a first planning route for controlling the ship unloader to enter and exit the operation area.
2. The unmanned ship unloader operation method according to claim 1, further comprising:
when the ship unloader enters the operation area, acquiring cargo ship cabin outer images acquired by all cabin outer sensing cameras on the ship unloader in real time, and acquiring all cargo ship cabin outer images at the current moment;
and according to the positions of all the cargo ship cabin outside image positioning operation hatches, generating a second planning route for controlling the material taking heads on the ship unloader to come in and go out of the operation hatches.
3. The unmanned ship unloader operation method according to claim 2, further comprising:
when the material taking head enters the operation hatch, acquiring cargo ship cabin images acquired by sensing cameras in each cabin on the ship unloader in real time, and obtaining all cargo ship cabin images at the current moment;
And generating a third planning route for controlling the material taking head to take materials in the operation cabin based on a predefined material taking strategy according to the positions of all the image positioning bulkheads in the cargo ship cabin.
4. The unmanned operation method of the ship unloader according to claim 1, wherein the positioning of the operation area according to all the ship body images specifically comprises:
performing image preprocessing, image registration and image fusion on all the cargo ship body images to obtain target cargo ship body images;
performing cargo ship detection according to the target cargo ship body image by adopting a machine vision technology;
identifying a ship form, a bow and a stern of the cargo ship upon detection of the cargo ship to determine a hold area of the cargo ship;
generating a ship body three-dimensional point cloud according to the ship body image of the target cargo ship by adopting a binocular vision technology;
carrying out three-dimensional space positioning on the cargo ship based on the ship body three-dimensional point cloud to obtain the position of the cargo ship;
and selecting an area around the cargo ship, which is close to the cabin area, as the operation area based on the position of the cargo ship, and positioning the position of the operation area.
5. The unmanned ship unloader operation method according to claim 4, wherein the positioning the position of the operation area according to all the ship body images, in particular, further comprises:
Reconstructing a three-dimensional grid model of the cargo ship based on the ship body three-dimensional point cloud;
determining the attitude of the cargo ship according to the three-dimensional grid model of the cargo ship;
and adjusting a predefined material taking strategy according to the attitude of the cargo ship.
6. The unmanned operation method of the ship unloader according to claim 2, wherein the positioning the position of the operation hatch according to all the cargo ship cabin exterior images specifically comprises:
performing image preprocessing, image registration and image fusion on all the cargo ship cabin images to obtain target cargo ship cabin images;
performing cabin section identification according to the images outside the target cargo ship cabin by adopting a machine vision technology to obtain each cabin section of the cargo ship;
generating a cabin section three-dimensional point cloud according to the target cargo ship cabin external image by adopting a multi-vision technology;
carrying out three-dimensional space positioning on the hatches of the cabin segments based on the three-dimensional point cloud of the cabin segments to obtain the hatch positions of the cabin segments;
and selecting a working cabin section from all the cabin sections based on a predefined working plan, and acquiring the hatch position of the working cabin section as the position of the working hatch.
7. The unmanned ship unloader operation method according to claim 6, wherein the positioning the position of the operation hatch according to all the cargo ship outside cabin images generates a second planned route for controlling the access of the pick-up head on the ship unloader to the operation hatch, further comprising:
And when the second planning route is generated, adopting a multi-vision technology, positioning the position of the obstacle on the second planning route according to the image outside the target cargo ship cabin, and adjusting the second planning route according to the position of the obstacle on the second planning route.
8. A method of unmanned ship unloader operation according to claim 3, wherein the positioning of the bulkhead according to the image of all cargo tanks comprises:
performing image preprocessing, image registration and image fusion on all the images in the cargo ship cabin to obtain an image in the target cargo ship cabin;
dividing the bulkhead and the material level according to the image in the cabin of the target cargo ship by adopting a machine vision technology;
generating a bulkhead three-dimensional point cloud according to the image in the target cargo ship cabin by adopting a multi-vision technology;
and carrying out three-dimensional space positioning on the bulkhead based on the bulkhead three-dimensional point cloud to obtain the position of the bulkhead.
9. The unmanned ship unloader operation method according to claim 8, wherein the generating a third planned route for controlling the pick head to pick in the operation cabin based on a predefined pick-up strategy according to the positions of all the cargo ship cabin image positioning bulkheads, further comprises:
And when the third planning route is generated, adopting a multi-vision technology, positioning the position of the obstacle on the third planning route according to the image in the cabin of the target cargo ship, and adjusting the third planning route according to the position of the obstacle on the third planning route.
10. The unmanned ship unloader operation method according to claim 8, wherein the positioning of the bulkhead according to the images of all cargo ship cabins, in particular, further comprises:
performing material type identification according to the images in the target cargo ship cabin by adopting a machine vision technology to obtain the material type of the cargo;
generating a material level three-dimensional point cloud according to the image in the target cargo ship cabin by adopting a multi-vision technology;
reconstructing a three-dimensional grid model of the cargo material based on the three-dimensional point cloud of the material surface;
and adjusting the material taking strategy by combining the material type of the cargo material and the three-dimensional grid model of the cargo material.
11. An unmanned operation device of ship unloader, characterized by comprising:
the image acquisition module is used for acquiring cargo ship body images acquired by all ship body sensing cameras on the ship unloader in real time to acquire all the cargo ship body images at the current moment;
and the navigation operation module is used for positioning the position of the operation area according to all the cargo ship body images and generating a first planning route for controlling the ship unloader to come in or go out of the operation area.
12. An electronic device comprising a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor; the memory is coupled to the processor and the processor, when executing the computer program, implements the unmanned operation method of the ship unloader according to any one of claims 1 to 10.
13. A computer readable storage medium, wherein the computer readable storage medium comprises a stored computer program; wherein the computer program, when run, controls the apparatus in which the computer readable storage medium is located to perform the unmanned operation method of the ship unloader according to any one of claims 1 to 10.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310783592.8A CN116700275A (en) | 2023-06-28 | 2023-06-28 | Unmanned operation method, unmanned operation device, unmanned operation equipment and unmanned operation storage medium for ship unloader |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310783592.8A CN116700275A (en) | 2023-06-28 | 2023-06-28 | Unmanned operation method, unmanned operation device, unmanned operation equipment and unmanned operation storage medium for ship unloader |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116700275A true CN116700275A (en) | 2023-09-05 |
Family
ID=87827542
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310783592.8A Pending CN116700275A (en) | 2023-06-28 | 2023-06-28 | Unmanned operation method, unmanned operation device, unmanned operation equipment and unmanned operation storage medium for ship unloader |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116700275A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117350177A (en) * | 2023-12-05 | 2024-01-05 | 西安热工研究院有限公司 | Training method and device for ship unloader path generation model, electronic equipment and medium |
-
2023
- 2023-06-28 CN CN202310783592.8A patent/CN116700275A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117350177A (en) * | 2023-12-05 | 2024-01-05 | 西安热工研究院有限公司 | Training method and device for ship unloader path generation model, electronic equipment and medium |
| CN117350177B (en) * | 2023-12-05 | 2024-03-22 | 西安热工研究院有限公司 | Training method and device for ship unloader path generation model, electronic equipment and medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| KR102231286B1 (en) | Ship surrounding monitoring device and ship surrounding monitoring method | |
| JP7252943B2 (en) | Object detection and avoidance for aircraft | |
| CN107844750B (en) | Water surface panoramic image target detection and identification method | |
| KR102530691B1 (en) | Device and method for monitoring a berthing | |
| Wei et al. | A non-contact measurement method of ship block using image-based 3D reconstruction technology | |
| US20220024549A1 (en) | System and method for measuring the distance to an object in water | |
| Park et al. | Development of an unmanned surface vehicle system for the 2014 Maritime RobotX Challenge | |
| JP2018177074A (en) | Autonomous type underwater robot and control method for the same | |
| KR101454855B1 (en) | Ship hull inspection and analysys system, and method thereof | |
| CN103336282B (en) | A kind of cabin locating device and localization method thereof automatically | |
| CN112150388A (en) | Continuous ship unloader ship and material identification sensing method | |
| KR20210110671A (en) | An image processing system, an image processing method, a method for generating a pre-trained model, and a data set for training | |
| CN116700275A (en) | Unmanned operation method, unmanned operation device, unmanned operation equipment and unmanned operation storage medium for ship unloader | |
| KR102520844B1 (en) | Method and device for monitoring harbor and ship considering sea level | |
| KR102530847B1 (en) | Method and device for monitoring harbor and ship | |
| Westman et al. | Wide aperture imaging sonar reconstruction using generative models | |
| KR102265980B1 (en) | Device and method for monitoring ship and port | |
| CN107831760A (en) | Robot barrier thing processing system and method | |
| CN110023947A (en) | The method and apparatus of vehicle-periphery view are generated in the car | |
| CN115131720A (en) | Ship berthing assisting method based on artificial intelligence | |
| CN116540724A (en) | Unmanned ship control method, device and system | |
| KR20230074438A (en) | Device and method for monitoring ship and port | |
| Karabchevsky et al. | AUV real-time acoustic vertical plane obstacle detection and avoidance | |
| CN111060091A (en) | Robot navigation system | |
| Saad et al. | Hybrid coordination strategy of a group of cooperating autonomous underwater vehicles |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |