CN111310646B - Method for improving navigation safety based on real-time detection of remote images - Google Patents
Method for improving navigation safety based on real-time detection of remote images Download PDFInfo
- Publication number
- CN111310646B CN111310646B CN202010089428.3A CN202010089428A CN111310646B CN 111310646 B CN111310646 B CN 111310646B CN 202010089428 A CN202010089428 A CN 202010089428A CN 111310646 B CN111310646 B CN 111310646B
- Authority
- CN
- China
- Prior art keywords
- video image
- ship
- control center
- real
- image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 35
- 238000011897 real-time detection Methods 0.000 title claims abstract description 23
- 238000001514 detection method Methods 0.000 claims abstract description 33
- 238000013527 convolutional neural network Methods 0.000 claims abstract description 16
- 238000012549 training Methods 0.000 claims abstract description 14
- 230000005540 biological transmission Effects 0.000 claims abstract description 12
- 230000006835 compression Effects 0.000 claims description 4
- 238000007906 compression Methods 0.000 claims description 4
- 230000000694 effects Effects 0.000 description 3
- 238000011161 development Methods 0.000 description 2
- 238000012360 testing method Methods 0.000 description 2
- 206010063385 Intellectualisation Diseases 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 239000003086 colorant Substances 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 230000004927 fusion Effects 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
- H04L65/60—Network streaming of media packets
- H04L65/70—Media network packetisation
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L69/00—Network arrangements, protocols or services independent of the application payload and not provided for in the other groups of this subclass
- H04L69/16—Implementation or adaptation of Internet protocol [IP], of transmission control protocol [TCP] or of user datagram protocol [UDP]
- H04L69/164—Adaptation or special uses of UDP protocol
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N7/00—Television systems
- H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Physics & Mathematics (AREA)
- Signal Processing (AREA)
- Theoretical Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- General Physics & Mathematics (AREA)
- General Health & Medical Sciences (AREA)
- Biophysics (AREA)
- Evolutionary Computation (AREA)
- Computational Linguistics (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Biomedical Technology (AREA)
- Artificial Intelligence (AREA)
- Life Sciences & Earth Sciences (AREA)
- Health & Medical Sciences (AREA)
- Computer Security & Cryptography (AREA)
- Traffic Control Systems (AREA)
Abstract
The invention belongs to the technical field of ship navigation, and particularly relates to a method for improving navigation safety based on real-time detection of remote images. Comprising the following steps: acquiring a video image shot by a ship end camera of the current ship; identifying a target object in a video image based on a pre-trained convolutional neural network model, and taking the position coordinates of the target object as detection data; the pre-trained convolutional neural network model is a model which is obtained by training based on pictures of ships, lands, buoys and the like in a training set and corresponding judgment results; the video image and the detection data are respectively transmitted to a control center, the control center fuses the video image and the detection data, whether the current ship needs to change a path or not is judged, and if so, an instruction is sent to the current ship; and receiving an instruction of a control center, and re-planning the route according to the current position of the ship and preset destination information. And the video image and detection data obtained by the video image are respectively transmitted to a control center, so that the stability and quality of information transmission are improved.
Description
Technical Field
The invention belongs to the technical field of ship navigation, and particularly relates to a method for improving navigation safety based on real-time detection of remote images.
Background
The ship intellectualization is the development trend of current ships, and in order to ensure the safety of the intelligent ship during sailing, accurate real-time detection and identification can be performed on the periphery of the ship in real time, so that the sailing safety coefficient is improved.
At present, the current surrounding conditions of the ship end are generally obtained through AIS, radar and other modes, and anti-collision operations are carried out on the ship. However, the AIS data cannot acquire an object which does not send AIS information, the radar only can show the outline of the object, and the objects such as reefs, land, ships and the like cannot be distinguished when the outlines are similar.
In order to avoid the situation, the current surrounding situation of the ship end can be obtained through the ship end camera shooting. However, video image transmission is low in stability and prone to errors.
Disclosure of Invention
First, the technical problem to be solved
Aiming at the existing technical problems, the invention provides a method for improving navigation safety based on real-time detection of remote images, which is used for respectively transmitting video images and detection data obtained from the video images to a control center and improving the stability and quality of information transmission.
(II) technical scheme
In order to achieve the above purpose, the main technical scheme adopted by the invention comprises the following steps:
the invention provides a method for improving navigation safety based on real-time detection of remote images, which comprises the following steps:
a1, acquiring a video image shot by a ship end shooting device of a current ship;
a2, identifying a target object in a video image based on a pre-trained convolutional neural network model, and taking the position coordinates of the target object as detection data;
the target is a marked target;
the pre-trained convolutional neural network model is a model obtained by training based on the training set ship, land and buoy pictures and corresponding judgment results;
a3, respectively transmitting the video image and the detection data to a control center, fusing the video image and the detection data by the control center, judging whether the current ship needs to change a path, and if so, sending an instruction to the current ship;
and step A4, receiving an instruction of a control center, and re-planning the route according to the current position of the ship and preset destination information.
Further, before the step A3, the method includes:
performing ffmpeg encoding on the video image to obtain a key frame image, and performing H265 encoding compression on the key frame image;
the key frame image is a video image that changes relative to a previous video image.
Further, the method for judging the key frame image comprises the following steps:
and judging the RGB data of the current video image and the previous video image, and if the RGB data are changed, judging that the current video image is a key frame image.
Further, in the step A3, the control center decodes the compressed key frame image and fuses the decoded key frame image with the detection data.
Further, in the step A1, the ship-end shooting device shoots and obtains the video image by adopting a frame skip transmission mode.
Further, the video image and the detection data are transmitted to the control center through UDP.
Further, the targets include vessels, land and buoys.
Further, the control center is located onshore or on another vessel.
The invention also provides a device for improving navigation safety based on real-time detection of remote images, which comprises a memory and a processor, wherein the memory stores instructions, and the processor executes the instructions stored in the memory, and particularly comprises the steps of executing the method according to any scheme.
The invention further comprises an intelligent ship, comprising a ship end camera device and the device for improving navigation safety based on real-time detection of remote images.
(III) beneficial effects
(1) In the method provided by the invention, the video image and the detection data obtained from the video image are respectively transmitted to the control center, so that the error probability of the video image and the detection data is reduced, and the transmission stability and the transmission quality of the video image and the detection data are improved.
(2) And the video image is encoded by applying the ffmpeg encoding technology, so that the transmission speed of the video image is improved.
(3) Compared with the method for acquiring the surrounding situation of the current ship end in the modes of AIS, radar and the like, the method provided by the invention can acquire the target object information more clearly and provide safety support for ship navigation.
Drawings
FIG. 1 is a flow chart of a method provided by the present invention;
FIG. 2 is a diagram showing the effect of identifying objects in the present invention;
fig. 3 is a flowchart of the control center acquiring video images in the present invention.
Detailed Description
The invention will be better explained by the following detailed description of the embodiments with reference to the drawings.
Example 1
The embodiment provides a method for improving navigation safety based on real-time detection of remote images, as shown in fig. 1, comprising the following steps:
the invention provides a method for improving navigation safety based on real-time detection of remote images, which comprises the following steps:
a1, acquiring a video image shot by a ship end shooting device of a current ship;
a2, identifying a target object in a video image based on a pre-trained convolutional neural network model, and taking the position coordinates of the target object as detection data;
the target is a marked target;
the pre-trained convolutional neural network model is a model obtained by training based on the training set ship, land and buoy pictures and corresponding judgment results;
a3, respectively transmitting the video image and the detection data to a control center, fusing the video image and the detection data by the control center, judging whether the current ship needs to change a path, and if so, sending an instruction to the current ship;
and step A4, receiving an instruction of a control center, and re-planning the route according to the current position of the ship and preset destination information.
Further, before the step A3, the method includes:
performing ffmpeg encoding on the video image to obtain a key frame image, and performing H265 encoding compression on the key frame image;
the key frame image is a video image that changes relative to a previous video image.
Further, the method for judging the key frame image comprises the following steps:
and judging the RGB data of the current video image and the previous video image, and if the RGB data are changed, judging that the current video image is a key frame image.
Further, in the step A3, the control center decodes the compressed key frame image and fuses the decoded key frame image with the detection data.
Further, the video image and the detection data are transmitted to the control center through UDP.
Further, the targets include vessels, land and buoys.
Further, the control center is located onshore or on another vessel.
The invention also provides a device for improving navigation safety based on real-time detection of remote images, which comprises a memory and a processor, wherein the memory stores instructions, and the processor executes the instructions stored in the memory, and particularly comprises the method for executing the scheme.
The invention further comprises an intelligent ship, comprising a ship end camera device and the device for improving navigation safety based on real-time detection of remote images.
Example 2
The embodiment provides a method for detecting remote images in real time to improve navigation safety, which comprises the following steps:
100. the control center sends an instruction for opening the camera shooting equipment to the ship end;
101. and the ship end receives the instruction, runs software remotecient and is based on the rtsp format video image acquired by the Zhejiang Dahua camera arranged at the ship end.
102. As shown in fig. 2, a ship in a video image is identified based on a convolutional neural network model trained in advance, and position coordinates of the ship are acquired as detection data.
The training process of the convolutional neural network model is as follows:
s1, acquiring a ship picture in a training set;
s2, inputting the ship pictures in the training set and the pre-known judgment results into a convolutional neural network model to obtain training parameters and a trained convolutional neural network model;
specifically, acquiring ship confidence coefficient according to the ship picture in the training set, and inputting the ship confidence coefficient and a pre-known judgment result into a convolutional neural network model to obtain training parameters and a trained convolutional neural network model;
the judgment basis is as follows: if the ship confidence coefficient is greater than or equal to the credible threshold value, judging that the ship is the ship;
s3, acquiring a ship picture in the test set;
and S4, inputting the ship pictures in the test set into a trained convolutional neural network model, and outputting a judging result.
103. And encoding the video image by using a ffmpeg encoding technology to obtain a key frame image, carrying out H265 encoding compression on the key frame image, transmitting the key frame image to a control center through UDP, and transmitting detection data to the control center through UDP in the form of a data packet. The header of the data packet can be used to distinguish different objects, such as ships, land, buoys, etc.
The key frame image is a video image which is changed relative to a previous video image, and the judging method comprises the following steps:
and judging the RGB data of the current video image and the previous video image, and if the RGB data are changed, judging that the current video image is a key frame image.
The principle is as follows: the image is composed of a plurality of pixel points, each pixel point has self RGB color development, so that the front image and the rear image are compared, different pixel point positions of the front RGB data and the rear RGB data and RGB data needing to be updated are transmitted, and the receiving end updates the corresponding RGB data according to the pixel point positions, thereby achieving the purpose of image updating. For example, 1080P images have 1920×1080 pixels, and if the entire image is transmitted, 1920×1080×3 data (RGB three colors) are required to be transmitted each time. In practice, the front and rear images may only have hundreds of pixels different (the image is unchanged because of the light problem), and even if the camera is rotated, the number of pixels changed is relatively large, and the transmission data may be relatively large, but the maximum data may not exceed 1920×1080×3.
Thus, key frames are acquired and transmitted through the ffmpeg coding technology, so that the transmission speed is improved and the traffic is saved.
104. The control center receives the compressed key frame image and the detection data in the form of the data packet, decompresses the compressed key frame image by utilizing a decoding means corresponding to H265, reads out the detection data by utilizing an analysis format corresponding to the data packet, fuses the decompressed video image with the read-out detection data to obtain real-time ship surrounding environment information, judges whether the current ship needs to change a path, and if so, sends an instruction to the current ship.
Specifically, the fusion includes: the control center decompresses the key frame image and directly outputs the key frame image to be displayed on the computer interface, and the position coordinates corresponding to the detection data are displayed on the computer interface in a rectangular frame mode.
Further, as shown in fig. 3, the control center may also determine whether to continue to receive the video image transmitted from the ship end.
105. And the current ship receives the instruction of the control center and re-plans the route according to the position of the current ship and preset destination information.
Example 3
The method for detecting the remote image in real time to improve navigation safety provided by the invention is different from the method in the embodiment 2 in that:
101. and running software RemoteClient based on the rtsp format video image acquired by the spherical camera arranged at the ship end.
Because the camera of the dome camera can rotate horizontally by 360 degrees and vertically by 180 degrees to acquire video images, the stability of detection data is ensured in the rotation process. Because the rotation of the dome camera is very fast, the definition of the picture is reduced or the recognition effect of the ship-end convolutional neural network model on the current rotation picture is reduced, in order to avoid or reduce the situation, the processing of the dome camera when in rotation is frame skipping transmission (stopping one frame when transmitting one frame of data, which is equivalent to transmitting 12.5 frames per second), and the reduction of the frame rate when the dome camera rotates has no influence on the watching effect of the video of the control center by combining the configuration of the dome camera (which can set how many frames of image data are read per second by the dome camera, and the invention sets 25 frames).
The accuracy of the detection data and the stability of the video data during rotation of the rotatable image pickup device are ensured by the frame skip transmission method.
The technical principles of the present invention have been described above in connection with specific embodiments, which are provided for the purpose of explaining the principles of the present invention and are not to be construed as limiting the scope of the present invention in any way. Other embodiments of the invention will be apparent to those skilled in the art from consideration of this specification without undue burden.
Claims (10)
1. The method for improving navigation safety based on real-time detection of the remote image is characterized by comprising the following steps:
a1, receiving an instruction of opening an image pickup device sent to a ship end by a control center, and acquiring a video image shot by the ship end image pickup device of the current ship;
a2, identifying a target object in a video image based on a pre-trained convolutional neural network model, acquiring a position coordinate of the target object, and taking the position coordinate of the target object as detection data;
the target is a marked target;
the pre-trained convolutional neural network model is a model obtained by training based on the training set ship, land and buoy pictures and corresponding judgment results;
a3, respectively transmitting the video image and the detection data to a control center, fusing the video image and the detection data by the control center, judging whether the current ship needs to change a path, and if so, sending an instruction to the current ship;
the control center fuses the video image and the detection data, the control center outputs the video image and displays the video image on a computer interface, and the position coordinates corresponding to the detection data are displayed on the computer interface in a rectangular frame mode;
and step A4, receiving an instruction of a control center, and re-planning the route according to the current position of the ship and preset destination information.
2. The method for improving navigation safety based on real-time detection of remote images according to claim 1, comprising, before said step A3:
performing ffmpeg encoding on the video image to obtain a key frame image, and performing H265 encoding compression on the key frame image;
the key frame image is a video image that changes relative to a previous video image.
3. The method for improving navigation safety based on real-time detection of remote images according to claim 2, wherein the method for determining the key frame image comprises:
and judging the RGB data of the current video image and the previous video image, and if the RGB data are changed, judging that the current video image is a key frame image.
4. The method for improving navigation safety based on real-time detection of remote images according to claim 3, wherein in the step A3, the control center decodes the compressed key frame image and fuses the decoded key frame image with the detection data.
5. The method for improving navigation safety based on real-time detection of remote images according to claim 1, wherein in the step A1, the ship-side shooting device shoots and obtains the video image by adopting a frame skip transmission mode.
6. The method for improving voyage safety based on real-time detection of remote images of claim 1, wherein the video image and the detection data are transmitted to a control center via UDP.
7. The method for improving navigational safety based on real-time detection of remote images according to claim 1, wherein the objects comprise vessels, land and buoys.
8. The method for improving navigational security based on real-time detection of remote images according to claim 1, wherein the control center is located on shore or on other vessels.
9. A device for improving navigation safety based on real-time detection of remote images is characterized by comprising a memory and a processor,
the memory stores instructions and the processor executes the instructions stored in the memory, including in particular the method of any one of the preceding claims 1 to 8.
10. An intelligent ship comprising a ship end camera device, characterized by further comprising the device for improving navigation safety based on real-time detection of remote images according to claim 9.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010089428.3A CN111310646B (en) | 2020-02-12 | 2020-02-12 | Method for improving navigation safety based on real-time detection of remote images |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010089428.3A CN111310646B (en) | 2020-02-12 | 2020-02-12 | Method for improving navigation safety based on real-time detection of remote images |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111310646A CN111310646A (en) | 2020-06-19 |
| CN111310646B true CN111310646B (en) | 2023-11-21 |
Family
ID=71145684
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010089428.3A Active CN111310646B (en) | 2020-02-12 | 2020-02-12 | Method for improving navigation safety based on real-time detection of remote images |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111310646B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116168333B (en) * | 2023-04-20 | 2023-08-22 | 华南理工大学 | Self-supervision visual language navigation pre-training method, device and storage medium |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101286272A (en) * | 2008-05-04 | 2008-10-15 | 北京海兰信数据科技股份有限公司 | Marine vehicle remote control administrative system |
| CN106713292A (en) * | 2016-12-13 | 2017-05-24 | 山东交通学院 | Ship real-time monitoring system |
| CN109842785A (en) * | 2018-12-25 | 2019-06-04 | 江苏恒澄交科信息科技股份有限公司 | Full visual field unmanned ship tele-control system |
| CN110008789A (en) * | 2018-01-05 | 2019-07-12 | 中国移动通信有限公司研究院 | Multiclass object detection and knowledge method for distinguishing, equipment and computer readable storage medium |
| CN110119757A (en) * | 2019-03-28 | 2019-08-13 | 北京奇艺世纪科技有限公司 | Model training method, video category detection method, device, electronic equipment and computer-readable medium |
| CN110263650A (en) * | 2019-05-22 | 2019-09-20 | 北京奇艺世纪科技有限公司 | Behavior category detection method, device, electronic equipment and computer-readable medium |
| CN110580043A (en) * | 2019-08-12 | 2019-12-17 | 中国科学院声学研究所 | Water surface target avoidance method based on image target identification |
-
2020
- 2020-02-12 CN CN202010089428.3A patent/CN111310646B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101286272A (en) * | 2008-05-04 | 2008-10-15 | 北京海兰信数据科技股份有限公司 | Marine vehicle remote control administrative system |
| CN106713292A (en) * | 2016-12-13 | 2017-05-24 | 山东交通学院 | Ship real-time monitoring system |
| CN110008789A (en) * | 2018-01-05 | 2019-07-12 | 中国移动通信有限公司研究院 | Multiclass object detection and knowledge method for distinguishing, equipment and computer readable storage medium |
| CN109842785A (en) * | 2018-12-25 | 2019-06-04 | 江苏恒澄交科信息科技股份有限公司 | Full visual field unmanned ship tele-control system |
| CN110119757A (en) * | 2019-03-28 | 2019-08-13 | 北京奇艺世纪科技有限公司 | Model training method, video category detection method, device, electronic equipment and computer-readable medium |
| CN110263650A (en) * | 2019-05-22 | 2019-09-20 | 北京奇艺世纪科技有限公司 | Behavior category detection method, device, electronic equipment and computer-readable medium |
| CN110580043A (en) * | 2019-08-12 | 2019-12-17 | 中国科学院声学研究所 | Water surface target avoidance method based on image target identification |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111310646A (en) | 2020-06-19 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10152665B2 (en) | Method and system for transmission of information | |
| CN107580717B (en) | Texture reconstruction from image sequences | |
| CN111246195A (en) | System for automatically defining user experience or system behavior related to scene image content | |
| US10535193B2 (en) | Image processing apparatus, image synthesizing apparatus, image processing system, image processing method, and storage medium | |
| CN111310646B (en) | Method for improving navigation safety based on real-time detection of remote images | |
| CN113056904A (en) | Image transmission method, movable platform and computer readable storage medium | |
| CN113112540B (en) | Method for positioning ship image target by using AIS (automatic identification system) Calibration CCTV (CCTV) camera in VTS (video tape server) system | |
| Zhao et al. | Laddernet: Knowledge transfer based viewpoint prediction in 360◦ video | |
| JP5950605B2 (en) | Image processing system and image processing method | |
| US10878577B2 (en) | Method, system and apparatus for segmenting an image of a scene | |
| CN113469869B (en) | Image management method and device | |
| US20210124174A1 (en) | Head mounted display, control method for head mounted display, information processor, display device, and program | |
| US9049495B2 (en) | Information processing system, information processing apparatus, control method of these, and storage medium | |
| WO2015053917A1 (en) | System, method and computer program product for facilitating optical data transfer to a mobile device | |
| CN112770095B (en) | Panoramic projection method and device and electronic equipment | |
| US20130315307A1 (en) | Processing and reproduction of frames | |
| US11470299B2 (en) | Methods and apparatus for encoding frames captured using fish-eye lenses | |
| US11216662B2 (en) | Efficient transmission of video over low bandwidth channels | |
| US20210118213A1 (en) | Systems and methods for three dimensional object scanning | |
| CN116309732A (en) | Ship motion visualization method based on digital twinning | |
| KR101915578B1 (en) | System for picking an object base on view-direction and method thereof | |
| CN113253619B (en) | Ship data information processing method and device | |
| CN115511870A (en) | Object detection method and device, electronic equipment and storage medium | |
| EP4142292A1 (en) | Image content transmitting method and device using edge computing service | |
| CN115086665A (en) | Error code masking method, device, system, storage medium and computer equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |