CN117671130A - Digital twin intelligent fishing port construction and use method based on oblique photography - Google Patents
Digital twin intelligent fishing port construction and use method based on oblique photography Download PDFInfo
- Publication number
- CN117671130A CN117671130A CN202311327255.4A CN202311327255A CN117671130A CN 117671130 A CN117671130 A CN 117671130A CN 202311327255 A CN202311327255 A CN 202311327255A CN 117671130 A CN117671130 A CN 117671130A
- Authority
- CN
- China
- Prior art keywords
- scene
- twin
- model
- fishing port
- fishing
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 25
- 238000010276 construction Methods 0.000 title abstract description 9
- 230000006870 function Effects 0.000 claims abstract description 21
- 229910052792 caesium Inorganic materials 0.000 claims abstract description 19
- TVFDJXOCXUVLDH-UHFFFAOYSA-N caesium atom Chemical compound [Cs] TVFDJXOCXUVLDH-UHFFFAOYSA-N 0.000 claims abstract description 19
- 238000009877 rendering Methods 0.000 claims abstract description 17
- 230000004927 fusion Effects 0.000 claims abstract description 13
- 238000012544 monitoring process Methods 0.000 claims description 21
- 230000000007 visual effect Effects 0.000 claims description 16
- 238000007726 management method Methods 0.000 claims description 15
- 238000012545 processing Methods 0.000 claims description 13
- 238000010586 diagram Methods 0.000 claims description 6
- 230000004044 response Effects 0.000 claims description 6
- 230000008439 repair process Effects 0.000 claims description 5
- 238000004422 calculation algorithm Methods 0.000 claims description 4
- 230000009466 transformation Effects 0.000 claims description 4
- 238000012952 Resampling Methods 0.000 claims description 3
- 238000004458 analytical method Methods 0.000 claims description 3
- 230000005540 biological transmission Effects 0.000 claims description 3
- 238000004364 calculation method Methods 0.000 claims description 3
- 239000003086 colorant Substances 0.000 claims description 3
- 238000004891 communication Methods 0.000 claims description 3
- 238000011835 investigation Methods 0.000 claims description 3
- 238000005259 measurement Methods 0.000 claims description 3
- 238000012805 post-processing Methods 0.000 claims description 3
- 238000007781 pre-processing Methods 0.000 claims description 3
- 238000012360 testing method Methods 0.000 claims description 3
- 230000001172 regenerating effect Effects 0.000 claims description 2
- 238000005507 spraying Methods 0.000 claims description 2
- 238000005516 engineering process Methods 0.000 abstract description 13
- 230000000694 effects Effects 0.000 description 8
- 230000008901 benefit Effects 0.000 description 5
- 238000011161 development Methods 0.000 description 4
- 238000006243 chemical reaction Methods 0.000 description 3
- 238000006467 substitution reaction Methods 0.000 description 3
- 238000007405 data analysis Methods 0.000 description 2
- 238000007689 inspection Methods 0.000 description 2
- 238000012800 visualization Methods 0.000 description 2
- 241000283070 Equus zebra Species 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 238000013497 data interchange Methods 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000009826 distribution Methods 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000012423 maintenance Methods 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000008447 perception Effects 0.000 description 1
- 230000008569 process Effects 0.000 description 1
- 238000007634 remodeling Methods 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T15/00—3D [Three Dimensional] image rendering
- G06T15/04—Texture mapping
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Computer Graphics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Geometry (AREA)
- Software Systems (AREA)
- Processing Or Creating Images (AREA)
Abstract
The invention relates to a digital twin intelligent fishing port construction and use method based on oblique photography, which belongs to the field of geographic information and comprises the following steps: (1) digital twin scene construction is carried out; (2) video fusion of fishing ports; (3) management of twin berths of the fishing port; 4) And (5) port area target radar tracking. The invention takes a three-dimensional live-action model produced by oblique photography as a main mode and a fine model produced by fine modeling as an auxiliary mode, combines and builds a high-efficiency and realistic digital twin fishing port scene, adopts an open source WebGIS (network geographic information system) frame such as Cesium or Maptalks and the like at the front end to conduct high-performance rendering and displaying on the twin scene, and completes the twin functions such as video fusion, berth management, radar tracking and the like by matching with mature internet technologies such as software and hardware equipment and IOT and the like.
Description
Technical Field
The invention relates to a digital twin intelligent fishing port construction and use method based on oblique photography, and belongs to the field of geographic information.
Background
The digital twin is a virtual scene constructed by digitizing a real scene by utilizing a plurality of new technologies such as a physical model, an internet of things technology, big data analysis, virtual simulation and the like, and the virtual world data and the operation are communicated, so that a user can know the state of an entity in a remote way through the virtual world and perform feedback operation by means of the digital twin technology. Compared with the two-dimensional flat world, the visual experience effect of the twin scene is more visual, the target management efficiency of a user on the twin scene can be effectively improved, and the maintenance work is simplified. Along with the improvement of the demand, the intelligent fishing port is developed, and although the port area visualization is primarily realized, a series of problems such as unrealistic display effect, less virtual-real interaction function and the like generally exist. The oblique photogrammetry is to carry a plurality of sensors on a flight platform, collect ground image data from multiple angles, obtain accurate and complete position information and texture data of ground objects through data processing, and produce a three-dimensional live-action model with realistic effect and high spatial precision.
In the aspect of digital twin fishing harbor, the digital twin fishing harbor is mainly created by technologies such as dynamic perception, digital twin, algorithm identification, data analysis, mechanism remodeling and digital reconstruction enabling, and the like, so that a series of problems of invisible, unclear, poor in management and the like of fishery safety control are solved, and the system is in a virtual simulation level. Wang Shuo, xuanying and the like apply the three-dimensional real-scene modeling technology to a power grid digital twin system, prove that the three-dimensional real-scene model can fill a display layer of a digital twin power grid, provide a placeable, displayable and measurable basic image layer for various sensor devices, power infrastructures and the like, and effectively improve the construction and display effects of the digital twin scene. In the aspect of oblique photogrammetry, along with the development of unmanned aerial vehicle technology, implementation threshold gradually reduces, and work efficiency promotes obviously, and the smart 4RTK is as a small-size many rotor high accuracy aerial survey unmanned aerial vehicle, towards low altitude photogrammetry application, possesses centimeter level navigation positioning system and high performance imaging system, and portable easy-to-use, aerial survey efficiency is higher, provides two kinds of oblique photogrammetry flight route planning schemes of well word flight and five-way flight simultaneously. Not only can a three-dimensional live-action model be produced, but also high-precision and low-cost modeling of the fishing port can be realized for providing centimeter-level high-resolution DOM (orthophoto) and DSM (digital surface model).
The difficulties of the digital twin technology at the browser end at present are mainly embodied in two aspects: 1. scene building, especially twin scene building, wherein the real scene area is usually larger, and how to quickly build a scene close to the real environment with low cost is the focus of digital twin project development; 2. scene rendering mainly comprises two main rendering schemes at present, one is rendering through a WebGL (3D drawing protocol) frame, such as three.js, processing and other open source frames, and the scheme has the advantages of low development and use cost, but has the problem of clamping when loading complex scenes; the other is developed by a game engine and is in the form of pixel stream for cloud rendering, and the scheme can realize front-end high-performance rendering, but has much higher use cost than WebGL. How to balance the cost and experience effect is the associated problem of the digital twin intelligent fishing port, and therefore, a digital twin intelligent fishing port system based on oblique photography is provided.
Disclosure of Invention
In order to overcome the defects of the prior art, the invention provides a digital twin intelligent fishing port construction and use method based on oblique photography, which comprises the following steps:
a digital twin intelligent fishing port constructing and using method based on oblique photography comprises the following steps:
(1) Digital twin scene building; (2) video fusion of fishing ports;
(3) Management of twin berths of a fishing port; 4) And (5) port area target radar tracking.
The step (1) specifically comprises the following steps:
(1-1) unmanned aerial vehicle aerial photography: after investigation of oblique photography flight conditions of the fishing port area, drawing an aerial photography area covering the fishing port through a mark language; in order to ensure the precision of the oblique photogrammetry result, the selection of the image control points should be uniformly distributed in the harbor area; after carrying out image control point marking by spraying or selecting a marked ground object on the ground, carrying out image control point measurement by using RTK equipment;
namely: checking the fishing port environment before flying, judging whether the current environment condition meets the flying or not, and if the current environment condition meets the flying, assembling the unmanned aerial vehicle, and starting aerial photography work when the unmanned aerial vehicle self-tests without errors and normally uses the RTK module;
(1-2) fishing port modeling: the method comprises the steps of deriving shot original data from an unmanned aerial vehicle, sequentially carrying out original data reading, first air triangulation, stabbing point, second air triangulation and modeling by using ContextCapture software, and generating original three-dimensional real scene model data in two formats of OSGB and OBJ, wherein the first air triangulation is used for calculating external azimuth elements of each photo; the second time of aerial triangulation is used for improving the calculation accuracy of the external azimuth angle elements of the photo and correcting the model;
(1-3) fishing port model repair: optimizing an original three-dimensional live-action model by using ModelFun software, importing an OSGB and OBJ format data set model, and regenerating OSGB data after finishing the model repair;
(1-4) fishing port fine modeling: determining a region needing to be modeled finely in a harbor area according to the modeling condition of the original three-dimensional live-action model, performing fine modeling by modeling software, and generating an OBJ model after modeling is finished;
(1-5) fishing port model rendering: converting the repaired result into a 3D tilles format; the browser end loads 3D Tiles data by using a Cesium or MapTalks open source GIS frame, and adjusts parameters according to project requirements, so that the front end renders a three-dimensional real scene.
The step (2) is specifically as follows:
(2-1) carrying out refined modeling on a camera in a twin scene, loading the camera into a corresponding position of the scene, and simultaneously adding a response event for the camera to control the opening or closing of the camera; when an opening event responds, a viewing cone is created by using a Cesium camera to simulate the visual range of a monitoring camera, and the visual angle of a twin scene is automatically switched to the corresponding position of a monitoring picture according to the position, azimuth angle and focal length parameters of the camera provided by a rear-end interface;
(2-2) adjusting a camera view cone to enable the view cone to correctly reflect the real focal length, angle and position conditions of the camera; performing visual field analysis processing on the camera view cone range by using a shadow map function of Cesium;
(2-3) creating a video tag, generating a video texture by playing a video stream picture; writing texture processing logic by using GLSL language, sequentially calculating relative coordinates and shadow coordinates of a visible area in a viewing cone range and a camera through video textures and corresponding vertexes to obtain corresponding gl_FragColor, and reserving original textures of a model in the shadow area;
(2-3) in order to avoid the influence of video deformation caused by projection on visual experience, additionally creating a group of video components to synchronously display the play pictures in a normal state, and binding the video player and the camera positions in the twin scene through a callback function provided by Cesium.
The step (3) is specifically as follows:
(3-1) carrying out intelligent twin management on berths in the harbor, providing a berth reservation function, and enabling fishermen to check regional division and use states of the berths based on a map in real time through a WeChat applet and select spare berths to carry out online reservation and intelligent path planning;
(3-2) displaying the use state of the berth in the current harbor in the form of a two-dimensional map combined with vector elements in a mini-program; selecting an idle berth for reservation, pushing reservation information to a twin scene through websocket after a background receives a reservation command, and displaying the real-time state of the berth by the twin scene through gradient color walls with different colors; clicking a berth to display the statistics information of the use of the berth in the near day; the background acquires the coordinates and azimuth information of the fishing boat in real time by using positioning equipment installed on the fishing boat and matching with a near-harbor radar or a monitoring picture, and the twin scene carries out route planning on the berthing of the fishing boat according to the transmission information;
and (3-3) providing necessary berth route planning for the fishing vessel by utilizing an A-algorithm, firstly carrying out data preprocessing, vectorizing berths through a remote sensing guard diagram of a harbor area range, converting vector elements into TIFF, carrying out binarization processing on the TIFF by utilizing a grid calculator, resampling the TIFF according to harbor berth complexity, reading a post-processing result by utilizing GDAL, deriving a two-dimensional array of x y, storing an array, range coordinates and pixel parameters into a database, transmitting longitude and latitude coordinates of the berth during route planning, converting the longitude and latitude coordinates into rectangular coordinate system coordinates according to the coordinate range and the pixel parameters corresponding to the array, reading the array by codes, carrying out breadth-first search, obtaining a route node set, converting the node set into a geographic coordinate system from the rectangular coordinate system, creating a GeoJson object as a result, and loading GeoJson on a twin scene to render the planned route.
The step (4) specifically comprises the following steps:
the method comprises the steps that a two-way communication protocol is established between a twin scene and a radar data server, and the radar data server pushes basic information and geospatial parameters of a current target to the twin scene in real time by analyzing the target of a marine interesting ship tracked by the current radar; after receiving data, the twin scene judges the rendering condition of a target ship in the scene, if the system tracks the target ship for the first time, a ship model is rendered, a response event is added for the ship model to display basic information of the ship, and if the target is in continuous tracking of the system, the geospatial parameters of the ship are dynamically adjusted, and the position and posture conditions of the ship are displayed in real time; calculating through a browser window coordinate, namely directly taking the screen coordinate of the current radar model as the screen coordinate of the video control, dynamically adjusting to a proper position through CSS, dynamically binding the video control to the position of the radar model in the twinning scene and automatically playing a real-time monitoring picture by taking the non-shielding other models as the standard; through the transformation of the marine geographic coordinates and the space coordinates of the twin scene, namely by utilizing a Cesium space coordinate transformation function tool kit, the geographic coordinate system is transformed into a Cartesian coordinate system, the space coordinates of the locked ship and the radar in the twin scene are calculated, a pyramid is drawn to serve as a tracking light cone, the bottom surface of the pyramid faces the locked ship, the vertex angle is positioned at the radar position, and the azimuth angle parameters of the light cone are adjusted in real time by utilizing a keyframe animation function, so that the light cone is continuously locked on the tracked ship.
The invention has the advantages that: according to the digital twin intelligent fishing port system based on oblique photography, the unmanned aerial vehicle with the RTK (real-time dynamic carrier phase difference technology) positioning module is used for carrying out oblique photography modeling on a port area, a three-dimensional scene of the fishing port is built in a combined mode by matching with a fine model manufactured by professional modeling software, rendering and displaying are carried out on the twin fishing port scene at a Web end, and low-cost, high-precision and rapid three-dimensional modeling of the fishing port is achieved. The intelligent fishing port system has the advantages that information intercommunication between a virtual scene and a real scene is realized by utilizing the IOT (Internet of things) technology, the data fusion technology and the like, the operations such as video fusion, radar tracking, berth management and the like in a harbor area are realized by matching with the mature artificial intelligence technology and the like, the intelligent fishing port system truly having digital twin capability is realized, a series of problems of disorder management, difficult supervision, high cost and the like in the traditional intelligent fishing port system are solved, and a new technical solving route is provided for the implementation scheme of the intelligent fishing port digital twin system.
The invention takes a three-dimensional live-action model produced by oblique photography as a main mode and a fine model produced by fine modeling as an auxiliary mode, combines and builds a high-efficiency and realistic digital twin fishing port scene, adopts an open source WebGIS (network geographic information system) frame such as Cesium or Maptalks and the like at the front end to conduct high-performance rendering and displaying on the twin scene, and completes the twin functions such as video fusion, berth management, radar tracking and the like by matching with mature internet technologies such as software and hardware equipment and IOT and the like.
The method also has the following advantages:
(1) The browser end performs high-performance rendering, data is transmitted through a network, seamless switching can be realized between the browser end and the traditional two-dimensional fishing port, and a user does not need to additionally install software or build an environment;
(2) The method has the capability of displaying the large-area fishing port three-dimensional model, the three-dimensional model is macroscopically built by adopting three-dimensional live-action, the loading efficiency is high, the space precision is high, the texture is real, the details of the region of interest are microscopically displayed through fine modeling, and the immersive experience is provided for the user;
(3) The virtual and real world data and operation intercommunication are realized, the digital twin intelligent fishing port scene has the functions of video fusion, radar linkage, berth management and the like, and the fishing port management and supervision capability is practically improved.
Drawings
Fig. 1 is a schematic flow chart of the present invention.
FIG. 2 is a schematic illustration of the route planning procedure of the present invention.
FIG. 3 is a schematic diagram of the modeling flow of the fishing port of the present invention.
Fig. 4 is a schematic diagram of a basic data processing flow of berth route planning according to the present invention.
Fig. 5 is a schematic diagram of a berth route planning procedure of the present invention.
Detailed Description
The invention will be further described with reference to specific embodiments, and advantages and features of the invention will become apparent from the description. These examples are merely exemplary and do not limit the scope of the invention in any way. It will be understood by those skilled in the art that various changes and substitutions of details and forms of the technical solution of the present invention may be made without departing from the spirit and scope of the present invention, but these changes and substitutions fall within the scope of the present invention.
Referring to fig. 1 to 5, the invention relates to a digital twin intelligent fishing port construction and use method based on oblique photography, which comprises the following steps: (1) digital twinning scene construction; (2) video fusion of fishing ports; (3) management of twin berths of the fishing port; 4) And (5) port area target radar tracking.
The step (1) specifically comprises the following steps:
(1-1) unmanned aerial vehicle flies: the method comprises the steps of performing flight condition investigation on a fishing port area, drawing an aerial photo area by using a markup language, ensuring to cover the fishing port, and automatically performing route planning in the area after an unmanned aerial vehicle loads the markup language KML; in order to ensure the accuracy of the oblique photogrammetry result, the image control point measurement is carried out by adopting a thousand-star-finding X RTK device, the selection of the phase control points is uniformly distributed in a harbor area, and the image control points are sprayed or ground markers such as zebra stripes and road indication arrows are selected; before flying, the fishing port environment is checked, whether the fishing port environment meets the flying or not is judged, an unmanned aerial vehicle is assembled if the condition is met, and after the unmanned aerial vehicle self-tests, whether the network RTK module is connected normally or not is checked.
(1-2) fishing port modeling: the method comprises the steps of deriving original data from an unmanned aerial vehicle, sequentially carrying out picture reading, first air triangulation, stabbing point, second air triangulation and modeling by using ContextCapture software, and generating original three-dimensional real-scene model data in two formats of OSGB and OBJ；In order to ensure the model precision, the number of the stabs is flexibly adjusted according to the area of the measuring area, and the final output result is data in two formats of OSGB (commonly used oblique photography data storage format) and OBJ (3D model file format); because this process is time consuming, clusters can be built to increase production efficiency when necessary. Wherein the first aerial triangulation is used to calculate the external orientation element for each photo; the second time of aerial triangulation is used for improving the calculation accuracy of the external azimuth angle elements of the photo and correcting the model;
(1-3) fishing port model repair: due to the technical characteristics of oblique photography, the details of a model part are lost, and the problems of hole breaking, deformation and the like of the model cannot be solved although the model precision can be improved by flying in modes of reducing the height, layering the height and the like. Therefore, the model needs to be repaired, the ModelFun software is used for optimizing the original three-dimensional live-action model, the OSGB and OBJ format data set model is imported, and OSGB data are regenerated after the model is repaired. Performing operations such as structure restoration, dock clearance, pavement screen placement, texture restoration and the like on the model by combining the existing data, ensuring that the fishing port dock has no movable interference objects after the model is repaired, and re-exporting the data;
(1-4) fishing port fine modeling: determining a region needing to be subjected to refined modeling in a harbor area according to the modeling condition of an original three-dimensional live-action model, performing the refined modeling by adopting 3Ds Max or Blender modeling software, and deriving an OBJ format for use after the modeling is finished;
(1-5) fishing port model rendering: converting the repaired result into a 3D tilles format; the browser end loads 3D Tiles data by using a Cesium or MapTalks open source GIS frame, and adjusts parameters according to project requirements, so that the front end renders a three-dimensional real scene.
The browser end cannot directly load slice data in OSGB and OBJ formats, and software such as Cesium Lab or DasViewer is used for converting the repaired result into a 3D Tiles format; 3D Tiles is an open three-dimensional data standard, and each 3D tile is composed of a JSON (lightweight data interchange format) file for storing node information and a plurality of slice package folders. The client side realizes the on-demand loading and rendering of the model by carrying out layered and hierarchical slicing treatment on the model; using a conversion result as a static resource to use servers such as Nginx (high-performance HTTP and reverse proxy web servers) and the like to realize the request access by the reverse proxy; and the browser end loads 3D Tiles data by using a Cesium or MapTalks open source GIS frame, and adjusts parameters of the data according to project requirements to complete the front-end high-performance rendering of the three-dimensional real scene.
The three-dimensional live-action scene has space information, space coordinates are given to the model which is produced in a refined mode, and the model is adjusted to be scaled, so that the model can be displayed in the three-dimensional live-action scene correctly, and the digital twin intelligent fishing port scene is built.
The step (2) is specifically as follows: the purpose of the video fusion of the fishing port is to carry out projection fusion on one or more monitoring videos in the fishing port area and a three-dimensional virtual scene related to the space of the monitoring videos, so that the virtual scene has the capability of reflecting monitoring pictures in real time and truly, and specifically comprises the following steps:
(2-1) carrying out refined modeling on a camera in a twin scene, loading the camera into a corresponding position of the scene, and simultaneously adding a response event for the camera to control the opening or closing of the camera; when an opening event responds, a viewing cone is created by using a Cesium camera to simulate the visual range of a monitoring camera, and the visual angle of a twin scene is automatically switched to the corresponding position of a monitoring picture according to the position, azimuth angle and focal length parameters of the camera provided by a rear-end interface;
(2-2) adjusting a camera view cone to enable the view cone to correctly reflect the real focal length, angle and position conditions of the camera; performing visual field analysis processing on the camera view cone range by using a shadow map function of Cesium;
(2-3) creating a video tag, generating video texture by playing a video streaming picture of HLS (streaming media network transport protocol of HTTP) or FLV (streaming media format); the method comprises the steps of writing texture processing logic by using GLSL language, sequentially calculating relative coordinates and shadow coordinates of a visible area in a viewing cone range and a camera through video textures and corresponding vertexes to obtain corresponding gl_FragColor, and reserving original textures of a model in the shadow area to realize virtual-real combination, so that a user can watch a real monitoring picture in a virtual scene;
(2-4) in order to avoid the influence of video deformation caused by projection on visual experience, additionally creating a group of video components to synchronously display the play pictures in a normal state, binding the video player and the camera position in the twin scene through a callback function provided by Cesium, and automatically moving along with the movement of the camera position in the scene when a user drags the virtual scene.
The video fusion using effect is related to the visual angle of the monitoring camera, and the closer the visual angle is to 90 degrees to the monitoring surface, the smaller the picture projection deformation is, and the more realistic the experience effect is. The video fusion effect is used for completing the upgrading of the monitoring picture from the plane to the three-dimensional, virtual-real combination is realized, meanwhile, the scene supports the monitoring video round inspection function, the manager can realize the comprehensive inspection of the port area monitoring without operation, and the management and supervision efficiency is effectively improved.
The step (3) is specifically as follows:
(3-1) carrying out intelligent twin management on the berths in the harbor, providing a berth reservation function, enabling fishermen to check the regional division and the use state of the berths based on a map in real time through a WeChat applet with the reservation function, selecting spare berths for online reservation and intelligent path planning, and realizing the visualization of berth management and the use efficiency of the berths;
and opening a WeChat applet, logging in, entering a berth reservation interface, determining the use state of the berth in the current harbor through a two-dimensional map and vector elements in the WeChat applet, and selecting a proper free berth for reservation.
(3-2) displaying the use state of the berth in the current harbor in the form of a two-dimensional map combined with vector elements; selecting an idle berth for reservation, pushing reservation information to a twin scene through websocket after a background receives a reservation command, and displaying the real-time state of the berth by the twin scene through gradient color walls with different colors; clicking a berth to display the statistics information of the use of the berth in the near day; the background acquires the coordinates and azimuth information of the fishing boat in real time by using positioning equipment installed on the fishing boat and matching with a near-harbor radar or a monitoring picture, and the twin scene carries out route planning on the berthing of the fishing boat according to the transmission information;
and (3-3) providing necessary berth route planning for the fishing vessel by utilizing an A-algorithm, firstly carrying out data preprocessing, vectorizing berths through a remote sensing guard diagram of a harbor area range, converting vector elements into TIFF, carrying out binarization processing on the TIFF by utilizing a grid calculator, resampling the TIFF according to harbor berth complexity, reading a post-processing result by utilizing GDAL, deriving a two-dimensional array of x y, storing an array, range coordinates and pixel parameters into a database, transferring longitude and latitude coordinates of the berth during route planning, converting the longitude and latitude coordinates into rectangular coordinate system coordinates according to a coordinate range and pixel parameters corresponding to the array, reading the array by codes, carrying out breadth-first search, obtaining a route node set, converting the node set into a geographic coordinate system from the rectangular coordinate system, creating a GeoJson object as a result, and returning the GeoJson object to the system, so that the route planning can be rendered by loading the twin scene.
The step (4) specifically comprises the following steps:
the method comprises the steps that a two-way communication protocol is established between a twin scene and a radar data server, and the radar data server pushes basic information and geospatial parameters of a current target to the twin scene in real time by analyzing the target of a marine interesting ship tracked by the current radar; after receiving data, the twin scene judges the rendering condition of a target ship in the scene, if the system tracks the target ship for the first time, a ship model is rendered, a response event is added for the ship model to display basic information of the ship, and if the target is in continuous tracking of the system, the geospatial parameters of the ship are dynamically adjusted, and the position and posture conditions of the ship are displayed in real time; calculating through a browser window coordinate, namely directly taking the screen coordinate of the current radar model as the screen coordinate of the video control, dynamically adjusting to a proper position through CSS, dynamically binding the video control to the position of the radar model in the twinning scene and automatically playing a real-time monitoring picture by taking the non-shielding other models as the standard; through marine geographic coordinates and twin scene space coordinate conversion, namely utilize Cesium space coordinate conversion function toolkit, convert geographic coordinate system into Cartesian coordinate system, calculate the space coordinates of locked ship and radar in twin scene, draw a pyramid as the tracking light cone, this pyramid bottom surface is towards being locked the ship, the apex angle is located the radar position, utilize key frame animation function real-time adjustment light cone azimuth angle parameter, make the light cone lock continuously by the tracked ship, even target area ship position distribution is intensive, the manager also can be clear to know current radar locking ship.
In view of development costs and hardware performance limitations, data transfer may not be continuous, and may be pushed once every few seconds. And carrying out interpolation processing on the points at the front and back times, so that the movement route of the ship in the scene is continuous and uniform, and the phenomenon of ship position jump is avoided.
The foregoing is only a preferred embodiment of the present invention, but the scope of the present invention is not limited thereto, and any person skilled in the art, who is within the scope of the present invention, should make equivalent substitutions or modifications according to the technical scheme of the present invention and the inventive concept thereof, and should be covered by the scope of the present invention.
Claims (5)
1. The method for constructing and using the digital twin intelligent fishing port based on oblique photography is characterized by comprising the following steps of:
(1) Digital twin scene building; (2) video fusion of fishing ports;
(3) Management of twin berths of a fishing port; 4) And (5) port area target radar tracking.
2. The method for constructing and using the digital twin intelligent fishing port based on oblique photography according to claim 1, wherein the step (1) specifically comprises:
(1-1) unmanned aerial vehicle aerial photography: after investigation of oblique photography flight conditions of the fishing port area, drawing an aerial photography area covering the fishing port through a mark language; in order to ensure the precision of the oblique photogrammetry result, the selection of the image control points should be uniformly distributed in the harbor area; after carrying out image control point marking by spraying or selecting a marked ground object on the ground, carrying out image control point measurement by using RTK equipment;
namely: checking the fishing port environment before flying, judging whether the current environment condition meets the flying or not, and if the current environment condition meets the flying, assembling the unmanned aerial vehicle, and starting aerial photography work when the unmanned aerial vehicle self-tests without errors and normally uses the RTK module;
(1-2) fishing port modeling: the method comprises the steps of deriving shot original data from an unmanned aerial vehicle, sequentially carrying out original data reading, first air triangulation, stabbing point, second air triangulation and modeling by using ContextCapture software, and generating original three-dimensional real scene model data in two formats of OSGB and OBJ, wherein the first air triangulation is used for calculating external azimuth elements of each photo; the second time of aerial triangulation is used for improving the calculation accuracy of the external azimuth angle elements of the photo and correcting the model;
(1-3) fishing port model repair: optimizing an original three-dimensional live-action model by using ModelFun software, importing an OSGB and OBJ format data set model, and regenerating OSGB data after finishing the model repair;
(1-4) fishing port fine modeling: determining a region needing to be modeled finely in a harbor area according to the modeling condition of the original three-dimensional live-action model, performing fine modeling by modeling software, and generating an OBJ model after modeling is finished;
(1-5) fishing port model rendering: converting the repaired result into a 3D tilles format; the browser end loads 3D Tiles data by using a Cesium or MapTalks open source GIS frame, and adjusts parameters according to project requirements, so that the front end renders a three-dimensional real scene.
3. The method for constructing and using the digital twin intelligent fishing port based on oblique photography according to claim 1 or 2, wherein the step (2) is specifically:
(2-1) carrying out refined modeling on a camera in a twin scene, loading the camera into a corresponding position of the scene, and simultaneously adding a response event for the camera to control the opening or closing of the camera; when an opening event responds, a viewing cone is created by using a Cesium camera to simulate the visual range of a monitoring camera, and the visual angle of a twin scene is automatically switched to the corresponding position of a monitoring picture according to the position, azimuth angle and focal length parameters of the camera provided by a rear-end interface;
(2-2) adjusting a camera view cone to enable the view cone to correctly reflect the real focal length, angle and position conditions of the camera; performing visual field analysis processing on the camera view cone range by using a shadow map function of Cesium;
(2-3) creating a video tag, and generating video textures by playing the monitor video stream picture; writing texture processing logic by using GLSL language, sequentially calculating relative coordinates and shadow coordinates of a visible area in a viewing cone range and a camera through video textures and corresponding vertexes to obtain corresponding gl_FragColor, and reserving original textures of a model in the shadow area;
(2-4) in order to avoid the influence of video deformation caused by projection on visual experience, additionally creating a group of video components to synchronously display the play pictures in a normal state, and binding the video player and the camera positions in the twin scene through a callback function provided by Cesium.
4. The method for constructing and using the digital twin intelligent fishing port based on oblique photography according to claim 1 or 2, wherein the step (3) is specifically:
(3-1) carrying out intelligent twin management on the berths in the harbor, providing a berth reservation function, checking the regional division and the use state of the berths based on a map in real time by fishermen, and selecting spare berths to carry out online reservation and intelligent path planning;
(3-2) displaying the use state of the berth in the current harbor in a two-dimensional map combined with vector elements in a WeChat applet; selecting an idle berth for reservation, pushing reservation information to a twin scene through websocket after a background receives a reservation command, and displaying the real-time state of the berth by the twin scene through gradient color walls with different colors; clicking a berth to display the statistics information of the use of the berth in the near day; the background acquires the coordinates and azimuth information of the fishing boat in real time by using positioning equipment installed on the fishing boat and matching with a near-harbor radar or a monitoring picture, and the twin scene carries out route planning on the berthing of the fishing boat according to the transmission information;
and (3-3) providing necessary berth route planning for the fishing vessel by utilizing an A-algorithm, firstly carrying out data preprocessing, vectorizing berths through a remote sensing guard diagram of a harbor area range, converting vector elements into TIFF, carrying out binarization processing on the TIFF by utilizing a grid calculator, resampling the TIFF according to harbor berth complexity, reading a post-processing result by utilizing GDAL, deriving a two-dimensional array of x y, storing an array, range coordinates and pixel parameters into a database, transmitting longitude and latitude coordinates of the berth during route planning, reading the array by codes, converting the longitude and latitude coordinates into rectangular coordinate system coordinates according to a coordinate range and pixel parameters corresponding to the array, carrying out breadth-first search in the array, obtaining a route node set, converting the node set into the geographic coordinate system from the rectangular coordinate system, creating a GeoJson object as a result, and returning the GeoJson object to the system, so that the route planning can be rendered by loading a twin scene.
5. The method for constructing and using the digital twin intelligent fishing port based on oblique photography according to claim 1 or 2, wherein the step (4) is specifically:
the method comprises the steps that a two-way communication protocol is established between a twin scene and a radar data server, and the radar data server pushes basic information and geospatial parameters of a current target to the twin scene in real time by analyzing the target of a marine interesting ship tracked by the current radar; after receiving data, the twin scene judges the rendering condition of a target ship in the scene, if the system tracks the target ship for the first time, a ship model is rendered, a response event is added for the ship model to display basic information of the ship, and if the target is in continuous tracking of the system, the geospatial parameters of the ship are dynamically adjusted, and the position and posture conditions of the ship are displayed in real time; calculating through a browser window coordinate, namely directly taking the screen coordinate of the current radar model as the screen coordinate of the video control, dynamically adjusting to a proper position through CSS, dynamically binding the video control to the position of the radar model in the twinning scene and automatically playing a real-time monitoring picture by taking the non-shielding other models as the standard; through the transformation of the marine geographic coordinates and the space coordinates of the twin scene, namely by utilizing a Cesium space coordinate transformation function tool kit, the geographic coordinate system is transformed into a Cartesian coordinate system, the space coordinates of the locked ship and the radar in the twin scene are calculated, a pyramid is drawn to serve as a tracking light cone, the bottom surface of the pyramid faces the locked ship, the vertex angle is positioned at the radar position, and the azimuth angle parameters of the light cone are adjusted in real time by utilizing a keyframe animation function, so that the light cone is continuously locked on the tracked ship.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311327255.4A CN117671130A (en) | 2023-10-13 | 2023-10-13 | Digital twin intelligent fishing port construction and use method based on oblique photography |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311327255.4A CN117671130A (en) | 2023-10-13 | 2023-10-13 | Digital twin intelligent fishing port construction and use method based on oblique photography |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117671130A true CN117671130A (en) | 2024-03-08 |
Family
ID=90081413
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311327255.4A Pending CN117671130A (en) | 2023-10-13 | 2023-10-13 | Digital twin intelligent fishing port construction and use method based on oblique photography |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117671130A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN118470173A (en) * | 2024-07-15 | 2024-08-09 | 安创启元(杭州)科技有限公司 | Underground pipe network display method and device based on cloud rendering and computer equipment |
-
2023
- 2023-10-13 CN CN202311327255.4A patent/CN117671130A/en active Pending
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN118470173A (en) * | 2024-07-15 | 2024-08-09 | 安创启元(杭州)科技有限公司 | Underground pipe network display method and device based on cloud rendering and computer equipment |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10297074B2 (en) | Three-dimensional modeling from optical capture | |
| US20190026400A1 (en) | Three-dimensional modeling from point cloud data migration | |
| CN111275750B (en) | Indoor space panoramic image generation method based on multi-sensor fusion | |
| US9996976B2 (en) | System and method for real-time overlay of map features onto a video feed | |
| Lin et al. | Capturing, reconstructing, and simulating: the urbanscene3d dataset | |
| JP2021111385A (en) | System and method enabling collaborative 3d map data fusion platform and virtual world system thereof | |
| WO2020192355A1 (en) | Method and system for measuring urban mountain viewing visible range | |
| CN109801358A (en) | A kind of substation's three-dimensional investigation method scanning and put cloud visual fusion based on SLAM | |
| CN104599243A (en) | Virtual and actual reality integration method of multiple video streams and three-dimensional scene | |
| CN114758337B (en) | Semantic instance reconstruction method, device, equipment and medium | |
| CN112652065A (en) | Three-dimensional community modeling method and device, computer equipment and storage medium | |
| CN110852952B (en) | Large-scale terrain real-time drawing method based on GPU | |
| CN117671130A (en) | Digital twin intelligent fishing port construction and use method based on oblique photography | |
| CN110310367A (en) | Based on large scene outdoor scene three-dimensional multi-angle 2.5D image lightweight browsing method | |
| CN115082254A (en) | Lean control digital twin system of transformer substation | |
| CN112580428A (en) | Power distribution network design method and device | |
| Koeva | 3D modelling and interactive web-based visualization of cultural heritage objects | |
| CN116129064A (en) | Electronic map generation method, device, equipment and storage medium | |
| Pyka et al. | LiDAR-based method for analysing landmark visibility to pedestrians in cities: case study in Kraków, Poland | |
| CN114359489A (en) | Method, device and equipment for making real-scene image in pipeline construction period and storage medium | |
| Stødle et al. | High-performance visualisation of UAV sensor and image data with raster maps and topography in 3D | |
| US20220131375A1 (en) | System and method for energy infrastructure and geospatial data visualization, management, and analysis using environment simulation and virtual realization | |
| KR20160120955A (en) | Module for multi diensional vedio information visualization | |
| Haibt | End-to-end digital twin creation of the archaeological landscape in Uruk-Warka (Iraq) | |
| Wu et al. | Research on the application of uav tilt photography technology in engineering project |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |