CN118877740A - A blind spot monitoring system for quay crane operation - Google Patents

Abstract

The present invention relates to the technical field of port automation equipment control and monitoring, and in particular to a blind spot monitoring system for quay crane operations. Firstly, multi-dimensional environmental data of the operating area is collected, and optimized three-dimensional scene data is generated through an adaptive modal distribution algorithm; then, a nonlinear prediction model is used to perform risk assessment on behavioral characteristics and equipment status, and multi-level risk assessment data is generated and an early warning is triggered; finally, real-time path adjustment and dynamic control of spreader operations are achieved through adaptive control and self-correction modules; the present invention significantly improves the safety, accuracy and ability to adapt to complex environments of spreader operations.

Description

A blind spot monitoring system for quay crane operation

Technical Field

The present invention relates to the technical field of port automation equipment control and monitoring, and in particular to a blind area monitoring system for quay crane operations.

Background Art

Due to the large size of containers, crane operators often cannot see the entire operating area during operation, resulting in visual blind spots, which can easily lead to collisions between containers and transport vehicles or ground personnel, affecting the safety and efficiency of operations. The prior art (Chinese invention patent, publication number: CN113848906A, name: Automated terminal crane safety operation system, control method and storage medium) generally uses cameras, infrared sensors and other detection methods to monitor the location information of vehicles and surrounding personnel. Although these systems can collect information about transport vehicles and surrounding environments, due to the relatively simple data integration and processing, it is difficult to cope with complex and changing operating environments, especially in dynamically changing blind spot operations, there are defects such as incomplete monitoring, delayed response, and inability to accurately track objects.

Summary of the invention

In view of the many problems existing in the above-mentioned prior art, the present invention provides a blind spot monitoring system for quay crane operations. The present invention generates accurate three-dimensional scene data and object tracking data by integrating multimodal data of vision, laser radar and depth sensor, dynamically adjusting the weight of data source, and combining nonlinear prediction model. Through real-time risk assessment and adaptive path optimization, the safety and accuracy of spreader operation are ensured, especially in blind spot operation, which effectively reduces safety hazards.

A blind area monitoring system for quay crane operation, comprising:

A data acquisition module collects multi-dimensional environmental data of the operating area through visual sensors, laser radars and depth sensors, and pre-processes the multi-dimensional environmental data to generate optimized fused perception data;

The scene construction module extracts visual feature data and depth feature data based on optimized fusion perception data, uses an adaptive modal distribution algorithm to dynamically adjust the weights of different data sources, and fuses multi-source data to generate three-dimensional scene data and object tracking data to build a real-time updated three-dimensional operation scene;

The risk assessment module extracts behavior feature data and device feature data based on 3D scene data and object tracking data, uses a nonlinear behavior prediction model to generate behavior prediction data and device trajectory prediction data, combines risk interaction analysis to generate multi-level risk assessment data, determines the risk level and triggers an early warning mechanism;

The control module generates optimized path data and multi-objective control data based on risk assessment data through path optimization algorithm and multi-objective decision-making algorithm, and generates dynamic control data in combination with the real-time operation status of the spreader to achieve adaptive control and real-time adjustment of the spreader operation;

The self-correction module collects feedback perception data and operation status data during the operation of the spreader, generates self-correction data through the feedback self-correction algorithm, and adjusts the operation path.

Preferably, the adaptive modal distribution algorithm in the scene construction module is used to dynamically adjust the weight of each data source according to the difference between the multi-dimensional environmental data collected by different sensors, so that when the environment changes, more reliable data sources can be given priority, thereby generating accurate three-dimensional scene data and object tracking data. The calculation expression of the weight is:

in, For the The weight of the data source, For the Data source and The degree of difference between the data sources; is the number of data sources.

Preferably, the fusion process of the visual feature data and the depth feature data is implemented by weighted combination, and the generated three-dimensional scene data can reflect the real state of the working area under complex environmental conditions. The specific calculation expression of the weighted combination is:

in, is the fused three-dimensional scene data; is the visual feature data; is the deep feature data; is the weight coefficient of visual feature data; is the weight coefficient of the deep feature data; and satisfies 1.

Preferably, the risk assessment module uses a nonlinear behavior prediction model to learn the historical behavior data of the operator to generate behavior prediction data reflecting the operator's possible future behavior. At the same time, by analyzing the equipment status, the equipment trajectory prediction data is generated to comprehensively assess the risk situation of the operating area. The calculation expression of the risk level is:

in, is the risk level; Predict data for operator behavior; is the motion trajectory of the device; is the risk assessment function.

Preferably, the risk interaction analysis identifies potential risk interaction areas by comparing the interaction points between the operator behavior prediction data and the equipment trajectory prediction data, and evaluates each interaction point according to the set risk level standard, thereby forming multi-level risk assessment data and triggering a corresponding early warning mechanism.

Preferably, the control module optimizes the path data and performs path planning based on the A-star algorithm to ensure that the spreader can effectively avoid obstacles during operation and make dynamic adjustments based on real-time feedback.

Preferably, the multi-objective decision-making algorithm is implemented by the following optimization function:

in, Optimize results for your goals; For the The weight of the control target; For the Control cost of each control target; To control the number of targets.

Preferably, the self-correction module monitors and records the operation status data and feedback perception data in real time during the operation of the spreader to generate self-correction data reflecting the current operation accuracy and environmental adaptability, and makes necessary adjustments to the spreader operation path based on the self-correction data. The calculation expression of the path adjustment is:

in, is the adjusted path; is the current path; is the path change amount adjusted according to the self-correction data.

Preferably, the self-correction module analyzes the performance of the spreader under different operating conditions in combination with historical operating data to optimize the parameter settings of each algorithm in the control module so that future operating paths can be self-adjusted based on the accumulated operating data and environmental changes.

Preferably, the self-correction module uses a recursive least squares method to perform optimization calculations to improve the adjustment accuracy of the spreader operation path and control strategy.

Compared with the prior art, the advantages and beneficial effects of the present invention are:

The present invention realizes multi-dimensional environmental monitoring of the working area through multi-modal data acquisition technology, and improves the accuracy of environmental perception by combining visual sensors, laser radars and depth sensors;

The present invention realizes the weight adjustment of dynamic data sources through an adaptive modal distribution algorithm, ensuring accurate scene construction in complex environments;

The present invention generates multi-level risk assessment data through nonlinear behavior prediction model and risk interaction analysis, solving the problem that the prior art cannot comprehensively predict risks.

The present invention provides real-time path optimization and automatic adjustment functions through an adaptive control module combined with the operating status of the spreader, greatly improving operation safety and operating efficiency.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG1 is a block diagram of the system of the present invention;

FIG2 is a flow chart of data collection and processing in the present invention;

FIG. 3 is a control and feedback loop diagram of the present invention.

DETAILED DESCRIPTION

Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the present disclosure. In the following detailed description, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of the embodiments of the present disclosure. However, it is obvious that one or more embodiments may also be implemented without these specific details. In addition, in the following description, descriptions of known structures and technologies are omitted to avoid unnecessary confusion of the concepts of the present disclosure.

As shown in FIG1 , a blind spot monitoring system for quay crane operation includes:

As shown in FIG2 , the data acquisition module collects multi-dimensional environmental data of the operating area through visual sensors, laser radars, and depth sensors, and pre-processes the multi-dimensional environmental data to generate optimized fused perception data;

Visual sensors are mainly used to obtain image and video information of the operating area and can capture the visual features of the scene in real time. The advantage of this sensor in the quay crane operation scene is that it can capture the movement status of equipment, personnel and objects in the operating area with high resolution, and is suitable for identifying fast-moving objects or obstacles. However, due to the limitations of visual sensors in low-light or occluded scenes, the present invention combines lidar and depth sensors for data fusion to make up for the shortcomings of a single sensor.

Laser radar (LiDAR) is a sensor commonly used for three-dimensional scanning. It generates a high-precision depth map by emitting lasers and measuring their return time. The function of this sensor is to provide accurate distance and spatial information, especially in poor lighting conditions. In the present invention, the combination of laser radar and visual sensor can build a three-dimensional model of the working area in real time, so that the system can accurately identify the relative positions of equipment, obstacles and personnel around the spreader, and then monitor potential risks in the blind spot.

Depth sensors generate depth data of the operating environment by capturing the distance between the object and the sensor, further enhancing the system's ability to perceive the shape and position of the object. In the blind spot monitoring of quay crane operations, depth sensors can not only help the system better perceive the spatial position of the object, but also assist in identifying the three-dimensional shape of the object. Unlike lidar, depth sensors usually use technologies such as structured light or time of flight (ToF) to obtain depth information, and can accurately scan complex environments.

Preprocessing in the data acquisition module is a key step in data fusion and optimization in the present invention. Data collected by different sensors differ in accuracy, format, and timestamp, so they must be processed uniformly before data fusion. The preprocessing process includes denoising of visual images, denoising and filtering of LiDAR point cloud data, and format standardization of depth data. Through these processing steps, the system can effectively eliminate data redundancy and noise, thereby improving the accuracy and speed of subsequent data fusion.

Finally, through the collaborative work of the above-mentioned multi-dimensional sensors and the preprocessing process, optimized fusion perception data is generated. This data not only integrates the advantages of various sensors and makes up for their respective shortcomings, but also provides a global perception of the quay crane operation area, enabling the system to monitor the blind spots in real time, identify potential dangers during the operation process and respond quickly. In the present invention, through the combination of these sensors and preprocessing modules, the monitoring system can effectively improve the visualization effect of the quay crane operation blind spots and ensure the safety and efficiency of the operation.

The scene construction module extracts visual feature data and depth feature data based on optimized fusion perception data, uses an adaptive modal distribution algorithm to dynamically adjust the weights of different data sources, and fuses multi-source data to generate three-dimensional scene data and object tracking data to build a real-time updated three-dimensional operation scene;

Visual feature data comes from visual sensors, and usually includes information such as color, shape, and texture. These data show the surface features of objects in the working area on a two-dimensional plane, but due to the limitations of visual sensors, it is difficult to accurately reflect the three-dimensional spatial position of objects by relying solely on visual data, especially in complex working environments or blind spots with limited vision.

Depth feature data is collected by LiDAR and depth sensors, which can provide information about the distance between the object and the sensor. These data enable the system to perceive the spatial position and contour of the object. In the blind spot monitoring of quay crane operations, the introduction of depth feature data greatly supplements the deficiencies of visual features, especially in the case of occlusion and drastic changes in light, depth data provides stable three-dimensional spatial information.

The core of the scene construction module is the use of an adaptive modal distribution algorithm, which is used to dynamically adjust according to the characteristics of different data sources (such as visual and depth data). This adaptability is reflected in the following: when there are differences between data sources (such as the reliability of a sensor decreases or the data accuracy is disturbed), the system can automatically adjust the weight of each data source so that more reliable and more accurate data dominates the scene construction process. For example, in poor lighting conditions, the quality of visual data decreases. At this time, the adaptive modal distribution algorithm will reduce the weight of visual data and increase the weight of depth feature data to ensure that the constructed three-dimensional scene data can maintain high accuracy. This dynamic adjustment mechanism greatly improves the robustness of the system and can maintain stable performance under different environmental conditions.

By adjusting the weights of data sources through the adaptive modal distribution algorithm, fusing multi-source data becomes one of the key steps of the present invention. In the process of fusing multi-source data, the system not only integrates visual features and depth features, but also ensures that the fused data can reflect the global state of the work area. The fused three-dimensional scene data shows the complete spatial information of the work area and accurately reflects the relative positions of slings, equipment, objects and personnel. At the same time, object tracking data provides the system with dynamic monitoring capabilities. By analyzing the trajectory of objects, the system can track the movement of slings in real time and identify possible collision risks or changes in distance from personnel.

In actual application, the effect of the scene building module is very significant. First, through the real-time updated three-dimensional scene, the system can accurately and timely reflect the spatial structure of the working area, especially the dynamic changes in the blind spot can be monitored. Secondly, the object tracking function enables the system to not only monitor static scenes, but also respond to dynamic changes. For example, when the spreader approaches a person in the blind spot, the object tracking data will reflect the relative position and trajectory of the spreader and the person. The system will issue a warning in time based on this data, thereby effectively avoiding the occurrence of safety accidents.

For example, suppose in a complex port operation scenario, the spreader is performing a lifting operation, and the visual sensor data is distorted due to strong sunlight reflection. Through the adaptive modal distribution algorithm, the system detects the degradation of visual data quality, automatically reduces its weight, and relies on the spatial data provided by the lidar and depth sensor. After data fusion, the generated three-dimensional scene can still accurately reflect the movement trajectory of the spreader and the position of surrounding objects, ensuring that the system's real-time monitoring of blind spots will not be affected by environmental changes.

Preferably, the adaptive modal distribution algorithm in the scene construction module is used to dynamically adjust the weight of each data source according to the difference between the multi-dimensional environmental data collected by different sensors, so that when the environment changes, more reliable data sources can be given priority, thereby generating accurate three-dimensional scene data and object tracking data. The calculation expression of the weight is:

in, For the The weight of the data source, For the Data source and The degree of difference between the data sources; is the number of data sources. The greater the difference, the greater the difference between a data source and other data sources, thus affecting its weight in the overall data fusion.

The adaptive modal distribution algorithm in the present invention plays a key role in the scene construction module. It is mainly used to dynamically adjust the weight of each data source according to the differences between the multi-dimensional environmental data collected by different sensors, thereby ensuring that when the environment changes, the system can give priority to more reliable data sources and ultimately generate accurate three-dimensional scene data and object tracking data.

The core of the algorithm is to dynamically adjust the weight of sensor data. Sensors perform differently in different environments. For example, the data quality of visual sensors may be affected in strong light or low light conditions, while lidar or depth sensors may provide more stable data in these environments. Therefore, through the adaptive modal distribution algorithm, the system can evaluate the reliability of each data source in real time and dynamically assign weights based on its difference from other data sources.

Through the above calculation formula, the system can achieve the following functions:

Dynamically adapt to different environmental conditions: When the data quality of some sensors deteriorates due to changes in environmental conditions (such as light intensity, weather, obstacles, etc.), the algorithm can automatically reduce the weight of these data sources. For example, in strong light conditions, the visual sensor data may be distorted, while the data of LiDAR and depth sensors are still reliable. At this time, the system will reduce the weight of visual data and increase the weight of LiDAR and depth sensor data to ensure that the generated 3D scene data is still accurate.

Automatic adjustment of weight distribution: By calculating the difference between different data sources, the system can automatically adjust the weight of each data source to adapt to the real-time changing environment. Such a dynamic adjustment mechanism can effectively respond to emergencies and ensure the continuity and accuracy of scene construction. For example, when there is thick fog or smoke in the port operation area, the reliability of visual data decreases, while lidar and depth sensors can still provide stable distance and depth information. The adaptive modal distribution algorithm will adjust the weight in this case to ensure the accuracy of data fusion.

The principle of the adaptive modal distribution algorithm relies on difference calculation, which evaluates the similarity and reliability between different data sources and assigns weights to more reliable data sources. This approach not only reduces the impact of single sensor failure, but also improves the overall robustness of the system. By dynamically adjusting the weights, the system always gives priority to the most reliable data source, thereby generating high-precision three-dimensional scene data and object tracking data. This is crucial for blind spot monitoring of quay crane operations, especially when monitoring complex dynamic environments in real time.

Example: In actual application, assume a port quay crane operation scenario, where the sensor data sources include visual sensors, lidars, and depth sensors. When the environment changes, such as when the sunlight suddenly increases, causing interference to the visual sensor data, the system detects an increase in the difference between the visual data and other sensor data through the adaptive modal distribution algorithm. At this time, the weight of the visual sensor is dynamically reduced, while the weights of the lidar and depth sensors are increased accordingly. Through this adjustment, the generated three-dimensional scene data can still accurately reflect the operating environment and is not affected by the distortion of the visual sensor data, thereby ensuring that the quay crane operator maintains real-time monitoring of objects and personnel activities in the blind spot. This example fully demonstrates the flexibility and accuracy of the adaptive modal distribution algorithm in dealing with complex environments.

Preferably, the fusion process of the visual feature data and the depth feature data is implemented by weighted combination, and the generated three-dimensional scene data can reflect the real state of the working area under complex environmental conditions. The specific calculation expression of the weighted combination is:

in, is the fused three-dimensional scene data; is the visual feature data; is the deep feature data; is the weight coefficient of visual feature data; is the weight coefficient of the deep feature data; and satisfies 1.

In the blind spot monitoring system for quay crane operation of the present invention, the fusion of visual feature data and depth feature data is a key step in generating high-precision three-dimensional scene data. The fusion process is achieved by weighted combination, the purpose of which is to combine the advantages of the two data sources, so as to construct a three-dimensional operation scene that can accurately reflect the actual situation in a complex operation environment.

Visual feature data mainly comes from visual sensors, which capture two-dimensional images or video information of the working area. These data can effectively provide visual features such as color, texture and shape of objects, and have great advantages in identifying object categories, contours and surface conditions. However, it is difficult to cope with light changes and occlusions in the working area by relying solely on visual feature data. In addition, visual data usually lacks depth information, making it difficult to accurately measure the spatial position between the object and the sensor.

Depth feature data comes from depth sensors and lidar, which can provide spatial distance information of objects. By measuring the distance between the object and the sensor, the depth data can clearly reflect the three-dimensional shape and relative position of the object in the operating area, especially in low light or complex environments, it can provide reliable depth perception. This is crucial for three-dimensional monitoring of the quay crane operating area, especially in the blind spots of the operating area. The depth data can provide the system with stable and continuous spatial information.

The weighted combination fusion method is to combine these two different types of data to generate more comprehensive 3D scene data. In this process, the following weighted calculation formula is used:

This formula assigns different weights to visual data and depth data, allowing the system to dynamically adjust the impact of both based on changes in the environment. For example, when the light is sufficient and the visual sensor is working well, the system can increase the weight of the visual data ( Larger values reduce the weight of depth data ( When the value is small, the 3D scene data generated can integrate the appearance and spatial position of the object to provide more complete monitoring information. When the light is insufficient or the quality of the visual sensor data is impaired, the system will increase the weight of the depth data so that the 3D scene data can still maintain high-precision spatial perception.

The fusion principle of visual feature data and depth feature data is to make full use of the advantages of the two types of sensors to form a complementary relationship. Visual sensors provide high-resolution two-dimensional image information, which is suitable for identifying the external features of objects; depth sensors provide three-dimensional spatial information, which can accurately capture the depth and distance of objects. Through weighted combination, the system can adaptively adjust under different environmental conditions to ensure that the generated three-dimensional scene data can not only show the surface details of the object, but also reflect its position in space.

Example: In a quay crane operation scenario, when the spreader is lifting cargo and approaches a blind spot, the system captures a real-time image of the spreader through a visual sensor and is able to identify the appearance and shape of the cargo and the movement of the spreader. However, due to direct sunlight, light spots appear in the visual data, affecting the accuracy of some data. At the same time, the depth sensor provides spatial distance data between the spreader and surrounding objects. Although it cannot show the appearance details of the object, it can accurately capture the relative position of the spreader. At this time, the system uses a weighted combination to reduce the weight of the visual data and increase the weight of the depth data to ensure that the generated three-dimensional scene data is still accurate and reliable. In this way, even though the visual data is interfered with, the system can still monitor obstacles and the working environment around the spreader to ensure safe operation.

Through this weighted fusion method, the system can not only generate accurate 3D scene data in an ideal environment, but also effectively make up for the defects of a single sensor in a harsh environment, ensuring monitoring accuracy and real-time performance. The final effect is to achieve continuous monitoring of the quay crane operation area. Especially in the complex environment of the blind spot, the system can dynamically adjust the weight of the sensor data, generate reliable 3D scene data, and help operators make accurate judgments on the situation in the blind spot to avoid safety accidents.

The risk assessment module extracts behavior feature data and device feature data based on 3D scene data and object tracking data, uses a nonlinear behavior prediction model to generate behavior prediction data and device trajectory prediction data, combines risk interaction analysis to generate multi-level risk assessment data, determines the risk level and triggers an early warning mechanism;

The risk assessment module monitors the behavior of personnel and the movement of equipment in the operating area in real time based on 3D scene data and object tracking data. 3D scene data can accurately reflect the relative position and spatial distribution of objects and personnel in the operating area, while object tracking data provides the system with dynamically updated trajectory information of objects and personnel. These data together constitute a comprehensive perception of the operating environment and provide a basis for subsequent risk assessment.

Behavior feature data and equipment feature data are the core information extracted from 3D scenes and object tracking data. Behavior feature data is mainly used to describe the action patterns, movement trajectories and position changes of personnel in the working area, while equipment feature data reflects the operating status and spatial position of spreaders and equipment during the operation process. By extracting these features, the system can build a dynamic model of the working area at the current moment and understand the interaction between personnel and equipment in real time.

The risk assessment module uses a nonlinear behavior prediction model to generate behavior prediction data and equipment trajectory prediction data. The advantage of the nonlinear behavior prediction model is that it can handle complex dynamic systems and capture the potential behavior trajectories of people and equipment in the future. This model predicts the possible movement paths of people and the future operating status of equipment by analyzing historical data and current features. For example, when the system recognizes that an operator is moving towards a hoist, the nonlinear behavior prediction model will predict his movement trajectory in the next few seconds based on his current speed, direction and other information, and compare it with the trajectory of the equipment.

Through risk interaction analysis, the system can combine behavior prediction data and equipment trajectory prediction data to identify potential collisions or dangerous situations between people and equipment. This process is similar to calculating the intersection point of the future trajectories of different objects and judging the risk level of the intersection point. The closer the intersection point is and the faster the relative speed between the equipment and the person, the higher the risk level determined by the system. Through this analysis, the system can identify potential dangerous areas in the operating area and issue corresponding warnings.

Multi-level risk assessment is another key feature of the risk assessment module. The system generates different levels of risk assessment data based on a combination of different risk factors, such as the speed of equipment movement, the behavior patterns of personnel, the relative distance of objects, etc. This multi-level risk assessment can refine the safety management of the operation scene. For example, the system can divide the risk level into low, medium, and high levels. Low-level risks may only require the operator's attention, while high-level risks require immediate measures to avoid accidents.

The risk assessment module activates the early warning mechanism according to the risk level. Once the system determines that there is a high risk in the operating area, the early warning mechanism will immediately issue an alarm to notify the operator or the automatic system to take corresponding protective measures. For example, when the system detects that the trajectories of the spreader and the operator may intersect in a short period of time and there is a risk of collision, the early warning mechanism will immediately issue an alarm to the operator and suggest suspending the spreader operation or issuing sound and light warnings to avoid accidents.

Example: In a complex port operation scenario, suppose an operator is approaching the spreader operation area in a blind spot. The system identifies the operator's movement trajectory through 3D scene data and object tracking data. Based on the nonlinear behavior prediction model, the system predicts that the operator will enter the spreader's operation trajectory within the next 3 seconds. At the same time, the spreader's equipment trajectory prediction data shows that the spreader will move to the area within 5 seconds. Through risk interaction analysis, the system calculates the intersection point of the trajectory of the operator and the spreader, and assesses the risk of this intersection point as a high level. Subsequently, the system triggered the early warning mechanism, issued a sound alarm, and notified the spreader operator to stop the operation in time to avoid accidents.

Preferably, the risk assessment module uses a nonlinear behavior prediction model to learn the historical behavior data of the operator to generate behavior prediction data reflecting the operator's possible future behavior. At the same time, by analyzing the equipment status, the equipment trajectory prediction data is generated to comprehensively assess the risk situation of the operating area. The calculation expression of the risk level is:

in, is the risk level; Predict data for operator behavior; is the motion trajectory of the device; is the risk assessment function.

The risk assessment module starts with the historical behavior data of the operators, and conducts deep learning and pattern recognition on these data through nonlinear behavior prediction models. Historical behavior data includes the action patterns, movement trajectories, and operating habits of operators under different environmental conditions. This prediction model can capture the behavioral trends of operators in the operating area. Even in complex and changing operating environments, the system can still predict their possible future behavior trajectories based on the past behavior of the personnel. For example, the behavior pattern of an operator may indicate that he usually stays in a specific area for a long time or often approaches a specific equipment area. By analyzing this data, the system can generate accurate behavior prediction data, thereby predicting the possible behavior of operators in the future in advance.

At the same time, the risk assessment module also conducts a comprehensive analysis of the equipment status, focusing on key features such as the equipment's motion trajectory, operating status, speed, direction, etc. By analyzing current and historical equipment data, the system is able to generate equipment trajectory prediction data, which helps the system understand the equipment's future movement path and operating mode. For example, when a spreader is moving forward at a certain speed, the system can predict the spreader's specific location in the next few seconds, as well as its possible interactions with other objects or people in the work area.

After combining the two, the system performs risk assessment through the following calculation expression:

This function comprehensively evaluates potential risks based on the relationship between behavior prediction data and device trajectory data. The evaluation function takes into account multiple factors, including the relative speed, distance, and direction of people and equipment, which together affect the system's risk level assessment results.

The core of the risk assessment function is to evaluate the possibility of danger between personnel and equipment by calculating the relative position and interaction points between the predicted behavior trajectory of the operator and the motion trajectory of the equipment. If the system detects that the behavior trajectory of the personnel and the trajectory of the equipment may intersect in the future, and the relative speed of the two is fast, the risk level will be assessed as high. The risk assessment function can dynamically adjust the risk level according to the characteristics of these interaction points, so that the system can issue different levels of warnings for different risk scenarios.

The actual effect of this risk assessment module is very significant. It can combine the dynamic behavior prediction data of personnel and equipment with environmental conditions to generate comprehensive risk assessment data, and guide operators to make reasonable decisions through quantified risk levels. For example, when the system determines that a person may enter the operating trajectory of the spreader in the next few seconds, the system will immediately issue a high-level risk alarm, prompting the operator to take emergency action to avoid accidents. In this way, the risk assessment module can not only detect potential dangers in advance, but also activate the corresponding early warning mechanism before the risk comes, providing a strong guarantee for the safe operation of the quay crane operation.

Example: Assume that at a port operation site, a worker is working in a blind spot, and the spreader is moving towards the area at a fast speed. The risk assessment module first generates the behavior prediction data of the worker through a nonlinear behavior prediction model, and concludes that he will move about 5 meters into the blind spot in the next 3 seconds. At the same time, the system generates the trajectory prediction data of the spreader by analyzing the state of the spreader, and finds that the spreader will enter the same area within 5 seconds. Through the risk assessment function, the system calculates the intersection point of the trajectory of the worker and the spreader, and assesses that the relative distance between the two is small and the risk of collision is high, so the risk level is set to "high". The system immediately triggers a high-level warning, alerts the operator, and requires the spreader operation to be suspended immediately to prevent possible collision accidents.

Through this mechanism, the system can not only accurately predict the dynamic behavior in the operating area, but also assess the possibility of future dangers in real time and issue early warnings. This module significantly improves the safety of quay crane operations, especially in blind areas. The system can accurately judge the potential dangers between personnel and equipment based on real-time monitoring and historical behavior data to ensure the safety of operators.

Preferably, the risk interaction analysis identifies potential risk interaction areas by comparing the interaction points between the operator behavior prediction data and the equipment trajectory prediction data, and evaluates each interaction point according to the set risk level standard, thereby forming multi-level risk assessment data and triggering a corresponding early warning mechanism.

Risk interaction analysis is based on two types of data that the system has generated: operator behavior prediction data and equipment trajectory prediction data. Behavior prediction data is the result of modeling and predicting the operator's historical behavior patterns, real-time locations, and movement trajectories, and can reflect the operator's future behavior trends. For example, if a worker usually moves along a specific path within the work area, the system can predict the worker's possible behavior route in the next few seconds. Equipment trajectory prediction data is generated by analyzing the operating status, speed, direction and other characteristics of the hoist or other equipment to generate the movement trajectory of the equipment in the next few seconds.

The identification of interaction points is the core of risk interaction analysis. The system compares the operator's behavior prediction data with the equipment's trajectory prediction data to find the intersection or overlapping area of the two in the future. These intersections are potential high-risk areas because in these areas, the paths of operators and equipment may overlap, leading to collisions or other dangerous events. For example, if the system predicts that a worker will enter the path of a spreader in the next 5 seconds, then the intersection of the two is an area that needs to be monitored.

Once the interaction points are identified, the system will assess each interaction point according to the set risk level criteria. The risk level assessment is usually based on the following factors:

Relative distance: The smaller the distance between the operator and the equipment, the higher the risk level. Interaction points with greater distances may only require low-level risk warnings, while interaction points with closer distances may require high-risk warnings.

Relative speed: If the personnel and equipment are moving fast, especially when they are approaching the intersection, the system will consider the risk to be greater. Fast-moving equipment and personnel are more difficult to react to in time, so the risk level will increase accordingly.

Environmental conditions: Environmental factors such as blind spots and poor lighting can also affect risk assessment. For example, if the interaction point is located in an operation blind spot, the system may assess it as a higher risk because the personnel and equipment cannot be detected by the visual monitoring system in time.

After the system comprehensively evaluates the interaction points based on these factors, it generates multi-level risk assessment data. This multi-level assessment can refine the handling of different risk situations. For example, low-risk interaction points may only require operators to pay attention, while high-risk interaction points will trigger more stringent early warning mechanisms, such as forcibly suspending equipment operations or sounding sound and light alarms.

The principle of risk interaction analysis is to identify in advance the areas where people and equipment may intersect through time series analysis and spatial comparison technology, and generate corresponding risk levels in a timely manner in combination with preset risk standards. The advantage of this mechanism is that it can provide operators with sufficient reaction time before an accident occurs, thereby avoiding the occurrence of dangerous events. Through multi-level risk assessment, the system can take different response measures according to different risk levels to ensure operational safety.

Example: Assume that during a port lifting operation, an operator is approaching the operating area of the spreader, and the spreader is moving forward at a relatively fast speed. The risk interaction analysis first uses the behavior prediction data and the equipment trajectory prediction data to find that the worker will enter the operating range of the spreader within the next 3 seconds, and the paths of the two will overlap at a certain moment. After the system identifies this interaction point, it assesses the interaction point as high risk based on factors such as distance and speed, and generates high-level risk assessment data. Subsequently, the system immediately triggers the early warning mechanism, sounds an alarm and notifies the spreader operator to temporarily stop the operation to ensure personnel safety.

Through this example, we can see that risk interaction analysis plays an important role in real-time monitoring and early warning. Through accurate identification and evaluation of interaction points, the system can not only predict possible dangers in advance, but also select appropriate response measures through a multi-level risk assessment mechanism to avoid accidents.

Risk interaction analysis can significantly improve the safety of quay crane operation blind spots. It can not only predict the potential collision risk between operators and equipment, but also refine the risk level through a multi-level evaluation mechanism to provide operators with more intuitive and timely safety tips. Combined with the automated early warning mechanism, the present invention can ensure the safe operation of personnel and equipment in complex blind spot environments and reduce the probability of accidents.

As shown in FIG3 , the control module generates optimized path data and multi-objective control data through a path optimization algorithm and a multi-objective decision algorithm based on risk assessment data, and generates dynamic control data in combination with the real-time operation status of the spreader to achieve adaptive control and real-time adjustment of the spreader operation;

The control module relies on risk assessment data. This data comes from the risk assessment module's assessment of the potential risks between operators and equipment in the operating area. The risk assessment data not only includes the classification of various risk levels, but also includes information such as the location of possible dangerous areas, time prediction, and relative distance from the spreader. Based on these assessment results, the control module can understand the potential risks in the current environment in real time and take these factors into account when formulating control strategies.

The path optimization algorithm plays a key role in the control module. Its main goal is to generate the optimal spreader operation path based on risk assessment data to ensure that the spreader can avoid dangerous areas and successfully complete the operation task. Path optimization algorithms usually consider multiple factors, including the distance between the equipment and the risk area, the speed and direction of the equipment movement, and the urgency of the operation task. For example, when the spreader is close to an operator, the path optimization algorithm will adjust the spreader's movement trajectory based on the current risk data to avoid contact with the operator. The algorithm can be combined with classic path planning methods such as the A* algorithm or the Dijkstra algorithm to ensure the efficiency and accuracy of path planning.

The multi-objective decision-making algorithm further improves the intelligence level of spreader operation on the basis of path optimization. Unlike traditional single-objective optimization, quay crane operation scenarios involve multiple objectives, such as ensuring the safety of operation and maximizing the operating efficiency of the spreader. The multi-objective decision-making algorithm generates multi-objective control data by comprehensively considering multiple constraints, that is, not only choosing a safe operating path, but also ensuring operating efficiency. Specifically, the algorithm will adjust the spreader's moving speed, direction and other parameters based on the risk assessment results in order to balance the needs of safety and efficiency. For example, when the spreader is in a high-risk area, the system may reduce its moving speed to prioritize safety; in lower-risk situations, the spreader's running speed can be increased to improve operating efficiency.

Dynamic control data is the final output of the control module. It combines the real-time operating status of the spreader and environmental changes to ensure that the spreader can make adaptive adjustments based on real-time monitoring results throughout the operation. For example, when the spreader's path is disturbed by temporary obstacles (such as other equipment or people who suddenly enter the blind spot), the control module can dynamically adjust the spreader's operating path and speed to ensure that it can avoid obstacles in time and continue to complete the task. The generation of dynamic control data relies on the response to real-time environmental changes to ensure the flexibility and safety of the spreader's operation.

The working principle of the control module is based on the comprehensive perception and evaluation of the working environment. Through the combination of path optimization algorithm and multi-objective decision algorithm, the optimized control strategy is generated in real time. The path optimization algorithm ensures that the spreader can find the safest and most effective path during the operation, while the multi-objective decision algorithm further considers the comprehensive needs of the operation on this basis to ensure that the operation of the spreader is safe and efficient. The dynamic control data finally generated can be adjusted according to real-time environmental changes, making the spreader operation adaptive and able to cope with complex and changing working environments.

Example: Assume that during a port spreader operation, the system identifies that an operator is working in a blind spot, and the risk assessment data indicates that there is a potential collision between the spreader and the operator. The control module first plans a safe path through a path optimization algorithm to avoid direct contact between the spreader and the operator. At the same time, the system comprehensively considers the urgency of the task through a multi-objective decision-making algorithm and adjusts the operating speed of the spreader to ensure that the spreader can complete the lifting task as soon as possible while avoiding the operator. The dynamic control data monitors the operating status of the spreader and the movement of personnel in real time. Once it detects that the operator is approaching the dangerous area further, the system will adjust the path again or even suspend the operation to ensure safety.

Through this flexible control method, the spreader can not only avoid potential risks smoothly, but also maximize operating efficiency while ensuring safety. This control mechanism significantly improves the automation level of quay crane operations, especially in blind spot monitoring and complex operation scenarios, which can greatly reduce the risk of collision between personnel and equipment.

The application of the control module is effective in blind spot monitoring of quay crane operations. It achieves precise control of spreader operation through a combination of path optimization and multi-objective decision-making algorithms, allowing the spreader to flexibly respond to risks in complex environments. At the same time, dynamic control data ensures that the system can adjust according to real-time conditions, making spreader operation safe and efficient. The module significantly reduces the probability of accidents while improving operational efficiency.

Preferably, the control module optimizes the path data and performs path planning based on the A-star algorithm to ensure that the spreader can effectively avoid obstacles during operation and make dynamic adjustments based on real-time feedback.

The control module initiates path planning based on the optimized path data. The optimized path data is generated by the system in combination with real-time risk assessment data, operating environment information, and equipment status. In actual applications, the optimized path data not only considers the destination that the spreader needs to reach, but also considers potential risk points in the operating area, obstacle locations, and operator behavior trajectories. For example, the risk assessment module may have identified the presence of operator or equipment activity trajectories in certain areas, and the optimized path data will mark these risk points as obstacles, requiring the path planning algorithm to avoid these areas.

The A-star algorithm (A)* is the core technology of path planning. It is a search algorithm widely used in navigation and robot control, which can find the optimal path from the starting point to the target in an environment with known obstacles. The A-star algorithm evaluates each possible path by combining heuristic estimation and actual path cost, and finally selects the path with the lowest cost. In the present invention, the control module uses the A-star algorithm to perform path planning based on the optimized path data to ensure that the sling can effectively avoid fixed obstacles (such as containers, buildings, etc.) and mobile obstacles (such as operators, mobile equipment, etc.) in the operating area. The advantage of the A-star algorithm lies in its efficiency and robustness, which can quickly find the shortest path in a complex environment while ensuring safety.

The real-time feedback mechanism plays an important role in the path planning process. Although the A-star algorithm can plan the optimal path based on the current environment, the environment may change during the operation, such as when a person suddenly enters the working area of the spreader or a new obstacle appears. At this time, the system continuously monitors the environment through sensors and monitoring data, and dynamically adjusts the path based on real-time feedback. The implementation of real-time feedback depends on the multi-dimensional data acquisition of sensors and the rapid response capability of the system. The feedback mechanism can re-evaluate the current state of the spreader and the surrounding environment within milliseconds and provide updated optimized path data to the control module. Based on this data, the A-star algorithm can quickly re-plan the path to ensure that the spreader can continue to operate safely without interrupting the operation. For example, when the spreader is close to a blind spot, the system may detect that a person is moving in the blind spot. The real-time feedback mechanism will immediately notify the control module to re-plan the path to avoid collision with the person.

The principle of path planning by the control module through the A-star algorithm relies on the input of combining optimized path data and real-time feedback. The A-star algorithm first generates the shortest path based on the optimized path data and ensures that the path can avoid all known obstacles. This planning process takes into account the movement costs between the starting point, the target point, and each node on the path, and finally selects the path with the lowest cost. Since the working environment is dynamically changing, the real-time feedback mechanism ensures that the system can adjust the path in time according to environmental changes, thereby avoiding safety hazards or work interruptions caused by emergencies.

The effect of this control module is very significant, especially in complex operating scenarios. Through path optimization and A-star algorithm, the spreader can operate flexibly in complex operating areas, not only effectively avoiding static obstacles, but also adjusting the path in time when personnel or equipment accidentally enter the operating area. Compared with the traditional fixed path planning method, this adaptive control greatly improves the flexibility and safety of the system, ensuring that the spreader can operate efficiently in blind spots and other areas with limited vision.

Example: In a typical port spreader operation scenario, assume that the spreader needs to move from the stacking area to the loading and unloading area, and there may be obstacles such as containers, other equipment, and operators on the path. The control module first plans an optimal path through the A-star algorithm based on the optimized path data to ensure that the spreader can reach the target location smoothly. During the operation, the system monitors the surrounding environment in real time through devices such as lidar, visual sensors, and depth sensors, and finds that an operator enters the travel path of the spreader. The real-time feedback mechanism immediately captures this change and transmits it to the control module. The control module re-plans the path through the A-star algorithm to ensure that the spreader can avoid the operator and complete the transportation task as soon as possible.

Through this flexible path adjustment mechanism, the spreader can not only safely avoid obstacles, but also ensure the continuity and efficiency of operations. Even in complex operating environments, the system can still respond quickly and adjust the path to deal with new emergencies.

The control module significantly improves the operating efficiency and safety of the spreader through the combination of A-star algorithm path planning and real-time feedback. Compared with the traditional fixed path control method, this adaptive path planning system can effectively respond to dynamic changes in the operating environment, ensure the safe operation of the spreader in the complex port environment, and reduce the occurrence of accidents. At the same time, the dynamic adjustment function ensures that the operation can be resumed quickly without long interruptions due to temporary obstacles or personnel movement.

Preferably, the multi-objective decision-making algorithm is implemented by the following optimization function:

in, Optimize results for your goals; For the The weight of the control target; For the Control cost of each control target; To control the number of targets.

During the operation of port spreaders, the spreaders must not only safely avoid operators and equipment, but also efficiently complete the lifting task to ensure the smooth progress of the operation. In order to meet these different control objectives at the same time, the multi-objective decision algorithm generates optimized control instructions by comprehensively considering multiple factors. These control objectives may include the following aspects:

Safety: Avoid collisions between the spreader and the operator or equipment, which is usually the target with higher weight (larger ); Operation efficiency: The spreader needs to complete the lifting operation as quickly as possible to ensure the progress of the operation, so the efficiency target also has a high weight; Energy consumption control: The operation of the spreader should minimize energy consumption, which is usually included in the optimization model as a secondary goal to control its cost (i.e. lower ).

The optimization function of the multi-objective decision-making algorithm finds a balance between these objectives. By adjusting the weight of each objective , the system can dynamically optimize the control strategy according to different operating environments and priorities. For example, when there is no human activity in the operating area, the system may prioritize improving operating efficiency and reducing the waiting time and operation cycle of the spreader. However, in an environment with dense personnel or complex equipment, the system will increase the weight of the safety goal to ensure that the spreader can prioritize avoiding personnel and equipment to reduce the risk of accidents.

Control costs It is the cost or expense of each control objective, which is usually calculated dynamically by the system based on real-time data. For example, when considering safety objectives, the system may evaluate factors such as the distance between the spreader and personnel or obstacles, the movement speed of the equipment, etc. If the spreader approaches a high-risk area, the control cost of the safety objective will increase ( Similarly, the cost of an energy consumption control target may be calculated based on the distance the spreader moves and the operating time. When the spreader is operated frequently or the path is too long, the energy consumption cost will increase.

By dynamically adjusting the weights and costs of each control objective, the multi-objective decision-making algorithm can achieve a balance between different objectives and output the optimal control strategy. This optimization method not only ensures the safety and efficiency of the spreader operation, but also takes into account additional factors such as energy consumption in actual operation, improving the overall performance of the system.

The effect of the multi-objective decision-making algorithm is that it provides the system with multi-dimensional decision-making capabilities and can flexibly adjust the optimization strategy according to different environments and operating requirements. Through the optimization function, the system can not only achieve excellent performance on a single goal (such as safety or efficiency), but also reasonably balance multiple goals. This flexibility enables the system to quickly adapt to changes in complex operating environments, avoiding safety accidents and ensuring the efficient completion of spreader operations.

Example: In an actual port spreader operation scenario, suppose the spreader is near a high-risk blind spot, and the system finds through risk assessment data that an operator is approaching the front area. At this time, the multi-objective decision algorithm will give priority to safety and weight the safety goal. Improve while reducing the weight of efficiency goals The system calculates through optimization functions that the spreader should reduce its moving speed and adjust its path to avoid potential dangerous areas. At the same time, the energy consumption target It may be appropriately reduced, allowing a slight increase in energy consumption in the short term to ensure safety. After the spreader successfully avoids the dangerous area, the system will readjust the weights of each control target, gradually improve efficiency and energy consumption control, and resume normal operation.

In this way, the multi-objective decision-making algorithm ensures the safe operation of the spreader in blind spots and high-risk environments, while also being able to flexibly adapt to different operational requirements to ensure the optimal balance between efficiency and safety.

Through the multi-objective decision-making algorithm, the quay crane operation blind spot monitoring system can intelligently manage all aspects of the spreader operation and dynamically balance safety, efficiency and energy consumption control. The system can flexibly adjust the optimization target according to the real-time operating environment and equipment status to ensure that the spreader can complete the task safely, efficiently and economically in a complex environment. This comprehensive decision-making ability greatly improves the adaptability and robustness of the system, enabling it to cope with various emergencies and ensure the smooth progress of operations.

The self-correction module collects feedback perception data and operation status data during the operation of the spreader, generates self-correction data through the feedback self-correction algorithm, and adjusts the operation path.

Preferably, the self-correction module monitors and records the operation status data and feedback perception data in real time during the operation of the spreader to generate self-correction data reflecting the current operation accuracy and environmental adaptability, and makes necessary adjustments to the spreader operation path based on the self-correction data. The calculation expression of the path adjustment is:

in, is the adjusted path; is the current path; is the path change amount adjusted according to the self-correction data.

The working principle of the self-correction module mainly relies on two important data inputs: operation status data and feedback perception data. Operation status data refers to the real-time monitoring results of various parameters such as position, speed, acceleration, etc. of the spreader during operation. These data are collected by sensors and reflect the actual situation of the spreader in the current operation. On the other hand, feedback perception data is the interaction information between the spreader and the external environment, including real-time feedback from other sensors (such as lidar, depth sensor, etc.). These feedback perception data help the system identify changes in the environment around the spreader, such as new obstacles or operators entering the working range of the spreader.

Based on this data, the system generates self-correction data, which is a reflection of the current operational deviation. For example, if the spreader deviates from the predetermined operating path, or the operating accuracy is affected by environmental changes, the system will calculate the amount of deviation that needs to be corrected based on the actual operation and feedback perception data. The generation of self-correction data takes into account the real-time position of the spreader, external environmental factors, and the accuracy required for the operation. Through calculation, the system can promptly detect and correct the operating deviation of the spreader to ensure that the spreader can always operate on the optimal path.

The working principle of the self-correction module is a continuous closed-loop feedback process. By monitoring the operating status and feeding back the perception data in real time, the system can quickly detect any deviation in the spreader operation and adjust the path in real time according to the deviation amount. This self-correction capability ensures that the spreader can always adapt to the complex operating environment and avoid the accumulation of deviations caused by environmental changes. The role of the self-correction module is particularly important in blind areas of operation or areas with frequent dynamic changes in the environment. It can make path adjustments within milliseconds to ensure that the spreader can respond quickly to emergencies.

The effects of this module are reflected in the following aspects:

Improved operational accuracy: Through real-time path adjustment, the self-correction module can ensure that the operating accuracy of the spreader is always maintained at a high level. Even in complex environments, the spreader can accurately perform operational tasks as planned.

Enhanced environmental adaptability: Since the feedback perception data can continuously reflect the interaction between the spreader and the surrounding environment, the self-correction module can adapt to dynamic changes in the environment, such as the sudden appearance of obstacles or the movement of operators. The system can respond quickly through path adjustment to avoid safety risks.

Real-time adjustment capability: The self-correction module can fine-tune the path based on real-time data, which means that even the smallest deviation can be corrected in the shortest time, ensuring that the spreader will not make large operating errors.

Example: Assume that a spreader is performing a lifting operation in a port operation area. During the spreader's movement, a forklift suddenly enters the spreader's working area, causing the spreader to deviate from the planned path. The system detects the deviation between the spreader's actual path and the planned path through the self-correction module. At the same time, the feedback perception data also shows that the forklift's entry has interfered with the spreader's path. Based on this data, the self-correction module calculates the path change that needs to be adjusted. , and immediately adjust the spreader path , redirecting it to a safe path to avoid collision with the forklift while continuing to complete the lifting task.

This example demonstrates the adaptability of the self-correction module in a dynamic environment. By quickly generating self-correction data and adjusting the path in real time, the system is able to ensure safe operation and efficient operation of the spreader in a complex and ever-changing operating environment.

Preferably, the self-correction module analyzes the performance of the spreader under different operating conditions in combination with historical operating data to optimize the parameter settings of each algorithm in the control module so that future operating paths can be self-adjusted based on the accumulated operating data and environmental changes.

The self-correction module records and analyzes historical operating data. This data includes the specific performance of the spreader under different operating conditions, such as how the spreader operates in different weather conditions, load conditions, and operating environments. Through in-depth analysis of these historical data, the system can identify the operating mode of the spreader under different environmental conditions and common deviations that occur under specific conditions. For example, in strong wind conditions, the spreader may be more likely to deviate from the intended path, while the operating speed and acceleration of the spreader will be greatly affected when fully loaded. The historical operating data provides important reference information for the system, enabling it to accurately understand the performance of the spreader under various operating conditions and improve it in future operations.

Based on the analysis of these historical data, the self-correction module can further optimize the parameter settings of each algorithm in the control module. The algorithms in the control module (such as the path optimization algorithm and the multi-objective decision algorithm) rely on multiple parameters to adjust the operation mode of the spreader, and the settings of these parameters directly affect the operating accuracy and efficiency of the spreader. For example, the target weight in the path optimization algorithm and the threshold setting in the decision algorithm will be adjusted according to the needs of different operating scenarios. The self-correction module analyzes the performance of the spreader under different operating conditions and dynamically adjusts these algorithm parameters, so that the control module can better adapt to real-time changing environmental conditions. This optimization process not only improves the accuracy of the current operation, but also provides better algorithm adaptability for future operations.

The self-correction module self-adjusts the future operation path according to the optimized algorithm parameters. The principle of this self-adjustment is based on a comprehensive analysis of the accumulated operation data and environmental changes. As the system runs more and more times in different operating environments, the accumulation of historical data enables the self-correction module to have a more comprehensive understanding of the environment, and thus be able to predict possible deviations of the spreader under specific conditions. For example, if the system learns through analysis that the spreader usually needs to reduce speed on slippery surfaces on rainy days to ensure safety, the self-correction module will automatically adjust the spreader's speed parameters to ensure path accuracy when encountering similar conditions in the future. At the same time, the dynamic changes in the environment will also be incorporated into the system's adjustment strategy. By combining real-time environmental feedback and historical data, the self-correction module can provide the best path adjustment solution for the spreader.

The self-correction module combines historical operation data and the self-adjustment function of environmental changes. In fact, it improves the robustness and operation efficiency of the system through continuous learning and optimization. Its working principle is similar to the iterative optimization process in machine learning. Through the accumulation of operation data each time, the system can gradually improve future decision-making and control methods. This self-correction mechanism based on historical data can not only reduce the operation error of the system, but also predict possible problems in the future and make adjustments before the problems occur.

The self-correction module plays a significant role in improving the operating efficiency and safety of the spreader. First, by analyzing historical data, the system can better understand the best operating strategy under different operating conditions, reduce human intervention in operations, and increase the degree of automation. Second, the self-correction module ensures that the system can respond to environmental changes in real time, quickly adjust the path when unexpected situations occur, and reduce the probability of accidents. For example, when operating in a port, if past data shows that the spreader often deviates when approaching certain obstacles, the system will automatically adjust the operation mode in these areas to avoid similar errors.

Example: Assume that during a certain spreader operation, the weather suddenly deteriorates. The system records the spreader's operating data in the rain through the self-correction module, including problems such as increased sliding distance and decreased operating accuracy. In subsequent spreader operations, when the system encounters similar weather again, the self-correction module will automatically reduce the spreader's operating speed through historical data analysis and adjust the relevant parameters in the path optimization algorithm to ensure that the spreader can operate safely under slippery conditions. As the number of operations increases, the adjustment effect of the self-correction module becomes more and more significant, and the system gradually becomes able to flexibly respond to and make precise adjustments under various harsh conditions.

Preferably, the self-correction module uses a recursive least squares method to perform optimization calculations to improve the adjustment accuracy of the spreader operation path and control strategy.

The principle of Recursive Least Squares (RLS) is to dynamically adjust the control parameters of the system by optimizing and estimating the operation data of the spreader (such as position, speed, acceleration, etc.) and the monitoring data of environmental changes in real time. When processing the input and output data of the system, the recursive least squares method can gradually optimize the system model parameters in a recursive manner to reduce errors, so that the system can maintain high control accuracy in a rapidly changing operating environment.

Compared with the traditional least squares method, the recursive least squares method can update the system model immediately after each new data is input, without recalculating all the data. This makes it particularly efficient in real-time control. For the operation of the spreader, the RLS method can optimize the spreader's operation path and control strategy through the continuously accumulated data, ensuring that the spreader always maintains an accurate operation path in a complex operating environment.

In the present invention, the operation path and control strategy of the spreader are adjusted according to the real-time collected operation status data and feedback perception data. These data may include the real-time position, speed, acceleration of the spreader, the distance between the spreader and surrounding objects (such as operators, obstacles), etc. The self-correction module processes these data through the recursive least squares method, calculates the current state of the system, and generates new path adjustment data or control instructions to ensure that the spreader can run according to the optimal path.

In the recursive least squares method, the goal of the spreader operation is to minimize the path error and control error, that is, to ensure that the deviation between the actual operation path of the spreader and the predetermined path is minimized, and the control output of the system can adapt to the dynamic environment. RLS continuously adjusts the model parameters of the system in an iterative manner, allowing the spreader to accurately self-adjust in response to environmental changes. For example, the actual path of the spreader may be offset due to external interference (such as wind, heavy load), and the system estimates the cause of this deviation through RLS, and promptly corrects the control strategy to pull the spreader back to the predetermined path.

Another advantage of recursive least squares optimization is its anti-interference ability. In the operation area, environmental factors (such as the movement of obstacles, the intervention of personnel, weather changes, etc.) may cause the operating accuracy of the spreader to decrease. Through recursive least squares optimization, the system can intervene in the early stage of deviation caused by these factors, and prevent the deviation from accumulating into a large operating error through small path and control adjustments.

RLS can quickly process and calculate each new data point, allowing the system to update the operation path and control parameters in real time, maintaining high flexibility and accuracy in complex environments; the system can recursively accumulate historical operation data, making future control strategies more intelligent. As the number of operations increases, the accuracy and efficiency of the spreader operation will continue to improve; the RLS algorithm can quickly identify interference factors in the operation and make timely adjustments to ensure that the spreader operation path is not overly affected by the external environment. This is especially critical in blind spot operations or areas with limited field of view, avoiding operational errors caused by limited visual monitoring or sudden changes in the environment.

In a typical port spreader operation scenario, assume that the spreader is performing a heavy load operation, and due to strong winds, the spreader deviates from the original operating path during the operation. The self-correction module monitors the deviation between the actual position of the spreader and the target path in real time through sensors, and calculates the path adjustment amount through recursive least squares method. Based on the calculation results, the system dynamically adjusts the operating parameters of the spreader and redirects the spreader back to the target path. In addition, the RLS algorithm can also adjust future operating paths and control strategies according to the pattern of wind changes to avoid similar deviations from happening again. In this way, the spreader can maintain high operating accuracy under strong wind conditions and continue to operate efficiently.

Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems or computer program products. Therefore, the present application may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware.

The above are only embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.

Claims (10)

1. A blind spot monitoring system for quay crane operation, characterized by comprising: A data acquisition module collects multi-dimensional environmental data of the operating area through visual sensors, laser radars and depth sensors, and pre-processes the multi-dimensional environmental data to generate optimized fused perception data; The scene construction module extracts visual feature data and depth feature data based on optimized fusion perception data, uses an adaptive modal distribution algorithm to dynamically adjust the weights of different data sources, and fuses multi-source data to generate three-dimensional scene data and object tracking data to build a real-time updated three-dimensional operation scene; The risk assessment module extracts behavior feature data and device feature data based on 3D scene data and object tracking data, uses a nonlinear behavior prediction model to generate behavior prediction data and device trajectory prediction data, combines risk interaction analysis to generate multi-level risk assessment data, determines the risk level and triggers an early warning mechanism; The control module generates optimized path data and multi-objective control data based on risk assessment data through path optimization algorithm and multi-objective decision-making algorithm, and generates dynamic control data in combination with the real-time operation status of the spreader to achieve adaptive control and real-time adjustment of the spreader operation; The self-correction module collects feedback perception data and operation status data during the operation of the spreader, generates self-correction data through the feedback self-correction algorithm, and adjusts the operation path. 2. The blind spot monitoring system for quay crane operations according to claim 1 is characterized in that the adaptive modal distribution algorithm in the scene construction module is used to dynamically adjust the weight of each data source according to the difference between the multi-dimensional environmental data collected by different sensors, so that when the environment changes, more reliable data sources can be given priority, thereby generating accurate three-dimensional scene data and object tracking data, and the calculation expression of the weight is: in, For the The weight of the data source, For the Data source and The degree of difference between the data sources; is the number of data sources. 3. The blind spot monitoring system for quay crane operation according to claim 1 is characterized in that the fusion process of the visual feature data and the depth feature data is realized by weighted combination, and the generated three-dimensional scene data can reflect the real state of the operation area under complex environmental conditions, and the specific calculation expression of the weighted combination is: in, is the fused three-dimensional scene data; is the visual feature data; is the deep feature data; is the weight coefficient of visual feature data; is the weight coefficient of the deep feature data; and satisfies 1. 4. The blind spot monitoring system for quay crane operation according to claim 1 is characterized in that the risk assessment module uses a nonlinear behavior prediction model to learn the historical behavior data of the operator to generate behavior prediction data reflecting the possible future behavior of the operator. At the same time, by analyzing the equipment status, the equipment trajectory prediction data is generated, so as to comprehensively evaluate the risk situation of the operation area. The calculation expression of the risk level is: in, is the risk level; Predict data for operator behavior; is the motion trajectory of the device; is the risk assessment function. 5. The blind spot monitoring system for quay crane operations according to claim 4 is characterized in that the risk interaction analysis identifies potential risk interaction areas by comparing the interaction points between the operator behavior prediction data and the equipment trajectory prediction data, and evaluates each interaction point according to the set risk level standard, thereby forming multi-level risk assessment data and triggering a corresponding early warning mechanism. 6. The blind spot monitoring system for quay crane operations according to claim 1 is characterized in that the control module optimizes path data and performs path planning based on the A-star algorithm to ensure that the spreader can effectively avoid obstacles during operation and make dynamic adjustments based on real-time feedback. 7. The blind spot monitoring system for quay crane operation according to claim 1, characterized in that the multi-objective decision algorithm is implemented by the following optimization function: in, Optimize results for your goals; For the The weight of the control target; For the Control cost of each control target; To control the number of targets. 8. The blind spot monitoring system for quay crane operation according to claim 1 is characterized in that the self-correction module monitors and records the operation status data and feedback perception data in real time during the operation of the spreader to generate self-correction data reflecting the current operation accuracy and environmental adaptability, and makes necessary adjustments to the spreader operation path based on the self-correction data, and the calculation expression of the path adjustment is: in, is the adjusted path; is the current path; is the path change amount adjusted according to the self-correction data. 9. The blind spot monitoring system for quay crane operations according to claim 1 is characterized in that the self-correction module analyzes the performance of the sling under different operating conditions in combination with historical operating data to optimize the parameter settings of each algorithm in the control module, so that the future operation path can be self-adjusted based on the accumulated operating data and environmental changes. 10. The blind spot monitoring system for quay crane operations according to claim 1, characterized in that the self-correction module uses recursive least squares method to perform optimization calculations to improve the adjustment accuracy of the spreader operation path and control strategy.