CN114604772B

CN114604772B - Intelligent tower crane cluster cooperative control method and system for task temporal model

Info

Publication number: CN114604772B
Application number: CN202210077273.0A
Authority: CN
Inventors: 陈德木; 蒋云; 赵晓东; 陆建江; 陈曦
Original assignee: Hangzhou Dajie Intelligent Transmission Technology Co Ltd
Current assignee: Hangzhou Dajie Intelligent Transmission Technology Co Ltd
Priority date: 2022-01-24
Filing date: 2022-01-24
Publication date: 2023-06-02
Anticipated expiration: 2042-01-24
Also published as: CN114604772A

Abstract

The application provides an intelligent tower crane cluster cooperative control method and system for a task temporal model. The intelligent tower crane cluster cooperative control method for the task temporal model comprises the following steps: task information of the latest task to be executed of each intelligent tower crane in the intelligent tower crane cluster is obtained to obtain a task information set, wherein the task information comprises task execution time and an action area of the intelligent tower crane for executing the task to be executed; task conflict information of each intelligent tower crane is determined according to the task information set; sequencing the tasks to be executed of each intelligent tower crane according to the task conflict information to obtain a task queue; and controlling each intelligent tower crane to execute the corresponding task to be executed according to the task queue. According to the intelligent tower crane cluster control method and device, cooperative control of the intelligent tower crane clusters can be achieved to avoid collision accidents, and therefore intelligent and automatic levels and safety of the intelligent tower crane are improved.

Description

Intelligent tower crane cluster cooperative control method and system for task temporal model

Technical Field

The application relates to the technical field of intelligent tower cranes, in particular to an intelligent tower crane cluster cooperative control method and system for a task temporal model.

Background

Along with the development of the building industry, the mechanization degree of building construction is improved year by year, and a tower crane (tower crane for short) is used as a machine capable of realizing vertical and horizontal material transportation, and is widely applied in the building industry particularly due to the characteristics of high lifting height, large lifting weight, large working range and the like.

Along with frequent occurrence of tower crane safety accidents, in order to protect personal safety of tower crane operators and span workers and reduce safety accidents caused by human errors, an unmanned tower crane, namely an intelligent tower crane, becomes a new research and development direction, and as a construction site is often a centralized operation of a plurality of intelligent tower cranes, how to realize cooperative control of an intelligent tower crane cluster so as to avoid collision accidents becomes a current problem to be solved urgently.

Disclosure of Invention

The purpose of the application is to provide an intelligent tower crane cluster cooperative control method and system for a task temporal model.

The first aspect of the application provides an intelligent tower crane cluster cooperative control method for a task temporal model, which comprises the following steps:

Task information of the latest task to be executed of each intelligent tower crane in the intelligent tower crane cluster is obtained to obtain a task information set, wherein the task information comprises task execution time and an action area of the intelligent tower crane for executing the task to be executed;

task conflict information of each intelligent tower crane is determined according to the task information set;

sequencing the tasks to be executed of each intelligent tower crane according to the task conflict information to obtain a task queue;

and controlling each intelligent tower crane to execute the corresponding task to be executed according to the task queue.

A second aspect of the present application provides an intelligent tower crane cluster cooperative control system for a task tense model, including:

the system comprises a task to be executed acquisition module, a task processing module and a task processing module, wherein the task acquisition module is used for acquiring task information of a latest task to be executed of each intelligent tower crane in an intelligent tower crane cluster to obtain a task information set, and the task information comprises task execution time and an action area of the intelligent tower crane for executing the task to be executed;

the conflict information determining module is used for determining task conflict information of each intelligent tower crane according to the task information set;

the task ordering module is used for ordering the tasks to be executed of each intelligent tower crane according to the task conflict information to obtain a task queue;

And the task execution module is used for controlling each intelligent tower crane to execute the corresponding task to be executed according to the task queue.

The third aspect of the application provides an intelligent tower crane, which is provided with the intelligent tower crane cluster cooperative control system for the task tense model provided by the first aspect of the application.

Compared with the prior art, the intelligent tower crane cluster cooperative control method and system for the task temporal model obtain a task information set by acquiring task information of the latest task to be executed of each intelligent tower crane in the intelligent tower crane cluster, wherein the task information comprises task execution time and an action area of the intelligent tower crane for executing the task to be executed; task conflict information of each intelligent tower crane is determined according to the task information set; sequencing the tasks to be executed of each intelligent tower crane according to the task conflict information to obtain a task queue; and controlling each intelligent tower crane to execute the corresponding task to be executed according to the task queue. Therefore, cooperative control of the intelligent tower crane clusters can be realized to avoid collision accidents, and the intelligent and automatic level and safety of the intelligent tower crane are improved.

Drawings

Various other advantages and benefits will become apparent to those of ordinary skill in the art upon reading the following detailed description of the preferred embodiments. The drawings are only for purposes of illustrating the preferred embodiments and are not to be construed as limiting the application. Also, like reference numerals are used to designate like parts throughout the figures. In the drawings:

FIG. 1 illustrates a flow chart of an intelligent tower crane cluster cooperative control method for a task temporal model provided by some embodiments of the present application;

fig. 2 illustrates a schematic diagram of an intelligent tower crane cluster cooperative control system for a task temporal model according to some embodiments of the present application.

Detailed Description

Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be embodied in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.

It is noted that unless otherwise indicated, technical or scientific terms used herein should be given the ordinary meaning as understood by one of ordinary skill in the art to which this application belongs.

In addition, the terms "first" and "second" etc. are used to distinguish different objects and are not used to describe a particular order. Furthermore, the terms "comprise" and "have," as well as any variations thereof, are intended to cover a non-exclusive inclusion. For example, a process, method, system, article, or apparatus that comprises a list of steps or elements is not limited to only those listed steps or elements but may include other steps or elements not listed or inherent to such process, method, article, or apparatus.

The embodiment of the application provides an intelligent tower crane cluster cooperative control method and system for a task temporal model, and the method and system are exemplified by the embodiment and the accompanying drawings.

Referring to fig. 1, a flowchart of an intelligent tower crane cluster cooperative control method for a task temporal model according to some embodiments of the present application is shown, as shown in fig. 1, where the intelligent tower crane cluster cooperative control method for a task temporal model may include the following steps:

Step S101: task information of the latest task to be executed of each intelligent tower crane in the intelligent tower crane cluster is obtained to obtain a task information set, wherein the task information comprises task execution time and an action area of the intelligent tower crane for executing the task to be executed;

step S102: task conflict information of each intelligent tower crane is determined according to the task information set;

step S103: sequencing the tasks to be executed of each intelligent tower crane according to the task conflict information to obtain a task queue;

step S104: and controlling each intelligent tower crane to execute the corresponding task to be executed according to the task queue.

Compared with the prior art, the intelligent tower crane cluster cooperative control method for the task temporal model obtains a task information set by obtaining task information of the latest task to be executed of each intelligent tower crane in the intelligent tower crane cluster, wherein the task information comprises task execution time and an action area of the intelligent tower crane for executing the task to be executed; task conflict information of each intelligent tower crane is determined according to the task information set; sequencing the tasks to be executed of each intelligent tower crane according to the task conflict information to obtain a task queue; and controlling each intelligent tower crane to execute the corresponding task to be executed according to the task queue. Therefore, cooperative control of the intelligent tower crane clusters can be realized to avoid collision accidents, and the intelligent and automatic level and safety of the intelligent tower crane are improved.

Wherein, in some embodiments, the task conflict information includes a number of task conflicts;

the task conflict information of each intelligent tower crane is determined according to the task information set, and the task conflict information comprises:

judging whether the action area corresponding to each intelligent tower crane has an intersecting area or not and corresponding to other surrounding intelligent tower cranes in sequence, if the action area does not have the intersecting area, determining that the task conflict number of the intelligent tower cranes is zero, if the action area does not have the intersecting area, judging whether the task execution time of the intelligent tower crane and the other intelligent tower cranes with the intersecting area has intersecting time, if the intelligent tower crane does not have the intersecting time, determining that the task conflict number of the intelligent tower cranes is zero, and if the intelligent tower cranes have the intersecting time, recording the task conflict number as the task conflict number.

Based on any of the foregoing embodiments, in some modified embodiments, the sorting the tasks to be performed of each intelligent tower crane according to the task conflict information includes:

the tasks to be executed of each intelligent tower crane are initially ordered according to the sequence of the task conflict quantity from small to large, and are grouped according to the different task conflict quantities, so that a task queue is obtained;

Aiming at tasks to be executed, the conflict quantity of which is nonzero and the same, the tasks to be executed are secondarily ordered in the group according to the sequence from short to long of the duration of the crossing time;

and for the tasks to be executed, which are adjacent and have the same crossing time after the secondary sequencing, sequencing for three times according to the sequence from short to long of the task execution time of the tasks to be executed, so as to obtain a task queue.

On the basis of the foregoing embodiments, in some modified embodiments, the controlling, according to the task queue, each intelligent tower crane to execute the corresponding task to be executed includes:

traversing each task to be executed in the task queue, triggering to immediately execute the task to be executed if the task conflict number corresponding to the task to be executed is zero, judging whether other tasks to be executed which conflict with the task to be executed exist or not to be executed before being executed and finishing, triggering to immediately execute the task to be executed if not, and temporarily not executing the task to be executed and skipping the task to be executed if not.

On the basis of the above embodiments, in some modified embodiments, the method further includes:

Monitoring the execution condition of each task to be executed in the task queue;

after the completion of the execution of any one of the tasks to be executed is monitored, deleting the task to be executed from the task queue, and triggering the execution of the step of acquiring the task information of the latest task to be executed of each intelligent tower crane in the intelligent tower crane cluster.

In the above embodiment, an intelligent tower crane cluster cooperative control method for a task temporal model is provided, and correspondingly, the application also provides an intelligent tower crane cluster cooperative control system for the task temporal model. The intelligent tower crane cluster cooperative control system for the task temporal model provided by the embodiment of the application can implement the intelligent tower crane cluster cooperative control method for the task temporal model, and the intelligent tower crane cluster cooperative control system for the task temporal model can be realized in a mode of software, hardware or combination of software and hardware. For example, the intelligent tower crane cluster cooperative control system for a task tense model may include integrated or separate functional modules or units to perform the corresponding steps in the methods described above. Referring to fig. 2, a schematic diagram of an intelligent tower crane cluster cooperative control system for a task temporal model according to some embodiments of the present application is schematically shown. Since the system embodiments are substantially similar to the method embodiments, the description is relatively simple, and reference should be made to the description of the method embodiments for relevant points. The system embodiments described below are merely illustrative.

As shown in fig. 2, an embodiment of the present application provides an intelligent tower crane cluster cooperative control system for a task temporal model, which may include:

the task to be executed acquisition module 101 is configured to acquire task information of a task to be executed latest for each intelligent tower crane in an intelligent tower crane cluster, and obtain a task information set, where the task information includes task execution time and an action area of the intelligent tower crane to execute the task to be executed;

the conflict information determining module 102 is configured to determine task conflict information of each intelligent tower crane according to the task information set;

the task ordering module 103 is configured to order tasks to be executed of each intelligent tower crane according to the task conflict information, so as to obtain a task queue;

and the task execution module 104 is configured to control each intelligent tower crane to execute a corresponding task to be executed according to the task queue.

In some variation of the embodiment of the present application, the task conflict information includes a task conflict number;

the conflict information determining module 102 includes:

the conflict information determining unit is used for sequentially determining whether the action area corresponding to each intelligent tower crane has an intersection area or not and the action areas corresponding to other surrounding intelligent tower cranes, if the action areas have no intersection area, determining that the task conflict number of the intelligent tower cranes is zero, if the action areas have intersection areas, determining whether the task execution time of the intelligent tower cranes and the task execution time of the other intelligent tower cranes with the intersection areas exist or not, if the action areas have no intersection time, determining that the task conflict number of the intelligent tower cranes is zero, and if the action areas have intersection time, recording the task conflict number of the intelligent tower cranes as the task conflict number.

In some variant implementations of the embodiments of the present application, the task ordering module 103 includes:

the primary sequencing unit is used for primarily sequencing the tasks to be executed of each intelligent tower crane according to the sequence of the task conflict quantity from small to large, and grouping the tasks to be executed according to the different task conflict quantities to obtain a task queue;

the secondary sequencing unit is used for aiming at tasks to be executed, the number of conflicts of which is non-zero and the same, and performing secondary sequencing on the tasks to be executed in the group according to the sequence from short to long of the duration of the crossing time;

and the third sequencing unit is used for sequencing the tasks to be executed, which are adjacent to each other and have the same crossing time after the second sequencing, for three times according to the sequence from short to long of the task execution time of the tasks to be executed, so as to obtain a task queue.

In some variant implementations of the embodiments of the present application, the task execution module 104 includes:

and the task execution unit is used for traversing each task to be executed in the task queue, triggering to execute the task to be executed immediately if the task conflict number corresponding to the task to be executed is zero, judging whether other tasks to be executed which conflict with the task to be executed exist and are not executed before being executed, if not, triggering to execute the task to be executed immediately, and if not, temporarily not executing the task to be executed and skipping the task to be executed.

In some variations of the embodiments of the present application, the apparatus further includes:

the execution condition monitoring module is used for monitoring the execution condition of each task to be executed in the task queue;

and the execution completion processing module is used for deleting the task to be executed from the task queue after monitoring that any task to be executed is completed, and triggering and executing the step of acquiring the task information of the latest task to be executed of each intelligent tower crane in the intelligent tower crane cluster.

The intelligent tower crane cluster cooperative control system for the task temporal model, which is provided by the embodiment of the application, has the same beneficial effects as the intelligent tower crane cluster cooperative control method for the task temporal model, which is provided by the previous embodiment of the application, is based on the same inventive concept.

The embodiment of the application also provides an intelligent tower crane corresponding to the intelligent tower crane cluster cooperative control method and system for the task temporal model provided by the previous embodiment, wherein the intelligent tower crane is provided with the intelligent tower crane cluster cooperative control system for the task temporal model provided by any of the previous embodiments, and the intelligent tower crane cluster cooperative control method for the task temporal model provided by any of the previous embodiments can be used for executing.

The intelligent tower crane provided by the embodiment of the application has the same beneficial effects as the intelligent tower crane cluster cooperative control method and system for the task temporal model provided by the previous embodiment of the application because of the same inventive concept.

In addition, in order to further perfect the intellectualization and unmanned of the intelligent tower crane, the intelligent tower crane can also reduce the lifting safety accident of the tower crane by configuring the following sensing equipment for the automatic grabbing process of the lifting hook of the tower crane, and the following description is made with reference to examples.

In some embodiments, the sensing device for an automatic grabbing process of a tower crane hook may include: the device comprises a controller, an automatic lifting hook, a lifting hook driving mechanism, a visual sensor and a sensor driving mechanism; wherein, the liquid crystal display device comprises a liquid crystal display device,

the automatic lifting hook is connected with the lifting hook driving mechanism, the visual sensor is connected with the sensor driving mechanism, and the lifting hook driving mechanism, the sensor driving mechanism and the visual sensor are all connected with the controller;

when the controller controls the automatic lifting hook to move through the lifting hook driving mechanism, the controller also controls the vision sensor to follow the automatic lifting hook to move through the sensor driving mechanism, and controls the vision sensor to acquire vision sensing signals towards the area where the automatic lifting hook is located, so that the automatic lifting hook is controlled to grab goods according to the vision sensing signals.

The controller can be implemented by a computer host, a microcontroller, a Programmable Logic Controller (PLC) and the like, and the automatic lifting hook can be implemented by any automatic lifting hook provided by the prior art.

It should be noted that, if the tower crane is an unmanned tower crane, the controller may be disposed on a control platform on the ground, and a display screen is disposed on the control platform, so that a tower crane operator views a live image around the lifting hook through the display screen, and accordingly, the automatic lifting hook is controlled to grasp goods according to the vision sensing signal, and the control of the automatic lifting hook by the vision sensing signal may be that the vision sensing signal is played through the display screen disposed on the control platform, so that the tower crane operator accurately knows the condition around the lifting hook and controls the automatic lifting hook to automatically hook the goods.

In addition, the controller can be connected with the vision sensor and the sensor driving mechanism in a wireless mode or in a wired mode, and in consideration of relatively poor stability of wireless signals, safety accidents possibly caused by signal interruption and errors, in some embodiments, the vision sensor and the sensor driving mechanism are preferably connected with the controller in a wired mode by using a cable, and particularly, the cable can be connected to a console on the ground along a crane arm and a standard section and connected with the controller on the console, so that signal quality and stability are improved, and the safety accidents caused by signal problems are avoided.

By configuring the intelligent tower crane with the sensing equipment for the automatic grabbing process of the tower crane lifting hook, the automatic lifting hook is connected with the lifting hook driving mechanism by arranging the controller, the automatic lifting hook, the lifting hook driving mechanism, the vision sensor and the sensor driving mechanism, the vision sensor is connected with the sensor driving mechanism, the lifting hook driving mechanism, the sensor driving mechanism and the vision sensor are all connected with the controller, and the controller also controls the vision sensor to follow the automatic lifting hook to move through the sensor driving mechanism when controlling the automatic lifting hook to move through the lifting hook driving mechanism, controls the vision sensor to acquire the vision sensing signal towards the area where the automatic lifting hook is positioned so as to control the automatic lifting hook to grab goods according to the vision sensing signal, therefore, the visual sensor can acquire visual sensing signals in a short distance along with the movement of the automatic lifting hook, compared with the mode of installing the zoom camera in the prior art, the problems that manual zooming affects the operation of a tower crane operator or the operation of the automatic zooming is out of alignment to cause the blurring of pictures can be avoided, the high-definition and accurate visual sensing signals can be automatically acquired without the additional operation of the tower crane operator, the tower crane operator can observe the conditions of lifting hook conditions, surrounding environment, obstacles and the like according to the visual sensing signals, the hidden safety hazards of 'mountain isolation lifting' and the like are solved, the lifting safety of the blind areas is ensured, the automatic lifting hook can be further utilized to automatically grab goods based on the visual sensing signals, the problems of inaccurate hook lifting and the like are solved, a span worker is not required to adopt a manual operation mode to lift the goods on the lifting hook, the participation of workers such as span worker, command and the like can be reduced, thereby further reducing the probability of accidental injury of the manual work by the goods and reducing the incidence rate of safety accidents.

In some variations of the present application, the hook drive mechanism includes a first trolley, the sensor drive mechanism includes a second trolley, and the first trolley and the second trolley are both disposed on and move along the boom of the tower crane.

Specifically, in some embodiments, the vision sensor is suspended on the second trolley by a rope pulley assembly, and moves in a horizontal direction according to the movement of the second trolley along the boom, and moves in a vertical direction according to the retracting action of the rope pulley assembly.

The first trolley and the second trolley can share one set of variable-amplitude steel wire rope to draw and move, under the condition that the first trolley and the second trolley need to keep a fixed distance, such as 3 meters, 5 meters and the like, in addition, the first trolley and the second trolley can also adopt two sets of different variable-amplitude steel wire ropes to draw and move respectively, so that the distance between the first trolley and the second trolley can be adjusted, the distance from the vision sensor to the automatic lifting hook can be adjusted conveniently, and a better observation field is obtained.

In addition, the first trolley and the second trolley need to adopt two sets of different lifting steel wire ropes to respectively pull the automatic lifting hook and the visual sensor to lift, so that the visual sensor can be leveled with the automatic lifting hook, can also be higher than the automatic lifting hook or lower than the automatic lifting hook to acquire signals, and can be applied to various working conditions to obtain a better observation field of view.

By arranging the second trolley to independently drive the vision sensor, the following relation between the vision sensor and the automatic lifting hook can be flexibly adjusted according to the actual working condition, for example, the vision sensor and the automatic lifting hook can be adjusted to keep 3 m intervals along the width-changing direction and keep m intervals along the height direction, or the vision sensor and the automatic lifting hook can be adjusted to keep m intervals along the width-changing direction and keep parallel (the interval is zero) along the height direction, and the like, so that a better observation field of view can be obtained.

After the following relation is determined, the controller can automatically control the vision sensor to carry out following movement according to the following relation when controlling the automatic lifting hook to move so as to keep the same observation field of view. In addition, the operator can also adjust the following relation according to the actual requirement, and the embodiment of the application is not limited to specific numerical values.

It should be noted that, the following related in the embodiment of the application refers to that the vision sensor and the automatic lifting hook keep a certain distance and angle when moving, so as to obtain the same observation field of view, improve the observation experience of the tower crane operator, and avoid the field of view transformation to influence the observation of the tower crane operator.

The vision sensor that this application embodiment provided can be connected with the controller through the cable, the cable can receive and release through the winder, the winder can be located on the second dolly, the winder can keep the cable in tightening state, avoids the cable to loosen and rocks and influence other part operations.

In other modified embodiments, the tower crane is provided with an amplitude sensor and a height sensor, wherein the amplitude sensor is used for detecting amplitude position information of the automatic lifting hook, and the height sensor is used for detecting height position information of the automatic lifting hook;

the controller controls the vision sensor to move along with the automatic lifting hook according to the amplitude position information and the height position information of the automatic lifting hook.

The amplitude sensor and the height sensor can be realized by using a sensor provided by the prior art, and the amplitude sensor and the height sensor can be a mechanical sensor, an infrared sensor or a laser sensor, which can realize the purposes of the embodiment of the application, and the embodiment of the application is not limited.

The amplitude-changing position information may include a horizontal distance between the automatic hook and the standard knot along the amplitude-changing direction (i.e., the horizontal direction of the boom), the height position information may include a vertical distance between the automatic hook and the boom along the vertical direction, and the amplitude-changing position information and the height position information may be used to determine amplitude-changing position information and height position information of a position where the visual sensor should be located according to amplitude-changing position information and height position information of the automatic hook and by combining predetermined following relationships, and control the visual sensor to move to the position where the visual sensor should be located according to the amplitude-changing position information and the height position information, so as to implement following motion with the automatic hook.

When the vision sensor is controlled to move along with the automatic lifting hook, the direction of the vision sensor is also required to be controlled (the vision sensor can be installed on the cloud deck through the control of the cloud deck, so that the direction is adjustable), the area where the automatic lifting hook is located can be shot, specifically, in some embodiments, the controller also determines the rough relative position relationship between the vision sensor and the automatic lifting hook according to the amplitude position information and the height position information of the automatic lifting hook, and the amplitude position information and the height position information of the vision sensor, and coarsely adjusts the steering of the vision sensor to the area where the automatic lifting hook is located according to the rough relative position relationship. Because the amplitude position information and the height position information of the automatic lifting hook and the visual sensor are already obtained during the following movement, the visual sensor can be quickly and coarsely adjusted to the area where the automatic lifting hook is located according to the existing data through the implementation mode.

In view of the fact that the automatic hook is not necessarily at a preferred position in the visual field of the vision sensor after coarse adjustment, and the vision sensor may swing along with air disturbance in the high altitude to fail to accurately capture an expected picture, in some modified embodiments, after coarsely adjusting the vision sensor to the area where the automatic hook is located, the controller further determines a fine relative position relationship between the vision sensor and the automatic hook by identifying the automatic hook in the vision sensor signal collected by the vision sensor, and fine-adjusts the vision sensor according to the fine relative position relationship, so that the fine-adjusted vision sensor collects a vision sensor signal meeting the expected requirement. According to the embodiment, the image recognition technology provided by the prior art can be utilized to recognize the automatic lifting hook in the vision sensing signal, so that the fine relative position relationship between the vision sensor and the automatic lifting hook is determined, and the vision sensor is finely tuned according to the fine relative position relationship, so that the vision sensor after fine tuning collects vision sensing signals meeting expectations, the expectations can be that the automatic lifting hook is positioned at the middle position of a picture of the vision sensing signal, or the automatic lifting hook and a suspended object are integrally positioned at the middle position of the picture of the vision sensing signal, and the embodiment of the application is not limited. According to the embodiment, the visual sensing signal meeting the expectation can be obtained through fine adjustment on the basis of coarse adjustment, and the accuracy of the visual sensing signal is improved, so that the automatic lifting hook action can be accurately controlled by using the visual sensing signal.

In any of the foregoing embodiments, the vision sensor may include a pan-tilt camera or a laser scanner, which may collect accurate vision sensing signals, so as to help a tower crane operator accurately understand the working condition of the lifting hook and accurately control the automatic lifting hook to lift the cargo.

It is easy to understand that if the vision sensor is lighter in weight, the vision sensor swings with air disturbance in the high altitude to affect the shooting effect, so that in some modified embodiments, the vision sensor can be further provided with a gesture stabilizing controller to help the vision sensor stabilize the gesture in the high altitude, reduce shaking, improve the shooting effect, further help the tower crane operator to accurately know the working condition of the lifting hook, and accurately control the automatic lifting hook to hoist the goods.

The attitude stabilization controller can be realized by at least one of a counterweight, a flywheel and a control moment gyro, and can be realized by one of the counterweights, the flywheel and the control moment gyro or a plurality of the counterweights, the flywheel and the control moment gyro. Wherein, the addition of the counterweight is most easy to realize and the implementation cost is lowest; if the flywheel is additionally arranged, the flywheel should be horizontally arranged, and the angular momentum generated by the flywheel can be helpful for keeping the posture of the vision sensor stable; in addition, the principle of the moment gyro is that when a gyro is given torque perpendicular to the rotation axis of the gyro, a precession moment perpendicular to the rotation axis and perpendicular to the torque axis is generated, and by using the principle, the vision sensor can be helped to keep stable posture by installing the moment gyro, and the effect of stabilizing the posture in the mode is the best.

In consideration, when lifting cargoes, the problems of falling of a lifting hook, falling of a rope and the like occur in the lifting process due to the fact that the cargoes are not tightly bound and are not accurately hooked and the like frequently, so that cargoes fall and injure ground workers such as a cable worker and the like due to falling of the cargoes, and safety accidents are caused, and therefore lifting abnormality of a tower crane needs to be further detected to reduce the lifting safety accidents of the tower crane. On the basis of any of the foregoing embodiments, in some modified embodiments, the sensing device for an automatic grabbing process of a tower crane hook may further include: a gesture sensor in communication with the controller;

the gesture sensor is fixedly arranged on the automatic lifting hook and is used for collecting gesture data of the automatic lifting hook in real time and sending the gesture data to the controller;

the controller determines the inclination information and the swing information of the automatic lifting hook according to the posture data, and judges whether the lifting state of the automatic lifting hook is abnormal according to the inclination information and the swing information.

The gesture sensor may include, but is not limited to, a motion sensor implementation such as a three-axis gyroscope, a three-axis accelerometer, a three-axis electronic compass, etc., which is not limited in this embodiment.

It should be noted that, if the tower crane is an unmanned tower crane, the controller may be disposed on a console on the ground, and a display screen and/or a sound connected to the controller are disposed on the console, so as to broadcast whether the lifting state of the automatic lifting hook is abnormal in an image and/or voice manner, so that a tower crane operator knows whether the lifting state of the automatic lifting hook is abnormal.

In addition, the controller and the attitude sensor can be connected in a wireless mode or a wired mode, and in consideration of relatively poor stability of wireless signals, safety accidents are possibly caused by signal interruption and errors, in some embodiments, the attitude sensor and the controller are preferably connected in a wired mode by using a cable, and the cable can be connected to a control table on the ground along a crane arm and a standard section and connected with the controller on the control table, so that signal quality and stability are improved, and abnormal lifting state of an automatic lifting hook cannot be found in time due to signal problems, and further safety accidents are caused.

Compared with the prior art, the sensing equipment for the automatic grabbing process of the tower crane lifting hook is further provided with the gesture sensor in communication connection with the controller, the gesture sensor is fixedly arranged on the automatic lifting hook and used for collecting gesture data of the automatic lifting hook in real time and sending the gesture data to the controller, and the controller determines inclination information and swing information of the automatic lifting hook according to the gesture data and judges whether lifting state of the automatic lifting hook is abnormal according to the inclination information and the swing information. Because the lifting hook often generates a larger inclination angle or shakes greatly before the lifting hook falls off and the rope falls off, the automatic lifting hook can accurately judge whether the lifting state of the tower crane is abnormal or not by utilizing the gesture data of the automatic lifting hook so as to timely carry out targeted treatment when detecting the abnormality, avoid scattering cargoes to injure workers by smashing, and reduce the safety accident occurrence rate caused by scattering of the lifted cargoes in the lifting stage of the tower crane.

The attitude sensor that this application embodiment provided can be connected with the controller through the cable, the cable can receive and release through the winder, the winder can be located and hang on the dolly of automatic lifting hook, the winder can keep the cable in the tightening state, avoids the cable to loosen and rocks and influence other part operations.

In some modified implementations of the embodiments of the present application, the controller stores no-load attitude data collected by the attitude sensor when the automatic hook is no-load, and determines inclination information and swing information of the automatic hook by comparing the load attitude data with the no-load attitude data after receiving load attitude data collected by the attitude sensor when the automatic hook is loaded.

The empty load attitude data are basic attitude data which are acquired in a static state that the automatic lifting hook is empty and the surrounding is windless and used for comparison, and after the automatic lifting hook is hooked to take the goods, namely the load, the inclination information and the swing information of the automatic lifting hook can be obtained by comparing the load attitude data with the empty load attitude data.

The inclination information refers to inclination angle and the like generated by the rotation of the automatic lifting hook by taking the automatic lifting hook as a reference, and the swinging information refers to swinging angle and the like generated by the swinging of the automatic lifting hook by taking a trolley suspending the automatic lifting hook as a reference, wherein the radius of a circle can be calculated according to a path (a section of arc line on the circle) of the automatic lifting hook passing through in the swinging process, the swinging angle can be further calculated according to the length of the arc line, and whether the state of the automatic lifting hook is abnormal can be judged and predicted according to the inclination information and the swinging information.

Specifically, the controller may determine tilt change information and swing change information of the unit time window according to the tilt information and the swing information by using a sliding time window method, and determine whether a lifting state of the automatic lifting hook is abnormal according to the tilt change information and the swing change information.

Wherein the tilt change information includes at least one of a tilt change amplitude and a tilt angle at the end of the unit time window, and the wobble change information includes at least one of a wobble change amplitude and a wobble angle at the end of the unit time window.

The inclination change amplitude is a difference value between the inclination angle at the end of the unit time window and the inclination angle at the initial stage, the inclination angle of the automatic lifting hook inclined towards the unhooking direction can be defined as positive, the inclination angle in the opposite direction is defined as negative, if the difference value is positive and larger than a preset threshold value, the abnormal probability is larger, and the abnormal lifting can be directly judged or further combined with other factors to judge whether the abnormal lifting exists.

The swing change amplitude is a difference value between the final inclination angle and the initial inclination angle of the unit time window, the swing angle of the automatic lifting hook swinging towards the unhooking direction can be defined as positive, the swing angle of the automatic lifting hook swinging towards the opposite direction is defined as negative, if the difference value is positive and is larger than a preset threshold value, the abnormal probability is larger, and the lifting abnormality can be directly judged or further combined with other factors to judge whether the lifting abnormality exists.

Considering that whether lifting is abnormal or not is judged by comparing the threshold values only, and the probability of misjudgment is caused, in order to improve the judgment accuracy, in some modified embodiments, the controller may input the inclination change information and the swing change information into a first neural network model trained in advance, and judge whether the lifting state of the automatic lifting hook is abnormal or not according to the first neural network model.

The first neural network model can be obtained by training a large number of training samples, the training samples comprise a plurality of groups of training data which are determined through experiments, each group of training data comprises inclination change information and swing change information, and whether an abnormal label exists or not.

The input data of the first neural network model comprises inclination change information and swing change information, the output data is a label (two kinds of labels) with abnormality, and the overall input parameters and output are relatively simple, so that the input data can be realized by adopting BP neural networks, convolutional neural networks CNN and other neural networks with simple structures, the input data can be formed by an input layer, a hidden layer and an output layer, the purpose of the embodiment of the application can be realized without complex design, the implementation difficulty is reduced, and relatively accurate judgment results are obtained. Wherein, the BP neural network and the convolutional neural network CNN are mature neural network models, and a person skilled in the art can flexibly construct the first neural network model by referring to the prior art and combining with actual requirements to achieve the purpose of the embodiment of the present application.

Through the embodiment, whether the lifting state of the automatic lifting hook is abnormal or not can be accurately judged by using the neural network model, and compared with a mode of judging according to a threshold value, the accuracy is higher.

Considering that the environmental wind force also can influence the hoist and mount firmness of goods, if the wind direction is the same with unhook direction, can increase the probability that the goods unhook, and the wind speed is bigger, and unhook probability is bigger, conversely, if the wind direction is opposite with unhook direction, can reduce the probability that the goods unhook, in order to more accurate judgement automatic lifting state of lifting hook is unusual, in some change embodiments, above-mentioned a sensing device for the automatic process of snatching of tower crane lifting hook still includes: the wind direction sensor and the wind speed sensor are arranged on the tower crane;

the wind direction sensor and the wind speed sensor are connected with the controller and are respectively used for collecting wind direction information and wind speed information around the automatic lifting hook and sending the wind direction information and the wind speed information to the controller;

the controller is also used for comprehensively judging whether the lifting state of the automatic lifting hook is abnormal according to the inclination change information, the swing change information, the wind direction information and the wind speed information.

The specific judging mode of the method can comprehensively judge whether the lifting state of the automatic lifting hook is abnormal based on the preset threshold value, or can judge whether the lifting state of the automatic lifting hook is abnormal by adopting a neural network, for example, in some embodiments, the controller can input the inclination change information, the swing change information, the wind direction information and the wind speed information into a pre-trained second neural network model, and judge whether the lifting state of the automatic lifting hook is abnormal according to the second neural network model.

The second neural network model can be obtained by training a large number of training samples, the training samples comprise a plurality of groups of training data which are determined through experiments, each group of training data comprises inclination change information, swing change information, wind direction information and wind speed information, and whether an abnormal label exists or not, through training, the second neural network model can output whether the abnormal label exists or not according to the input inclination change information, swing change information, wind direction information and wind speed information, and then whether the lifting state of the automatic lifting hook is abnormal or not can be judged by using the second neural network model.

Similar to the first neural network model, the input data of the second neural network model includes inclination change information, swing change information, wind direction information and wind speed information, the output data is a label (two kinds of labels) with abnormality, and the overall input parameters and output are simpler, so that the input data can be realized by adopting BP neural networks, convolutional neural networks CNN and other neural networks with simple structures, and the input data can be formed by an input layer, a hidden layer and an output layer, and the aim of the embodiment of the application can be realized without complex design, thereby reducing implementation difficulty and obtaining more accurate judgment results. Wherein, the BP neural network and the convolutional neural network CNN are mature neural network models, and a person skilled in the art can flexibly construct the first neural network model by referring to the prior art and combining with actual requirements to achieve the purpose of the embodiment of the present application.

Through the embodiment, whether the lifting state of the automatic lifting hook is abnormal can be accurately judged by using the second neural network model, and whether the lifting state of the automatic lifting hook is abnormal can be accurately judged by using the neural network model due to the fact that the influence of wind power on unhooking abnormality is considered.

On the basis of any of the foregoing embodiments, in other modified embodiments, the sensing device for an automatic grabbing process of a tower crane hook may further include: the alarm device is arranged on the automatic lifting hook;

the alarm device is connected with the controller, and when the controller detects that the lifting state of the automatic lifting hook is abnormal, the controller broadcasts abnormal alarm information through the alarm device.

The alarm device can comprise a buzzer, a sound box and other voice alarm devices, and can warn surrounding workers to evacuate by broadcasting abnormal alarm information to the surrounding, so that the condition that the surrounding workers are injured due to unhooked cargoes is avoided, and accident loss is reduced.

In addition, after lifting the goods, the controller can comprehensively judge whether the lifting state of the tower crane lifting hook is abnormal according to the visual sensing signals and the gesture data acquired by the gesture sensor, for example, by taking the visual sensing signals as real-time pictures shot by a tripod head camera, through image recognition, the lifting hook and the rope can be recognized, and whether the lifting state of the tower crane lifting hook is abnormal or not can be judged through the relative positions of the lifting hook and the rope and the movement trend of the rope in the pictures shot successively, for example, if the rope moves to a preset range at an outlet of the lifting hook and has a trend of moving continuously towards the unhooking direction, the unhooking risk is judged, namely the lifting state of the tower crane lifting hook is judged to be abnormal; otherwise, the unhooking risk can be judged, namely, the lifting state of the tower crane lifting hook is judged to be abnormal. The image recognition technology is a mature technology in the prior art, and a person skilled in the art can directly apply the prior art to the present application to achieve the purpose of the embodiment of the present application.

It should be noted that, if it is comprehensively determined whether the lifting state of the tower crane lifting hook is abnormal according to the gesture data and the visual sense signal, the specific determination mode may be: if any one of the gesture data and the visual sensing signals is adopted to judge that the lifting state of the tower crane lifting hook is abnormal, the lifting state of the tower crane lifting hook is judged to be abnormal as a whole, otherwise, the lifting state of the tower crane lifting hook is judged to be abnormal. Therefore, whether the lifting state of the tower crane lifting hook is abnormal or not is comprehensively and accurately judged by comprehensively utilizing the attitude data and the visual sense signals, and the accuracy is improved.

In addition, in order to further perfect the intellectualization and unmanned of the intelligent tower crane, the intelligent tower crane can also reduce the lifting safety accident of the tower crane by configuring the following sensing internet of things system for sensing the lifting abnormal state of the intelligent tower crane, and the following description is made with reference to examples.

In some embodiments, the sensing internet of things system for sensing abnormal lifting state of an intelligent tower crane may include: a controller and an attitude sensor in communication with the controller;

the attitude sensor is fixedly arranged on the tower crane lifting hook and is used for acquiring the attitude data of the tower crane lifting hook in real time and sending the attitude data to the controller;

The controller determines inclination information and swing information of the tower crane lifting hook according to the attitude data, and judges whether the lifting state of the tower crane lifting hook is abnormal according to the inclination information and the swing information.

The controller may be implemented by a computer host, a microcontroller, a programmable logic controller PLC, etc., and the gesture sensor may include, but is not limited to, a motion sensor such as a three-axis gyroscope, a three-axis accelerometer, a three-axis electronic compass, etc., which is not limited in this embodiment of the present application.

It should be noted that, if the tower crane is an unmanned tower crane, the controller may be disposed on a console on the ground, and a display screen and/or a sound connected to the controller are disposed on the console, so as to broadcast whether the lifting state of the tower crane lifting hook is abnormal in an image and/or voice manner, so that a tower crane operator knows whether the lifting state of the tower crane lifting hook is abnormal.

In addition, the controller and the attitude sensor can be connected in a wireless mode or a wired mode, and in consideration of relatively poor stability of wireless signals, safety accidents are possibly caused by signal interruption and errors, in some embodiments, the attitude sensor and the controller are preferably connected in a wired mode by using a cable, and the cable can be connected to a control table on the ground along a crane arm and a standard section and connected with the controller on the control table, so that signal quality and stability are improved, and abnormal lifting state of a tower crane lifting hook can not be found in time due to signal problems, and the safety accidents are further caused.

Compared with the prior art, the sensor internet of things system for sensing the abnormal lifting state of the intelligent tower crane is provided, the controller is arranged, the attitude sensor is in communication connection with the controller, the attitude sensor is fixedly arranged on the tower crane lifting hook and is used for collecting the attitude data of the tower crane lifting hook in real time and sending the attitude data to the controller, and the controller determines the inclination information and the swing information of the tower crane lifting hook according to the attitude data and judges whether the lifting state of the tower crane lifting hook is abnormal according to the inclination information and the swing information. Because the lifting hook often generates a larger inclination angle or shakes greatly before the lifting hook falls off and the rope falls off, the lifting hook can accurately judge whether the lifting state of the tower crane is abnormal or not by utilizing the attitude data of the lifting hook of the tower crane, so that targeted treatment is timely carried out when the abnormality is detected, the scattering of cargoes is avoided, the injury to workers is avoided, and the safety accident occurrence rate caused by the scattering of the cargoes in the lifting stage of the tower crane is reduced.

The attitude sensor that this application embodiment provided can be connected with the controller through the cable, the cable can receive and release through the winder, the winder can be located and hang on the dolly of tower crane lifting hook, the winder can keep the cable in the state of tightening, avoids the cable to loosen and rocks and influence other part operations.

In some modified implementations of the embodiments of the present application, the controller stores idle load attitude data collected by the attitude sensor when the tower crane hook is idle, and determines inclination information and swing information of the tower crane hook by comparing the load attitude data with the idle load attitude data after receiving load attitude data collected by the attitude sensor when the tower crane hook is loaded.

The empty load attitude data are basic attitude data which are acquired in a static state that the tower crane lifting hook is empty and the surrounding is windless and used for comparison, and after the tower crane is hooked to a load, the load attitude data and the empty load attitude data are compared, so that the inclination information and the swing information of the tower crane lifting hook can be obtained.

The above-mentioned inclination information refers to information such as inclination angle generated by the tower crane hook rotating by taking the tower crane hook as a reference, and the above-mentioned swing information refers to information such as swing angle generated by the tower crane hook swinging by taking a trolley suspending the tower crane hook as a reference, wherein the radius of a circle can be calculated according to the path (a section of arc line on the circle) of the tower crane hook passing through in the swing process, and further, the swing angle can be calculated according to the length of the arc line, and whether the state of the tower crane hook is abnormal can be judged and predicted according to the inclination information and the swing information.

Specifically, the controller may determine tilt change information and swing change information of the unit time window according to the tilt information and the swing information by using a sliding time window method, and determine whether a lifting state of the tower crane lifting hook is abnormal according to the tilt change information and the swing change information.

The inclination change amplitude is a difference value between an inclination angle at the end of a unit time window and an inclination angle at the initial stage, the inclination angle of the tower crane lifting hook inclined towards the unhooking direction can be defined as positive, the inclination angle in the opposite direction is negative, if the difference value is positive and is larger than a preset threshold value, the abnormal probability is larger, and the lifting abnormality can be directly judged or further combined with other factors to judge whether the lifting abnormality exists.

The swing change amplitude is a difference value between the final inclination angle and the initial inclination angle of the unit time window, the swing angle of the tower crane lifting hook swinging towards the unhooking direction can be defined as positive, the swing angle of the tower crane lifting hook swinging towards the opposite direction is defined as negative, if the difference value is positive and is larger than a preset threshold value, the abnormal probability is larger, and the lifting abnormality can be directly judged or further combined with other factors to judge whether the lifting abnormality exists.

Considering that whether lifting abnormality exists or not is judged by comparing the threshold values only, and the probability of misjudgment exists, in order to improve judgment accuracy, in some modified embodiments, the controller may input the inclination change information and the swing change information into a first neural network model trained in advance, and judge whether the lifting state of the tower crane lifting hook is abnormal according to the first neural network model.

The first neural network model can be obtained by training a large number of training samples, the training samples comprise multiple groups of training data which are determined through experiments, each group of training data comprises inclination change information and swing change information, and whether an abnormal label exists or not.

Through the embodiment, whether the lifting state of the tower crane lifting hook is abnormal or not can be accurately judged by using the neural network model, and compared with a mode of judging according to a threshold value, the accuracy is higher.

Considering that the environmental wind force also can influence the hoist and mount firmness of goods, if the wind direction is the same with unhook direction, can increase the probability that the goods unhook, and the wind speed is bigger, and unhook probability is bigger, conversely, if the wind direction is opposite with unhook direction, can reduce the probability that the goods unhook, in order to more accurate judgement the lifting state of tower crane lifting hook is unusual, in some change embodiments, above-mentioned a sensing internet of things system for intelligent tower crane lifting unusual state perception still includes: the wind direction sensor and the wind speed sensor are arranged on the tower crane;

the wind direction sensor and the wind speed sensor are connected with the controller and are respectively used for collecting wind direction information and wind speed information around the tower crane lifting hook and sending the wind direction information and the wind speed information to the controller;

the controller is also used for comprehensively judging whether the lifting state of the tower crane lifting hook is abnormal according to the inclination change information, the swing change information, the wind direction information and the wind speed information.

The specific judging mode of the method can comprehensively judge whether the lifting state of the tower crane lifting hook is abnormal based on the preset threshold value, or can judge whether the lifting state of the tower crane lifting hook is abnormal by adopting a neural network, for example, in some embodiments, the controller can input the inclination change information, the swing change information, the wind direction information and the wind speed information into a pre-trained second neural network model, and judge whether the lifting state of the tower crane lifting hook is abnormal according to the second neural network model.

The second neural network model can be obtained by training a large number of training samples, the training samples comprise a plurality of groups of training data which are determined through experiments, each group of training data comprises inclination change information, swing change information, wind direction information and wind speed information, and whether an abnormal label exists or not, through training, the second neural network model can output whether the abnormal label exists or not according to the input inclination change information, swing change information, wind direction information and wind speed information, and then whether the lifting state of the tower crane lifting hook is abnormal or not can be judged by using the second neural network model.

Through the embodiment, whether the lifting state of the tower crane lifting hook is abnormal can be accurately judged by using the second neural network model, and whether the lifting state of the tower crane lifting hook is abnormal can be accurately judged by using the neural network model due to the fact that the influence of wind power on unhooking abnormality is considered.

Based on any of the foregoing embodiments, in other modified embodiments, the sensing internet of things system for sensing an abnormal lifting state of an intelligent tower crane may further include: the alarm device is arranged on the tower crane lifting hook;

the alarm device is connected with the controller, and when the controller detects that the lifting state of the tower crane lifting hook is abnormal, the controller broadcasts abnormal alarm information through the alarm device.

Based on any of the foregoing embodiments, in some modified embodiments, the sensing internet of things system for sensing an abnormal lifting state of an intelligent tower crane may further include: the device comprises a lifting hook, a lifting hook driving mechanism, a visual sensor and a sensor driving mechanism; wherein, the liquid crystal display device comprises a liquid crystal display device,

the lifting hook is connected with the lifting hook driving mechanism, the visual sensor is connected with the sensor driving mechanism, and the lifting hook driving mechanism, the sensor driving mechanism and the visual sensor are all connected with the controller;

When the controller controls the lifting hook to move through the lifting hook driving mechanism, the controller also controls the vision sensor to follow the lifting hook to move through the sensor driving mechanism, and controls the vision sensor to acquire vision sensing signals towards the area where the lifting hook is located, so that whether the lifting state of the tower crane lifting hook is abnormal or not is comprehensively judged according to the vision sensing signals and the gesture data acquired by the gesture sensor.

In addition, the controller can be connected with the vision sensor and the sensor driving mechanism in a wireless mode or in a wired mode, and the vision sensor and the sensor driving mechanism are preferably connected with the controller in a wired mode in some embodiments in consideration of relatively poor stability of wireless signals, and specifically, the cable can be connected to a console on the ground along a crane arm and a standard section and connected with the controller on the console, so that signal quality and stability are improved, and perception errors caused by signal problems are avoided.

Compared with the prior art, the intelligent tower crane provided by the embodiment of the application can be used for further adding the visual sensor and the sensor driving mechanism through the sensor internet of things system for sensing the lifting abnormal state of the intelligent tower crane, and controlling the lifting hook to move through the lifting hook driving mechanism, controlling the visual sensor to follow the lifting hook to move through the sensor driving mechanism, controlling the visual sensor to face the area where the lifting hook is located to collect visual sensing signals, comprehensively judging whether the lifting state of the lifting hook of the tower crane is abnormal according to the visual sensing signals and the gesture data collected by the gesture sensor, so that the visual sensor can collect the visual sensing signals in a short distance along with the movement of the lifting hook, high-definition and accurate visual sensing signals can be automatically collected without additional operation of a tower crane operator, and whether the lifting state of the lifting hook of the tower crane is abnormal or not can be comprehensively and accurately judged according to the visual sensing signals, thereby further reducing the probability of being mistakenly injured by goods, and reducing the occurrence rate of safety accidents.

The first trolley and the second trolley can share one set of variable-amplitude steel wire rope to draw and move, under the condition that fixed distances, such as rice and the like, are needed to be kept, in addition, the first trolley and the second trolley can also adopt two sets of different variable-amplitude steel wire ropes to draw and move respectively, so that the distance between the first trolley and the second trolley can be adjusted, the distance from the visual sensor to the lifting hook can be adjusted conveniently, and a good observation view can be obtained.

In addition, the first trolley and the second trolley need to adopt two sets of different lifting steel wire ropes to respectively drag the lifting hook and the vision sensor to lift, so that the vision sensor can be leveled with the lifting hook, can also be higher than the lifting hook or lower than the lifting hook to acquire signals, and can be applied to various working conditions to obtain a better observation field.

By arranging the second trolley to independently drive the vision sensor, the following relation between the vision sensor and the lifting hook can be flexibly adjusted according to the actual working condition, for example, the distance between the vision sensor and the lifting hook along the width-changing direction can be adjusted, the distance between the vision sensor and the lifting hook along the height direction can be adjusted, or the distance between the vision sensor and the lifting hook along the width-changing direction can be adjusted, the distance between the vision sensor and the lifting hook along the height direction can be kept parallel (the distance is zero), and the like, so that a better observation field of vision can be obtained.

After the following relation is determined, the controller can automatically control the vision sensor to carry out following movement according to the following relation when controlling the lifting hook to move so as to keep the same observation field of view. In addition, the operator can also adjust the following relation according to the actual requirement, and the embodiment of the application is not limited to specific numerical values.

It should be noted that, the following related to the embodiment of the present application means that the visual sensor and the lifting hook keep a certain distance and angle when moving, so as to obtain the same observation field of view, so as to facilitate judging whether the lifting state of the tower crane lifting hook is abnormal through image comparison and recognition.

For example, taking a visual sensing signal as a real-time picture shot by a tripod head camera, the lifting hook and the rope can be identified through image recognition, whether an abnormality exists or not can be judged through the relative positions of the lifting hook and the rope and the movement trend of the rope in the pictures shot successively, for example, if the rope moves to a preset range at the outlet of the lifting hook and has a trend of continuing to move towards the unhooking direction, the unhooking risk is judged, namely the lifting state of the lifting hook of the tower crane is judged to be abnormal; otherwise, the unhooking risk can be judged, namely, the lifting state of the tower crane lifting hook is judged to be abnormal. The image recognition technology is a mature technology in the prior art, and a person skilled in the art can directly apply the prior art to the present application to achieve the purpose of the embodiment of the present application.

In other modified embodiments, the tower crane is provided with an amplitude sensor and a height sensor, wherein the amplitude sensor is used for detecting amplitude position information of the lifting hook, and the height sensor is used for detecting height position information of the lifting hook;

the controller controls the vision sensor to move along with the lifting hook according to the amplitude position information and the height position information of the lifting hook.

The amplitude-changing position information can comprise the horizontal distance between the lifting hook and the standard joint along the amplitude-changing direction (namely the horizontal direction of the lifting arm), the height position information can comprise the vertical distance between the lifting hook and the lifting arm along the vertical direction, and the amplitude-changing position information and the height position information can be used for determining the amplitude-changing position information and the height position information of the position where the vision sensor should be positioned according to the amplitude-changing position information and the height position information of the lifting hook and combining the predetermined following relation and controlling the vision sensor to move to the position where the vision sensor should be positioned according to the amplitude-changing position information and the height position information.

When the vision sensor is controlled to move along with the lifting hook, the direction of the vision sensor is also required to be controlled (the vision sensor can be installed on the cloud deck through the cloud deck control, so that the direction is adjustable), the area where the lifting hook is located can be shot, specifically, in some embodiments, the controller also determines the rough relative position relation between the vision sensor and the lifting hook according to the amplitude position information and the height position information of the lifting hook, and the amplitude position information and the height position information of the vision sensor, and coarsely adjusts the area where the vision sensor is turned to the lifting hook according to the rough relative position relation. Because the amplitude position information and the height position information of the lifting hook and the visual sensor are already obtained during the following movement, the visual sensor can be quickly and coarsely adjusted to the area where the lifting hook is positioned according to the existing data through the implementation mode.

In view of the fact that the hook is not necessarily at a preferred position in the visual field of the vision sensor after coarse adjustment, and the vision sensor may swing along with air disturbance in the high altitude to fail to accurately capture an expected picture, in some modified embodiments, after coarsely adjusting the vision sensor to the area where the hook is located, the controller further determines a fine relative positional relationship between the vision sensor and the hook by identifying the hook in the vision sensor signal collected by the vision sensor, and fine-adjusts the vision sensor according to the fine relative positional relationship, so that the fine-adjusted vision sensor collects the vision sensor signal meeting the expected requirement. According to the embodiment, the lifting hook in the visual sensing signal can be identified by utilizing the image identification technology provided by the prior art, so that the fine relative position relationship between the visual sensor and the lifting hook is determined, and the visual sensor is finely tuned according to the fine relative position relationship, so that the visual sensor after fine tuning acquires the visual sensing signal which accords with the expectation, wherein the expectation can be that the lifting hook is positioned at the middle position of the picture of the visual sensing signal or that the lifting hook and the suspended goods are integrally positioned at the middle position of the picture of the visual sensing signal, and the embodiment of the application is not limited. Through this embodiment, can be on coarse tuning the basis further through fine setting and just recall and accord with anticipated vision sensing signal, improve vision sensing signal accuracy to utilize this vision sensing signal to judge accurately the play of lifting state of tower crane lifting hook is unusual.

In any of the foregoing embodiments, the visual sensor may include a pan-tilt camera or a laser scanner, which may collect accurate visual sensing signals, so as to determine whether a lifting state of a tower crane hook is abnormal by combining gesture data.

Specifically, if the lifting state of the tower crane lifting hook is comprehensively judged according to the gesture data and the visual sense signal, the specific judging mode can be as follows: if any one of the gesture data and the visual sensing signals is adopted to judge that the lifting state of the tower crane lifting hook is abnormal, the lifting state of the tower crane lifting hook is judged to be abnormal as a whole, otherwise, the lifting state of the tower crane lifting hook is judged to be abnormal. Therefore, whether the lifting state of the tower crane lifting hook is abnormal or not is comprehensively and accurately judged by comprehensively utilizing the attitude data and the visual sense signals, and the accuracy is improved.

It is easy to understand that if the vision sensor is light, the vision sensor swings with air disturbance in the high air to affect the shooting effect, so that in some modification embodiments, the vision sensor can be further provided with a gesture stabilizing controller to help the vision sensor stabilize the gesture in the high air, so as to reduce shaking, improve the shooting effect and further improve the accuracy of abnormal judgment of the lifting state of the tower crane lifting hook.

In addition, in order to further perfect the intellectualization and unmanned of above-mentioned intelligent tower crane, the intelligent tower crane can also realize the comprehensive monitoring and the discernment to job site operating mode through the following three-dimensional augmented reality video control device that is used for intelligent tower crane to control through the configuration, need not the tower crane driver and carries out the overhead operation and just can realize the control to intelligent tower crane according to this three-dimensional augmented reality video, reduces staff's participation to can effectively reduce accident rate and avoid staff's casualties, the explanation is described below in connection with the example.

In some embodiments, the three-dimensional augmented reality video control device for intelligent tower crane control may include: a controller, a global camera, and a plurality of local cameras;

the global camera and the local camera are connected with the controller;

the global camera is downwards arranged on the intelligent tower crane boom and is used for shooting a global image of the intelligent tower crane working scene and sending the global image to the controller;

the plurality of local cameras are uniformly distributed on the periphery of the lifting hook of the intelligent tower crane, and are used for shooting local images from different directions on the periphery of the lifting hook and sending the local images to the controller;

the controller generates a three-dimensional augmented reality video representing the real-time working scene of the intelligent tower crane according to the global image and the local image, and controls the intelligent tower crane to operate according to the three-dimensional augmented reality video.

Compared with the prior art, the intelligent tower crane provided by the embodiment of the application can be controlled by arranging the controller, the global camera and the local cameras through the three-dimensional augmented reality video control device for controlling the intelligent tower crane; wherein the global camera and the local camera are both connected with the controller; the global camera is downwards arranged on the intelligent tower crane boom and is used for shooting a global image of the intelligent tower crane working scene and sending the global image to the controller; the plurality of local cameras are uniformly distributed on the periphery of the lifting hook of the intelligent tower crane, and are used for shooting local images from different directions on the periphery of the lifting hook and sending the local images to the controller; the controller generates a three-dimensional augmented reality video representing the real-time working scene of the intelligent tower crane according to the global image and the local image, and controls the intelligent tower crane to operate according to the three-dimensional augmented reality video. Therefore, real-time images of the working scene of the tower crane can be acquired by using the global camera and the local camera, and then a three-dimensional augmented reality video is generated, so that the working condition of a construction site is comprehensively monitored and identified, a tower crane driver does not need to carry out overhead operation, the intelligent tower crane can be controlled according to the three-dimensional augmented reality video, the participation of staff is reduced, and the accident occurrence rate is effectively reduced, and the casualties of the staff are avoided.

Regarding the installation manner of the local camera, in some modified implementations of the embodiments of the present application, the three-dimensional augmented reality video control device for controlling an intelligent tower crane may further include: a multi-branch support;

the multi-branch support frame is arranged on the shell of the lifting hook and is opened in an umbrella shape, and the plurality of local cameras are arranged at the tail ends of all branches of the multi-branch support frame.

In some variations, the multi-branch support frame may include a bottom fixed portion, a sleeve, a plurality of branches, and an adjustable portion movable up and down along the sleeve;

the bottom fixing part is arranged on the shell of the lifting hook, and the sleeve is sleeved on the steel wire rope of the lifting hook;

each branch comprises a supporting rod and a pull rod, one end of the supporting rod is connected with the bottom fixing part, and the other end of the supporting rod is used for installing the local camera;

one end of the pull rod is connected with the adjustable part, and the other end of the pull rod is connected with the middle part of the support rod.

Through setting up above-mentioned multi-branch support frame, can install local camera around the lifting hook, make local camera can be along with the lifting hook removal, obtain stable, clear shooting picture, help generating accurate three-dimensional augmented reality video.

It should be noted that, the above is merely a simple schematic structure of the multi-branch support frame, and in practical application, the structure of the multi-branch support frame may be changed according to practical requirements to obtain a better implementation effect, which all do not depart from the inventive concept of the present embodiment, and all should be within the scope of protection of the present application.

On the basis of the above embodiments, in some modified embodiments, the sleeve outer surface is provided with external threads, and the adjustable part comprises a gear bearing provided with internal threads and a driving motor, and the external threads are matched with the internal threads;

the driving motor is meshed with the gear bearing through a gear and is electrically connected with the controller and used for driving the gear bearing to rotate around the sleeve to move up and down under the control of the controller.

Through the above-mentioned embodiment, can realize the electric drive of adjustable portion, can drive the pull rod motion when adjustable portion reciprocates, and then drive local camera reciprocates and be close to or keep away from the lifting hook to realize the automatically controlled regulation of local camera, help tower crane control personnel to combine the convenient, nimble position of adjusting local camera of actual scene in order to obtain comparatively ideal shooting effect, and then generate accurate three-dimensional augmented reality video.

In addition, in order to improve the usability of local camera, in some change embodiments, the local camera pass through the cloud platform install in each branch end of multi-branch support frame, through setting up the cloud platform, can more nimble control local camera gathers required image, on the one hand, can be when the shooting angle appears deviating, correct the angle deviation through cloud platform control local camera to more accurate required image of gathering, on the other hand, can control local camera and cruise the shooting, gather the image in the bigger scope around, so that further carry out the three-dimensional reconstruction of full scene, improve intelligent level.

For the number of the local cameras, more than one local camera can be generally set in consideration of balance problems and shielding problems caused by surrounding arrangement of a plurality of local cameras, and in consideration of the fact that the number is too large, the system load and the implementation cost for generating the three-dimensional augmented reality video can be improved, preferably, the number of the local cameras is one or more, so that implementation cost and implementation effect are both considered, and a higher input-output ratio is obtained.

It should be noted that, the embodiment of the application adopts the mode that global camera and local camera combine together to carry out image acquisition, wherein, global camera can shoot and obtain the global image that construction scene is more comprehensive, but because its mounted position is higher, can exist and shelter from and shoot the less than of low object definition in the picture, consequently, through introducing the local camera that encircles the lifting hook setting, can gather the picture of shelter from the department, reduce shielding problem, and because local camera is along with the lifting hook removal, can closely gather the picture that the definition is higher, like this, through global camera and local camera's cooperation, with global image and local image fusion, can obtain comprehensive, clear, accurate image data, thereby ensure that the three-dimensional augmented reality video that generates can restore the true condition of construction scene more accurately, help the intelligent tower crane to realize accurate operation based on three-dimensional augmented reality video, the intelligent of intelligent tower crane, the automation level and operation precision are improved.

The controller and the local camera can be connected in a wireless mode or a wired mode, and the safety accidents caused by signal interruption and errors are possibly caused by the fact that the stability of wireless signals is relatively poor are considered. On this basis, the three-dimensional augmented reality video control device for intelligent tower crane control can further comprise: the winder is arranged on the trolley for hanging the lifting hook; the local cameras are connected with the controller through cables, and the cables are wound and unwound through the winder. By the embodiment, the cable can be kept in the tightened state by the reel, and the cable is prevented from loosening and shaking to influence the operation of other parts.

The three-dimensional augmented reality video can be realized by adopting a three-dimensional reconstruction technology, and in some embodiments, the controller generates the three-dimensional augmented reality video representing the real-time working scene of the intelligent tower crane through three-dimensional reconstruction specifically according to the global image and the local image.

For example, the controller determines the position information of each pixel point corresponding to a three-dimensional point in a world coordinate system by adopting a dense reconstruction algorithm according to the camera position information of the global camera and the local camera and the pixel position information of each pixel point in the global image and the local image, and determines the three-dimensional augmented reality video of the real-time working scene of the intelligent tower crane according to the three-dimensional point cloud formed by the three-dimensional points. The three-dimensional reconstruction based on multiple images is already a mature prior art, so specific processes thereof are not repeated herein, and a person skilled in the art can flexibly alter and implement the three-dimensional reconstruction with reference to the prior art, and the embodiments of the present application are not limited and are all within the protection scope of the present application.

In addition, building information model (Building Information Modeling, BIM) tools may also be employed to generate three-dimensional augmented reality video based on the global and local images, which may also achieve the purposes of the embodiments of the present application, and should also be within the scope of the present application.

In addition, in order to further perfect the intellectualization and unmanned of above-mentioned intelligent tower crane, the intelligent tower crane can also be through disposing the following sensor internet of things equipment that is used for intelligent tower crane to get and put motion detection, realize automatic detection and relative position calculation to lifting hook and goods, and then realize the automation of lifting hook and get and put the operation, need not manual control and participate in and can realize automatic hook crane to effectively improve the automation of intelligent tower crane, intelligent level, reduce accident occurrence and avoid the casualties of staff, the explanation is described below in connection with the example.

In some embodiments, the sensing internet of things device for intelligent tower crane pick-and-place motion detection may include: a controller and a plurality of miniature image sensors connected with the controller;

the intelligent tower crane comprises a plurality of miniature image sensors, a controller and a plurality of intelligent image sensors, wherein the miniature image sensors are all arranged on a lifting hook of the intelligent tower crane, at least one miniature image sensor is arranged on the inner side of a hook body of the lifting hook, and the miniature image sensors are respectively used for acquiring image information of a lifting part of goods to be loaded and unloaded at different positions of the lifting hook and sending the image information to the controller;

the controller is used for detecting relative position information between the hoisting part and the lifting hook according to the image information and controlling the lifting hook to move according to the relative position information so as to pick and place the goods to be loaded and unloaded.

Compared with the prior art, the intelligent tower crane provided by the embodiment of the application can be used for setting the controller and the plurality of miniature image sensors connected with the controller by configuring the sensing internet of things equipment for detecting the picking and placing movement of the intelligent tower crane; the intelligent tower crane comprises a plurality of miniature image sensors, a controller and a plurality of intelligent image sensors, wherein the miniature image sensors are all arranged on a lifting hook of the intelligent tower crane, at least one miniature image sensor is arranged on the inner side of a hook body of the lifting hook, and the miniature image sensors are respectively used for acquiring image information of a lifting part of goods to be loaded and unloaded at different positions of the lifting hook and sending the image information to the controller; the controller is used for detecting relative position information between the hoisting part and the lifting hook according to the image information and controlling the lifting hook to move according to the relative position information so as to pick and place the goods to be loaded and unloaded. Therefore, the automatic detection and the relative position calculation of the lifting hook and the cargoes can be realized, the automatic picking and placing operation of the lifting hook can be realized, the automation and intelligent level of the intelligent tower crane can be improved, the accident occurrence rate can be reduced, and the casualties of workers can be avoided.

The relative position information includes at least one information of a relative direction, a relative distance, a relative angle, and the like, and the embodiment of the application is not limited to the specific content, and can be flexibly selected and used by those skilled in the art according to actual requirements.

In some modified implementations of the embodiments of the present application, at least one of the plurality of micro image sensors is disposed on a side of a shank portion of the hook facing the opening of the hook, and is configured to collect image information outside the hook of the hoisting portion before the hoisting portion enters the hook of the hook; the controller is used for detecting the relative position information of the lifting hook outside the lifting hook according to the image information outside the lifting hook, and controlling the lifting hook to be close to the lifting hook according to the relative position information outside the lifting hook.

Through this embodiment, can gather the outer image information of hook through setting up at the outer miniature image sensor of hook, and then utilize this outer image information of hook to confirm that hoist and mount portion is in the outer relative position information of hook of lifting hook to can ensure that the controller can control the lifting hook accurate motion is to hoist and mount portion place, need not manual control and participate in and can realize automatic hook and hang.

In some modified implementations of the embodiments of the present application, a miniature image sensor disposed inside the hook body is used to collect in-hook image information of the lifting part after the lifting part enters the hook of the lifting hook;

the controller is used for detecting the relative position information in the hook after the lifting part enters the hook according to the image information in the hook, and controlling the lifting hook to hook the lifting part according to the relative position information in the hook.

Through this embodiment, can gather the image information in the hook through the miniature image sensor who sets up in the hook, and then utilize this image information in the hook to confirm that hoist and mount portion is in the relative position information in the hook of lifting hook to can ensure that the controller can control the lifting hook fine setting makes hoist and mount portion accurately fall in the hook of lifting hook, need not manual control and participate in and can realize automatic hook and hang.

For example, 4 miniature image sensors can be arranged on the lifting hook, wherein the miniature image sensors A and B are arranged outside the lifting hook and used for collecting image information outside the lifting hook, and the miniature image sensors C and D are arranged in the lifting hook and used for collecting image information inside the lifting hook, so that comprehensive monitoring on the working conditions inside and outside the lifting hook is realized.

In some modified implementations of the embodiments of the present application, the controller is further configured to detect whether there is a risk of unhooking the load from the intra-hook relative information after lifting the load. With the above example, by using the image information in the hook collected by the micro image sensor C disposed with the top end in the hook facing downward, it can be accurately determined whether the lifting part is located in a safety area (for example, the preset range of the hook bottom is the safety area), if the lifting part is located in the safety area, lifting can be performed, if the lifting part is located in the safety area, it is indicated that there is a unhooking danger, and the position of the lifting hook needs to be readjusted until the lifting is performed after the safety is ensured.

In some modified implementations of the embodiments of the present application, the information collecting end face of the miniature image sensor is provided with a transparent protective cover, and the transparent protective cover is used for protecting the miniature image sensor from being polluted and/or damaged by impact of the hoisting part. The transparent protective cover can be made of glass or transparent acrylic, and the specific materials are not limited in the examples. Considering that the building site is comparatively abominable, the dust is more, and the lifting hook bumps with hoist and mount portion easily when hoist and mount, through this embodiment, can effectively protect miniature image sensor is polluted by dust, rainwater etc. and effectively protect miniature image sensor is by hoist and mount portion striking damage.

Considering that when the impact force is large, the transparent protective cover such as glass or acrylic is likely to be crashed and damage the miniature image sensor, therefore, in some modified implementations of the embodiment of the application, a plurality of grooves are formed on the lifting hook, the miniature image sensor is embedded in the grooves, and the outer surface of the transparent protective cover is flush with or lower than the upper surface of the grooves. Through embedding miniature image sensor and transparent safety cover in the recess, even hoist and mount portion takes place the striking with the lifting hook, the impact force that the striking produced also is born by the lifting hook body, and can not damage miniature image sensor to can effectively improve miniature image sensor's life.

On the basis of any of the above embodiments, in some modified embodiments, the lifting portion of the cargo to be loaded and unloaded is provided with a preset pattern different from other portions;

the controller is used for identifying the hoisting part by detecting the preset pattern in the image information.

The preset pattern can be sprayed on the hoisting part by adopting a spray gun, can be attached to the hoisting part in a form of a sticker, or is arranged on the marking plate, and then the marking plate is clamped on the hoisting part by a clamp.

Since the preset pattern is different from other parts, the preset pattern can be detected by pattern matching recognition in the image information, and the hanging part is recognized in the image information. The image recognition technology based on the pattern is a mature technology at present, so that a detailed description is omitted herein, and a person skilled in the art can use any image recognition technology based on the pattern disclosed in the prior art to achieve the purpose of the embodiments of the present application, which is all within the protection scope of the present application.

Through this embodiment, can utilize the pattern of predetermineeing to realize the distinguishing sign to hoist and mount portion, help improving the rate of accuracy and the efficiency of controller discernment hoist and mount portion, and then improve intelligent level and the security of intelligent tower crane.

In addition, since one of the purposes of the embodiments of the present application is to identify the relative position information between the lifting part and the lifting hook, in order to obtain the depth information and further determine the relative position information of the three-dimensional space, in some embodiments, the above-mentioned miniature image sensor includes a binocular camera, and the controller is specifically configured to calculate the relative position information between the lifting part and the lifting hook by adopting a binocular camera ranging algorithm.

The ranging algorithm based on the binocular camera is a current mature technology, so that the description is omitted here, and a person skilled in the art can adopt any binocular camera ranging algorithm disclosed in the prior art to achieve the purpose of the embodiment of the application, and the range of the binocular camera ranging algorithm is within the protection scope of the application.

Through this embodiment, can utilize binocular camera and the range finding algorithm that corresponds to accurately calculate the relative position information between hoist and mount portion and the lifting hook, and then control the lifting hook motion accurately in order to realize automatic getting and put, can effectively improve lifting hook and get accuracy and the efficiency of putting the motion.

It should be noted that the above-mentioned micro image sensor may be implemented by a CCD sensor or a CMOS sensor, which may all achieve the purpose of the embodiments of the present application, and is not limited herein.

In addition, considering that under the condition that miniature image sensor imbeds the recess, wireless communication signal can receive the shielding of lifting hook main part metallic structure and can't effective transmission, consequently, in order to improve the security of intelligent tower crane, in some embodiments, the sensor thing networking equipment that is used for intelligent tower crane to get and puts motion detection still includes: the winder is arranged on the trolley for hanging the lifting hook;

the plurality of miniature image sensors are connected with the controller through cables, and the cables are wound and unwound through the winder.

Specifically, the cable can be connected to the console on the ground along the crane arm and the standard section and is connected with a controller on the console, so that the signal quality and stability are improved, and safety accidents caused by signal problems are avoided. The winder can keep the cable in a tightening state, avoid the cable from loosening and shaking to influence the operation of other parts, and improve the safety of the intelligent tower crane.

In addition, the controller can be realized by a computer host, a microcontroller, a Programmable Logic Controller (PLC) and the like, and the lifting hook can be realized by any automatic lifting hook provided by the prior art, so that the embodiment of the application is not limited.

In addition, in order to further perfect the intellectualization and unmanned of above-mentioned intelligent tower crane, intelligent tower crane can also realize the assistance-localization of material through the intelligent tower crane material location auxiliary device who disposes below based on internet of things communication, and then realize the automation of material and get and put the operation, need not manual control and participate in and can realize automatic hook and hang to effectively improve automation, intelligent level and the operating efficiency of intelligent tower crane, reduce accident occurrence and avoid the staff casualties, the explanation is described below in connection with the example.

In some embodiments, the intelligent tower crane material positioning auxiliary device based on internet of things communication may include: a controller, a radio frequency signal transmitter and a plurality of radio frequency signal receivers;

the radio frequency signal transmitter is arranged on the material when in use and broadcasts radio frequency signals to the surrounding;

the plurality of radio frequency signal receivers are respectively arranged at a plurality of different positions on the intelligent tower crane and are all in communication connection with the controller;

each radio frequency signal receiver is used for receiving the radio frequency signal broadcast by the radio frequency signal transmitter and sending the radio frequency signal and the arrival time thereof to the controller;

The controller is used for calculating the positioning information of the materials by adopting a TDOA algorithm according to the position information of each radio frequency signal receiver and the arrival time of the radio frequency signals sent by the radio frequency signal receiver.

Compared with the prior art, the intelligent tower crane material positioning auxiliary device based on the internet of things communication is characterized in that a controller, a radio frequency signal transmitter and a plurality of radio frequency signal receivers are arranged, and the radio frequency signal transmitter is arranged on a material when in use and broadcasts radio frequency signals to the surrounding; the system comprises a controller, a plurality of radio frequency signal receivers, a controller, a plurality of time difference of arrival TDOA algorithm, a plurality of intelligent towers, a plurality of wireless sensors and a plurality of wireless sensors, wherein the plurality of radio frequency signal receivers are respectively arranged at a plurality of different positions on the intelligent towers and are in communication connection with the controller, each radio frequency signal receiver is used for receiving radio frequency signals broadcast by the radio frequency signal transmitter and transmitting the radio frequency signals to the controller, and the controller is used for calculating positioning information of the materials according to position information of each radio frequency signal receiver and the arrival time of the transmitted radio frequency signals by the radio frequency signal receiver and adopting the arrival time difference TDOA algorithm, so that the positioning of the materials on a construction site can be realized, the automatic picking and placing operation of the lifting hooks can be realized, the automation, the intelligent level and the operation efficiency of the intelligent towers are improved, the accident rate is reduced, and the casualties of workers are avoided.

The embodiment of the application calculates the positioning information of the material by using a time difference of arrival (Time Difference of Arrival, TDOA) algorithm, where TDOA is a method for positioning by using a time difference, and by measuring the time when a signal (e.g., a radio frequency signal) arrives at a monitoring station (e.g., a radio frequency signal receiver in the embodiment), the distance of a signal source (e.g., a radio frequency signal transmitter in the embodiment) can be determined. The location of the signal can be determined using the distances of the signal source to the plurality of radio monitoring stations (the distances being circles around the radio monitoring stations) and the radius. By comparing the time differences of the signals reaching the monitoring stations, a hyperbola taking the monitoring stations as focuses and the distance differences as long axes can be made, and the intersection point of the hyperbolas is the position of the radio frequency signal transmitter, namely the position of the material. Since the TDOA algorithm is a current and mature positioning algorithm, which is not described herein, a person skilled in the art can flexibly apply and change the application in combination with the prior art to achieve the purpose of the embodiments of the present application, which are all within the protection scope of the present application.

It should be noted that, in order to implement the TDOA algorithm, a plurality of radio frequency signal receivers need to be clocked, and in particular, in some modified embodiments, the plurality of radio frequency signal receivers are connected to the controller in a wired manner, and the controller sends clock synchronization signals to the plurality of radio frequency signal receivers at preset time intervals, so that the plurality of radio frequency signal receivers keep clocked. Therefore, the position of the radio frequency signal transmitter, namely the positioning information of the material, can be accurately calculated by using the TDOA algorithm according to the arrival time of the radio frequency signal without clock synchronization with the radio frequency signal transmitter.

Considering that a construction site often has a plurality of materials and a plurality of intelligent towers, how to identify the materials needed to be hoisted by the current intelligent towers from the plurality of materials on the construction site of the intelligent towers and how to hoist the materials by the proper intelligent towers are further problems to be solved, in order to solve the problems, in some modified implementations of the embodiments of the present application, the radio frequency signals transmitted by the radio frequency signal transmitter carry the identification information of the intelligent towers;

the controller is also used for screening radio frequency signals carrying intelligent tower crane identification information of the current intelligent tower crane from all received radio frequency signals, and calculating positioning information of materials needing to be hoisted by the current intelligent tower crane according to arrival time of the screened radio frequency signals so as to identify the materials needing to be hoisted by the current intelligent tower crane from a plurality of materials on a construction site.

For example, distinguishing identification can be carried out on each intelligent tower crane, for example, codes, numbers and the like are adopted as intelligent tower crane identification information, when in operation, a user places or installs a radio frequency signal emitter on a material, radio frequency signals transmitted by the radio frequency signal emitter carry intelligent tower crane identification information, and therefore the intelligent tower crane can be ensured to screen out radio frequency signals of the radio frequency signal emitter according to the intelligent tower crane identification information so as to position the radio frequency signal emitter, and accordingly materials needing to be lifted by the current intelligent tower crane are identified from a plurality of materials on a construction site.

The radio frequency signal transmitter can be manufactured into an electronic tag and is arranged on materials in a manner of attaching, clamping, binding and the like. The radio frequency signal transmitter can be pre-bound with the intelligent tower crane, and can also be temporarily paired during construction, for example, in some modified embodiments, an input module is arranged on the radio frequency signal transmitter;

the radio frequency signal transmitter is also used for generating intelligent tower crane identification information according to the intelligent tower crane identification input by the user through the input module.

The input module may be implemented by a keyboard, a touch screen, etc., which is not limited in this embodiment. Through this embodiment, the workman can be in the real-time input intelligent tower crane sign in order to select suitable intelligent tower crane to hoist on site, and compatibility and flexibility are better.

In addition, if a plurality of materials need to be hoisted by the same intelligent tower crane on site, the hoisting order needs to be distinguished, so that in some modified embodiments, the radio frequency signals transmitted by the radio frequency signal transmitter carry hoisting order information;

the controller is also used for determining the lifting order of a plurality of materials needing to be lifted by the current intelligent tower crane according to the lifting order information, and sequentially lifting the materials according to the lifting order and the positioning information of the materials needing to be lifted by the current intelligent tower crane.

The hoisting order information may include one of sequence number information input by a user through an input module on the radio frequency signal transmitter, current start time information of the radio frequency signal transmitter, or start broadcasting time of the radio frequency signal.

If the hoisting order information is sequence number information input by a user through an input module on the radio frequency signal transmitter, the user can flexibly set and adjust the hoisting order of each material through the input module, and the device has the advantages of convenience, rapidness, flexibility and adjustability.

If the hoisting order information is the current starting time information of the radio frequency signal transmitter or the starting broadcast time of the radio frequency signal, the user does not need to perform manual input operation, and only needs to start or trigger the radio frequency signal transmitter to broadcast, the corresponding hoisting order information can be automatically generated according to the starting time or the broadcast trigger time and the like, so that the use is more convenient and faster.

It should be noted that, for positioning by using the TDOA method, the number of the radio frequency signal receivers is at least 4, and the radio frequency signal receivers may be distributed at a plurality of positions on the boom, the tower body and the hook of the intelligent tower crane. It is easy to understand that by measuring and recording the rotation angle, amplitude information and lifting information of the intelligent tower crane and combining the specific installation position of the radio frequency signal receiver, the real-time position information of the radio frequency signal receiver can be calculated at any time (the calculation can be performed by combining the geometric relationship specifically, and the repeated description is omitted here), and then the TDOA algorithm is adopted to realize the positioning of the materials.

In addition, the radio frequency signal transmitter and the radio frequency signal receiver can be realized by adopting any medium-long distance wireless communication module such as a GPRS/4G wireless communication module, a 2.4G wireless communication module, a WiFi wireless communication module and the like, and the communication distance of radio frequency signals needs to be as far as possible and has stronger penetrating power and anti-interference capability in consideration of the fact that the height of the tower crane is higher, so that the radio frequency signal transmitter and the radio frequency signal receiver are realized by adopting the 433M wireless module in the embodiment, and the 433M wireless module has the advantages of strong signal, long transmission distance, ideal transmission distance of about 3 km, strong penetrating and diffracting power, smaller attenuation in the transmission process and the like, so that the radio frequency signal transmitter and the radio frequency signal receiver can be well applied to the working scene of the intelligent tower crane, thereby obtaining better signal transmission effect and improving the stability and reliability of the implementation of the scheme.

In addition, in order to further perfect the intellectualization and unmanned of the intelligent tower crane, the intelligent tower crane can automatically identify materials by configuring the following tower crane material classification and identification system based on image analysis, so that the materials in different types can be automatically selected and loaded and unloaded in a reasonable mode, the occurrence of loading and unloading accidents is reduced, the unmanned and intelligent development of the tower crane is promoted, and the following description is made by combining with examples.

In some embodiments, the tower crane material classification and identification system based on image analysis may include:

the system comprises an image group acquisition module, a control module and a control module, wherein the image group acquisition module is used for acquiring a material image group acquired by a camera group arranged on an intelligent tower crane, the camera group comprises a plurality of cameras with different shooting angles, and the material image group comprises material images acquired by each camera aiming at materials;

the attribute information determining module is used for determining attribute information of the materials according to the material image group, wherein the attribute information comprises shape information, size information and texture information;

the material category matching module is used for matching the category of the material from a tower crane material database according to the attribute information of the material, wherein the tower crane material database stores the attribute information of different tower crane materials in advance.

Compared with the prior art, the tower crane material classification recognition system based on image analysis provided by the embodiment of the application is characterized in that the material image group collected by the camera group arranged on the intelligent tower crane is obtained, the camera group comprises a plurality of cameras with different shooting angles, each camera is used for recognizing the material image collected by the material according to the material image group, the attribute information of the material is determined according to the material image group, the attribute information comprises shape information, size information and texture information, the material is obtained by matching from the tower crane material database according to the attribute information of the material, wherein the tower crane material database stores the attribute information of different tower crane materials in advance, and the material type loaded and unloaded by the tower crane is clear and characterized in that the material type loaded and unloaded by the tower crane is clear, so that the type of the material can be rapidly and accurately recognized by matching the database, the intelligent tower crane can be used for automatically selecting and reasonably loading and unloading the different types of materials, the occurrence of accidents is reduced, and unmanned intelligent development of the tower crane is promoted.

The tower crane material classification and identification system based on image analysis can be realized by a controller of an intelligent tower crane, the controller can realize automatic classification and identification of tower crane materials, and further, the tower crane materials are automatically and reasonably selected for loading and unloading according to different types of materials, for example, a proper lifting hook is selected for lifting, different lifting speeds are selected according to glass and steel materials, and the like, so that loading and unloading accidents are reduced, and unmanned and intelligent development of the tower crane is promoted.

In some modification of the embodiment of the present application, the attribute information determining module includes:

the initial image inquiring unit is used for inquiring an initial image corresponding to the gesture information from an initial image database according to the gesture information of each camera for acquiring the material image, wherein the gesture information comprises shooting position information and shooting angle information of the camera, the initial image database stores initial images acquired by each camera corresponding to the gesture information, and the initial images are acquired before the material enters the field;

the image comparison unit is used for respectively comparing each material image in the material image group with the initial image which is acquired in advance by a camera for acquiring the material image under the same gesture, so as to identify a material main body in each material image;

And the attribute information determining unit is used for determining the attribute information of the materials according to the identified material main bodies in each material image.

On the basis of the above embodiments, for extracting shape information and size information, in some modified implementations of the embodiments of the present application, the attribute information determining unit includes:

the coordinate conversion relation determining subunit is used for determining the coordinate conversion relation between the pixel coordinate system corresponding to each camera and the world coordinate system;

the coordinate conversion subunit is used for converting the pixel coordinates of the material main body in each material image into world coordinates in a world coordinate system according to the coordinate conversion relation;

and the shape and size determining subunit is used for determining shape information and size information of the material according to the world coordinates.

The above coordinate conversion relation of the camera is determined, and the pixel coordinate is converted into the world coordinate by the mature prior art, so that the specific process is not described herein, and a person skilled in the art can flexibly change and implement the coordinate conversion relation by referring to the prior art, which is not limited in the embodiment of the present application, and is within the protection scope of the present application.

On the basis of the foregoing embodiments, for extracting texture information, in some modified implementations of the embodiments of the present application, the attribute information determining unit includes:

and the texture determining subunit is used for identifying texture information of the material main body in each material image by adopting a texture identification algorithm.

The above texture recognition algorithm may be implemented by any texture feature extraction algorithm provided in the prior art, for example, a local binary pattern (Local Binary Patterns, LBP) algorithm, an OpenCV-based texture recognition algorithm, etc., which may all achieve the purpose of the embodiments of the present application, and since they are all the existing mature technologies, the specific process thereof will not be described in detail herein, and a person skilled in the art may refer to the existing technologies to flexibly alter and implement the texture feature extraction algorithm, which is not limited in the scope of protection of the present application.

In practical application, the materials commonly used for hoisting the tower crane are mainly raw materials for construction such as steel bars, wood edges, concrete, steel pipes and glass, and the shape, the size and the texture of the materials are greatly different, so that the types of the materials can be quickly and accurately identified through database matching only by pointedly extracting the attribute information such as the shape, the size and the texture of the materials, the intelligent tower crane can automatically select a reasonable mode for loading and unloading the materials of different types, the loading and unloading accidents are reduced, and unmanned and intelligent development of the tower crane is promoted.

In some modified implementations of the embodiments of the present application, the camera includes a binocular camera, and the material image acquired by the binocular camera carries depth information;

the attribute information determining module includes:

and the depth-of-field-based determining unit is used for determining attribute information of the materials according to the depth information carried by each material image in the material image group.

In addition, in order to further perfect the intellectualization and unmanned of the intelligent tower crane, the intelligent tower crane can also predict the fault occurrence rate according to the daily instruction execution in-place rate of the intelligent tower crane by configuring the following intelligent tower crane maintenance management system based on the fault identification model, and output a corresponding working condition detection strategy when the warning value is exceeded, so that the working condition detection is carried out on the possibly faulty component at any time and in a targeted manner before the fault occurs, the fault is eliminated in a sprouting state, the occurrence of the fault can be effectively reduced, the intellectualization level, the automation level and the safety of the intelligent tower crane are improved, and the following description is made by combining examples.

In some embodiments, the intelligent tower crane maintenance management system based on the fault identification model may include:

The execution information acquisition module is used for acquiring execution monitoring information corresponding to the control instruction sent by the intelligent tower crane controller in real time;

the arrival rate calculation module is used for calculating the instruction execution arrival rate of the control instruction according to the execution monitoring information;

the fault identification module is used for inputting the control instruction and the instruction thereof into a pre-trained fault identification model at a bit rate and obtaining the fault occurrence rate predicted by the fault identification model;

and the maintenance strategy output module is used for inquiring and outputting a working condition detection strategy of the executing mechanism corresponding to the control instruction if the fault occurrence rate is larger than a preset warning value.

Compared with the prior art, the intelligent tower crane maintenance management system based on the fault identification model provided by the embodiment of the application is characterized in that by acquiring the execution monitoring information corresponding to the control instruction sent by the intelligent tower crane controller in real time, calculating the instruction execution in-place rate of the control instruction according to the execution monitoring information, inputting the control instruction and the instruction execution in-place rate thereof into the pre-trained fault identification model, acquiring the fault occurrence rate predicted by the fault identification model, inquiring the working condition detection strategy of the executing mechanism corresponding to the control instruction and outputting the fault occurrence rate if the fault occurrence rate is larger than the preset warning value, and because most faults are caused by small products, fine abnormal performance such as the fact that the instruction execution is not in place, the instruction execution in-place rate is low and the like is often present before the faults occur, the scheme can predict the fault occurrence rate according to the daily instruction execution in-place rate of the intelligent tower crane, and output the corresponding working condition detection strategy when the warning value is exceeded, further carrying out working condition detection on possibly occurring parts at any time and pertinently before the faults occur, and eliminating the faults in an automatic state of the intelligent tower crane, so that the intelligent tower crane can be reduced in the effective fault occurrence level and the intelligent safety level is improved.

Wherein, the operating mode detection strategy that this application embodiment inquired can export display screen etc. display device, so that tower crane control personnel maintains with the manual mode according to this operating mode detection strategy, in addition, the operating mode detection strategy that inquires also can export operating mode detection robot (the intelligent auxiliary robot that is used for intelligent tower crane operating mode to detect), utilize this operating mode detection robot to realize the automatic maintenance to intelligent tower crane, realize the automation of intelligent tower crane, unmanned, intelligent operating mode detects, promote unmanned, the intelligent development of tower crane, above-mentioned mode all can realize the purpose of this application embodiment, all should be within the scope of protection of this application.

The execution monitoring information comprises action information of an executing mechanism corresponding to the control instruction, wherein the action information is detected from the control instruction, and the action information comprises at least one of action amplitude, action time and action speed. The action information can be used for judging the instruction execution condition of the executing mechanism and calculating the instruction execution arrival rate of the executing mechanism. The actuating mechanism comprises a lifting mechanism, a slewing mechanism and an amplitude changing mechanism, wherein the intelligent tower crane can be provided with corresponding sensors for monitoring the action information of the actuating mechanism, for example, a camera is arranged for monitoring the action information of the actuating mechanism in an image recognition analysis mode, and for example, a laser range finder, an accelerometer, a gyroscope and other sensors are arranged for monitoring the action information of the actuating mechanism.

After the above action information is collected, the instruction execution arrival rate of the corresponding control instruction may be determined, in this embodiment, the instruction execution arrival rate may be calculated by the execution duration, and in some modification implementations of this embodiment, the arrival rate calculating module includes:

the execution time length calculation unit is used for determining the actual execution time length from the start of issuing the control instruction to the condition that the execution condition of the control instruction accords with a preset in-place condition according to the execution monitoring information;

and the arrival rate calculation unit is used for determining the instruction execution arrival rate of the control instruction according to the ratio of the actual execution arrival time length to the standard execution arrival time length corresponding to the control instruction.

For example, the control instruction is to lift the lifting hook at a speed of 1 m/s, the standard execution in-place duration of the lifting hook from static acceleration to 1 m/s is 2 seconds, and if the actual execution in-place duration is 4 seconds, the corresponding instruction execution in-place rate is 2 seconds/4 seconds=50%; for another example, the control command is to control the luffing trolley to brake, the standard execution time length is 0.5 seconds, and if the actual execution time length is 0.5 seconds, the corresponding command execution time length is 0.5 seconds/0.5 seconds=100%. The foregoing are exemplary descriptions, and a person skilled in the art may flexibly set, according to actual situations, a specific value of the standard execution in-place duration and a specific calculation manner of the instruction execution in-place rate, which are all within the scope of protection of the present application.

After calculating the instruction execution arrival rate, a pre-trained fault recognition model may be input to predict the fault occurrence rate, where it should be noted that, in some modified implementations of the embodiments of the present application, the intelligent tower crane maintenance management system based on the fault recognition model further includes:

the historical information acquisition module is used for acquiring all historical control instructions of the intelligent tower crane before the fault occurs and corresponding historical execution monitoring information of the intelligent tower crane;

the historical arrival rate calculation module is used for calculating the historical execution arrival rate corresponding to the historical control instruction according to the historical control instruction and the historical execution monitoring information;

the historical arrival rate ordering module is used for ordering the historical execution arrival rates according to time to obtain a historical execution arrival rate set;

the fault rate assignment module is used for assigning the fault occurrence rate to each historical execution arrival rate in the historical execution arrival rate set according to the time from the occurrence of the fault, wherein the shorter the time from the occurrence of the fault is, the higher the assigned fault occurrence rate is;

the learning sample generation module is used for generating machine learning samples, wherein each machine learning sample comprises one history control instruction and corresponding history execution monitoring information and failure occurrence rate;

And the model training module is used for training the fault recognition model according to the machine learning sample to obtain a pre-trained fault recognition model.

In this embodiment, a large amount of history data is collected to calculate a history execution arrival rate (i.e., a history command execution arrival rate) and order the history execution arrival rate according to a time of issuing a control command, then, for each actually occurring fault, an execution mechanism and a control command related to the fault are identified through fault detection, then, according to a time from occurrence of the fault, the history execution arrival rate corresponding to the control command related to the fault is assigned, a value of assignment is between 0 and 1, the closer to the fault is, the higher the value of assignment is, for example, the fault occurrence rate corresponding to the history execution arrival rate within one day of the fault is assigned to be 0.9, the fault occurrence rate corresponding to the history execution arrival rate within 7 days of the fault is assigned to be 0.5, and the specific assignment manner is not limited, for example, a period from history execution to occurrence rate of less than 100% to occurrence of the fault is also noted as a hidden fault period, a duration is L, and a time interval from the history execution arrival rate corresponding to occurrence of a control command is assigned to occurrence of the fault is a, then, the history execution arrival rate corresponding to the fault occurrence rate may be calculated by adopting the following formula:

m ₁ ＝1-a/L

In the above, m ₁ The failure occurrence rate is represented by a, the time interval from the historical execution to the bit rate corresponding to the control instruction to the failure is represented by L, and the time interval from the historical execution to the bit rate lower than 100% to the failure is represented by L.

In addition, the above assignment may be performed in combination with a specific historical execution arrival rate value, for example, the historical execution arrival rate when a fault occurs is b, and the historical execution arrival rate when the execution mechanism works well is c, and then the fault occurrence rate corresponding to a certain historical execution arrival rate d may be calculated by the following formula:

in the above, m ₂ The failure occurrence rate corresponding to the historical execution arrival rate d is represented by b, the historical execution arrival rate when the failure occurs is represented by c, and the historical execution arrival rate when the execution mechanism works well is represented by c.

In addition, the failure rate can be calculated by summing the time factor and the historical execution rate value factor, for example, the failure rate can be calculated according to the m ₁ And m ₂ The product of (2) further determines the failure occurrence m, as follows:

m＝m ₁ ×m ₂

in the above formula, m is the total failure occurrence rate, m ₁ Representing the failure occurrence rate, m of the assignment according to the time of occurrence of the distance failure ₂ The failure occurrence rate of assignment according to the specific historical execution arrival rate value is represented, the total failure occurrence rate obtained by multiplying the failure occurrence rate and the actual failure occurrence rate is larger in change amplitude through the embodiment, the difference of the failure occurrence rate can be more remarkably represented on the numerical value, the prediction accuracy of the failure is higher, the sensitivity and accuracy of a failure recognition model trained after assignment are higher, and the overall implementation accuracy of the scheme is improved.

In addition, the fault recognition model is a model for predicting the fault occurrence rate according to the actual instruction execution arrival rate, and may be obtained by training a machine learning sample generated according to historical data.

The input data of the fault recognition model comprises a control instruction code and the instruction execution in-place rate thereof, the output data is the fault occurrence rate, and the whole input parameters and output are relatively simple, so that the fault recognition model can be realized by adopting BP neural networks, convolutional neural networks CNN and other neural networks with simple structures, and can be composed of an input layer, a hidden layer and an output layer, the aim of the embodiment of the application can be realized without complex design, the implementation difficulty is reduced, and relatively accurate judgment results are obtained. The BP neural network and the convolutional neural network CNN are mature neural network models, and a person skilled in the art can flexibly construct the fault identification model by referring to the prior art and combining with actual requirements to achieve the purpose of the embodiment of the application, which are all within the protection scope of the application.

After the failure occurrence rate is predicted, a corresponding working condition detection strategy can be further determined, and in some modification implementations of the embodiments of the present application, the intelligent tower crane maintenance management system based on the failure recognition model further includes:

the strategy determining module is used for determining an executing mechanism corresponding to the control instruction and a corresponding working condition detection strategy of the executing mechanism aiming at various control instructions sent by the controller;

the mapping table generation module is used for generating a working condition detection strategy mapping table according to various control instructions, corresponding execution mechanisms and working condition detection strategies;

the maintenance policy output module includes:

and the mapping table inquiring module is used for inquiring the executing mechanism and the working condition detection strategy corresponding to the control instruction from the working condition detection strategy mapping table and outputting the executing mechanism and the working condition detection strategy.

According to the embodiment, corresponding working condition detection strategies can be preset for various situations, a mapping table is established for storage, and when the failure occurrence rate is detected to be larger than the preset warning value, an executing mechanism and the working condition detection strategies corresponding to the control instruction can be inquired from the mapping table and output, so that the corresponding working condition detection can be conveniently carried out for the executing mechanism.

It should be noted that, the above-mentioned working condition detection policy may be set by a technician in combination with actual flexibility, for example, the failure occurrence rate corresponding to the control instruction for controlling the braking of the luffing trolley exceeds the corresponding warning value, and the working condition detection policy is to detect the working condition of the braking system of the trolley; the failure occurrence rate corresponding to the control instruction for controlling the starting of the variable-amplitude trolley exceeds the corresponding warning value, and the working condition detection strategy is to detect the working condition of the driving system of the trolley; the embodiment of the present application is not limited to the specific content of the above-mentioned working condition detection strategy, and when it is used in the scheme of the present application, it should be within the protection scope of the present application.

In addition, in order to further perfect the intellectualization and unmanned of the intelligent tower crane, the intelligent tower crane can realize unmanned, intelligent and automatic lifting appliance selection by configuring a tower crane lifting appliance selection device based on the three-dimensional material form model simulation, and improve the intelligent level, the automatic level and the safety of the intelligent tower crane, and the intelligent tower crane is described below with reference to examples.

In some embodiments, the intelligent tower crane material positioning auxiliary device based on internet of things communication may include:

The material image acquisition module is used for acquiring a material image group acquired by a camera group arranged on the intelligent tower crane, wherein the camera group comprises a plurality of cameras arranged at different positions of the intelligent tower crane, and the material image group comprises material images acquired by each camera aiming at materials;

the three-dimensional reconstruction module is used for carrying out three-dimensional reconstruction on the materials according to the material image group to obtain three-dimensional simulation materials;

the lifting appliance matching module is used for sequentially matching a plurality of three-dimensional simulation lifting appliances to be replaced with the three-dimensional simulation materials and selecting the three-dimensional simulation lifting appliance with the highest matching degree;

and the lifting appliance selecting module is used for controlling the intelligent tower crane to select the lifting appliance corresponding to the three-dimensional simulation lifting appliance from the lifting appliance pool.

Compared with the prior art, the tower crane lifting tool selecting device based on the three-dimensional material form model simulation is provided, the material image group collected by the camera group arranged on the intelligent tower crane is obtained, the camera group comprises a plurality of cameras arranged at different positions of the intelligent tower crane, each camera is used for carrying out three-dimensional reconstruction on the material according to the material image group, three-dimensional simulation materials are obtained, a plurality of three-dimensional simulation lifting tools to be selected are sequentially matched with the three-dimensional simulation materials, the three-dimensional simulation lifting tool with the highest matching degree is selected, finally the intelligent tower crane is controlled to select the lifting tool corresponding to the three-dimensional simulation lifting tool from the pool, the three-dimensional simulation lifting tool corresponding to the material can be constructed through three-dimensional reconstruction, then the three-dimensional simulation lifting tool can be directly and pertinently selected from the lifting tool pool, unmanned, intelligent and automatic lifting of lifting tool selection can be realized, and intelligent lifting tool level, automation level and safety of lifting tool selection can be improved.

The tower crane lifting appliance selection method based on the three-dimensional material morphology model simulation provided by the embodiment of the application can be realized by a controller of an intelligent tower crane, the controller can be realized by a computer host, a microcontroller, a programmable logic controller PLC and the like, and the embodiment of the application is not limited.

In the embodiment of the application, three-dimensional reconstruction refers to establishing a mathematical model suitable for computer representation and processing on a three-dimensional object, is a basis for processing, operating and analyzing the three-dimensional object in a computer environment, and is also a key technology for establishing virtual reality expressing an objective world in a computer. In computer vision, three-dimensional reconstruction refers to a process of reconstructing three-dimensional information from single-view or multi-view images, which may be performed by calibrating a camera, that is, calculating a relationship between a pixel coordinate system of the camera and a world coordinate system, and then reconstructing three-dimensional information using information in a plurality of two-dimensional images, for example, three-dimensional reconstruction process based on two-dimensional images is illustrated in the following (1) - (5):

(1) Image acquisition: prior to image processing, a two-dimensional image of a three-dimensional object (e.g., a material image of the subject application) is acquired using an imaging device (e.g., a camera).

(2) Calibrating a camera: an effective imaging model is established through camera calibration, internal and external parameters of a camera are solved, and therefore three-dimensional point coordinates in space can be obtained by combining the matching result of images, and the purpose of three-dimensional reconstruction is achieved.

(3) Feature extraction: the features mainly comprise feature points, feature lines and regions. In most cases, feature points are taken as matching primitives, and the form of feature point extraction is closely related to the matching strategy. Feature point extraction algorithms may include, but are not limited to: a method based on directional derivative, a method based on image brightness contrast relation, a method based on mathematical morphology and the like.

(4) Stereo matching: the stereo matching is to establish a corresponding relation between different images according to the extracted features, that is, imaging points of the same physical space point in two different images are in one-to-one correspondence. Attention is paid to disturbances in the scene due to factors such as light conditions, noise disturbances, scene geometry distortions, surface physical properties, and camera characteristics.

(5) Three-dimensional reconstruction: the three-dimensional scene information can be recovered by combining the internal and external parameters calibrated by the camera with a relatively accurate matching result. Because the three-dimensional reconstruction precision is influenced by factors such as matching precision, internal and external parameter errors of a camera and the like, the work of the previous steps is needed to be done, so that the precision of each link is high, the error is small, and the three-dimensional reconstruction can be realized more accurately.

The foregoing illustrates a three-dimensional reconstruction process based on a two-dimensional image, and a person skilled in the art may refer to the foregoing exemplary description, and flexibly change and implement, in combination with an actual scene, to perform three-dimensional reconstruction according to a material image to obtain a three-dimensional simulated material, so as to achieve the purposes of the embodiments of the present application, and further described below with reference to examples:

in some variations of the embodiments of the present application, the three-dimensional reconstruction module includes:

the camera position determining unit is used for determining camera position information corresponding to each material image in the material image group;

and the three-dimensional reconstruction unit is used for carrying out three-dimensional reconstruction according to the position information of the camera corresponding to each material image and the position information of the corresponding pixel point of the material in each material image to obtain the three-dimensional simulation material corresponding to the material.

The camera can be used for shooting a global image of a working scene of the intelligent tower crane, so that the position of the material can be determined by preliminarily positioning the position of the material according to the global image, and the position information of the material can be calculated and determined according to the rotation information and the installation position of the tower crane; for another example, the camera can also comprise a local camera installed near the lifting appliance, the local camera is used for shooting materials only by way of example, a material image with higher definition and higher accuracy is obtained, and the position information of the local camera can be calculated and determined according to the rotation information, the amplitude information, the lifting information and the installation position of the tower crane; the calculation of the camera position information can be implemented according to the geometric relationship, and will not be described herein.

On the basis of the above embodiments, in some modified embodiments, the three-dimensional reconstruction unit includes:

and the dense reconstruction subunit is used for determining the position information of each pixel point corresponding to a three-dimensional point in a world coordinate system by adopting a dense reconstruction algorithm according to the position information of a camera corresponding to each material image and the position information of the pixel point corresponding to the material in each material image, and determining the three-dimensional simulation material corresponding to the material according to the three-dimensional point cloud formed by the three-dimensional points.

The dense reconstruction (Multiple View Stereo, MVS) algorithm is multi-view solid geometry, and aims to calculate three-dimensional points corresponding to each pixel point in an image pixel by pixel on the premise that the pose of a camera is known, so as to obtain dense three-dimensional point clouds on the surface of a scene object.

Through the implementation mode, three-dimensional reconstruction can be accurately and rapidly realized, and the whole labeling accuracy and the whole labeling efficiency are improved.

In some variations of the embodiments of the present application, the spreader matching module includes:

the lifting appliance matching unit is used for sequentially matching a plurality of three-dimensional simulation lifting appliances to be replaced with the three-dimensional simulation materials in building information model BIM software, and determining matching degree according to the priority of the lifting appliances and the coupling degree of the lifting parts, wherein different priorities are preset for different lifting appliances, and the coupling degree of the lifting parts is determined according to the shape and size coupling information;

And the lifting appliance selecting unit is used for selecting the three-dimensional simulation lifting appliance with the highest matching degree from the plurality of three-dimensional simulation lifting appliances according to the matching degree.

When the three-dimensional simulation lifting appliance matching method is implemented, corresponding three-dimensional simulation lifting appliances can be preset in building information model (Building Information Modeling, BIM) software corresponding to each lifting appliance in the lifting appliance pool, attribute information and priority information are set for each three-dimensional simulation lifting appliance, and after three-dimensional simulation materials are generated, the three-dimensional simulation lifting appliances can be matched with the three-dimensional simulation materials in sequence in the BIM software, so that the matching degree of each three-dimensional simulation lifting appliance is determined.

It is easy to understand that different priorities can be set for different spreaders, for example, the safety of the lifting hook is higher, the priority is larger than that of the hanging pliers, the clamp, the hanging beam, etc., and the person skilled in the art can flexibly set the priority of each spreader according to the actual requirement, which is not limited herein.

In addition, the coupling degree of the hoisting part (i.e. the contact part of the hoisting tool and the material) can be determined according to the shape and size coupling information, for example, the coupling degree of the circle and the circle is larger than that of the circle and the rectangle, the coupling degree of the small-size hoisting tool and the large-size material cannot be coupled, i.e. is zero, and the like.

Through the mode, BIM software can be utilized to automatically select a proper three-dimensional simulation lifting appliance for materials, so that the corresponding lifting appliance can be selected in a targeted manner, and the accuracy and the efficiency are high.

In some modified implementations of the embodiments of the present application, a spreader pool is configured for the intelligent tower crane, a plurality of different spreaders are disposed in the spreader pool, and each spreader is disposed at a preset position in the spreader pool according to a corresponding spreader identifier;

the lifting appliance selecting module comprises:

and the lifting appliance selection module unit is used for controlling the intelligent tower crane to select a corresponding lifting appliance from a preset position corresponding to the lifting appliance identification in the lifting appliance pool according to the lifting appliance identification of the three-dimensional simulation lifting appliance with the highest selected matching degree.

According to the embodiment, the lifting appliance identification can be set for different lifting appliances, and the lifting appliances are arranged at the appointed positions in the lifting appliance pool, so that after the three-dimensional simulation lifting appliances are matched and determined, the corresponding lifting appliances can be selected quickly according to the lifting appliance identification, and the lifting appliance selection accuracy and efficiency are improved.

It should be noted that, the lifting appliance related to the embodiment of the present application includes, but is not limited to, a lifting hook, a lifting clamp, a lifting beam, a clamp, a steel plate lifting appliance, a steel ingot lifting appliance, a vertical coil lifting appliance, a C-shaped lifting appliance, a round steel lifting appliance, an electric horizontal coil lifting appliance, a container lifting appliance, a roller lifting appliance, and the like.

In addition, in order to further perfect the intellectualization and unmanned of the intelligent tower crane, the intelligent tower crane can realize automatic inspection of maintenance points by configuring the intelligent auxiliary robot for maintenance of the tower crane, improve the timeliness of maintenance, simultaneously shoot manual maintenance videos and concurrent certificates, automatically generate maintenance records, and help to trace back after accidents.

In some embodiments, the intelligent auxiliary robot for maintenance of the tower crane may include: a control device, a driving device and an imaging device;

the driving device and the image pickup device are connected with the control device;

the driving device is used for driving the intelligent auxiliary robot to move along a preset maintenance path on the intelligent tower crane;

the control device is used for controlling the driving device to drive the intelligent auxiliary robot to inspect and maintain the maintenance points according to a preset maintenance schedule;

The camera device is used for shooting a manual maintenance video and concurrent evidence when the camera device stays at the maintenance point requiring manual maintenance;

the control device is also used for generating maintenance records aiming at all maintenance points on the maintenance schedule after the maintenance is completed.

Compared with the prior art, the intelligent auxiliary robot for maintenance of the tower crane, provided by the embodiment of the application, is provided with the control device, the driving device and the camera device, wherein the driving device and the camera device are connected with the control device, the driving device is used for driving the intelligent auxiliary robot to move along a preset maintenance path on the intelligent tower crane, the control device is used for controlling the driving device to drive the intelligent auxiliary robot to patrol and examine maintenance points according to a preset maintenance schedule, the camera device is used for shooting an artificial maintenance video and a concurrent certificate when the maintenance points requiring artificial maintenance are stopped, and the control device is further used for generating maintenance records for each maintenance point on the maintenance schedule after the maintenance is completed. Therefore, the intelligent auxiliary robot can be utilized to carry out automatic inspection of maintenance points, the timeliness of maintenance is improved, meanwhile, the concurrent evidence of manual maintenance videos can be shot, and the maintenance records can be automatically generated, so that the maintenance points are traced after an accident, in addition, maintenance personnel can be stimulated to carefully and responsible to carry out maintenance through shooting the manual maintenance videos, the occurrence of the problem that the maintenance is not in place is reduced, and the intelligent level and the safety of the intelligent tower crane are improved as a whole.

The control device can be implemented by a computer host, a microcontroller, a Programmable Logic Controller (PLC) and the like, and the image pickup device can be implemented by any camera provided by the prior art, so that the embodiment of the application is not limited.

In some modified implementations of the embodiments of the present application, the intelligent auxiliary robot for maintenance of a tower crane further includes: an electronic tag reader connected with the control device;

the maintenance points are provided with electronic tags which are used for distinguishing and marking different maintenance points;

the electronic tag reader is used for detecting an electronic tag in the moving process of the intelligent auxiliary robot, and sending tag information of the electronic tag to the control device when the electronic tag is detected;

the control device is also used for identifying the maintenance point according to the label information.

Through this embodiment, can utilize electronic tags to realize the sign to maintaining the maintenance point for intelligent auxiliary robot can be through reading this electronic tags discernment maintain the maintenance point, and then carry out the targeted maintenance to and accomplish the inspection, avoid missing simultaneously and maintain the maintenance point.

In some modified implementations of the embodiments of the present application, the intelligent auxiliary robot for maintenance of a tower crane further includes: a display device connected to the control device;

the control device is also used for controlling the display device to play the maintenance guide information of the current maintenance point when the control device stays at the maintenance point which needs manual maintenance.

By playing the maintenance guide information, maintenance personnel can be guided to complete maintenance work correctly and efficiently, and the problems of wrong maintenance or improper maintenance are avoided.

In some modified implementations of the embodiments of the present application, a steel chain is provided along a preset overhaul path on the intelligent tower crane;

the driving device comprises a gear structure coupled with the steel chain and is used for driving the intelligent auxiliary robot to move along the steel chain.

Because a large number of maintenance paths need to be climbed for the intelligent tower crane, the intelligent auxiliary robot can climb along the maintenance paths to reach maintenance points at all positions through the embodiment, and safe and smooth implementation of the scheme is ensured.

In some modified implementations of the embodiments of the present application, the intelligent auxiliary robot for maintenance of a tower crane further includes: a housing;

the control device, the driving device and the image pickup device are fixedly installed through the shell;

the surface of casing is provided with gyro wheel and rubidium iron boron magnet, ru Tiepeng magnet is used for with intelligent auxiliary robot inhales on intelligent tower crane, the gyro wheel be used for with intelligent tower crane contact so that intelligent auxiliary robot can follow intelligent tower crane removes.

Through this embodiment, can ensure that intelligent auxiliary robot can adsorb and remove on intelligent tower crane body of tower, avoid intelligent auxiliary robot at the emergence of the condition such as high altitude wind swing, improve job stabilization nature and security.

In some modified implementations of the embodiments of the present application, the intelligent auxiliary robot for maintenance of a tower crane further includes: a communication device;

the communication device is connected with the control device;

the control device is also used for sending a maintenance schedule of the inspection to a terminal carried by a maintenance person through the communication device before the inspection, so that the maintenance person inspects maintenance points along with the intelligent auxiliary robot according to the maintenance schedule.

According to the embodiment, maintenance personnel can be timely and effectively reminded to carry out maintenance, the maintenance personnel can know the maintenance content by sending the maintenance schedule, the maintenance efficiency is improved, and the condition of untimely maintenance is avoided.

In some modified implementations of the embodiments of the present application, the control device is further configured to send, during the inspection, warning information to the controller of the intelligent tower crane through the communication device, where the warning information indicates that maintenance is being performed, so that the controller avoids performing a construction operation during the maintenance.

Through this embodiment, can avoid intelligent tower crane construction operation in the period of patrolling and examining, improve the security, avoid construction and maintenance to go on simultaneously and the incident emergence that leads to.

In some modified implementations of the embodiments of the present application, the control device is further configured to control the camera device to collect a face image of a maintenance person when the camera device stays at the maintenance point where manual maintenance is required, identify and authenticate the maintenance person according to the face image, and play warning information indicating authentication failure to the maintenance person after authentication failure.

By the method, identity authentication of maintenance personnel can be realized, the problems of wrong maintenance, out-of-place maintenance and the like caused by replacement operation of the maintenance personnel, maintenance by illegal personnel (the personnel without maintenance qualification) and the like are avoided, and the maintenance quality and the in-place rate are improved.

In addition, in order to further perfect the intellectualization and unmanned of the intelligent tower crane, the intelligent tower crane can also realize the automatic judgment of whether the construction site is suitable for construction by configuring the intelligent auxiliary robot for the maintenance of the tower crane, so as to ensure that the construction is restarted under the condition that no personnel are around the materials, avoid personnel in the vicinity of the materials during the construction and cause personal injury to the personnel, and integrally improve the intellectualization level and the safety of the intelligent tower crane, and the intelligent tower crane is described below with reference to examples.

In some embodiments, the intelligent auxiliary robot for detecting the working condition of the intelligent tower crane may include a housing, a moving module, a control module, an environment scanning module and a wireless communication module; wherein, the liquid crystal display device comprises a liquid crystal display device,

the shell is arranged on the mobile module, the control module, the environment scanning module and the wireless communication module are arranged on the shell, and the mobile module, the environment scanning module and the wireless communication module are connected with the control module;

The mobile module is used for driving the intelligent auxiliary robot to move on the ground around the intelligent tower crane under the control of the control module;

the environment scanning module is connected with the control module and is used for scanning environment data around the intelligent tower crane under the control of the control module along with the movement of the moving module before the construction of the intelligent tower crane and sending the scanned environment data to the control module;

the control module is used for determining working condition information representing whether the construction is suitable currently or not according to the environmental data, and sending the working condition information to a controller of the intelligent tower crane through the wireless communication module so that the controller determines whether to start the construction according to the working condition information.

Compared with the prior art, the intelligent auxiliary robot for intelligent tower crane working condition detection is provided, through setting up casing, mobile module, control module, environment scanning module and wireless communication module, wherein, the casing is located on the mobile module, control module the environment scanning module with wireless communication module all locates on the casing, mobile module the environment scanning module with wireless communication module all with control module connects, mobile module is used for driving under control module's control intelligent auxiliary robot moves on the subaerial around the intelligent tower crane, the environment scanning module with control module connects, is used for follow before the construction of intelligent tower crane the mobile module's removal, scan under control module's control the environmental data around the intelligent tower crane, and send the environmental data who obtains of scanning to control module, control module is used for according to environmental data confirm whether the characterization is currently fit for the working condition information, and will the working condition information is passed through wireless communication module sends to the controller around the intelligent tower crane so that whether the working condition information begins to confirm. Therefore, the intelligent auxiliary robot can automatically detect the material and personnel conditions of a construction site before construction, generate working condition information representing whether the construction is suitable for the current time and send the working condition information to the controller, and further enable the controller to determine whether the construction is started according to the working condition information, so that the intelligent tower crane can know whether the working condition of the construction site is suitable for the construction or not, the construction is restarted under the condition that no personnel are around the material, personal injury caused by personnel around the material during the construction is avoided, and the intelligent level and safety of the intelligent tower crane are integrally improved.

The control module may be implemented by a computer host, a microcontroller, a programmable logic controller PLC, etc., which is not limited in this embodiment.

The above-mentioned environment scanning module may be implemented by using at least one of any laser scanner, binocular camera and depth camera provided in the prior art, for example, the above-mentioned environment scanning module may be implemented by using any one of any laser scanner, binocular camera and depth camera provided in the prior art, or may be implemented by using any two or three of them in combination.

The wireless communication module can be realized by adopting the LoRa wireless communication module, the 433M wireless module and the like, and the advantages of strong signals, long transmission distance, ideal transmission distance of about 3 km, penetration, strong diffraction capacity, small attenuation in the transmission process, easiness in networking, low cost and the like of the LoRa wireless communication module and the 433M wireless module are also realized, so that the wireless communication module is well applicable to the working scene of the intelligent tower crane, thereby obtaining better signal transmission effect, improving the stability and reliability of the implementation of the scheme, and the embodiment of the application is not limited.

In some modified implementations of the embodiments of the present application, the control module is specifically configured to perform synchronous mapping on a surrounding environment according to the environmental data by using a synchronous positioning and map building (Simultaneous Localization And Mapping, SLAM) algorithm corresponding to the environmental scanning module, and determine, according to the constructed environmental map, working condition information that characterizes whether the current construction is suitable.

The synchronous positioning and map construction SLAM refers to a process of realizing self positioning of a moving object carrying a sensor in the moving process and synchronously constructing a map of the surrounding environment in a proper mode, can be regarded as a combination of two technologies of self positioning and map construction, and is a mature algorithm applied to the field of robots. The explanation of a specific point is as follows: a movable robot starts to move from any place in a completely unknown environment, and continuously uses a sensor (such as the environment scanning module) to observe surrounding environment characteristics in the process of movement, so that the position and the angle of the robot are positioned, and meanwhile, the environment incremental map is continuously updated and constructed according to the relative pose information of the robot and the environment, and the movable robot is helped to construct a perception system for generating surrounding three-dimensional environment data, so that the autonomous movement and environment perception of the movable robot are realized.

At present, SLAM is a mature technology in the robot field, and a person skilled in the art can directly or alternatively implement any SLAM algorithm provided by the prior art to achieve the purpose of the embodiment of the present application, which is not limited in this application.

In consideration of the fact that different SLAM algorithms are required to be adopted for different data acquired by different environment scanning modules, according to the environment scanning modules actually adopted, the embodiment of the application can synchronously map the surrounding environment by adopting the SLAM algorithm corresponding to the environment scanning modules, for example, if the environment scanning modules adopt laser scanners, the surrounding environment is required to be synchronously mapped by adopting the SLAM algorithm corresponding to the laser scanners and based on laser point cloud data; if the environment scanning module adopts a binocular camera, synchronous mapping is required to be carried out on the surrounding environment by adopting a SLAM algorithm which corresponds to the binocular camera and is based on binocular vision images; if the environment scanning module adopts a depth camera, synchronous mapping is required to be carried out on the surrounding environment by adopting a SLAM algorithm based on a depth image, which corresponds to the depth camera; if the environment scanning module is realized by adopting a plurality of laser scanners, binocular cameras and depth cameras, the surrounding environment is required to be synchronously mapped by adopting a corresponding fusion SLAM algorithm; the foregoing are all mature technical means in the prior art, and any SLAM algorithm provided by the prior art may be directly or alternatively implemented by a person skilled in the art to achieve the purpose of the embodiments of the present application, which is not described herein again.

Through the implementation mode, the SLAM algorithm can be adopted to realize real-time construction of the surrounding environment map, so that whether the current working condition information suitable for construction is accurately determined.

On the basis of the above embodiments, in some modified embodiments, the control module is specifically configured to identify materials and personnel in the constructed environment map, and generate working condition information indicating that the construction is currently suitable if the materials are identified and no personnel exist in a preset range around the materials, otherwise generate working condition information indicating that the construction is not currently suitable.

The materials are materials which need to be hoisted by an intelligent tower crane, such as raw materials for building construction, including steel bars, wood ribs, concrete, steel pipes, glass and the like, if the materials are identified on a construction site, basic construction conditions are provided, in addition, whether personnel are around the materials or not needs to be identified for protecting the personal safety of ground staff, if no personnel are in a preset range around the materials, the materials are suitable for construction, and if the personnel are, the materials are unsuitable for construction. The preset range can be flexibly set according to actual needs, for example, 3 meters, 5 meters and the like, and the embodiment of the application is not limited.

According to the method and the device, the working condition information which indicates that the current construction is suitable can be generated and sent to the controller under the condition that the materials are identified and no personnel exist in the preset range around the materials, and the safety problem caused by the construction of the controller under the unsuitable working condition is avoided.

On the basis of the above embodiments, in some modified embodiments, the material is provided with a material identification pattern;

the environment scanning module comprises a binocular camera or a depth camera, and the environment data comprises image data;

the control module is specifically configured to identify the material by detecting the material identification pattern in the image data.

The material identification pattern needs to be realized by adopting a pattern which can be distinguished from the surrounding environment, the material identification pattern can be sprayed on the surface of the material by adopting a spray gun, and can be attached to the surface of the material in a form of a sticker, or the preset pattern is arranged on the identification plate, and then the identification plate is clamped on the material by the clamp.

Since the material identification pattern is different from the surrounding environment, the material identification pattern can be detected by pattern matching recognition in the image data, and thus the material is recognized in the image data. The image recognition technology based on the pattern is a mature technology at present, so that a detailed description is omitted herein, and a person skilled in the art can use any image recognition technology based on the pattern disclosed in the prior art to achieve the purpose of the embodiments of the present application, which is all within the protection scope of the present application.

Through this embodiment, can utilize material identification pattern to realize distinguishing the sign to the material, help improving intelligent auxiliary robot and discerned rate of accuracy and efficiency of material.

In addition, different material identification patterns corresponding to different materials can be set for the intelligent auxiliary robot to identify the types of the materials according to the material identification patterns, so that the intelligent auxiliary robot can further help the controller to make more accurate decisions, for example, when multiple materials exist on a construction site, the controller can determine the lifting priority of the various materials according to the identified types of the materials, and the construction rationality and the automatic construction efficiency are improved.

In addition, the identification of the material can be realized by adopting other embodiments, for example, in some modified embodiments, an electronic tag for identifying the material is arranged on the material;

the intelligent auxiliary robot further comprises: an electronic tag reader connected with the control module;

the electronic tag reader is used for detecting the electronic tag in the moving process of the intelligent auxiliary robot, and sending tag information of the electronic tag to the control module when the electronic tag is detected;

The control module is specifically used for identifying the materials according to the tag information.

Through this embodiment, can adopt electronic tags's mode to sign the material, electronic tags can carry more material information, for example information such as category, quantity, weight, hoist and mount priority, can make the more reasonable planning construction plan of controller and carry out reasonable construction, improves intelligent and the automation level of intelligent tower crane.

For identifying the person, in some modified embodiments, the control module is specifically configured to detect the person in the image data by using a person identification algorithm, determine the location of the material and the person in the environment map, and determine whether there is a person in a preset range around the material.

Image-based person recognition algorithms are already mature technologies, and therefore will not be described in detail herein, and any person recognition algorithm based on image disclosed in the prior art may be adopted by a person skilled in the art to achieve the purpose of the embodiments of the present application, which are all within the protection scope of the present application.

After detecting the materials and the personnel, the positions of the materials and the personnel can be identified in the constructed environment map, and the distance between the materials and the personnel is calculated, so that whether the personnel exist in a preset range around the materials is determined.

Through this embodiment, can comparatively accurate discernment material and personnel and confirm the distance between the two to whether have personnel in the preset scope around the accurate judgement material, and then confirm whether at present is fit for the construction, improve the security of construction.

The above-mentioned mobile module can be realized by any wheel type mobile mechanism, foot type mobile mechanism or crawler type mobile mechanism provided in the prior art. For the mobile robot, the wheel type moving mechanism is the most applied structure, on a flat ground, the wheel type moving mode is optimal, the efficient moving speed can be guaranteed, for a more complex ground, the crawler type moving mechanism can be used for obtaining better penetrating performance and stability, and for a ground with high uneven fluctuation, the foot type moving mechanism can be used for realizing. Those skilled in the art may implement the purpose of the embodiments of the present application by using a suitable mobile module according to different construction environments, and the present application is not limited to the specific implementation manner of the mobile module.

In addition, in order to further perfect the intelligent level and the safety of the intelligent tower crane, the intelligent tower crane can monitor whether the motion state of the boom is abnormal by configuring the following intelligent tower crane boom state internet of things sensing monitoring system, so that the automatic diagnosis of the abnormal state of the boom is realized, the automation level, the intelligent level and the safety of the intelligent tower crane are improved, and the following description is made by combining with an example.

In some embodiments, the intelligent tower crane boom condition internet of things sensing and monitoring system may include: the system comprises a controller and an inertial sensor in communication connection with the controller, wherein the inertial sensor comprises an accelerometer and a gyroscope;

the inertial sensor is fixedly arranged on a trolley of the intelligent tower crane and used for collecting inertial motion data of the trolley in real time and sending the inertial motion data to the controller, the trolley is arranged on a suspension arm of the intelligent tower crane and moves along the suspension arm, and the inertial motion data comprise acceleration data and angular velocity data;

the controller is used for determining the inertial motion information of the intelligent tower crane according to the inertial motion data, and judging whether the motion state of the suspension arm is abnormal or not by comparing the inertial motion information with the motion information fed back by the driving mechanism of the intelligent tower crane, wherein the motion information comprises rotation motion information and/or amplitude motion information.

Compared with the prior art, the intelligent tower crane boom condition internet of things sensing monitoring system provided by the embodiment of the application is characterized in that a controller and an inertial sensor in communication connection with the controller are arranged, the inertial sensor comprises an accelerometer and a gyroscope, the inertial sensor is fixedly arranged on a trolley of the intelligent tower crane and is used for collecting inertial motion data of the trolley in real time and sending the inertial motion data to the controller, the trolley is arranged on a boom of the intelligent tower crane and moves along the boom, the inertial motion data comprise acceleration data and angular velocity data, the controller is used for determining inertial motion information of the intelligent tower crane according to the inertial motion data, judging whether the motion state of the boom is abnormal or not through comparing the inertial motion information with motion information fed back by a driving mechanism of the intelligent tower crane, wherein the motion information comprises swing motion information and/or amplitude motion information, so that the inertial motion data of the trolley on the boom can be collected, the inertial motion information is calculated according to the inertial motion information, and then the inertial motion information is compared with motion information fed back by the driving mechanism to judge whether the motion state of the boom is abnormal or not, and the intelligent tower crane is automatically diagnosed, and the abnormal state is automatically improved.

The above-mentioned controller can adopt host computer, microcontroller, programmable logic controller PLC etc. to realize, and above-mentioned inertial sensor can adopt accelerometer and gyroscope to realize, and this application embodiment is not limited, and above-mentioned davit is the jib loading boom.

The controller and the inertial sensor can be connected in a wired mode or in a wireless mode, when the controller and the inertial sensor are connected in a wireless mode, the controller and the inertial sensor can be connected in a communication mode by adopting a LoRa wireless communication module or 433M wireless module, and the advantages of strong signals, long transmission distance, ideal transmission distance of about 3 km, penetration, strong diffraction capacity, small attenuation of the transmission process, easiness in networking, low cost and the like of the LoRa wireless communication module and the 433M wireless module are achieved, so that the controller and the inertial sensor are well applicable to working scenes of the intelligent tower crane, thereby obtaining better signal transmission effect, improving the stability, reliability and networking flexibility of the implementation of the scheme.

The motion information fed back by the driving mechanism is obtained by the controller from the driving mechanism, wherein the driving mechanism comprises an amplitude-variable driving mechanism and/or a rotation driving mechanism, and in some embodiments, the driving mechanism comprises a variable frequency motor and an encoder;

The variable frequency motor and the encoder are connected with each other and are connected with the controller;

the variable frequency motor is used for driving the action mechanism of the intelligent tower crane to move under the control of the controller;

the encoder is used for measuring the action information output by the variable frequency motor and feeding back the measured action information to the controller.

Correspondingly, the variable frequency motor comprises an amplitude variable frequency motor and a rotary variable frequency motor, the encoder comprises an amplitude variable encoder and a rotary encoder, the amplitude variable encoder is used for measuring the amplitude variable position, the amplitude variable speed and other action information of the amplitude variable mechanism, and the rotary encoder is used for measuring the rotary angle, the rotary speed and other action information of the rotary mechanism.

In this embodiment, the variable frequency motor is used to accurately control the motion of the motion mechanism (including the slewing mechanism and/or the luffing mechanism), and the encoder is used to accurately measure the output of the variable frequency motor, so as to obtain the output motion information.

In some variations, the drive mechanism comprises a rotary drive mechanism, the drive mechanism comprises a luffing drive mechanism, the motion information comprises luffing motion information, and the luffing motion information comprises luffing distance and/or luffing speed of the trolley.

In some variations, the drive mechanism comprises a swing drive mechanism, the motion information comprises swing motion information comprising a swing angle and/or a swing speed of the boom.

In some modified embodiments, the controller is specifically configured to calculate a similarity between the inertial motion information and the feedback motion information, and determine that the motion state of the boom is abnormal when the similarity is less than a preset threshold.

The preset threshold value can be flexibly set according to actual requirements, and the embodiment of the application is not limited.

On the basis of the above embodiment, in some modified embodiments, if the motion information has multiple items, the controller is specifically configured to calculate a similarity of each item of motion information, and determine that the motion state of the boom is abnormal if the similarity of any item of motion information is less than a preset threshold.

For example, if the motion information includes four items of amplitude variation distance, amplitude variation speed, rotation angle and rotation speed, the similarity between each item of inertia value and feedback value is calculated, and if any one of the similarities is smaller than a preset threshold value, the abnormal motion state of the suspension arm can be judged.

The inertial motion data comprise acceleration data acquired by an accelerometer and angular velocity data acquired by a gyroscope, the acceleration data can be integrated according to time to obtain velocity data, displacement data can be obtained by integrating the velocity data again, and angle rotation data can be obtained by integrating the angular velocity data, so that inertial motion information can be obtained.

In some embodiments, the above motion information may be calculated by using an inertial navigation algorithm, and correspondingly, the controller is specifically configured to calculate the motion information of the intelligent tower crane inertia according to the inertial motion data through the inertial navigation algorithm.

The inertial navigation algorithm is a mature technical means in the prior art, and compensates calculation data according to factors such as earth rotation and the like in the process of integrating, so that a more accurate calculation result is obtained, a specific implementation mode of the inertial navigation algorithm is not limited in the art, and a person skilled in the art can adopt any inertial navigation algorithm provided by the prior art to directly or change and implement the inertial navigation algorithm so as to achieve the purpose of the embodiment of the application, and the inertial navigation algorithm is within the protection scope of the application.

In some modified implementations of the embodiments of the present application, the controller is further configured to control the trolley to move from an initial end to a final end of the boom when the intelligent tower crane is empty, determine a motion track of the trolley according to inertial motion data generated by the inertial sensor during the moving process, and determine whether deformation abnormality occurs to the boom according to the motion track.

The initial end is one end of the suspension arm, which is close to the tower body, the tail end is one end of the suspension arm, which is far away from the tower body, and according to inertial motion data acquired in the moving process, the position of the trolley at each moment can be calculated by adopting an inertial navigation algorithm, so that the motion track of the trolley is determined, the motion track is connected into a line segment, the bending condition of the line segment can accurately reflect the deformation condition of the suspension arm, and therefore, the deformation information such as the deformation rate of the suspension arm can be calculated, and then whether the suspension arm is deformed abnormally is judged by utilizing whether the deformation information such as the deformation rate accords with a preset deformation standard or not.

Through this embodiment, can also detect the deformation abnormality of davit to the state of comprehensive diagnosis davit, in order to in time discover the abnormality, avoid work and produce the incident when the davit is unusual.

In addition, in order to further perfect the intelligent level and the safety of the intelligent tower crane, the intelligent tower crane can further realize remote monitoring of the intelligent tower crane by configuring the following state data monitoring transmission system for remote control of the intelligent tower crane, so that the automation and the intelligent level of the intelligent tower crane are improved, and the intelligent tower crane is described below with reference to examples.

In some embodiments, the status data monitoring and transmitting system for remote control of the intelligent tower crane may include: the system comprises a controller, a LoRa wireless communication module and a remote monitoring terminal;

the controller is connected with the LoRa wireless communication module, and the LoRa wireless communication module is connected with the remote monitoring terminal in a wireless communication mode;

the controller is used for collecting state data of the intelligent tower crane and sending the state data to the remote monitoring terminal through the LoRa wireless communication module;

the remote monitoring terminal is used for displaying the state data of the intelligent tower crane to a user.

The Long Range Radio (LoRa) is an ultra-Long Range wireless transmission scheme based on a spread spectrum technology, has the advantages of strong signal, long transmission distance, ideal transmission distance of about 3 km, strong penetration and diffraction capacity, small attenuation in a transmission process, easiness in networking, low cost and the like, and can be well applied to a working scene of an intelligent tower crane, so that a good signal transmission effect is obtained, the implementation stability and reliability of the scheme are improved, and the embodiment of the application is not limited.

Compared with the prior art, the state data monitoring transmission system for intelligent tower crane remote control provided by the embodiment of the application is characterized in that the controller is connected with the LoRa wireless communication module, the LoRa wireless communication module is connected with the remote monitoring terminal in a wireless communication mode, the controller is used for collecting state data of the intelligent tower crane and transmitting the state data to the remote monitoring terminal through the LoRa wireless communication module, and the remote monitoring terminal is used for displaying the state data of the intelligent tower crane to a user, so that remote monitoring of the intelligent tower crane can be realized.

In some modified implementations of the embodiments of the present application, the working frequency band of the LoRa wireless communication module adopts a 470-510MHz frequency band. The 470-510MHz frequency band is a free frequency band applied to networking in small areas such as building, residential communities and villages, the implementation cost can be reduced, and the frequency band has longer signal wavelength and better penetrating and diffracting performance, thereby obtaining better signal transmission effect.

In some modification of the embodiment of the present application, the status data monitoring and transmitting system for remote control of an intelligent tower crane further includes: a LoRa gateway;

the LoRa gateway is arranged between the LoRa wireless communication module and the remote monitoring terminal;

and the LoRa wireless communication modules of the intelligent towers are connected with the remote monitoring terminal through the LoRa gateway.

It should be noted that, loRa is a modulation technique of a physical layer, and may be used in different protocols, such as LoRa wan protocol, CLAA network protocol, loRa private network protocol, and LoRa data transparent transmission. When the intelligent tower crane monitoring system is implemented, the LoRa wireless communication module and the LoRa gateway are connected through the LoRa wireless communication, and the LoRa gateway and the remote monitoring terminal can be connected through a cellular network, so that a farther monitoring range is obtained, and a user monitors the intelligent tower crane at a farther distance.

And through passing through the loRa wireless communication module of a plurality of intelligent tower cranes the loRa gateway with remote monitoring terminal is connected, can realize the remote monitoring of a remote monitoring terminal to a plurality of intelligent tower cranes, realize centralized control management, make the user know the situation of job site comprehensively.

In some embodiments, the status data includes idle status data, luffing motion data, slewing motion data, and lifting motion data of the intelligent tower crane, so that a user knows the real-time running status of the intelligent tower crane.

Based on the above embodiments, in some modified embodiments, the status data further includes a type and a weight of the material lifted by the intelligent tower crane each time;

the remote monitoring terminal is further used for counting the workload information of each intelligent tower crane in a preset time period according to the state data and displaying the workload information to a user, wherein the workload information comprises the type and the weight of the materials which are hoisted in the preset time period.

Through this embodiment, can also realize the remote monitoring to intelligent tower crane work load, make the user more comprehensive understanding construction condition, manage the construction progress better.

In some modified embodiments, the intelligent tower crane is provided with a camera group connected with the controller, the camera group is used for collecting a material image group of a material hoisted by the intelligent tower crane and sending the material image group to the controller, wherein the camera group comprises a plurality of cameras with different shooting angles, and the material image group comprises a material image collected by each camera for the material;

The controller is further used for determining attribute information of the materials according to the material image group, and obtaining the category of the materials by matching from a tower crane material database according to the attribute information of the materials, wherein the attribute information comprises shape information, size information and texture information, and the tower crane material database stores the attribute information of different tower crane materials in advance.

Because the materials loaded and unloaded by the tower crane are clear in category and distinct in characteristics, the materials commonly used for hoisting by the tower crane are mainly raw materials for building construction such as steel bars, wood ridges, concrete, steel pipes and glass, and the shape, the size and the texture of the materials are greatly different, so that the category of the materials can be rapidly and accurately identified through database matching only by pointedly extracting the attribute information such as the shape, the size and the texture of the materials. Through the embodiment, the types of the materials can be accurately identified, so that a user can know the types of the materials hoisted by the intelligent tower crane through the remote monitoring terminal, and reasonable management and control are performed.

In some modified embodiments, the determining attribute information of the material according to the material image group includes:

inquiring an initial image corresponding to the gesture information from an initial image database according to the gesture information of each camera for acquiring the material image, wherein the gesture information comprises shooting position information and shooting angle information of the camera, the initial image database stores initial images acquired by each camera corresponding to the gesture information, and the initial images are acquired before the material enters the field;

Respectively comparing each material image in the material image group with the initial image which is acquired in advance by a camera for acquiring the material image under the same gesture, and identifying a material main body in each material image;

and determining attribute information of the materials according to the identified material main bodies in each material image.

On the basis of the above embodiment, for extracting the shape information and the size information, in some modified embodiments, the determining, according to the identified material body in each material image, attribute information of the material includes:

determining the coordinate conversion relation between the pixel coordinate system corresponding to each camera and the world coordinate system;

according to the coordinate conversion relation, converting the pixel coordinates of the material main body in each material image into world coordinates in a world coordinate system;

and determining shape information and size information of the material according to the world coordinates.

On the basis of the foregoing embodiments, for extracting texture information, in some modified embodiments, the determining, according to the identified material body in each material image, attribute information of the material includes:

and identifying texture information of the material main body in each material image by adopting a texture identification algorithm.

For the collection of the material weight information, in some modified embodiments, a force sensor is arranged on a guide pulley mechanism of the intelligent tower crane;

the force sensor is used for acquiring real-time force sensing data and sending the force sensing data to the controller;

The controller is also used for calculating the weight of the materials hoisted by the intelligent tower crane according to the force sensing data.

The force sensor can be realized by adopting a pin force sensor, the weight of the materials hoisted by the intelligent tower crane is calculated by utilizing force sensing data, which is a mature technical scheme in the field, and the force sensor is not repeated here, so that the aim of the embodiment of the application can be realized by the technical staff in the field by referring to the prior art.

Through this embodiment, can realize the measurement to material weight, the user of being convenient for knows the weight of the material that intelligent tower crane hoisted through remote monitoring terminal to carry out reasonable management and control.

In some modified implementations of the embodiments of the present application, the remote monitoring terminal is specifically configured to construct a real-time simulation scene in building information model BIM software according to attribute information and the state data of the intelligent tower crane, and display the simulation scene through a display, so that a user can know a working state of the intelligent tower crane by watching the simulation scene, where the attribute information includes position information, altitude information and specification model information of the intelligent tower crane.

Building information model (Building Information Modeling, BIM) software is a graphical tool for architecture, engineering, and civil engineering. The core of BIM is to build a virtual three-dimensional building engineering model and provide a complete building engineering information base consistent with the actual situation for the model by utilizing a digitizing technology. The information base contains not only geometric information, professional attributes and state information describing building elements, but also state information of non-element objects (such as space, sports behavior). By means of the three-dimensional model containing the construction engineering information, the information integration degree of the construction engineering is greatly improved, and therefore a platform for engineering information exchange and sharing is provided for relevant stakeholders of the construction engineering project.

According to the embodiment, the BIM can be used for constructing a simulation scene of the intelligent tower crane on the construction site, so that a user can intuitively and rapidly know the working state of the intelligent tower crane.

In addition, in order to further perfect the intelligent level and the safety of the intelligent tower crane, the intelligent tower crane can also realize the warning of personnel and vehicles on the operation site of the intelligent tower crane by configuring the safety warning auxiliary system for the intelligent tower crane, so that the safety level of the intelligent tower crane is improved, and the following description is made with reference to an example.

In some embodiments, the safety warning auxiliary system for an intelligent tower crane may include: the intelligent auxiliary robot is in communication connection with the controller;

the controller is used for determining the position information of the lifting hook of the intelligent tower crane according to the rotation information, the amplitude information and the lifting information of the intelligent tower crane, and sending the position information of the lifting hook to the intelligent auxiliary robot;

the intelligent auxiliary robot is arranged on the ground near the intelligent tower crane, and is used for moving along with the lifting hook of the intelligent tower crane according to the position information of the lifting hook, determining a warning area around the lifting hook, and sending out an alarm after detecting that a dynamic object enters the warning area.

Compared with the prior art, the safety warning auxiliary system for the intelligent tower crane provided by the embodiment of the application is characterized in that the controller is used for determining the position information of the lifting hook of the intelligent tower crane according to the rotation information, the amplitude information and the lifting information of the intelligent tower crane through the arrangement of the controller and the intelligent auxiliary robot in communication connection with the controller, the intelligent auxiliary robot is arranged on the ground near the intelligent tower crane and used for following the lifting hook of the intelligent tower crane to move according to the position information of the lifting hook, determining the warning area around the lifting hook and sending out an alarm after detecting that a dynamic object enters the warning area, so that the dynamic object such as personnel and vehicles entering the warning area can be automatically detected and timely sent out an alarm, the safety of operation of the intelligent tower crane is improved, and the accuracy and the timeliness of safety warning can be improved because the intelligent auxiliary robot follows the movement of the lifting hook and can be updated in real time.

In some embodiments, the intelligent auxiliary robot includes: control module and environment scanning module:

the environment scanning module is connected with the control module and is used for scanning the environment data around the intelligent auxiliary robot and sending the scanned environment data to the control module;

the control module is used for detecting whether a dynamic object enters the warning area according to the environmental data.

By the implementation mode, whether a dynamic object enters the warning area can be accurately detected by scanning surrounding environment data.

Furthermore, in some variant embodiments, the intelligent auxiliary robot further comprises: a wireless communication module connected with the control module;

the control module is in communication connection with the controller through the wireless communication module.

In some modified implementations of the embodiments of the present application, the control module is specifically configured to perform synchronous mapping on a surrounding environment according to the environmental data by using a synchronous positioning and mapping SLAM algorithm corresponding to the environmental scanning module, determine a warning area around the hook according to the constructed environmental map, and detect whether a dynamic object enters the warning area.

The synchronous positioning and map construction (Simultaneous Localization And Mapping, SLAM) refers to a process of realizing self positioning of a moving object carrying a sensor in the moving process and synchronously constructing a map of the surrounding environment in a proper mode, can be regarded as a combination of two technologies of self positioning and map construction, and is a mature algorithm applied to the field of robots. The explanation of a specific point is as follows: a movable robot starts to move from any place in a completely unknown environment, and continuously uses a sensor (such as the environment scanning module) to observe surrounding environment characteristics in the process of movement, so that the position and the angle of the robot are positioned, and meanwhile, the environment incremental map is continuously updated and constructed according to the relative pose information of the robot and the environment, and the movable robot is helped to construct a perception system for generating surrounding three-dimensional environment data, so that the autonomous movement and environment perception of the movable robot are realized.

Through the embodiment, the real-time construction of the surrounding environment map can be realized by adopting the SLAM algorithm, so that the warning area around the lifting hook is determined based on the constructed environment map, and whether a dynamic object enters the warning area is detected.

Considering that a construction site often has a plurality of intelligent towers working together, in order to enable the intelligent auxiliary robot to identify the intelligent towers paired with the intelligent auxiliary robot and perform a targeted alarm, in some modified implementations of the embodiments of the present application, the safety warning auxiliary system for the intelligent towers further includes: a robot warehouse paired with the intelligent auxiliary robot;

the robot warehouse is arranged below the tower body of the intelligent tower crane and is used for parking the intelligent auxiliary robot and charging the intelligent auxiliary robot;

the intelligent auxiliary robot starts to move by taking the position of the robot warehouse as an initial position;

the control module is specifically configured to determine the initial position as a tower body position of the intelligent tower crane, mark the tower body in the constructed environment map according to the tower body position, mark the lifting hook in the environment map according to the position information of the lifting hook relative to the tower body, and mark a preset area around the lifting hook as a warning area in the environment map.

Based on the above embodiment, the control module is specifically configured to update the environment map according to a preset time interval, and identify the dynamic object by comparing changes of the environment map before and after the update.

The dynamic objects can be personnel, vehicles and the like, the time interval can be flexibly set according to actual demands, the embodiment of the application is not limited, the dynamic objects in the warning area can be identified through map comparison, on the basis, in order to reduce the operation load of the controller, the operation efficiency is improved so as to send out warning in real time, the environment map in the warning area can be only compared so as to identify the dynamic objects, the operation load of the controller is reduced, the operation efficiency is improved, the warning can be sent out more timely, and the real-time performance of safety warning is improved.

In addition, because the dynamic objects entering the construction site are single and have obvious characteristics, for example, people wear safety helmets and wearing tools, and vehicles are limited to vehicles with obvious characteristics and few categories, such as transport vehicles, stirring vehicles and the like, therefore, on the basis of the embodiment, the characteristic information of the dynamic objects can be extracted in advance, after the dynamic objects are identified, the dynamic objects and the categories thereof can be further determined clearly by carrying out characteristic matching according to the characteristic information, and the dynamic objects and the categories thereof can be realized by referring to any human body recognition algorithm, vehicle recognition algorithm and the like based on image characteristics provided by the prior art, which are not repeated herein, and are all within the protection scope of the application.

In some modified embodiments, the warning area may include a circular area centered on the hook and a protruding area along the movement direction of the hook. The circular area is determined to be a warning area, people or vehicles are prevented from being injured by the unhooking of the material at the current position, when the lifting hook moves, the unhooking of the material can continue to move forward in the lifting process to injure the people or vehicles in front, and certain reaction time is also needed for the alarm to be sent and the people to leave after hearing the alarm, so that the protruding area is determined to be the warning area, a certain amount of advance can be provided, the people and the vehicles entering the protruding area are ensured to leave for a sufficient time, and the effectiveness of the alarm is improved.

In addition, to implement the alert function, in some embodiments, the intelligent auxiliary robot further includes: the alarm module is connected with the control module;

the control module is specifically used for controlling the alarm module to send out an alarm to the dynamic object after detecting that the dynamic object enters the alarm area.

The alarm module can be realized by adopting a sound box, a loudspeaker and the like, and alarm content, such as voice content such as please note that you have entered a dangerous area of tower crane construction and please get away from a lifting hook, can be input in advance in actual implementation, and then the alarm module can be controlled to broadcast the voice content so as to give an alarm to a dynamic object. With the present embodiment, the alarm can be effectively given by voice or the like, and the alarm effectiveness can be improved.

On the basis of any of the above embodiments, the intelligent auxiliary robot may further include: and the mobile module is connected with the control module and is used for driving the intelligent auxiliary robot to move on the ground around the intelligent tower crane under the control of the control module.

In addition, in order to further perfect the intelligent level and safety of the intelligent tower crane, the intelligent tower crane can also predict and warn the collision accident of the tower crane and improve the safety level of the intelligent tower crane by configuring the safety warning auxiliary system for the intelligent tower crane, and the following description is made with reference to examples.

In some embodiments, the control safety warning system for implementing intelligent tower crane assistance may include: a controller and a safety monitoring terminal of the intelligent tower crane;

the controller is used for collecting state data and control instructions of the intelligent tower crane and sending the state data and the control instructions to the remote monitoring terminal;

the safety monitoring terminal is in communication connection with a plurality of controllers of the intelligent tower cranes, and is used for simulating the running states of the intelligent tower cranes in three-dimensional simulation software and predicting running tracks according to the state data and the control instructions sent by the controllers, and feeding back safety warning information for the control instructions to the controllers according to the prediction results.

Compared with the prior art, the embodiment of the application provides an implementation intelligent tower crane auxiliary control safety warning system, through setting up intelligent tower crane's controller and safety monitoring terminal, the controller is used for gathering intelligent tower crane's state data and control the instruction, and will state data with control the instruction and send for remote monitoring terminal, safety monitoring terminal with a plurality of intelligent tower crane's controller communication connection is used for according to a plurality of the controller sends state data with control the instruction, simulate in three-dimensional simulation software a plurality of intelligent tower crane's running state and carry out the moving track prediction, and according to the prediction result to the controller feedback is directed at control the safety warning information of instruction, thereby can utilize safety monitoring terminal to realize the prediction to a plurality of intelligent tower cranes emergence collision accident, make the controller can control the instruction according to the rational handling of safety warning information, realize the emergence to intelligent tower crane's safety control, improve intelligent tower crane's security and intelligent level.

In some modified implementations of the embodiments of the present application, the security monitoring terminal is specifically configured to construct a real-time simulation scene in building information model BIM software according to attribute information of each intelligent tower crane, and simulate, in the simulation scene, operation states of the plurality of intelligent tower cranes according to state data and control instructions of each intelligent tower crane, and perform operation track prediction.

The attribute information can include, but is not limited to, position information, height information and specification and model information of the intelligent tower crane, a real-time simulation scene can be constructed in building information model BIM software by using the attribute information, the state data include data describing real-time working states of the intelligent tower crane such as luffing motion data, rotation motion data and lifting motion data, the state data can be set in the BIM software to simulate the real-time motion state of the intelligent tower crane, on the basis, the control instruction is input, the control instruction can be further executed to predict the running track of the intelligent tower crane for executing the control instruction, and under the condition that the running tracks of the intelligent tower cranes are all predicted, whether the intelligent tower cranes have collision risks can be predicted.

In some variations of the embodiments of the present application, the safety warning information includes safety indication information and hazard indication information;

the safety monitoring terminal is specifically configured to generate safety instruction information for the control instruction according to the situation that the prediction result is that the intelligent tower crane cannot collide, and generate danger instruction information for the control instruction according to the situation that the prediction result is that the intelligent tower crane can collide.

On the basis of any of the above embodiments, in some modified embodiments, the controller does not execute immediately after generating the manipulation instruction;

the controller is also used for triggering and executing the control instruction after receiving the safety indication information which is fed back by the safety monitoring terminal and is specific to the control instruction.

It should be noted that, the controller is used as a control mechanism of the intelligent tower crane, and control instructions for all the motion mechanisms (such as a lifting mechanism, a slewing mechanism, an amplitude changing mechanism, an automatic lifting hook, etc.) are generated by the controller.

On the other hand, in some embodiments, the controller does not execute immediately after generating the manipulation instruction;

and the controller is also used for changing the control instruction and resending the control instruction to the safety monitoring terminal for prediction after receiving the dangerous instruction information which is fed back by the safety monitoring terminal and aims at the control instruction.

In this embodiment, after the controller receives the danger indication information, the controller does not execute the control instruction any more, but sends the control instruction to the safety monitoring terminal again to predict the danger indication information until the controller receives the safety indication information, so that the controller executes the corresponding control instruction again after ensuring safety, and accidents caused by the operation under the condition of potential safety hazards are avoided.

It should be noted that, the danger indication information may carry information such as an identifier and a position of the intelligent tower crane that collides with the intelligent tower crane, and information such as time and a position of the collision, and the controller may correspondingly change the control instruction according to the information, for example, change the amplitude distance in the control instruction so as to make the trolley approach the tower body to avoid the collision, change the lifting height in the control instruction so as to make the material rise to avoid the collision, change the rotation speed in the control instruction so as to slow down the rotation speed and avoid the collision.

In other embodiments, the controller executes immediately after generating the manipulation instruction;

And the controller is also used for triggering to stop executing the control instruction after receiving the danger indication information which is fed back by the safety monitoring terminal and is specific to the control instruction.

In this embodiment, the controller can execute immediately after generating the control instruction, and stop after receiving the danger indication information, so that the time for waiting for the safety indication information can be saved, the working efficiency of the intelligent tower crane is improved, and as the intelligent tower crane has a certain time delay from starting operation to collision, the safety monitoring terminal can complete the prediction of the running track and return the safety warning information within the delay time, so that the occurrence of collision accidents can be effectively avoided, and the working efficiency can be considered while the safety is improved.

Based on any of the foregoing embodiments, in some modified embodiments, the security monitoring terminal is further configured to generate, according to a preset yield rule, yield indication information and send the yield indication information to the controller, where the prediction result is that the intelligent tower crane will have a collision accident;

the controller is further configured to delay or slow down execution of the control instruction according to the yield indication information.

The above yielding rule can be flexibly set according to actual requirements, for example, the intelligent tower crane is not limited by the foregoing yielding rule, and the intelligent tower crane is stopped and is waited for a period of time (the intelligent tower crane can be flexibly set according to actual requirements and can also be real-time determined according to collision time information fed back by a safety monitoring terminal), and then is restored to the operation, and the slowing down of the execution means slowing down of the movement speed of the movement mechanism to avoid collision. The aim of the embodiment of the application can be achieved by the two embodiments, wherein the safety of the delayed execution is higher, and the influence of avoidance on the operation efficiency can be reduced by slowing down the execution, so that the avoidance is achieved with higher operation efficiency.

Based on any of the foregoing embodiments, in some modified embodiments, the control safety warning system for implementing intelligent tower crane assistance further includes: a LoRa wireless communication module;

the controller is connected with the LoRa wireless communication module, and the LoRa wireless communication module is connected with the safety monitoring terminal in a LoRa wireless communication mode.

The Long Range Radio (LoRa) is an ultra-Long Range wireless transmission scheme based on a spread spectrum technology, has the advantages of strong signal, long transmission distance, ideal transmission distance of about 3 km, strong penetration and diffraction capacity, small attenuation in a transmission process, easiness in networking, low cost and the like, is suitable for working scenes of intelligent towers, realizes networking of a plurality of intelligent towers and safety monitoring terminals, and accordingly achieves good signal transmission effect, stability and reliability of implementation of the scheme are improved, and the embodiment of the application is not limited.

In addition, in order to further perfect the intelligent and automatic level of the intelligent tower crane, the intelligent tower crane can further realize detection of the lifting direction of the material lifting part and automatic guiding rotation of the lifting hook by configuring the following electromagnetic positioning device for guiding the lifting hook of the intelligent tower crane, so that the lifting hook is quickly aligned to the material lifting part for lifting, the lifting efficiency and the lifting accuracy are improved, the safety level of the intelligent tower crane is improved, and the following description is made by combining an example.

In some embodiments, the electromagnetic positioning device for intelligent tower crane hook guiding may include a controller, an electric rotating hook, and an electromagnetic positioning system; wherein, the liquid crystal display device comprises a liquid crystal display device,

the electric rotary lifting hook and the electromagnetic positioning system are connected with the controller;

the material side emitter of the electromagnetic positioning system is detachably arranged on the hoisting part of the material to be hoisted and emits magnetic force signals outwards;

the system electronic unit of the electromagnetic positioning system is used for determining pose information of the material side transmitter according to the magnetic force signal and sending the pose information of the material side transmitter to the controller;

the controller is used for determining the hooking direction of the hoisting part according to the pose information of the material side emitter and controlling the electric rotary lifting hook to adjust the hooking direction according to the hooking direction.

Compared with the prior art, the electromagnetic positioning device for guiding the intelligent tower crane lifting hook is provided with the controller, the electric rotating lifting hook and the electromagnetic positioning system; wherein, the electric rotary lifting hook and the electromagnetic positioning system are both connected with the controller; the material side emitter of the electromagnetic positioning system is detachably arranged on the hoisting part of the material to be hoisted and emits magnetic force signals outwards; the system electronic unit of the electromagnetic positioning system is used for determining pose information of the material side transmitter according to the magnetic force signal and sending the pose information of the material side transmitter to the controller; the controller is used for determining the hooking direction of the hoisting part according to the pose information of the material side emitter and controlling the electric rotary lifting hook to adjust the hooking direction according to the hooking direction. Thereby can utilize electromagnetic positioning system to realize the detection to material hoist and mount portion hoist and mount direction to control electronic rotatory lifting hook adjustment hook direction, realize the automatic guidance rotation of lifting hook, make the lifting hook aim at material hoist and mount portion fast and hoist and mount, improve hoist and mount efficiency and degree of accuracy, and then improve intelligent and the automation level of intelligent tower crane.

The controller refers to a controller of the intelligent tower crane, and can be realized by a computer host, a microcontroller, a Programmable Logic Controller (PLC) and the like, the electric rotating lifting hook refers to a lifting hook of the intelligent tower crane, any electric control rotating lifting hook component provided by the prior art can be adopted to realize, and the electromagnetic positioning system can be realized by any electromagnetic position tracking system provided by the prior art.

In some variations of the present application, the electromagnetic positioning system includes a system electronics unit, a receiver, and a material side transmitter; wherein, the liquid crystal display device comprises a liquid crystal display device,

the system electronic unit and the receiver are both arranged on the fixed part of the electric rotary lifting hook;

the receiver is connected with the system electronic unit and is used for collecting magnetic force signals emitted by the material side emitter and sending collected information to the system electronic unit;

the system electronic unit is used for calculating pose information of the material side transmitter according to the acquired information and the position information of the receiver, and sending the pose information to the controller.

The material side emitter is an emitter arranged on one side of the material, and the emitter can be arranged on the material lifting part so as to determine the hook feeding direction of the lifting part according to pose information of the emitter.

Taking Polhemus LIBERTY LATUS wireless large-range tracking system as an example, polhemus LIBERTY LATUS wireless large-range tracking system is a wireless magnetic tracking solution with 6 degrees of freedom, the system can track 12 independent markers (namely transmitters) at most, has the characteristics of wide tracking range, high speed and simple use, is provided with an intuitive user interface and a perfect software development kit, can output data to a computer host, a microcontroller, a PLC (programmable logic controller) and other controllers, has extremely high stability and extremely low signal noise ratio, and can provide consistent high-quality data.

The specific mode of calculating the pose information of the material side transmitter by the system electronic unit according to the collected magnetic force signals and the position information of the receiver can be realized by referring to the magnetic force signal-based positioning algorithm provided by the prior art, the specific algorithm is not limited in the embodiment of the application, and the system electronic unit can be realized by directly adopting the existing product (such as the Polhemus LIBERTY LATUS wireless large-range tracking system) without paying attention to the specific positioning algorithm during specific implementation.

Taking the Polhemus LIBERTY LATUS wireless wide-range tracking system as an example, in some modified embodiments, the material side transmitter includes a triad electromagnetic source, a control circuit, and a power supply battery;

The power supply battery is used for supplying power to the triple electromagnetic source and the control circuit;

the control circuit is used for controlling the triad electromagnetic source to externally emit magnetic force signals.

Through the above-mentioned embodiment, can adopt the transmitter of independent power supply and work to realize the transmission of magnetic force signal, the transmitter need not to carry out physical cable connection with the system electronic unit for ground staff can install the transmitter in the hoist and mount portion of material convenient and fast at any time, thereby has higher convenience and flexibility.

In some modified embodiments, the receiver includes a triad electromagnetic receiving element for detecting a magnetic force signal emitted from the material side emitter;

and the system electronic unit is used for calculating the 6-degree-of-freedom pose information of the material side transmitter according to the magnetic force signals and the position information of the receiver.

Because the transmitter and the receiver both adopt the three-element electromagnetic element, the system electronic unit can calculate pose information with 6 degrees of freedom, including the position (X, Y, Z Cartesian coordinates) and the direction (azimuth angle, elevation angle and roll angle) of the transmitter, thus, the hook-in direction of the hoisting part can be determined in various ways, and rich and flexible implementation modes are provided so as to be suitable for various complex scenes.

For example, if the material side emitters are mounted on the hoisting part at a fixed angle (parallel or perpendicular to the hooking direction), the hooking direction of the hoisting part can be determined by using the azimuth angle of one material side emitter; in addition, the hooking direction of the hoisting part can also be determined by using the position information of the plurality of material side transmitters and the respective transmitters, for example:

in some variations, the number of material side emitters is two;

the two material side emitters are respectively arranged at two sides of the hoisting position on the hoisting part, and the connecting line of the two material side emitters is perpendicular to the hooking direction;

the controller is specifically used for determining a connecting line between the two material side transmitters according to pose information of the two material side transmitters, making a vertical line along the horizontal direction from the position of the electric rotary lifting hook to the connecting line, and determining the hook feeding direction of the lifting part according to the direction of the vertical line.

Through this embodiment, can utilize two material side emitter to realize advancing the sign and confirm of hook direction to hoist and mount portion, consider that material side emitter has certain volume, be difficult to be fixed in on hoist and mount portion such as rope, rings, be difficult to adopt the direction angle of emitter to confirm into hook direction, to this kind of situation, can install two material side emitters in rope hoist and mount position both sides, so, material side emitter can bind with arbitrary gesture, hang, attach on the rope and need not to stabilize its gesture, then utilize the positional information of two emitters can confirm into hook direction, have simple operation, easy implementation, the higher advantage of degree of accuracy.

In some variations on any of the above embodiments, the electromagnetic positioning system further comprises a hook side emitter;

the lifting hook side emitter is fixedly arranged on the electric rotary lifting hook and is used for emitting magnetic force signals outwards;

the receiver is also used for collecting magnetic force signals transmitted by the lifting hook side transmitter and transmitting collected information to the system electronic unit;

the system electronic unit is used for calculating pose information of the lifting hook side transmitter according to the magnetic force signal transmitted by the lifting hook side transmitter and the position information of the receiver, and transmitting the pose information of the lifting hook side transmitter to the controller;

the controller is also used for determining the current hooking direction of the electric rotating hook according to the pose information of the hook side emitter, and controlling the electric rotating hook to adjust the hooking direction to be consistent with the hooking direction according to the hooking direction.

Considering that the purpose of the rotary hook is to make the hook hanging direction of the hook consistent with the hook entering direction of the hanging part, therefore, the real-time hook hanging direction of the hook needs to be determined.

It should be noted that, through the above embodiment, the above coordinate system of the hooking direction and the hooking direction can be unified, that is, the hooking direction and the hooking direction are obtained in the cartesian coordinate system, so that the pertinence and the accuracy of the hook steering are improved, and the consistency of the hooking direction and the hooking direction is ensured.

In some modified embodiments, the hook side emitter is fixedly mounted on the hook body of the electric rotating hook;

the controller is specifically used for calculating the current hooking direction of the electric rotary hook according to the relative pose information of the hook side emitter and the hook body and the pose information of the hook side emitter.

Through this embodiment, can with lifting hook side transmitter fixed mounting in on the hook body of electronic rotatory lifting hook to can be simple, the current hook direction of lifting hook is calculated to direct utilization lifting hook side transmitter's position appearance information.

In other modified embodiments, the hook side transmitter is fixedly mounted to the fixing portion of the electric rotating hook, considering that if the transmitter is mounted to the hook body, the transmitter is easily damaged due to collision or the like;

The electric rotary lifting hook comprises an electric cabinet, a motor, a gear and a hook body which are arranged on the fixing part;

the gear is sleeved on the handle of the hook body and meshed with the output gear of the motor;

the electric control box is electrically connected with the motor and is used for controlling the motor to rotate so as to drive the hook body to rotate through the gear, calculating relative orientation information of the hook body relative to the fixed part according to rotation position information of the motor and gear ratio of an output gear of the motor and a gear sleeved on a handle of the hook body, and sending the relative orientation information to the controller;

the controller is specifically configured to calculate a current hooking direction of the electric rotating hook according to the relative orientation information, the pose information of the hook side emitter, and the relative pose information of the hook side emitter and the fixing portion.

The motor may be implemented by a servo motor or a stepper motor, which is not limited in this embodiment.

Through this embodiment, can also realize the unanimity of hook hanging direction and advance hook direction under the same coordinate system to realize the accurate rotation and the counterpoint of lifting hook, and can effectively reduce the probability that the transmitter is bumped, improve life and system stability.

It should be noted that, the electromagnetic positioning device for guiding an intelligent tower crane lifting hook provided in the embodiment of the application can be matched with or implemented in a compatible manner with the sensing internet of things device for capturing and placing movement detection of an intelligent tower crane provided in the foregoing embodiment of the application, for example, firstly, the following sensing internet of things device for capturing and placing movement detection of an intelligent tower crane is utilized to control the lifting hook (namely, the automatic lifting hook comprises the above-mentioned electric rotating lifting hook) to move to the vicinity of the material lifting part, and then the electromagnetic positioning device for guiding an intelligent tower crane lifting hook provided in the embodiment of the application is adopted to adjust the lifting hook direction of the lifting hook so as to align with the lifting part, and then the lifting hook is controlled to carry out automatic lifting, so that automatic lifting of materials is completed.

In addition, in order to further perfect the level of intellectualization and automation of the intelligent tower crane, the intelligent tower crane can also realize the abnormal detection of the swing action of the intelligent tower crane to perform intelligent early warning by configuring the following intelligent monitoring early warning system based on the swing action model of the tower crane, so that the safety of the intelligent tower crane is improved, and the following description is made with reference to examples.

In some embodiments, the intelligent monitoring and early warning system based on the tower crane rotation action model can comprise a controller, a gravity sensor and a wind sensor; wherein, the liquid crystal display device comprises a liquid crystal display device,

The gravity sensor is arranged at a lifting hook of the intelligent tower crane and is used for collecting weight information of materials lifted by the lifting hook and sending the weight information to the controller;

the wind sensor is arranged at the top of the intelligent tower crane and is used for collecting wind information around the intelligent tower crane and sending the wind information to the controller;

the controller is used for calculating a theoretical rotation angle of the intelligent tower crane after a preset time length according to rotation angle calculation parameters, comparing the theoretical rotation angle with an actual rotation angle of the intelligent tower crane, and carrying out early warning on the rotation action of the intelligent tower crane according to a comparison result, wherein the rotation angle calculation parameters comprise weight information and wind power information.

Compared with the prior art, the intelligent monitoring and early warning system based on the tower crane rotation action model is provided by arranging a controller, a gravity sensor and a wind sensor; the gravity sensor is arranged at a lifting hook of the intelligent tower crane and is used for collecting weight information of materials lifted by the lifting hook and sending the weight information to the controller; the wind sensor is arranged at the top of the intelligent tower crane and is used for collecting wind information around the intelligent tower crane and sending the wind information to the controller; the controller is used for calculating a theoretical rotation angle of the intelligent tower crane after a preset time length according to rotation angle calculation parameters, comparing the theoretical rotation angle with an actual rotation angle of the intelligent tower crane, and carrying out early warning on the rotation action of the intelligent tower crane according to a comparison result, wherein the rotation angle calculation parameters comprise weight information and wind power information. Therefore, the theoretical rotation angle can be calculated by using rotation angle calculation parameters such as weight, wind force and the like, whether the rotation action is abnormal or not is judged and early warning is carried out by comparing the theoretical rotation angle with the actual rotation angle, the abnormal detection of the rotation action of the intelligent tower crane is realized to carry out intelligent early warning, and the safety of the intelligent tower crane is further improved.

The controller refers to a controller of an intelligent tower crane, and can be realized by a computer host, a microcontroller, a Programmable Logic Controller (PLC) and the like, the gravity sensor and the wind sensor can be directly realized by a finished product sensor provided by the prior art, the embodiment of the application is not limited to specific models and specifications, the gravity sensor is used for collecting weight information of the hoisted materials, and the wind sensor is used for collecting wind force information of the hoisted materials, including wind direction, wind speed and the like.

In some modified implementations of the embodiments of the present application, the rotation angle calculation parameters of the intelligent tower crane further include an output torque of a rotation mechanism of the intelligent tower crane, an amplitude position, and a cross-sectional area of the material in a windward direction.

On the basis of any of the above embodiments, in some modified embodiments, the controller is specifically configured to calculate wind force to which the material is subjected according to a cross-sectional area of a windward direction of the material and wind speed information in the wind force information, calculate wind resistance torque of the material according to the wind force to which the material is subjected and the luffing position, calculate rotational torque of the material according to the output torque and the wind resistance torque, and calculate a theoretical rotation angle within a preset duration according to the rotational torque and a weight of the material.

For example, the wind force exerted on the materials can be calculated according to the following formula:

wind force = windward area x windward area coefficient x wind speed resistance of the material.

Wherein wind speed drag = coefficient K wind speed.

The windward area coefficient and the coefficient K may be determined according to experiments, and the embodiment of the present application is not limited.

Through the implementation mode, the theoretical rotation angle can be obtained through calculation of the mathematical relationship, so that whether the rotation action of the intelligent tower crane is abnormal or not can be judged.

Because the material can rotate in the hoisting process, the windward area of the material is inconvenient to measure and the corresponding windward area coefficient is difficult to determine, the windward sectional area of the material is preferably adopted as the windward area for calculation in the embodiment of the application, and the relatively accurate calculation result can be obtained. The cross-sectional area in the windward direction may be obtained by the following embodiments, and in some modified embodiments, the controller is further configured to determine the cross-sectional area in the windward direction of the material according to a three-dimensional image of the material acquired in advance, orientation information of the material acquired in real time, and wind direction information in the wind force information.

The three-dimensional image of the material can be acquired in advance, and in particular, in some embodiments, the intelligent monitoring and early warning system based on the tower crane rotation motion model further comprises: the intelligent auxiliary robot is used for detecting the working condition of the intelligent tower crane;

the intelligent auxiliary robot is in communication connection with the controller and is used for moving around the material on the ground to shoot a three-dimensional image of the material and sending the three-dimensional image to the controller.

Based on any of the foregoing embodiments, in some modified embodiments, the intelligent monitoring and early warning system based on the tower crane rotation motion model further includes: a global camera connected to the controller;

the controller is also used for identifying the orientation information of the materials in the working scene in real time according to the global image.

On the basis of any of the foregoing embodiments, in some modified embodiments, the controller is specifically configured to normalize the rotation angle calculation parameter, input a preset rotation action model, and output a theoretical rotation angle after a preset time period by using the rotation action model, where the rotation action model is obtained by training in advance by using normal tower crane operation data based on a neural network.

Considering that the windward area coefficient and the coefficient K can change frequently in practical application to influence the accuracy of mathematical calculation, the embodiment of the application is changed to realize the determination of the theoretical rotation angle by means of a neural network, so that the more accurate theoretical rotation angle can be obtained, and the accuracy of rotation action detection and the accuracy of early warning are improved.

Based on the above embodiments, in some modified embodiments, the controller is further configured to obtain multiple sets of operation data of a normal tower crane, where each set of operation data includes a rotation angle calculation parameter of the intelligent tower crane acquired at any time, and an actual rotation angle after a preset duration at any time, and take each rotation angle calculation parameter in each set of operation data after normalization processing and the actual rotation angle in the set of operation data together as a set of training samples, and train a neural network by using multiple sets of training samples to obtain a rotation action model, where the rotation angle calculation parameter after normalization processing in each set of training samples is taken as an input factor of the neural network, and the actual rotation angle in each set of training samples is taken as an output factor of the neural network.

Through the implementation mode, a more accurate rotation action model can be obtained through training, and further more accurate theoretical rotation angles are obtained.

In addition, in order to further perfect the intelligent and automatic level of the intelligent tower crane, the intelligent tower crane can further realize automatic detection of the intelligent tower crane on the materials by configuring the following automatic material characteristic recognition device for the unmanned intelligent tower crane, judge whether the materials accord with hoisting conditions to carry out quick and accurate hoisting, further improve the intelligent and automatic level and safety of the intelligent tower crane, and be described below with reference to examples.

In some embodiments, the automatic material characteristic recognition device for the unmanned intelligent tower crane may include:

the image acquisition module is used for acquiring the global image of the intelligent tower crane working scene shot by the global camera;

the image recognition module is used for recognizing the carrier and the material in the global image through an image recognition algorithm;

the material type determining module is used for determining the type of the material according to preset carrier use conditions, wherein different carriers are used for loading different types of materials;

The shielding judging module is used for judging whether the identified materials are shielded or not according to the standard size of the type of materials;

and the hoisting condition judging module is used for determining that the material meets hoisting conditions under the condition that the material is not shielded.

In some variations of the embodiments of the present application, the image recognition module includes:

the feature matching unit is used for identifying a carrier in the global image through image feature matching, wherein the carrier has image features which are different from materials and surrounding environments;

the growth point determining unit is used for determining growth points in the bearing range of the carrier of the global image;

and the region growing unit is used for dividing and determining the material in the global image by using a region growing method by taking the growing points as references.

In some modification of the embodiment of the present application, the growth point determining unit includes:

a growing point determining subunit, configured to query growing point selection information corresponding to the type of the material in a preset growing point selection mapping table, where the growing point selection information corresponding to different material types is recorded in the growing point selection mapping table, and the positions, sizes or numbers of growing points corresponding to different material types are different;

And determining a growth point in the bearing range of the carrier of the global image according to the growth point selection information.

In some modified implementations of the embodiments of the present application, if the determined number of growing points is a plurality of, the area growing unit includes:

the multi-region growth subunit is used for dividing a plurality of growth regions in the global image by using a region growth method respectively with each growth point as a reference;

and the multi-region merging subunit is used for merging the plurality of growing regions to determine the materials.

In some modification of the embodiment of the present application, the occlusion determining module includes:

the actual size calculating unit is used for calculating the actual size of the identified material;

the standard size inquiring unit is used for inquiring the standard size of the type of material from a preset material information base;

and the size comparison unit is used for comparing the difference between the actual size and the standard size, judging that the material is not shielded if the difference is smaller than a preset threshold value, and judging that the material is shielded if the difference is not smaller than the preset threshold value.

According to the material automatic feature recognition device for the unmanned intelligent tower crane, the carrier and the material are recognized in the global image through the image recognition algorithm by acquiring the global image of the intelligent tower crane working scene shot by the global camera, and the type of the material is determined according to the preset carrier use condition, wherein different carriers are used for loading different types of materials; judging whether the identified material is shielded or not according to the standard size of the type of material; and under the condition that the material is not shielded, determining that the material meets hoisting conditions. Therefore, the materials can be accurately identified and judged to be shielded, whether the materials meet the hoisting conditions or not is further determined, so that quick and accurate hoisting is performed, and the intelligent and automatic level and safety of the intelligent tower crane are further improved.

In addition, in order to further perfect the intelligent and automatic level of the intelligent tower crane, the intelligent tower crane can further realize automatic detection of the intelligent tower crane on the material stacking state by configuring the following material stacking space image recognition analysis device for the intelligent tower crane so as to carry out quick and accurate lifting, thereby improving the intelligent and automatic level and safety of the intelligent tower crane, and the following description is made by combining an example.

In some embodiments, the device for identifying and analyzing a material stacking space image of an intelligent tower crane may include:

the intelligent auxiliary robot is arranged on the ground near the intelligent tower crane and moves near the intelligent tower crane to acquire the working scene image;

the global image acquisition module is used for acquiring a global image of the intelligent tower crane working scene shot by a global camera, wherein the global camera is downwards arranged on a crane arm of the intelligent tower crane;

the material identification module is used for identifying materials in each of the working scene images and the global images through an image identification algorithm and determining the size and stacking position of each material;

And the three-dimensional reconstruction module is used for carrying out three-dimensional reconstruction according to the fusion of the working scene image and the global image to obtain a three-dimensional simulation scene, and marking the materials in the three-dimensional simulation scene according to the sizes and stacking positions of the materials.

In some modification of the embodiment of the present application, the global image obtaining module includes:

the rotation shooting unit is used for controlling the crane boom to rotate and controlling the global camera to shoot a plurality of images in the rotation process;

and the image merging unit is used for merging the plurality of images to generate a global image of the intelligent tower crane working scene.

In some modified implementations of the embodiments of the present application, each pile of materials is stacked on a working site of the intelligent tower crane after being loaded by a corresponding carrier, where the carrier has image features different from the materials and surrounding environments;

the material identification module comprises:

the carrier identification unit is used for identifying the carrier in each working scene image and the global image through image feature matching according to the image features of the carrier;

and the space position determining unit is used for dividing the identified stacked materials according to the position of the carrier after identifying the stacked materials so as to determine the space position of each stack of the materials.

In some variations of the embodiments of the present application, the material identification module further includes:

the growing point determining unit is used for determining growing points in the bearing range of the carrier for each working scene image and each global image;

and the region growing unit is used for dividing and determining the materials in the working scene image and the global image respectively by using a region growing method by taking the growing points as references.

In some modified implementations of the embodiments of the present application, the working scene image and the global image are both acquired by a binocular camera or a TOF depth camera and carry depth of field information;

the three-dimensional reconstruction module comprises:

the global image processing unit is used for constructing a three-dimensional simulation model corresponding to the global image in three-dimensional simulation software according to depth information carried by the global image and the position of the global camera when the global image is shot;

and the scene image fusion unit is used for projecting each pixel point in the working scene image into the three-dimensional simulation model according to the depth of field information carried by the working scene image and the position of a camera of the intelligent auxiliary robot when the working scene image is shot, so as to obtain a three-dimensional simulation scene.

According to the material stacking space image recognition analysis device for the intelligent tower crane, the intelligent auxiliary robot scans the ground to obtain the working scene images, wherein the intelligent auxiliary robot is arranged on the ground near the intelligent tower crane and moves near the intelligent tower crane to collect the working scene images, the global images of the working scene of the intelligent tower crane, which are shot by the global camera, are obtained, the global camera is downwards arranged on the lifting arm of the intelligent tower crane, the materials are recognized in each working scene image and the global images through the image recognition algorithm, the size and stacking position of each material are determined, three-dimensional reconstruction is carried out after the working scene images and the global images are fused, a three-dimensional simulation scene is obtained, and the materials are marked in the three-dimensional simulation scene according to the size and stacking position of the materials. Therefore, the automatic detection of the material stacking state by the intelligent tower crane can be realized so as to carry out quick and accurate lifting, and the intelligent and automatic level and safety of the intelligent tower crane are further improved.

In addition, in order to further perfect the intelligent and automatic level of the intelligent tower crane, the intelligent tower crane can also realize automatic planning of a material transmission path by configuring the following material transmission optimization path planning system for the intelligent tower crane so as to avoid collision accidents and further improve the intelligent and automatic level and safety of the intelligent tower crane, and the following description is made with reference to examples.

In some embodiments, the material transfer optimization path planning system for an intelligent tower crane may include:

the scene plan view acquisition module is used for acquiring a plan view of the working scene of the intelligent tower crane; the plan view may be a global image of a working scene of the intelligent tower crane.

A gray level layer determining module, configured to determine a gray level layer based on the plan view, where the gray level layer includes a plurality of gray level regions, and a gray level value of each gray level region is proportional to a height of a collision hidden danger object on the plan view;

the gray level region selection module is used for selecting the gray level region according to the priority from low to high of the gray level value until the selected gray level region is communicated with the starting point and the end point of material hoisting, and marking the selected combined region of the gray level region as a safe transmission region;

and the transmission path determining module is used for determining the transmission path of the material according to the safe transmission area.

In some modification of the embodiment of the present application, the gray layer determining module includes:

a blank layer newly-built unit for newly-building a blank layer on the plane diagram;

the blank layer dividing unit is used for dividing the blank layer into a plurality of blank areas corresponding to the coverage areas of the collision hidden danger objects on the plan, wherein the collision hidden danger objects comprise buildings and other intelligent tower cranes around the intelligent tower crane;

And the gray value assignment unit is used for carrying out gray value assignment on each blank area according to the height of each collision hidden danger object to form a plurality of gray areas, wherein the gray areas form a gray image layer, and the gray value of the gray area corresponding to the collision hidden danger object with higher height is higher.

and the collision hidden danger object information determining module is used for determining the height and the coverage area of each collision hidden danger object below the crane arm in the coverage area of the intelligent tower crane based on the plan.

In some modification of the embodiments of the present application, the transmission path determining module includes:

and the transmission path determining unit is used for determining a material transmission path for connecting a material hoisting starting point and a material hoisting end point according to the side line of the safe transmission area, which is close to one side of the intelligent tower crane.

In some modified implementations of the embodiments of the present application, the material conveying path is located in the safe conveying area, and a distance between the material conveying path and the edge is greater than a safe distance, where the safe distance is determined according to a radius of a circumcircle of the material.

According to the material transmission optimization path planning system for the intelligent tower crane, a plan view of a working scene of the intelligent tower crane is obtained; determining a gray scale layer based on the plane graph, wherein the gray scale layer comprises a plurality of gray scale areas, and the gray scale value of each gray scale area is proportional to the height of the collision hidden danger object on the plane graph; selecting the gray scale region according to the priority from low to high of the gray scale value until the selected gray scale region is communicated with a starting point and an end point of material hoisting, and marking the selected combined region of the gray scale region as a safe transmission region; and determining the conveying path of the material according to the safe conveying area. Therefore, the automatic planning of the material transmission path can be realized, so that collision accidents are avoided, and the intelligent and automatic level and safety of the intelligent tower crane are improved.

In addition, in order to further perfect the level of intellectualization and automation of the intelligent tower crane, the intelligent tower crane can realize centralized management and remote monitoring of the operation information of the intelligent tower crane by configuring the cloud information system for the operation data of the intelligent tower crane, and the level of automation and intellectualization of the intelligent tower crane is improved, and the following description is made with reference to examples.

In some embodiments, the cloud information system for intelligent tower crane operation data may include: the controller and the LoRa wireless communication module are arranged on the intelligent tower crane, and the LoRa gateway, the cloud server and the remote monitoring terminal are arranged on the intelligent tower crane;

the controller is connected with the LoRa wireless communication modules, and the LoRa wireless communication modules of the plurality of intelligent towers are connected with the LoRa gateway based on a LoRa wireless communication protocol;

the LoRa gateway is arranged at construction sites of the intelligent tower cranes and is connected with the cloud server through a cellular network and/or a broadband network;

the controller is used for collecting operation data of the intelligent tower crane and sending the operation data to the cloud server for storage through the LoRa wireless communication module and the LoRa gateway;

the remote monitoring terminal is connected with the cloud server through a cellular network and/or a broadband network and is used for acquiring and displaying operation data of the intelligent tower crane from the cloud server.

Compared with the prior art, the cloud information system for intelligent tower crane operation data provided by the embodiment of the application is characterized in that a controller and a LoRa wireless communication module are arranged, and a LoRa gateway, a cloud server and a remote monitoring terminal are arranged; the controller is connected with the LoRa wireless communication modules, and the LoRa wireless communication modules of the plurality of intelligent towers are connected with the LoRa gateway based on a LoRa wireless communication protocol; the LoRa gateway is arranged at construction sites of the intelligent tower cranes and is connected with the cloud server through a cellular network and/or a broadband network; the controller is used for collecting operation data of the intelligent tower crane and sending the operation data to the cloud server for storage through the LoRa wireless communication module and the LoRa gateway; the remote monitoring terminal is connected with the cloud server through a cellular network and/or a broadband network and is used for acquiring and displaying operation data of the intelligent tower crane from the cloud server. Therefore, the centralized management and the remote monitoring of the operation information of the intelligent tower crane can be realized, and the automation and the intelligent level of the intelligent tower crane are improved.

In some modified implementations of the embodiments of the present application, the controller is specifically configured to write the running data into a JSON file and then send the running data to the cloud server;

the cloud server is used for analyzing the JSON file to obtain and store the operation data of the intelligent tower crane.

In some modified implementations of the embodiments of the present application, the working frequency band of the LoRa wireless communication module adopts a 470-510MHz frequency band.

In some modified implementations of the embodiments of the present application, the operation data includes idle state data, luffing motion data, slewing motion data, lifting motion data of the intelligent tower crane, and a type and weight of the material to be lifted each time.

In some modified implementations of the embodiments of the present application, the cloud server is further configured to count workload information in a preset time period of each intelligent tower crane according to the operation data, and send the workload information to the remote monitoring terminal for display, where the workload information includes a type and a weight of a material that has been lifted in the preset time period.

In some modified implementations of the embodiments of the present application, the intelligent tower crane is provided with a camera group connected to the controller, where the camera group is configured to collect a material image group of a material hoisted by the intelligent tower crane and send the material image group to the controller, where the camera group includes a plurality of cameras with different shooting angles, and the material image group includes a material image collected by each of the cameras for the material;

In some modified implementations of the embodiments of the present application, the guide pulley mechanism of the intelligent tower crane is provided with a force sensor;

In some modified implementations of the embodiments of the present application, the remote monitoring terminal is further configured to construct a real-time simulation scene in building information model BIM software according to attribute information and the operation data of the intelligent tower crane, and display the simulation scene through a display, so that a user can know the operation information of the intelligent tower crane by watching the simulation scene, where the attribute information includes position information, altitude information and specification model information of the intelligent tower crane.

It is noted that the flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present application. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.

It will be clear to those skilled in the art that, for convenience and brevity of description, specific working procedures of the above-described systems, apparatuses and units may refer to corresponding procedures in the foregoing method embodiments, and are not repeated herein.

In the several embodiments provided in this application, it should be understood that the disclosed apparatus and method may be implemented in other ways. The above-described apparatus embodiments are merely illustrative, for example, the division of the units is merely a logical function division, and there may be other manners of division in actual implementation, and for example, multiple units or components may be combined or integrated into another system, or some features may be omitted, or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed with each other may be through some communication interface, device or unit indirect coupling or communication connection, which may be in electrical, mechanical or other form.

The units described as separate units may or may not be physically separate, and units shown as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

In addition, each functional unit in each embodiment of the present application may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit.

The functions, if implemented in the form of software functional units and sold or used as a stand-alone product, may be stored in a computer-readable storage medium. Based on such a theory

The solution of the present application may be embodied in the form of a software product stored on a storage medium, including instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of a method according to various embodiments of the present application. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a random access Memory (RAM, random Access Memory), a magnetic disk, or an optical disk, or other various media capable of storing program codes.

Finally, it should be noted that: the above embodiments are only for illustrating the technical solution of the present application, and not for limiting the same; although the present application has been described in detail with reference to the foregoing embodiments, it should be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some or all of the technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit of the embodiments, and are intended to be included within the scope of the claims and description.

Claims

1. The intelligent tower crane cluster cooperative control method for the task temporal model is characterized by comprising the following steps of:

controlling each intelligent tower crane to execute a corresponding task to be executed according to the task queue;

the task conflict information comprises a task conflict number;

judging whether the action area corresponding to each intelligent tower crane has an intersecting area or not and corresponding to other surrounding intelligent tower cranes in sequence, if the action area does not have the intersecting area, determining that the task conflict number of the intelligent tower crane is zero, if the action area does not have the intersecting area, judging whether the task execution time of the intelligent tower crane and other intelligent tower cranes with the intersecting area exists or not, if the action area does not have the intersecting time, determining that the task conflict number of the intelligent tower crane is zero, and if the action area does not have the intersecting time, recording the task conflict number of the intelligent tower crane as the task conflict number;

The sorting of the tasks to be executed of each intelligent tower crane according to the task conflict information comprises the following steps:

for adjacent tasks to be executed, which have the same crossing time after secondary sequencing, sequencing for three times according to the sequence from short to long of the task execution time of the tasks to be executed, so as to obtain a task queue;

and controlling each intelligent tower crane to execute the corresponding task to be executed according to the task queue, wherein the task queue comprises the following steps:

2. The intelligent tower crane cluster cooperative control method for a task temporal model according to claim 1, further comprising:

3. An intelligent tower crane cluster cooperative control device for a task temporal model, which is characterized by comprising:

The task execution module is used for controlling each intelligent tower crane to execute the corresponding task to be executed according to the task queue;

the task conflict information comprises a task conflict number;

the conflict information determining module includes:

the conflict information determining unit is used for sequentially determining whether the action area corresponding to each intelligent tower crane has an intersection area with action areas corresponding to other surrounding intelligent tower cranes, if the action area does not have the intersection area, determining that the task conflict number of the intelligent tower crane is zero, if the action area does not have the intersection area, determining whether the task execution time of the intelligent tower crane and the task execution time of the other intelligent tower cranes with the intersection area exist or not, if the action area does not have the intersection time, determining that the task conflict number of the intelligent tower crane is zero, and if the action area does not have the intersection time, recording the task conflict number of the intelligent tower crane as the task conflict number;

the service ordering module comprises:

The third sequencing unit is used for sequencing the tasks to be executed, which are adjacent to each other and have the same crossing time after the second sequencing, for three times according to the sequence from short to long of the task execution time of the tasks to be executed, so as to obtain a task queue;

the task execution module includes:

4. An intelligent tower crane, characterized in that the intelligent tower crane is provided with the intelligent tower crane cluster cooperative control device for the task temporal model according to claim 3.