CN117726882A - Tower crane object identification method, system and electronic equipment - Google Patents
Tower crane object identification method, system and electronic equipment Download PDFInfo
- Publication number
- CN117726882A CN117726882A CN202410172331.7A CN202410172331A CN117726882A CN 117726882 A CN117726882 A CN 117726882A CN 202410172331 A CN202410172331 A CN 202410172331A CN 117726882 A CN117726882 A CN 117726882A
- Authority
- CN
- China
- Prior art keywords
- area
- hook
- identified
- image
- lifting hook
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 66
- 238000001514 detection method Methods 0.000 claims abstract description 66
- 238000000605 extraction Methods 0.000 claims abstract description 49
- 238000013145 classification model Methods 0.000 claims abstract description 26
- 238000012549 training Methods 0.000 claims description 18
- 239000013598 vector Substances 0.000 claims description 11
- 230000008569 process Effects 0.000 claims description 10
- 238000004590 computer program Methods 0.000 claims description 7
- 238000010586 diagram Methods 0.000 description 6
- 238000005516 engineering process Methods 0.000 description 5
- 230000008901 benefit Effects 0.000 description 4
- 238000004364 calculation method Methods 0.000 description 4
- 230000006870 function Effects 0.000 description 3
- 239000004566 building material Substances 0.000 description 2
- 238000010276 construction Methods 0.000 description 2
- 238000004519 manufacturing process Methods 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 238000003491 array Methods 0.000 description 1
- 238000004422 calculation algorithm Methods 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 239000000284 extract Substances 0.000 description 1
- 230000002452 interceptive effect Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 238000011410 subtraction method Methods 0.000 description 1
Landscapes
- Image Analysis (AREA)
Abstract
The application relates to a tower crane object identification method, a system and electronic equipment. The method for identifying the tower crane object comprises the following steps: acquiring hook image data of continuous frames, identifying the hook image data, and judging whether the hook is in a moving state or not; responding to the lifting hook in a moving state, acquiring an image to be detected containing the lifting hook and a lifting object, and detecting the image to be detected through a main body detection model to obtain a lifting hook area and a lifting object area, wherein the main body detection model is a classification model; determining a region to be identified according to the lifting hook region and the lifting object region, and extracting the features to be identified of the region to be identified through a feature extraction network; calculating the feature similarity of the feature to be identified and the image feature in the comparison library, and determining an identification result corresponding to the feature to be identified according to the feature similarity.
Description
Technical Field
The application relates to the field of intelligent recognition, in particular to a method, a system and electronic equipment for recognizing a tower crane object.
Background
The intelligent recognition of the type of the tower crane object in the construction industry is of great significance in preventing the object from collision, estimating the construction progress and other tasks.
In the prior art, mainly, a laser radar is used as a sensor to detect the type of an object based on the generated 3D information, or an RGB camera is used as a sensor, and a deep learning detection network is used to detect the position and the type of the suspended object in a picture. The mode based on the laser radar has large calculated amount and higher cost; based on the mode of RGB camera, because of the variety of hanging object types and shapes, the requirement on the algorithm model is higher, the complexity of the model is higher, and the untrained hanging object types can not be identified.
When the prior art is applied, the problems of large calculated amount and low recognition accuracy exist.
Disclosure of Invention
The embodiment of the application provides a method, a system and electronic equipment for identifying a tower crane object, which are used for at least solving the problems of large calculated amount and low identification accuracy in the related technology.
In a first aspect, an embodiment of the present application provides a method for identifying an object of a tower crane, including:
acquiring hook image data of continuous frames, identifying the hook image data, and judging whether the hook is in a moving state or not;
responding to the lifting hook in a moving state, acquiring an image to be detected comprising the lifting hook and a lifting object, and detecting the image to be detected through a main body detection model to obtain a target lifting hook area and a lifting object area, wherein the main body detection model is a classification model;
determining a region to be identified according to the target lifting hook region and the lifting object region, and extracting the features to be identified of the region to be identified through a feature extraction network;
calculating the feature similarity of the feature to be identified and the image feature in the comparison library, and determining an identification result corresponding to the feature to be identified according to the feature similarity.
In an embodiment, the detecting the image to be detected by the main body detection model to obtain a target hook area and a suspended object area includes:
detecting the image to be detected to obtain a lifting hook area and a hanging object area in the image to be detected, wherein the confidence coefficient of the lifting hook area and the hanging object area is larger than or equal to a first threshold value;
and determining the lifting hook area with the highest confidence from the lifting hook areas as a target lifting hook area.
In an embodiment, the determining the area to be identified according to the target hook area and the suspended object area includes:
acquiring relative position parameters from each object hanging area to the target lifting hook area;
and determining an area to be identified in the object hanging area according to the relative position parameters and the target lifting hook length.
In an embodiment, the relative position parameters include: a first distance between a center point of the sling area and a center point of the target hook area,
determining an area to be identified in the object hanging area according to the relative position parameter and the target hook length, including:
responding to the proportion of the first distance to the target lifting hook length within a preset threshold range, and taking the corresponding lifting object area as an area to be identified; or (b)
The relative position parameters include: a second distance between a center point of the hanger area and a midpoint of a bottom edge of the target hook area,
determining an area to be identified in the object hanging area according to the relative position parameter and the target hook length, including:
and responding to the proportion of the second distance to the target lifting hook length within a preset threshold range, and taking the corresponding lifting object area as the area to be identified.
In an embodiment, the feature extraction network is configured to obtain by:
acquiring a classification model with training completed, wherein the classification model comprises a feature extraction part and a classification part, the feature extraction part is used for acquiring a feature vector of input data, and the classification part is used for outputting a classification result based on the feature vector;
and taking a characteristic extraction part of the classification model as the characteristic extraction network.
In an embodiment, the comparison library is configured to be obtained by:
acquiring first suspended object images of different types, extracting features of the first suspended object images to obtain image features, and storing the first suspended object images, the image features and the corresponding suspended object types into the comparison library; and/or
Receiving a second suspended object image uploaded by a user terminal, and detecting the second suspended object image through the main body detection model to obtain a suspended object area;
determining a target suspended object area and a category corresponding to the suspended object in the target suspended object area according to a user instruction;
and carrying out feature extraction on the target suspended object region to obtain suspended object features, and storing the second suspended object image, the suspended object features and the categories into the comparison library.
In an embodiment, the method further comprises: and in the process of one-time lifting, carrying out multiple times of tower crane object recognition, and responding to the recognition times being greater than or equal to a second threshold value, wherein the result with the largest occurrence times is taken as a final recognition result in the results obtained by multiple times of recognition.
In a second aspect, an embodiment of the present application provides a tower crane object identifying system, including:
the mobile detection module: the method comprises the steps of acquiring hook image data of continuous frames, identifying the hook image data, and judging whether a hook is in a moving state or not;
and a detection module: responding to the lifting hook in a moving state, acquiring an image to be detected comprising the lifting hook and a lifting object, and detecting the image to be detected through a main body detection model to obtain a target lifting hook area and a lifting object area, wherein the main body detection model is a classification model;
and the feature extraction module is used for: the method comprises the steps of determining a region to be identified according to a target lifting hook region and a lifting object region, and extracting the features to be identified of the region to be identified through a feature extraction network;
and an identification module: and the method is used for calculating the feature similarity of the feature to be identified and the image feature in the comparison library, and determining an identification result corresponding to the feature to be identified according to the feature similarity.
In a third aspect, an embodiment of the present application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and capable of running on the processor, where the processor implements the method for identifying an object of an tower crane according to the first aspect when executing the computer program.
In a fourth aspect, an embodiment of the present application provides a computer readable storage medium, on which a computer program is stored, where the program when executed by a processor implements the tower crane object identifying method according to the first aspect.
The tower crane object identification method, the system and the electronic equipment have at least the following technical effects.
The state of lifting hook is judged to this application using removal detection technique, when the lifting hook is in the mobile state, acquires the image of waiting to detect that contains lifting hook and hanging the thing, treats the image of detecting through main part detection model and detects the position of lifting hook and hanging the thing to do not distinguish specific type of hanging the thing. According to the method and the device, when the lifting hook is in a moving state, main body detection is only carried out, so that the calculated amount in the identification process is reduced, and the load of a processor is reduced. Meanwhile, the main body detection model only distinguishes two categories of hanging objects and lifting hooks, so that the requirement of data annotation and the difficulty of model training are reduced. According to the method, the to-be-identified characteristics of the to-be-identified area are extracted through the characteristic extraction network, the to-be-identified characteristics are compared with the image characteristics in the comparison library, and the suspended object category is determined according to the characteristic similarity. The specific result is not output through the model classification model, but the suspended object type is obtained through searching similar characteristics from the comparison library. In this way, even if the suspended object type is not trained by the feature extraction network, the suspended object type can be identified by only adding the suspended object image features into the comparison library, so that the accuracy, convenience and generalization of suspended object identification are effectively improved.
The details of one or more embodiments of the application are set forth in the accompanying drawings and the description below to provide a more thorough understanding of the other features, objects, and advantages of the application.
Drawings
The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this application, illustrate embodiments of the application and together with the description serve to explain the application and do not constitute an undue limitation to the application. In the drawings:
FIG. 1 is a flow chart illustrating a method of identifying an tower crane object according to an exemplary embodiment;
FIG. 2 is a block diagram of a feature extraction network, shown in accordance with an exemplary embodiment;
FIG. 3 is a block diagram illustrating a tower crane identification system according to an exemplary embodiment;
fig. 4 is a schematic structural diagram of an electronic device according to an embodiment of the present application.
Detailed Description
In order to make the objects, technical solutions and advantages of the present application more apparent, the present application is described and illustrated below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the present application. All other embodiments, which can be made by one of ordinary skill in the art without undue burden on the person of ordinary skill in the art based on the embodiments provided herein, are intended to be within the scope of the present application.
It is apparent that the drawings in the following description are only some examples or embodiments of the present application, and it is possible for those of ordinary skill in the art to apply the present application to other similar situations according to these drawings without inventive effort. Moreover, it should be appreciated that while such a development effort might be complex and lengthy, it would nevertheless be a routine undertaking of design, fabrication, or manufacture for those of ordinary skill having the benefit of this disclosure, and thus should not be construed as having the benefit of this disclosure.
Reference in the specification to "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment may be included in at least one embodiment of the application. The appearances of such phrases in various places in the specification are not necessarily all referring to the same embodiment, nor are separate or alternative embodiments mutually exclusive of other embodiments. It is to be expressly and implicitly understood by those of ordinary skill in the art that the embodiments described herein can be combined with other embodiments without conflict.
Unless defined otherwise, technical or scientific terms used herein should be given the ordinary meaning as understood by one of ordinary skill in the art to which this application belongs. Reference to "a," "an," "the," and similar terms herein do not denote a limitation of quantity, but rather denote the singular or plural. The terms "comprising," "including," "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, article, or apparatus that comprises a list of steps or modules (elements) is not limited to only those steps or elements but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus. The terms "connected," "coupled," and the like in this application are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. The term "plurality" as used herein refers to two or more. "and/or" describes an association relationship of an association object, meaning that there may be three relationships, e.g., "a and/or B" may mean: a exists alone, A and B exist together, and B exists alone. The character "/" generally indicates that the context-dependent object is an "or" relationship. The terms "first," "second," "third," and the like, as used herein, are merely distinguishing between similar objects and not representing a particular ordering of objects.
In a first aspect, an embodiment of the present application provides a method for identifying an object of a tower crane, and fig. 1 is a flowchart of a method for identifying an object of a tower crane, as shown in fig. 1, where the method includes:
step S101, hook image data of continuous frames are acquired, the hook image data are identified, and whether the hook is in a moving state or not is judged.
Alternatively, successive frames of hook image data are acquired by an image acquisition assembly, such as a camera mounted below the tower crane trolley to capture hooks vertically. The state of the hook is obtained through a motion detection technology, such as a neighboring frame difference method, a background subtraction method, an optical flow method and the like. In this way, it is determined whether the hook is in a moving state, and when the hook is in a moving state, the subsequent steps are performed. And a large amount of calculation caused by real-time identification or uninterrupted identification is avoided, the calculation amount in the identification process is reduced, and the load of a processor is reduced.
Step S102, in response to the lifting hook being in a moving state, obtaining an image to be detected comprising the lifting hook and the lifting object, and detecting the image to be detected through a main body detection model to obtain a target lifting hook area and a lifting object area, wherein the main body detection model is a classification model.
Optionally, when the lifting hook is in a moving state, the image acquisition component acquires an image to be detected, and the image to be detected is detected through the main body detection model. The target hook region may be obtained by setting a higher hook detection threshold or selecting the target hook region from a plurality of hook regions detected. Illustratively, the subject detection model is selected from the group consisting of the YOLO-v7 and YOLO-v5 detection models. The training set of subject detection models includes building material pictures in web pages acquired using crawler technology, and object pictures taken in actual scenes. When the training set is marked, only two categories of the lifting hook and the lifting object are distinguished, the specific category of the lifting object is not identified, and the data marking requirement and the model training difficulty are reduced. And training the monitoring model by using multi-source data to improve generalization of the detection model and realize detection of new suspended object types.
In one example, step S102 includes:
detecting an image to be detected to obtain a lifting hook area and a hanging object area in the image to be detected, wherein the confidence coefficient of the lifting hook area and the hanging object area is larger than or equal to a first threshold value. And determining the lifting hook area with the highest confidence from the lifting hook areas as a target lifting hook area.
Optionally, when the image to be detected is detected, thresholds of hook detection and suspended object detection are set so as to obtain a hook area and a suspended object area with confidence coefficient greater than or equal to a first threshold. The confidence coefficient is between 0 and 1, the first threshold is set to be a lower detection threshold, so that a suspended object area in an image to be detected is detected as far as possible, suspended objects except a training sample are detected, and specific suspended object types are identified in the follow-up steps. Meanwhile, the lifting hook area with the highest confidence coefficient in the lifting hook area is used as a target lifting hook area, so that the lifting hook area can be accurately detected. In this way, the recognition of the suspended objects outside the training sample is realized, the condition of missing error detection and detection when the suspended objects outside the training sample are encountered is avoided, and the accuracy of the suspended object recognition is improved.
Step S103, determining a region to be identified according to the target lifting hook region and the lifting object region, and extracting the features to be identified of the region to be identified through a feature extraction network.
Optionally, the detection result in step S102 generally includes a hook area and a plurality of hanging object areas, and the area to be identified is determined from the plurality of hanging object areas according to the positions or the relative positions of the hook area and the hanging object areas, so as to screen out the interfering object areas, and improve the accuracy of the subsequent hanging object identification.
In one example, step S103 includes: acquiring relative position parameters from each object hanging area to a target hanging hook area; and determining a region to be identified in the suspended object region according to the relative position parameters and the target hook length.
Optionally, the area to be identified is determined according to the relative position of each suspended-object area and the target hook area and the proportion of the hook length. Wherein the relative positional parameters of the sling area and the target hook area include, but are not limited to: the distance from the center point of the object hanging area to the center point of the target lifting hook area; the distance from the center point of the object hanging area to the midpoint of the bottom edge of the target hanging hook area; the distance from the center point of the lifting hook region to the midpoint of the top edge of the target lifting hook region. In this way, the interference object area is filtered, and the accuracy of hanging object identification is improved.
As an alternative, the relative position parameters include: a first distance between a center point of the sling area and a center point of the target hook area. In step S103, determining a region to be identified in the suspended object region according to the relative position parameter and the target hook length, including: and responding to the proportion of the first distance to the target lifting hook length within a preset threshold range, and taking the corresponding lifting object area as the area to be identified.
As another alternative, the relative position parameters include: a second distance between the center point of the sling area and the midpoint of the bottom edge of the target hook area. In step S103, determining a region to be identified in the suspended object region according to the relative position parameter and the target hook length, including: and responding to the proportion of the second distance to the target lifting hook length within a preset threshold range, and taking the corresponding lifting object area as the area to be identified.
By adopting the two modes, the threshold range of the ratio of the relative distance between the suspended object area and the target lifting hook area to the length of the lifting hook is set, the interference of building materials on the ground is filtered, and the accuracy of suspended object identification is improved.
In one example, the feature extraction network in step S103 is configured to obtain by:
a classification model after training is obtained, the classification model including a feature extraction section for obtaining feature vectors of input data, and a classification section for outputting classification results based on the feature vectors. The feature extraction part of the classification model is taken as a feature extraction network.
Alternatively, fig. 2 is a block diagram of a feature extraction network shown according to an exemplary embodiment, and as shown in fig. 2, the trained classification model is divided into a classification section and a feature extraction section, and the feature extraction section is all layers except the classification layer. When the method is used, the classifying layer is removed to obtain a feature extraction network, the feature layer in front of the classifying layer is used as an output layer, the output of the feature layer is a feature vector with fixed dimension, the output feature vector is normalized, and the dimension of the feature vector is generally 128, 256, 512 and the like. Alternatively, the classification model includes one of ResNet50, resNet101, and Mobilene models, but is not limited thereto. The training set of classification models includes: articles such as commodities, vehicles, buildings, flowers, furniture and the like, and building hanging objects and the like. The loss function at training is a weighted combination of cross entropy loss and trippletangullar marginless.
In this way, the classification model is trained by acquiring various types of data, the distinguishing degree of the features learned by the feature extraction network and the generalization of the feature extraction network are improved, so that the classification of suspended objects in a training set, the identification of new suspended objects not in the training set and the accuracy and generalization of a suspended object identification method are realized.
With continued reference to fig. 1, step S104 is performed after step S103, specifically as follows:
step S104, calculating the feature similarity between the feature to be identified and the image feature in the comparison library, and determining the identification result corresponding to the feature to be identified according to the feature similarity.
Optionally, calculating the feature similarity between the feature to be identified and the image feature in the comparison library, and taking the category corresponding to the image feature with the maximum feature similarity and larger than the set threshold value as the identification result. The feature similarity calculation method can select cosine feature similarity, euclidean distance and other similarity calculation methods.
In this way, the classification model does not directly output the recognition result, but recognizes the type of the suspended object by performing similarity comparison after feature extraction. Even if the suspended object type is not trained, the suspended object image of the type can be identified after the suspended object type image is stored in the comparison library, so that the accuracy and generalization of the suspended object identification method are effectively improved.
In one example, the comparison library is configured to be obtained by: and acquiring different types of first suspended object images, extracting the characteristics of the first suspended object images to obtain image characteristics, and storing the first suspended object images, the image characteristics and the corresponding suspended object types into the comparison library. In this way, common object images and features are added to the comparison library for feature comparison during feature recognition.
In one example, the comparison library is configured to be obtained by:
and receiving a second suspended object image uploaded by the user terminal, and detecting the second suspended object image through the main body detection model to obtain a suspended object area. Optionally, the user and the implementation personnel can upload a new type of suspended object image which is not in the comparison library, detect the suspended object image through the main body detection model, and sort all the detected suspended objects according to the order of confidence from high to low.
And determining a target suspended object area and a category corresponding to the suspended object in the target suspended object area according to the user instruction. Optionally, confirming the target suspended object area which needs to be added into the comparison warehouse from all the detected suspended object areas by a user, marking the type of suspended objects in the target suspended object area,
and carrying out feature extraction on the target suspended object region to obtain suspended object features, and storing the second suspended object image, the suspended object features and the categories into a comparison warehouse.
Optionally, the comparison library consists of common suspended objects and new suspended objects uploaded by users, and for suspended object types which are not trained by the feature extraction network, the new suspended object types can be identified after the new suspended objects are stored in the comparison library, so that generalization, accuracy and flexibility of suspended object identification are improved.
In one example, the method further comprises: and in the process of one-time lifting, carrying out multiple times of tower crane object recognition, and responding to the recognition times being greater than or equal to a second threshold value, wherein the result with the largest occurrence times is taken as a final recognition result in the results obtained by multiple times of recognition.
Optionally, in the moving process of the suspended object, the suspended object is continuously identified, and after the identification times reach a second threshold value, the category with the largest occurrence times of the identification result is taken as the final suspended object identification result. In this way, the accuracy of the suspended object identification is further improved, and false detection is avoided.
To sum up, the state of the lifting hook is judged by using the mobile detection technology, when the lifting hook is in a mobile state, an image to be detected containing the lifting hook and the suspended object is obtained, the image to be detected is detected by the main body detection model, and the positions of the lifting hook and the suspended object are detected without distinguishing the specific suspended object types. According to the method and the device, when the lifting hook is in a moving state, main body detection is only carried out, so that the calculated amount in the identification process is reduced, and the load of a processor is reduced. The main body detection model only distinguishes two categories of hanging objects and lifting hooks, so that the data marking requirement and the model training difficulty are reduced, and a proper detection threshold is set during main body detection, so that the detection of missing errors is avoided. According to the method, the to-be-identified characteristics of the to-be-identified area are extracted through the characteristic extraction network, the to-be-identified characteristics are compared with the image characteristics in the comparison library, and the suspended object category is determined according to the characteristic similarity. The specific result is not output through the model classification model, but the suspended object type is obtained through searching similar characteristics from the comparison library. In this way, even if the suspended object type is not trained by the feature extraction network, the suspended object type can be identified by only adding the suspended object image features into the comparison library, so that the accuracy, convenience and generalization of suspended object identification are effectively improved. The method and the device can also continuously identify the suspended objects, and the category with the largest occurrence number of the identification result is used as the final suspended object identification result, so that the accuracy of suspended object detection is further improved.
In a second aspect, an embodiment of the present application provides a tower crane object identifying system, fig. 3 is a block diagram of a tower crane object identifying system according to an exemplary embodiment, and as shown in fig. 3, the system includes:
the motion detection module 100: the method comprises the steps of acquiring hook image data of continuous frames, identifying the hook image data, and judging whether a hook is in a moving state or not;
detection module 200: responding to the state that the lifting hook is in a moving state, acquiring an image to be detected containing the lifting hook and the lifting object, and detecting the image to be detected through a main body detection model to obtain a target lifting hook area and a lifting object area, wherein the main body detection model is a two-class model;
feature extraction module 300: the method comprises the steps of determining a region to be identified according to a target lifting hook region and the lifting object region, and extracting the features to be identified of the region to be identified through a feature extraction network;
the identification module 400: and the method is used for calculating the feature similarity of the features to be identified and the image features in the comparison library, and determining the identification result corresponding to the features to be identified according to the feature similarity.
In one example, the detection module 200 includes: detecting an image to be detected to obtain a lifting hook area and a hanging object area in the image to be detected, wherein the confidence coefficient of the lifting hook area and the hanging object area is larger than or equal to a first threshold value; and determining the lifting hook area with the highest confidence from the lifting hook areas as a target lifting hook area.
In one example, the feature extraction module 300 includes: acquiring relative position parameters from each object hanging area to a target hanging hook area; and determining a region to be identified in the suspended object region according to the relative position parameters and the target hook length.
In one example, the feature extraction module 300 includes: the relative position parameters include: a first distance between the center point of the sling area and the center point of the target hook area,
determining an area to be identified in the object hanging area according to the relative position parameters and the target hook length, wherein the method comprises the following steps:
responding to the proportion of the first distance to the length of the target lifting hook within a preset threshold range, and taking the corresponding lifting object area as an area to be identified; or (b)
The relative position parameters include: a second distance between the center point of the sling area and the midpoint of the bottom edge of the target hook area,
root relative position parameter and target lifting hook length, confirm to wait to discern the area in hanging the thing area, include:
and responding to the proportion of the second distance to the target lifting hook length within a preset threshold range, and taking the corresponding lifting object area as the area to be identified.
In one example, the feature extraction network is configured to obtain by: a classification model after training is obtained, the classification model including a feature extraction section for obtaining feature vectors of input data, and a classification section for outputting classification results based on the feature vectors.
The feature extraction part of the classification model is taken as a feature extraction network.
In one example, the comparison library is configured to be obtained by:
acquiring different types of first suspended object images, extracting features of the first suspended object images to obtain image features, and storing the first suspended object images, the image features and corresponding suspended object types into a comparison library; and/or
And receiving a second suspended object image uploaded by the user terminal, and detecting the second suspended object image through the main body detection model to obtain a suspended object area.
And determining a target suspended object area and a category corresponding to the suspended object in the target suspended object area according to the user instruction.
And carrying out feature extraction on the target suspended object region to obtain suspended object features, and storing the second suspended object image, the suspended object features and the categories into a comparison warehouse.
In one example, the method further comprises: and carrying out multiple times of tower crane object recognition, and responding to the recognition times being greater than or equal to a second threshold value, wherein the result with the largest occurrence times is taken as a final recognition result in the results obtained by multiple times of recognition.
In summary, the present application determines the state of the hook by using the motion detection technology through the motion detection module 100, when the hook is in the motion state, the detection module 200 obtains the image to be detected including the hook and the suspended object, and the main body detection model detects the image to be detected, so as to detect the positions of the hook and the suspended object, and not distinguish the specific suspended object types. According to the method and the device, when the lifting hook is in a moving state, main body detection is only carried out, so that the calculated amount in the identification process is reduced, and the load of a processor is reduced. The main body detection model only distinguishes two categories of hanging objects and lifting hooks, so that the data marking requirement and the model training difficulty are reduced, and a proper detection threshold is set during main body detection, so that the detection of missing errors is avoided. The method extracts the to-be-identified characteristics of the to-be-identified area through the characteristic extraction module 300, compares the to-be-identified characteristics with the image characteristics in the comparison library through the identification module 400, and determines the hanging object category according to the characteristic similarity. The specific result is not output through the model classification model, but the suspended object type is obtained through searching similar characteristics from the comparison library. In this way, even if the suspended object type is not trained by the feature extraction network, the suspended object type can be identified by only adding the suspended object image features into the comparison library, so that the accuracy, convenience and generalization of suspended object identification are effectively improved. The method and the device can also continuously identify the suspended objects, and the category with the largest occurrence number of the identification result is used as the final suspended object identification result, so that the accuracy of suspended object detection is further improved.
In a third aspect, an embodiment of the present application provides an electronic device, and fig. 4 is a schematic structural diagram of the electronic device provided in the embodiment of the present application. The electronic device includes a memory, a processor, and a computer program stored on the memory and capable of running on the processor, where the processor implements the method for identifying an object of an tower crane provided in the first aspect when executing the program, and the electronic device 60 shown in fig. 4 is only an example, and should not impose any limitation on the functions and the application scope of the embodiments of the present application.
The electronic device 60 may be in the form of a general purpose computing device, which may be a server device, for example. Components of electronic device 60 may include, but are not limited to: the at least one processor 61, the at least one memory 62, a bus 63 connecting the different system components, including the memory 62 and the processor 61.
The bus 63 includes a data bus, an address bus, and a control bus.
Memory 62 may include volatile memory such as Random Access Memory (RAM) 621 and/or cache memory 622, and may further include Read Only Memory (ROM) 623.
Memory 62 may also include a program/utility 625 having a set (at least one) of program modules 624, such program modules 624 including, but not limited to: an operating system, one or more application programs, other program modules, and program data, each or some combination of which may include an implementation of a network environment.
The processor 61 executes various functional applications and data processing, such as the tower crane object identification method of the first aspect of the present application, by running a computer program stored in the memory 62.
The electronic device 60 may also communicate with one or more external devices 64 (e.g., keyboard, pointing device, etc.). Such communication may occur through an input/output (I/O) interface 65. Also, the model-generating device 60 may also communicate with one or more networks, such as a Local Area Network (LAN), a Wide Area Network (WAN), and/or a public network, such as the internet, through a network adapter 66. As shown, the network adapter 66 communicates with other modules of the model-generating device 60 via the bus 63. It should be appreciated that although not shown in the figures, other hardware and/or software modules may be used in connection with the model-generating device 60, including, but not limited to: microcode, device drivers, redundant processors, external disk drive arrays, RAID (disk array) systems, tape drives, data backup storage systems, and the like.
It should be noted that although several units/modules or sub-units/modules of an electronic device are mentioned in the above detailed description, such a division is merely exemplary and not mandatory. Indeed, the features and functionality of two or more units/modules described above may be embodied in one unit/module in accordance with embodiments of the present invention. Conversely, the features and functions of one unit/module described above may be further divided into ones that are embodied by a plurality of units/modules.
In a fourth aspect, embodiments of the present application provide a computer readable storage medium having a program stored thereon, where the program, when executed by a processor, implements the tower crane object identification method provided in the first aspect.
More specifically, among others, readable storage media may be employed including, but not limited to: portable disk, hard disk, random access memory, read only memory, erasable programmable read only memory, optical storage device, magnetic storage device, or any suitable combination of the foregoing.
In a possible embodiment, the invention may also be implemented in the form of a program product comprising program code for causing a terminal device to carry out the steps of implementing the method for identifying an object of an tower crane provided in the first aspect, when said program product is run on the terminal device.
Wherein the program code for carrying out the invention may be written in any combination of one or more programming languages, which program code may execute entirely on the user device, partly on the user device, as a stand-alone software package, partly on the user device and partly on the remote device or entirely on the remote device.
The technical features of the above-described embodiments may be arbitrarily combined, and all possible combinations of the technical features in the above-described embodiments are not described for brevity of description, however, as long as there is no contradiction between the combinations of the technical features, they should be considered as the scope of the description.
The above examples merely represent a few embodiments of the present application, which are described in more detail and are not to be construed as limiting the scope of the invention. It should be noted that it would be apparent to those skilled in the art that various modifications and improvements could be made without departing from the spirit of the present application, which would be within the scope of the present application. Accordingly, the scope of protection of the present application is to be determined by the claims appended hereto.
Claims (10)
1. The method for identifying the tower crane object is characterized by comprising the following steps of:
acquiring hook image data of continuous frames, identifying the hook image data, and judging whether the hook is in a moving state or not;
responding to the lifting hook in a moving state, acquiring an image to be detected comprising the lifting hook and a lifting object, and detecting the image to be detected through a main body detection model to obtain a target lifting hook area and a lifting object area, wherein the main body detection model is a classification model;
determining a region to be identified according to the target lifting hook region and the lifting object region, and extracting the features to be identified of the region to be identified through a feature extraction network;
calculating the feature similarity of the feature to be identified and the image feature in the comparison library, and determining an identification result corresponding to the feature to be identified according to the feature similarity.
2. The method for identifying an object of a tower crane according to claim 1, wherein the detecting the image to be detected by the main body detection model to obtain a target hook area and an object hanging area comprises:
detecting the image to be detected to obtain a lifting hook area and a hanging object area in the image to be detected, wherein the confidence coefficient of the lifting hook area and the hanging object area is larger than or equal to a first threshold value;
and determining the lifting hook area with the highest confidence from the lifting hook areas as a target lifting hook area.
3. The method for identifying an object of tower crane according to claim 2, wherein the determining an area to be identified according to the target hook area and the object hanging area comprises:
acquiring relative position parameters from each object hanging area to the target lifting hook area;
and determining an area to be identified in the object hanging area according to the relative position parameters and the target lifting hook length.
4. A method of identifying an object of tower crane according to claim 3, wherein the relative position parameters include: the first distance between the center point of the object hanging area and the center point of the target lifting hook area, the determining the area to be identified in the object hanging area according to the relative position parameter and the target lifting hook length includes:
responding to the proportion of the first distance to the target lifting hook length within a preset threshold range, and taking the corresponding lifting object area as an area to be identified; or (b)
The relative position parameters include: the second distance between the center point of the object hanging area and the midpoint of the bottom edge of the target lifting hook area, wherein the determining the area to be identified in the object hanging area according to the relative position parameter and the target lifting hook length comprises the following steps:
and responding to the proportion of the second distance to the target lifting hook length within a preset threshold range, and taking the corresponding lifting object area as the area to be identified.
5. The tower crane identification method according to claim 1, wherein the feature extraction network is configured to obtain by:
acquiring a classification model with training completed, wherein the classification model comprises a feature extraction part and a classification part, the feature extraction part is used for acquiring a feature vector of input data, and the classification part is used for outputting a classification result based on the feature vector;
and taking a characteristic extraction part of the classification model as the characteristic extraction network.
6. The tower crane identification method of claim 1, wherein the comparison library is configured to be obtained by:
acquiring first suspended object images of different types, extracting features of the first suspended object images to obtain image features, and storing the first suspended object images, the image features and the corresponding suspended object types into the comparison library; and/or
Receiving a second suspended object image uploaded by a user terminal, and detecting the second suspended object image through the main body detection model to obtain a suspended object area;
determining a target suspended object area and a category corresponding to the suspended object in the target suspended object area according to a user instruction;
and carrying out feature extraction on the target suspended object region to obtain suspended object features, and storing the second suspended object image, the suspended object features and the categories into the comparison library.
7. The method of identifying an object of a tower crane according to claim 1, further comprising:
and in the process of one-time lifting, carrying out multiple times of tower crane object recognition, and responding to the recognition times being greater than or equal to a second threshold value, wherein the result with the largest occurrence times is taken as a final recognition result in the results obtained by multiple times of recognition.
8. A tower crane object identification system, comprising:
the mobile detection module: the method comprises the steps of acquiring hook image data of continuous frames, identifying the hook image data, and judging whether a hook is in a moving state or not;
and a detection module: responding to the lifting hook in a moving state, acquiring an image to be detected comprising the lifting hook and a lifting object, and detecting the image to be detected through a main body detection model to obtain a target lifting hook area and a lifting object area, wherein the main body detection model is a classification model;
and the feature extraction module is used for: the method comprises the steps of determining a region to be identified according to a target lifting hook region and a lifting object region, and extracting the features to be identified of the region to be identified through a feature extraction network;
and an identification module: and the method is used for calculating the feature similarity of the feature to be identified and the image feature in the comparison library, and determining an identification result corresponding to the feature to be identified according to the feature similarity.
9. An electronic device, comprising
The memory device is used for storing the data,
processor and method for controlling the same
Computer program stored on the memory and executable on the processor, which when executed implements the tower crane object identification method according to any of claims 1 to 7.
10. A computer-readable storage medium, on which a computer program is stored, characterized in that the program, when executed by a processor, implements the tower crane object identification method according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410172331.7A CN117726882A (en) | 2024-02-07 | 2024-02-07 | Tower crane object identification method, system and electronic equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410172331.7A CN117726882A (en) | 2024-02-07 | 2024-02-07 | Tower crane object identification method, system and electronic equipment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117726882A true CN117726882A (en) | 2024-03-19 |
Family
ID=90209219
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202410172331.7A Pending CN117726882A (en) | 2024-02-07 | 2024-02-07 | Tower crane object identification method, system and electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117726882A (en) |
Citations (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107298381A (en) * | 2017-08-08 | 2017-10-27 | 王修晖 | The slow control method and device in place of tower crane |
| CN107879260A (en) * | 2017-10-11 | 2018-04-06 | 武汉科技大学 | Tower crane collision avoidance system and safety-protection system based on BIM |
| WO2019232831A1 (en) * | 2018-06-06 | 2019-12-12 | 平安科技（深圳）有限公司 | Method and device for recognizing foreign object debris at airport, computer apparatus, and storage medium |
| CN111392619A (en) * | 2020-03-25 | 2020-07-10 | 广东博智林机器人有限公司 | Tower crane early warning method, device and system and storage medium |
| CN112001282A (en) * | 2020-08-12 | 2020-11-27 | 腾讯音乐娱乐科技（深圳）有限公司 | Image recognition method |
| CN112347985A (en) * | 2020-11-30 | 2021-02-09 | 广联达科技股份有限公司 | Material type detection method and device |
| CN112884085A (en) * | 2021-04-02 | 2021-06-01 | 中国科学院自动化研究所 | Method, system and equipment for detecting and identifying contraband based on X-ray image |
| WO2021169161A1 (en) * | 2020-02-26 | 2021-09-02 | 上海商汤智能科技有限公司 | Image recognition method, recognition model training method and apparatuses related thereto, and device |
| CN113780429A (en) * | 2021-09-14 | 2021-12-10 | 杭州大杰智能传动科技有限公司 | Tower crane material classification and identification method and system based on image analysis |
| CN116385485A (en) * | 2023-03-13 | 2023-07-04 | 腾晖科技建筑智能(深圳)有限公司 | Video tracking method and system for long-strip-shaped tower crane object |
| CN116395567A (en) * | 2023-02-27 | 2023-07-07 | 腾晖科技建筑智能(深圳)有限公司 | Tower crane control method and system based on camera and laser radar |
| CN116503850A (en) * | 2023-03-17 | 2023-07-28 | 北京水狸智能建筑科技有限公司 | Method, device and equipment for identifying suspended object outline and readable storage medium |
-
2024
- 2024-02-07 CN CN202410172331.7A patent/CN117726882A/en active Pending
Patent Citations (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107298381A (en) * | 2017-08-08 | 2017-10-27 | 王修晖 | The slow control method and device in place of tower crane |
| CN107879260A (en) * | 2017-10-11 | 2018-04-06 | 武汉科技大学 | Tower crane collision avoidance system and safety-protection system based on BIM |
| WO2019232831A1 (en) * | 2018-06-06 | 2019-12-12 | 平安科技（深圳）有限公司 | Method and device for recognizing foreign object debris at airport, computer apparatus, and storage medium |
| WO2021169161A1 (en) * | 2020-02-26 | 2021-09-02 | 上海商汤智能科技有限公司 | Image recognition method, recognition model training method and apparatuses related thereto, and device |
| CN111392619A (en) * | 2020-03-25 | 2020-07-10 | 广东博智林机器人有限公司 | Tower crane early warning method, device and system and storage medium |
| CN112001282A (en) * | 2020-08-12 | 2020-11-27 | 腾讯音乐娱乐科技（深圳）有限公司 | Image recognition method |
| CN112347985A (en) * | 2020-11-30 | 2021-02-09 | 广联达科技股份有限公司 | Material type detection method and device |
| CN112884085A (en) * | 2021-04-02 | 2021-06-01 | 中国科学院自动化研究所 | Method, system and equipment for detecting and identifying contraband based on X-ray image |
| CN113780429A (en) * | 2021-09-14 | 2021-12-10 | 杭州大杰智能传动科技有限公司 | Tower crane material classification and identification method and system based on image analysis |
| CN116395567A (en) * | 2023-02-27 | 2023-07-07 | 腾晖科技建筑智能(深圳)有限公司 | Tower crane control method and system based on camera and laser radar |
| CN116385485A (en) * | 2023-03-13 | 2023-07-04 | 腾晖科技建筑智能(深圳)有限公司 | Video tracking method and system for long-strip-shaped tower crane object |
| CN116503850A (en) * | 2023-03-17 | 2023-07-28 | 北京水狸智能建筑科技有限公司 | Method, device and equipment for identifying suspended object outline and readable storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111127513B (en) | Multi-target tracking method | |
| CN108734162B (en) | Method, system, equipment and storage medium for identifying target in commodity image | |
| CN109003390B (en) | Commodity identification method, unmanned vending machine and computer-readable storage medium | |
| CN111008597B (en) | Space identification method and device for CAD drawing, electronic equipment and storage medium | |
| CN108256479B (en) | Face tracking method and device | |
| CN111931764B (en) | Target detection method, target detection frame and related equipment | |
| KR100612858B1 (en) | Method and apparatus for tracking human using robot | |
| US20230007831A1 (en) | Method for warehouse storage-location monitoring, computer device, and non-volatile storage medium | |
| CN113537070B (en) | Detection method, detection device, electronic equipment and storage medium | |
| CN114022508A (en) | Target tracking method, terminal and computer readable storage medium | |
| CN111160169A (en) | Face detection method, device, equipment and computer readable storage medium | |
| CN111797826A (en) | Large aggregate concentration area detection method and device and network model training method thereof | |
| CN113420682A (en) | Target detection method and device in vehicle-road cooperation and road side equipment | |
| CN116579616A (en) | Risk identification method based on deep learning | |
| CN111783910A (en) | Building project management method, electronic equipment and related products | |
| CN115690514A (en) | Image recognition method and related equipment | |
| EP4102463A1 (en) | Image processing method and related device | |
| CN117372928A (en) | Video target detection method and device and related equipment | |
| CN111709346A (en) | Historical building identification and detection method based on deep learning and high-resolution images | |
| CN117726882A (en) | Tower crane object identification method, system and electronic equipment | |
| CN111539390A (en) | Small target image identification method, equipment and system based on Yolov3 | |
| CN113869163B (en) | Target tracking method and device, electronic equipment and storage medium | |
| CN115527083A (en) | Image annotation method and device and electronic equipment | |
| CN115223173A (en) | Object identification method and device, electronic equipment and storage medium | |
| CN115115857A (en) | Image matching method and device and computer equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB02 | Change of applicant information | ||
| CB02 | Change of applicant information |
Country or region after: China Address after: 310000, 24th, 25th, and 26th floors of Building 3, Fashion Wantong City, Cangqian Street, Yuhang District, Hangzhou City, Zhejiang Province Applicant after: Hangzhou Yufan Intelligent Technology Co.,Ltd. Address before: 310000, 24th, 25th, and 26th floors of Building 3, Fashion Wantong City, Cangqian Street, Yuhang District, Hangzhou City, Zhejiang Province Applicant before: UNIVERSAL UBIQUITOUS TECHNOLOGY Co.,Ltd. Country or region before: China |