WO2021000948A1

WO2021000948A1 - Counterweight weight detection method and system, and acquisition method and system, and crane

Info

Publication number: WO2021000948A1
Application number: PCT/CN2020/100176
Authority: WO
Inventors: 徐柏科; 范卿; 曾杨; 谭智仁; 雷美玲
Original assignee: 中联重科股份有限公司
Priority date: 2019-07-04
Filing date: 2020-07-03
Publication date: 2021-01-07
Also published as: CN110956180B; CN110956180A

Abstract

A counterweight weight detection method and system, and acquisition method and system, and a crane, relating to the field of counterweight identification. The counterweight weight detection method comprises: acquiring an area to be detected in an image of a counterweight block on the basis of a structural feature and a color feature in the image of the counterweight block (S101); performing binarization processing on said area (S102); extracting a quasi-target area in said area on the basis of the binarized area to be detected (S103); and processing the extracted quasi-target area by adopting a trained classifier so as to detect the counterweight weight of the counterweight block (S104). The detection method can rapidly lock and extract the area where the counterweight weight is located, and has good reliability and robustness, thereby implementing automatic identification and high-precision detection of the counterweight weight, and further implementing automatic identification of the total counterweight weight in a counterweight assembling process.

Description

Counterweight weight detection method and system, acquisition method and system and crane

Technical field

The invention relates to the field of counterweight identification, in particular to a method for detecting the weight of a counterweight, an obtaining method, a detection system, an obtaining system and a crane.

Background technique

At this stage, the weight of the crane counterweight is mainly recognized by humans. Workers use video devices or directly visually measure the weight of the counterweight and calculate the total weight of the counterweight and match it with the crane. This method requires manual execution and is easy Errors, especially when the counterweight calculation error leads to the wrong selection of working conditions, the corresponding lifting weight table corresponding to the working conditions is wrong, which is very easy to cause accidents such as overloading and overturning of the crane.

The paper "Research and Design of Embedded Crane Counterweight Automatic Identification System" discloses the following content: first detect the position and size of the white paper and the sign on the counterweight, and then use the white paper and the sign as the basis to determine the position and weight of the counterweight. The weight is detected and identified. However, due to the fact that the white paper and the signs on the counterweight are easy to wear and fall off during the long-term use of the counterweight, the white paper and the signs are used as the premise, and the detection of the counterweight text lacks reliability and is not practical.

Summary of the invention

The purpose of the present invention is to provide a counterweight weight detection method, acquisition method, detection system, acquisition system and crane, which can quickly lock and extract the area where the counterweight weight is located, and has better reliability and robustness, So as to realize the automatic identification and high-precision detection of the weight of the counterweight.

In order to achieve the above object, the present invention provides a method for detecting the weight of a counterweight. The detection method includes: acquiring the area to be detected in the image of the counterweight based on the structural features and color features in the image of the counterweight. Binarize the area to be detected; extract the quasi-target area in the area to be detected based on the to-be-detected area after the binarization processing; and use the trained classifier to process the extracted quasi-target area , To detect the weight of the counterweight.

Preferably, the detection method further includes: before performing the step of binarizing the area to be detected, performing the following operations: calculating the average gray value of the area to be detected; When the gray average value of the area is less than the preset average value, the image texture enhancement is performed on the area to be detected.

Preferably, said performing image texture enhancement on the area to be detected includes: using a first structural element to perform opening and closing operations on the area to be detected; obtaining the first image based on the area to be detected and the image after the opening operation ; Obtain a second image based on the area to be detected and the image after the closing operation; and Obtain a fusion image corresponding to the area to be detected based on the first image and the second image.

Preferably, the obtaining the fused image corresponding to the area to be detected based on the first image and the second image includes: calculating edge information entropy of the first image and the second image respectively; and Weighted fusion is performed on the edge information entropy of the first image and the second image to obtain the fused image corresponding to the region to be detected.

Preferably, the detection method further includes: before performing the step of calculating the average gray value of the area to be detected, using a second structural element to perform opening and closing operations on the area to be detected to achieve filtering and denoising .

Preferably, the extracting the quasi-target area in the to-be-detected area based on the to-be-detected area after binarization processing includes: adopting an image processing method to obtain connected areas in the to-be-detected area after the binarization processing; And extracting the quasi-target area based on the location information of the connected area.

Preferably, the detection method further comprises: before performing the step of extracting the quasi-target area based on the position information of the connected area, performing the following operation: based on the concave-convex curvature of the connected area, The regions are divided to remove interference points; the area and height-to-width ratio of each sub-connected region divided in the connected region are estimated; and the area and height-to-width ratio of the specific sub-connected region in each sub-connected region satisfy In the case of any of the following removal conditions, the specific sub-connected area is removed: the area of the specific sub-connected area is smaller than a first preset area; the area of the specific sub-connected area is larger than a second preset area; and The height-to-width ratio of the specific sub-connected area is greater than the preset ratio, wherein the first preset area is smaller than the second preset area.

Preferably, the obtaining the area to be detected in the image of the counterweight based on the structural feature and the color feature in the image of the counterweight includes: obtaining the image based on the structural feature of the image of the counterweight The part of the image in which includes the area to be detected; and based on the acquired color features of the part of the image, according to the magnitude of the horizontal grayscale gradient complexity mutation and the vertical grayscale gradient complexity mutation, the execution of the partial image And the cutting of columns to obtain the area to be inspected.

Preferably, the performing row and column cutting of the partial image includes: calculating the horizontal gray gradient complexity and the vertical gray gradient complexity based on the color characteristics of the partial image; based on the horizontal gray gradient complexity The maximum and minimum values of the horizontal gray-scale gradient complexity mutation and the maximum and minimum values of the vertical gray-scale gradient complexity mutation are respectively obtained based on the horizontal gray-scale gradient complexity. The column corresponding to the maximum value and the minimum value of the sudden change, and the row corresponding to the maximum value and the minimum value of the vertical gray gradient complexity sudden change, cut the partial image to obtain the area to be detected.

Through the above technical solution, the present invention creatively obtains the area to be inspected including the weight of the weight block based on the structural features and color characteristics in the image of the weight block, and then from the binarized area to be inspected The quasi-target area about the weight of the counterweight is extracted in the, and finally, the extracted quasi-target area is processed by the classifier that has been trained in advance to detect the weight of the counterweight, which can quickly lock and extract the weight The area where the heavy weight is located has good reliability and robustness, so as to realize the automatic identification and high-precision detection of the weight of the weight.

Correspondingly, the present invention also provides a method for obtaining the weight of the counterweight, the obtaining method includes: detecting the counterweight weight of the first counterweight according to the above-mentioned method for detecting the weight of the counterweight; The detection method includes detecting the weight of the second weight; and obtaining the total weight of the weight based on the weight of the first weight and the weight of the second weight.

Preferably, the acquisition method further comprises: acquiring images of the first weight and the second weight; after performing the step of detecting the weight of the first weight, and the acquired The image of shows that when the second weight is installed on the positioning tip, the column corresponding to the maximum value of the vertical gradient mutation and the row corresponding to the maximum value of the horizontal gradient mutation based on the image of the first weight block, Assign a value of 0 to the pixels of the image of the first weight block.

Through the above technical solution, the present invention creatively detects the weight of the first and second weights through the above-mentioned method for detecting the weight of the weight, and obtains the total weight of the weight based on the weight of the first and second weights. Therefore, the total weight of the counterweight can be effectively identified with high accuracy, so that the automatic identification of the total counterweight can be realized in the process of assembling the counterweight.

Correspondingly, the present invention also provides a detection system for the weight of a counterweight. The detection system includes: an area to be detected acquisition device for obtaining the counterweight based on the structural features and color features in the image of the counterweight. The area to be detected in the image of the; binarization processing device for binarizing the area to be detected; quasi-target area extraction device for extracting the area to be detected based on the binarization processing A quasi-target area in the area to be detected; and a detection device for processing the extracted quasi-target area with a trained classifier to detect the weight of the counterweight.

For the specific details and benefits of the detection system for the weight of the counterweight provided by the present invention, please refer to the description of the detection method for the weight of the counterweight, which will not be repeated here.

Correspondingly, the present invention also provides a system for obtaining the weight of the counterweight, the obtaining system includes: the detection system for the weight of the counterweight described above, for detecting the weight of the first counterweight and the second counterweight And a total counterweight weight obtaining device for obtaining the total counterweight weight of the counterweight based on the counterweight weight of the first counterweight and the second counterweight.

Preferably, the acquisition system further includes: a collection device for collecting images of the first counterweight and the second counterweight; and an assignment device for detecting the first counterweight after the detection system After the weight of the counterweight of the block, and the image collected by the acquisition device shows that the second counterweight is installed on the positioning pin, the maximum value of the vertical gradient mutation based on the image of the first counterweight corresponds to In the column of and the row corresponding to the maximum value of the horizontal gradient mutation, the pixel of the image of the first weight block is assigned a value of 0.

Preferably, the collection device includes: a camera for collecting images of the first counterweight and the second counterweight; and a telescopic control module for controlling the telescopic and/or rotation of the camera to make The viewing angle of the camera is greater than or equal to the range of the area where the first weight block and the second weight block are located.

Regarding the benefits of the system for obtaining the weight of the counterweight provided by the present invention, please refer to the above description of the method for obtaining the weight of the counterweight, which will not be repeated here.

Correspondingly, the present invention also provides a crane, which is configured with the aforementioned system for obtaining the weight of the counterweight.

Correspondingly, the present invention also provides a machine-readable storage medium having instructions stored on the machine-readable storage medium for causing a machine to execute the aforementioned method for detecting the weight of a counterweight or the aforementioned method for obtaining the weight of a counterweight .

Other features and advantages of the present invention will be described in detail in the following specific embodiments.

Description of the drawings

The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the following specific embodiments, they are used to explain the present invention, but do not constitute a limitation to the present invention. In the attached picture:

FIG. 1 is a flowchart of a method for detecting the weight of a counterweight provided by an embodiment of the present invention;

FIG. 2 is a flowchart of obtaining a region to be detected according to an embodiment of the present invention;

Figure 3 is a schematic structural diagram of a counterweight provided by an embodiment of the present invention;

4 is a flowchart of extracting a quasi-target area provided by an embodiment of the present invention;

FIG. 5 is a flow chart of removing non-counterweight weight areas in the process of extracting quasi-target areas according to an embodiment of the present invention;

6 is a flowchart of a method for detecting the weight of a counterweight provided by an embodiment of the present invention;

Figure 7 is a structural diagram of a system for detecting the weight of a counterweight provided by an embodiment of the present invention;

Figure 8 is a structural diagram of a system for obtaining a counterweight weight provided by an embodiment of the present invention;

FIG. 9 is a flowchart of a method for obtaining a weight of a counterweight provided by an embodiment of the present invention; and

FIG. 10 is a schematic diagram of the installation position of the camera and the counterweight provided by the embodiment of the present invention.

Description of reference signs

1 Counterweights Counterweights 2 Counterweights

70 Obtaining device for the area to be inspected 71 Binarization processing device

72 Quasi-target area extraction device 73 Detection device

80 Detection system 800 image analysis processor

801 On-board display 810 Cameras

Detailed ways

The specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described here are only used to illustrate and explain the present invention, and are not used to limit the present invention.

Fig. 1 is a flowchart of a method for detecting the weight of a counterweight provided by an embodiment of the present invention. As shown in FIG. 1, the detection method may include the following steps: step S101, based on the structural feature and color feature in the image of the weight block, obtain the area to be detected in the image of the weight block; step S102, right The area to be detected is subjected to binarization processing; step S103, based on the area to be detected after the binarization processing, a quasi-target area in the area to be detected is extracted; and step S104, a trained classifier is used to process the extracted Quasi-target area to detect the weight of the weight block.

In a preferred embodiment, in order to increase the processing speed of the image, the image of the weight block may be scaled and grayed before step S101 is executed.

The above detection method can be executed by a weight detection system, and the detection system can be an image analysis processor 800, as shown in FIG. 8. In addition, in order to facilitate inspection by operators such as the operator, the detection system may further include: a vehicle-mounted display 801 for real-time display of the weight of the counterweight, as shown in FIG.

The step S101 may include the following steps: based on the structural features in the image of the weight block, acquiring a part of the image including the area to be detected in the image; and based on the acquired color features of the part of the image, According to the magnitude of the horizontal gray gradient complexity mutation and the vertical gray gradient complexity mutation, row and column cutting is performed on the partial image to obtain the region to be detected.

Specifically, the process of acquiring a part of the image including the area to be detected in the image includes the following content: As shown in FIG. 3, the area where the weight of the counterweight is located is generally at the buckle rigging (recess) The counterweight block (weight block 1, counterweight block 2) is a symmetric feature, so the left half image or the right half image of the counterweight block image is selected as the research object (ie, partial image). The embodiment of the present invention is mainly, but not limited to, taking the image of the left half (ie, the left side) of the column center line of the image as the research object (ie, partial image).

The process of performing row and column cutting on this part of the image may include the following steps, as shown in Fig. 2:

Step S201, based on the color features of the partial images, calculate the horizontal gray gradient complexity and the vertical gray gradient complexity respectively.

Perform horizontal and vertical gray gradient complexity calculations on the research object to analyze the gray structure characteristics of the weight.

The definition of image gradient complexity is as follows:

Specifically, the vertical and horizontal gradient complexity are defined as follows:

Among them, I(x,y) is the image of the weight block (or original grayscale image), row number i=1, 2,...m, column number j=1, 2,...n, C ₁ , C ₂ respectively Represents the vertical and horizontal gradient complexity of the image.

Step S202, based on the complexity of the horizontal gray gradient and the complexity of the vertical gray gradient, obtain the maximum and minimum of the horizontal gray gradient complexity mutation, and the maximum and the minimum of the vertical gray gradient complexity mutation. value.

Based on the horizontal gray gradient complexity and the vertical gray gradient complexity, the maximum and minimum values of the horizontal gray gradient complexity mutation Hori_grad(max _j1 ,min _j2 ) are filtered out, and Hori_grad(max _j1 ,min _j2 ) corresponds to the column numbers j1 and j2. Similarly, filter the maximum and minimum values of the vertical gray gradient complexity mutation Verti_grad(max _i1 ,min _i2 ), and record the row numbers i1 and i2 corresponding to Verti_grad(max _i1 ,min _i2 ).

Step S203: Perform processing on the partial image based on the columns corresponding to the maximum and minimum values of the horizontal gray gradient complexity mutation, and the rows corresponding to the maximum and minimum values of the vertical gray gradient complexity mutation. Cutting to obtain the area to be inspected.

Cut with the column numbers j1, j2 corresponding to Hori_grad(max _j1 , min _j2 ) recorded in step S202, and cut with the row numbers i1, i2 corresponding to Verti_grad(max _i1 , min _i2 ) recorded in step S202 , So as to obtain the area to be detected in the image of the weight block, such as the circular area A in FIG. 3.

For step S102, after acquiring the area to be detected, binarization can be performed on the area to be detected by analyzing the gray distribution of the area to be detected, for example, in the case where the gray level is greater than a preset gray level Next, it is assigned a value of 0; when the grayscale is less than or equal to the preset grayscale, it is assigned a value of 1.

As shown in FIG. 4, the step S103 may include the following steps:

In step S401, the image processing method is adopted to obtain the connected areas in the area to be detected after the binarization processing.

Step S402: Extract the quasi-target area based on the location information of the connected area.

In fact, the area where interference points such as dense impurities or stains are located (ie, non-digital connected areas) will adhere to the quasi-target area, resulting in an increase in the area of the quasi-target area identified, and ultimately affecting the accuracy and timeliness of the weight. Sex. Therefore, in order to eliminate the adverse effects of the interference points, preferably, before performing step S402, the coordinate points of the bump hull of the connected area can be calculated, and the connected area is divided based on the calculated coordinate points to divide into several sub-connected areas . Finally, by analyzing the specific conditions of the sub-connected areas, a large number of non-digital connected areas are eliminated, so as to accurately identify the quasi-target area, and lay a solid foundation for quickly and accurately identifying the weight of the counterweight.

Specifically, as shown in Figure 5, the above process may include the following steps:

Step S501, based on the concave-convex curvature of the connected area, divide the connected area to remove interference points. Analyze the concave and convex curvature of the connected region, extract peak points (for example, the points corresponding to the maximum value and the minimum value), and perform segmentation processing on the connected connected domains based on the coordinate points of the peak point, thereby removing impurities and stains Wait for interference. At the same time, the connected area is divided into sub-connected areas

Step S502, estimating the area and height-to-width ratio of each sub-connected area divided in the connected area.

Step S503: In the case where the area and the height-to-width ratio of the specific sub-connected area in each of the sub-connected areas meet any of the following elimination conditions, the specific sub-connected area is eliminated.

The elimination condition may be that the area of the specific sub-connected area is smaller than the first preset area; the area of the specific sub-connected area is greater than the second preset area; or the height-to-width ratio of the specific sub-connected area is greater than the preset Ratio, wherein the first predetermined area is smaller than the second predetermined area.

The area of the area where the weight of the counterweight (such as 8t) is located usually meets certain specifications. For example, the area of the area where the weight of the counterweight (such as 8t) is greater than or equal to 150, less than or equal to 3000, and the aspect ratio is less than or equal to 1.5, corresponding Ground, the first preset area may be 150, the second preset area may be 3000, and the preset ratio may be 1.5. Of course, the first preset area, the second preset area, and the preset ratio in the embodiment of the present invention are not limited to the aforementioned values, and any other values within a reasonable range are feasible.

When the area of a sub-connected area is too large (for example, more than 3000), too small (for example, less than 150), or the aspect ratio is too large (for example, more than 1.5), it indicates that the sub-connected area is not the area where the counterweight is located. The sub-connected region is de-binarized to eliminate the sub-connected region. For example, if the sub-connected area is 1, it is reversed to 0, that is, the value of the sub-connected area and the background (non-weighted area) are the same.

After removing the non-counterweight weight area, based on the position information of the remaining connected areas, extract the quasi-target area, that is, extract the area where the counterweight weight of the roughly positioned counterweight block is located. The weight of the counterweight and the structural position and color characteristics of the sling are roughly extracted from the area where the weight is located.

For the step S104, a certain number of positive and negative samples of the weight of the counterweight (ie, the digital area) can be collected in advance. The positive and negative samples are the area where the weight of the weight is located (ie, the target area) and the non- The area where the weight of the counterweight is located (that is, the non-target area). Use the positive and negative samples to train the classifier (for example, vector machine SVM), and then use the trained classifier to process the quasi-target region extracted in step S103, so as to realize the allocation of the weight block Real-time detection of heavy weight.

If the light is dark when the image of the counterweight is collected, the texture of the counterweight text will not be prominent under the condition of the weak difference between the counterweight text and the background color. In this embodiment, the thresholding decision can be used to enhance the counterweight text texture. Effectively highlight the texture change characteristics of the weighted text for easy detection.

Before performing binarization processing on the area to be detected, the following operations may be performed: calculating the average gray value of the area to be detected; and in the case that the average gray value of the area to be detected is less than a preset average value, Perform image texture enhancement on the area to be detected. Wherein, said performing image texture enhancement on the area to be detected may include: using a first structural element to perform opening and closing operations on the area to be detected; obtaining a first image based on the area to be detected and the image after the opening operation ; Obtain a second image based on the area to be detected and the image after the closing operation; and Obtain a fusion image corresponding to the area to be detected based on the first image and the second image.

Wherein, the obtaining the fused image corresponding to the area to be detected based on the first image and the second image may include: calculating edge information entropy of the first image and the second image respectively; and Weighted fusion is performed on the edge information entropy of the first image and the second image to obtain the fused image corresponding to the region to be detected.

Specifically, the grayMean value of the gray value of the area to be detected is calculated, and the preset average value (or gray value threshold grayValue _thred ) is used to decide whether to perform image texture (detail) enhancement processing. If the average gray value is less than the gray threshold, the image texture detail enhancement is performed, otherwise it is not executed.

The process of image texture enhancement is as follows:

Perform an open operation on the original grayscale image I(x, y) of the area to be detected. Compared with the original grayscale image, some data in the image after the open operation have changed, while other data remain unchanged ( For example, the gray value at a large gray value changes greatly, and the gray value at a small gray value remains unchanged or changes less). At this time, the data whose gray value changes greatly after the open operation is set to 0, and the data whose gray value does not change or changes little remains the original value, and the changed image is obtained. In order to improve the edge texture of the dark area, the original gray image and the changed image are subtracted to obtain an image f ₁ (x, y).

Perform a closed operation on the original grayscale image I(x, y) of the area to be detected, and compare the two images before and after the operation to obtain a difference image. The data in the difference image that is less than the set threshold is calibrated to 1, and the data that is greater than the set threshold is calibrated to 0, thereby obtaining a binary image. The binary image is multiplied with the original gray image to obtain the image f ₂ (x, y), thereby improving the edge texture contrast of the bright area.

Calculate the edge information entropy of f ₁ (x, y) and f ₂ (x, y), and perform weighted fusion of the edge information entropy of the two images (that is, fusion according to the entropy weight of the two images) , The fusion image f _fusion (x, y) corresponding to the region to be detected is obtained.

In addition, in a preferred embodiment, before the step of calculating the average gray value of the area to be detected, the second structural element is used to perform opening and closing operations on the area to be detected to achieve positive and negative filtering. noise.

Specifically, as shown in Figure 6, the detection process of the counterweight weight is as follows:

In step S601, the image of the weight block is scaled and grayed.

Step S602: Obtain the area to be detected in the image of the weight block by analyzing the gray gradient complexity of the zoomed and grayed image.

Step S603: Perform opening and closing operations on the area to be detected.

The purpose of this step is to filter and denoise the area to be detected.

Step S604: Obtain the average gray value of the area to be detected.

Step S605: It is judged whether the average gray value is greater than the preset average value, if it is greater, step S607 is executed, otherwise, step S606 is executed.

Step S606: Perform image texture enhancement processing on the area to be detected, and perform step S607.

In the above steps S604-S606, according to the global gray distribution of the area to be detected, it is decided to perform the image texture enhancement of the morphological transformation.

Step S607: Binarize the area to be detected.

In step S608, the connected area in the area to be detected after the binarization process is obtained, and the peak point of the bump hull in the connected area is calculated.

In step S609, the connected area is divided based on the peak points of the bump hull in the connected area to obtain multiple sub-connected areas.

Step S610: It is judged whether the area and aspect ratio of each sub-connected region in the multiple sub-connected regions meet the elimination condition, if they are satisfied, step S611 is executed; otherwise, step S612 is executed.

In step S611, the sub-connected regions meeting the elimination condition are eliminated, and step S612 is executed.

Step S612, extracting the area where the weight of the roughly positioned weight block is located.

Step S613: Use a trained classifier to process the extracted area where the weight of the roughly positioned weight block is located to detect the weight of the weight block.

In summary, the present invention is creatively based on the structural features and color features in the image of the weight block to obtain the area to be inspected including the weight of the weight block, and then from the binarized area to be inspected The quasi-target area about the weight of the counterweight is extracted in the, and finally, the extracted quasi-target area is processed by the classifier that has been trained in advance to detect the weight of the counterweight, which can quickly lock and extract the weight The area where the heavy weight is located has good reliability and robustness, so as to realize the automatic identification and high-precision detection of the weight of the weight.

Correspondingly, as shown in FIG. 7, the present invention also provides a weight detection system. The detection system may include: an area-to-be-detected area acquisition device 70 for structural features and colors in the image based on the weight. Feature, obtain the area to be detected in the image of the weight block; binarization processing device 71, for binarizing the area to be detected; quasi-target area extraction device 72, for binarization based After the processed area to be detected, extracting the quasi-target area in the area to be detected; and the detection device 73 is used to process the extracted quasi-target area by using a trained classifier to detect the weight of the weight block weight.

Optionally, the detection system further includes: a gray-scale mean value calculation device for calculating the gray-scale mean value of the area to be detected; and a texture enhancement device for performing the binarization processing device on the to-be-detected area. Before the region is binarized and the average gray value of the region to be detected is less than the preset average value, image texture enhancement is performed on the region to be detected.

Optionally, the texture enhancement device includes: an arithmetic module, configured to use a first structural element to perform opening and closing operations on the region to be detected; a first image acquisition module, configured to perform opening and closing operations based on the region to be detected and After the image, the first image is acquired; the second image acquisition module is used to acquire the second image based on the area to be detected and the image after the closing operation; and the fusion image acquisition module is used to acquire the second image based on the first image and The second image acquires a fusion image corresponding to the area to be detected.

Optionally, the fusion image acquisition module includes: an edge information entropy calculation unit for calculating the edge information entropy of the first image and the second image respectively; and a fusion image acquisition unit for calculating the first image and the second image. The edge information entropy of an image and the second image is weighted and fused to obtain the fused image corresponding to the region to be detected.

Optionally, the detection system further includes: an arithmetic device, configured to use a second structural element to perform a calculation on the area to be detected before the step of calculating the average gray value of the area to be detected by the gray average value calculation device Open and close operations to achieve filtering and denoising.

Optionally, the device for extracting the quasi-target region includes: a connected region acquisition module for acquiring connected regions in the region to be detected after binarization processing by using an image processing method; and a quasi-target region extraction module for Extracting the quasi-target area based on the location information of the connected area.

The detection system further includes: a segmentation device, which is configured to: before the quasi-target region extraction module extracts the quasi-target region based on the position information of the connected region, based on the concave-convex curvature of the connected region, compare the connected region The area is divided to remove interference points; an estimation device is used to estimate the area and height-to-width ratio of each sub-connected area divided in the connected area; and a culling device is used for a specific sub-connected area in each sub-connected area. The specific sub-connected area is removed when the area and height-to-width ratio of the connected area meet any of the following removal conditions: the area of the specific sub-connected area is smaller than the first preset area; the area of the specific sub-connected area is greater than A second preset area; and the height-to-width ratio of the specific sub-connected area is greater than a preset ratio, wherein the first preset area is smaller than the second preset area.

Optionally, the device for acquiring the region to be detected includes: a partial image acquisition module for acquiring a partial image of the image including the region to be detected based on structural features in the image of the weight block; and The detection area module is used to perform row and column cutting of the partial image based on the color features of the acquired part of the image, according to the magnitude of the horizontal gray gradient complexity mutation and the vertical gray gradient complexity mutation, to obtain The area to be detected. Optionally, the acquisition module for the area to be detected includes: a complexity calculation unit, configured to calculate the horizontal gray gradient complexity and the vertical gray gradient complexity based on the color characteristics of the partial image; the gray gradient complexity A sudden change maximum value acquisition unit for acquiring the maximum and minimum values of the horizontal gray gradient complexity and the vertical gray gradient complexity respectively based on the horizontal gray gradient complexity and the vertical gray gradient complexity And the area to be detected acquisition unit, which is used for columns corresponding to the maximum and minimum of the horizontal grayscale gradient complexity mutation, the maximum and the vertical grayscale gradient complexity mutation The row corresponding to the minimum value is used to cut the partial image to obtain the area to be detected.

The above process is a detection process for the weight of a single counterweight, but in reality, multiple counterweights are often required to meet engineering needs. In the embodiment of the present invention, two counterweights (as shown in FIG. 8) are mainly taken as an example to describe the process of obtaining the total counterweight weight of the counterweight.

As shown in FIG. 9, the method for obtaining the weight of the counterweight may include the following steps: step S901, detecting the weight of the first counterweight according to the above-mentioned method for detecting the weight of the counterweight; step S902, according to the above-mentioned counterweight The weight detection method is to detect the counterweight weight of the second counterweight; and step S903, based on the counterweight weight of the first counterweight and the second counterweight, obtain the total counterweight of the counterweight weight.

When the weight of each weight is detected in turn, because the side structure of each weight is the same, the maximum and minimum vertical gradient mutations can be determined based on Verti_grad (max _i1 , min _i2 ). Cut the image to obtain For the area to be detected, there is no need to repeat the gradient complexity calculation, which reduces the calculation complexity. Thus, the acquisition method may further include: acquiring images of the first weight and the second weight; after performing the step of detecting the weight of the first weight, and The captured image shows that when the second counterweight is installed on the positioning pins A1 and A2 (as shown in Fig. 10), the column corresponding to the maximum value of the vertical gradient mutation based on the image of the first counterweight and In the row corresponding to the maximum value of the horizontal gradient mutation, the pixel of the image of the first weight block is assigned a value of 0.

Specifically, the process of obtaining the total counterweight weight is explained and described in detail by taking the counterweight weight obtaining system shown in FIG. 8 and FIG. 10 as an example.

Before explaining and explaining the process of obtaining the total counterweight weight, let's introduce the counterweight weight obtaining system.

As shown in FIG. 8, the acquisition system may include: a weight detection system, the detection system 80 includes: an image analysis processor 800; and a vehicle-mounted display 801, a total weight acquisition device (not shown), It is used to obtain the total weight of the weight based on the weight 2 of the first weight 1 and the weight 2 of the second weight.

As shown in FIG. 8, the acquisition system may further include: an acquisition device 81 for acquiring images of the first weight block 1 and the second weight block 2; and an assignment device (not shown), After the detection system detects the weight of the first counterweight 1, and the image collected by the collection device 81 shows that the second counterweight 2 is installed on the positioning pins A1, A2 (as shown in Fig. 10 Shown), based on the column corresponding to the maximum value of the vertical gradient sudden change of the image of the first weight block 1 and the row corresponding to the maximum value of the horizontal gradient sudden change, the first weight block 1 The pixels of the image are assigned the value 0. Wherein, the collecting device 81 collects the video image of the counterweight and transmits it to the image analysis processor 800 for real-time detection via wireless WiFi, and feeds back the detection result to the on-board display 801 to notify the operator of the total weight of the counterweight mounted. When the total counterweight reaches the demand, the on-board display 801 shows that the counterweight is full, and then the counterweight cylinder of the crane is activated to mount the counterweight.

As shown in FIG. 10, the collection device 81 may include a camera 810; and a telescopic control module (not shown) for controlling the telescopic and/or rotation of the camera so that the viewing angle of the camera is greater than or equal to the first The area where a counterweight 1 and the second counterweight (not shown) are located. Wherein, the camera 810 may be a webcam Camera. The camera 810 is installed in a protective shell in the direction of the front of the car. A telescopic control module (not shown) can control the up and down expansion and/or rotation of the camera 810. When the weight of the counterweight 1 is detected, the telescopic control module ( (Not shown) the camera 810 can be controlled to tilt upwards, and the video image of the counterweight is collected from the front. After the detection is completed, the camera 810 is controlled to be zoomed and placed in the protective shell, so as to realize the safety protection and effective operation of the camera.

The process of obtaining the total counterweight weight is as follows:

The video image of the first counterweight 1 is collected by the collecting device 81, and based on the video image of the first counterweight 1, the above detection method is used to detect the counterweight of the first counterweight 1. In this process, Record the column number j corresponding to the maximum value of the vertical gradient mutation and the row number i corresponding to the maximum value of the horizontal gradient mutation in the image of the first weight block 1. At this point, the second counterweight 2 is lifted.

In the case where the image collected by the collecting device 81 shows that the second counterweight 2 is installed on the positioning pins A1 and A2 (as shown in FIG. 10), the column number corresponding to the maximum value of the pre-recorded vertical gradient sudden change is extracted j and the row number i corresponding to the maximum value of the horizontal gradient mutation, use i and j as the horizontal and vertical thresholds respectively, that is, T _{Horizontal_thred} = j and T _{Vertical_thred} = i. Assign all the image pixel values of the part below the T _{Vertical_thred} row and the left part of the T _{Horizontal_thred} column (the previous weight block) to 0 to reduce the interference of the detection and eliminate the image, that is, to detect the second weight block 2 separately. Heavy weight. Based on the video image of the second weight block 2, the above detection method is used to detect the weight of the second weight block 2

The total weight value of the first counterweight 1 and the second counterweight 2 is accumulated to calculate the total weight of the counterweight.

The above-mentioned method for obtaining the weight of the counterweight based on machine vision involves relatively low computational complexity and can achieve an effective and high-precision counterweight recognition effect. However, the present invention is not limited to obtaining the counterweight weight of two counterweights, and the process of obtaining the counterweight weight of any other multiple counterweights is similar to the above-mentioned process, and will not be repeated here.

In summary, the present invention creatively detects the weight of the first and second counterweights through the above-mentioned method for detecting the weight of the counterweight, and obtains the total counterweight weight of the counterweight based on the weight of the first and second counterweights. Therefore, the total weight of the counterweight can be effectively identified with high accuracy, so that the automatic identification of the total counterweight can be realized in the process of assembling the counterweight.

Of course, the present invention is not limited to the above crane, and is also applicable to any other construction machinery that requires counterweight and needs to obtain the weight of the counterweight.

The machine-readable storage medium includes, but is not limited to, phase change memory (Phase Change Random Access Memory, PRAM, also known as RCM/PCRAM), static random access memory (SRAM), dynamic Random access memory (DRAM), other types of random access memory (RAM), read only memory (ROM), electrically erasable programmable read only memory (EEPROM), flash memory or other memory Technology, CD-ROM, Digital Versatile Disk (DVD) or other optical storage, magnetic cassette tape, magnetic tape disk storage or other magnetic storage devices and other media that can store program codes.

The preferred embodiments of the present invention are described in detail above with reference to the accompanying drawings. However, the present invention is not limited to the specific details in the above embodiments. Within the scope of the technical concept of the present invention, many simple modifications can be made to the technical solutions of the present invention. These simple modifications belong to the protection scope of the present invention.

In addition, it should be noted that the various specific technical features described in the foregoing specific embodiments can be combined in any suitable manner without contradiction. In order to avoid unnecessary repetition, various possible combinations are not described separately in the present invention.

In addition, various different embodiments of the present invention can also be combined arbitrarily, as long as they do not violate the idea of the present invention, they should also be regarded as the content disclosed in the present invention.

Claims

A method for detecting the weight of a counterweight, characterized in that the detecting method includes:

Acquiring the area to be detected in the image of the weight block based on the structural feature and the color feature in the image of the weight block;

Performing binarization processing on the area to be detected;

Extracting the quasi-target area in the to-be-detected area based on the to-be-detected area after binarization processing; and

The trained classifier is used to process the extracted quasi-target area to detect the weight of the weight block.
The method for detecting the weight of a counterweight according to claim 1, wherein the detecting method further comprises:

Before performing the step of binarizing the area to be detected, perform the following operations:

Calculating the average gray value of the area to be detected; and

In the case where the average gray value of the area to be detected is less than the preset average value, image texture enhancement is performed on the area to be detected.
The method for detecting the weight of a counterweight according to claim 2, wherein said performing image texture enhancement on the area to be detected comprises:

Using the first structural element to perform opening and closing operations on the area to be detected;

Acquiring a first image based on the area to be detected and the image after the opening operation;

Acquiring a second image based on the area to be detected and the image after the closing operation; and

Based on the first image and the second image, a fusion image corresponding to the area to be detected is acquired.
The method for detecting the weight of a counterweight according to claim 3, wherein the acquiring, based on the first image and the second image, the fusion image corresponding to the area to be detected comprises:

Respectively calculating the edge information entropy of the first image and the second image; and

Weighted fusion is performed on the edge information entropy of the first image and the second image to obtain the fused image corresponding to the region to be detected.
The method for detecting the weight of a counterweight according to claim 2, wherein the detecting method further comprises:

Before performing the step of calculating the average gray value of the area to be detected, a second structural element is used to perform opening and closing operations on the area to be detected to achieve filtering and denoising.
The method for detecting the weight of a counterweight according to claim 1, wherein the extracting the quasi-target area in the area to be detected based on the area to be detected after binarization processing comprises:

Using an image processing method to obtain the connected areas in the area to be detected after binarization processing; and

Extracting the quasi-target area based on the location information of the connected area.
The method for detecting the weight of a counterweight according to claim 6, wherein the detecting method further comprises:

Before performing the step of extracting the quasi-target area based on the location information of the connected area, perform the following operations:

Segmenting the connected area based on the concave and convex curvature of the connected area to remove interference points;

Estimating the area and height-to-width ratio of each sub-connected area divided in the connected area; and

In the case where the area and the height-to-width ratio of the specific sub-connected area in each sub-connected area satisfy any of the following elimination conditions, the specific sub-connected area is eliminated:

The area of the specific sub-connected area is smaller than the first preset area;

The area of the specific sub-connected area is greater than the second predetermined area; and

The height-to-width ratio of the specific sub-connected area is greater than a preset ratio,

Wherein, the first predetermined area is smaller than the second predetermined area.
The method for detecting the weight of a counterweight according to any one of claims 1 to 7, wherein the structural feature and color feature in the image based on the counterweight block are used to obtain The area to be inspected includes:

Based on the structural features in the image of the weight block, acquiring a partial image of the image including the area to be detected; and

Based on the acquired color features of the partial image, according to the magnitude of the horizontal gray gradient complexity mutation and the vertical gray gradient complexity mutation, the partial image is cut into rows and columns to obtain the region to be detected.
The method for detecting the weight of a counterweight according to claim 8, wherein the performing row and column cutting on the partial image comprises:

Based on the color features of the part of the image, respectively calculate the horizontal grayscale gradient complexity and the vertical grayscale gradient complexity;

Based on the horizontal grayscale gradient complexity and the vertical grayscale gradient complexity, respectively obtaining the maximum and minimum values of the horizontal grayscale gradient complexity mutation and the maximum and minimum values of the vertical grayscale gradient complexity mutation; and

Based on the columns corresponding to the maximum and minimum values of the horizontal grayscale gradient complexity mutation, and the rows corresponding to the maximum and minimum values of the vertical grayscale gradient complexity mutation, the partial image is cut to Obtain the area to be detected.
A method for obtaining the weight of a counterweight, characterized in that the obtaining method includes:

According to the method for detecting the weight of the counterweight according to any one of claims 1-9, detecting the weight of the counterweight of the first counterweight;

The method for detecting the weight of the counterweight according to any one of claims 1-9, detecting the counterweight weight of the second counterweight; and

Obtain the total weight of the weight based on the weight of the first weight and the weight of the second weight.
The method for obtaining the weight of a counterweight according to claim 10, wherein the obtaining method further comprises:

Acquiring images of the first counterweight and the second counterweight;

After the step of detecting the weight of the first counterweight is performed, and the captured image shows that the second counterweight is installed to the positioning pin, the vertical direction of the image based on the first counterweight is In the column corresponding to the maximum value of the gradient mutation and the row corresponding to the maximum value of the horizontal gradient mutation, the pixel of the image of the first weight block is assigned a value of 0.
A detection system for the weight of a counterweight, characterized in that the detection system includes:

A device for obtaining a region to be detected, configured to obtain the region to be detected in the image of the counterweight based on the structural feature and the color feature in the image of the counterweight;

A binarization processing device, configured to perform binarization processing on the area to be detected;

A quasi-target region extraction device, which is used to extract a quasi-target region in the region to be detected based on the region to be detected after binarization processing; and

The detection device is used for processing the extracted quasi-target area with a trained classifier to detect the weight of the counterweight block.
The counterweight weight detection system according to claim 12, wherein the detection system further comprises:

A gray average value calculation device for calculating the gray average value of the area to be detected; and

The texture enhancement device is configured to perform the detection on the area to be detected before the binarization processing device performs the binarization process on the area to be detected and the average gray value of the area to be detected is less than the preset average value. Image texture enhancement.
The counterweight weight detection system according to claim 13, wherein the texture enhancement device comprises:

An arithmetic module, configured to use the first structural element to perform opening and closing operations on the area to be detected;

The first image acquisition module is configured to acquire a first image based on the area to be detected and the image after the opening operation;

The second image acquisition module is configured to acquire a second image based on the area to be detected and the image after the closing operation; and

The fusion image acquisition module is configured to acquire a fusion image corresponding to the area to be detected based on the first image and the second image.
The counterweight weight detection system according to claim 14, wherein the fusion image acquisition module comprises:

An edge information entropy calculation unit, configured to calculate the edge information entropy of the first image and the second image respectively; and

The fusion image acquisition unit is configured to perform weighted fusion on the edge information entropy of the first image and the second image to acquire the fusion image corresponding to the area to be detected.
The counterweight weight detection system according to claim 13, wherein the detection system further comprises:

The arithmetic device is used for performing opening and closing operations on the area to be detected by using a second structural element before the step of calculating the average gray value of the area to be detected by the gray average value calculating device, so as to achieve filtering and denoising.
The counterweight weight detection system according to claim 12, wherein the quasi-target area extraction device comprises:

The connected region acquisition module is used to adopt the image processing method to acquire the connected region in the to-be-detected region after binarization processing; and

The quasi-target region extraction module is configured to extract the quasi-target region based on the position information of the connected region.
The counterweight weight detection system according to claim 17, wherein the detection system further comprises:

A segmentation device for segmenting the connected area based on the concavity and convexity of the connected area before the quasi-target area extraction module extracts the quasi-target area based on the position information of the connected area to remove interference point;

An estimation device for estimating the area and height-to-width ratio of each sub-connected area divided in the connected area; and

The culling device is used for culling the specific sub-connected area when the area and the height-to-width ratio of the specific sub-connected area in each of the sub-connected areas meet any of the following removal conditions:

The area of the specific sub-connected area is smaller than the first preset area;

The area of the specific sub-connected area is greater than the second predetermined area; and

The height-to-width ratio of the specific sub-connected area is greater than a preset ratio,

Wherein, the first predetermined area is smaller than the second predetermined area.
The system for detecting the weight of a counterweight according to any one of claims 12 to 18, wherein the device for obtaining the area to be detected comprises:

A partial image acquisition module for acquiring a partial image of the image including the area to be detected based on the structural features in the image of the weight block; and

The to-be-detected area module is used to perform row and column cutting of the partial image based on the acquired color features of the partial image, according to the magnitude of the horizontal grayscale gradient complexity mutation and the vertical grayscale gradient complexity mutation Obtain the area to be detected.
The counterweight weight detection system according to claim 19, wherein the to-be-detected area acquisition module comprises:

The complexity calculation unit is configured to calculate the horizontal gray gradient complexity and the vertical gray gradient complexity based on the color features of the partial image;

The maximum value acquisition unit of gray gradient complexity mutation is used to obtain the maximum and minimum values of the horizontal gray gradient complexity mutation and the vertical gray gradient based on the horizontal gray gradient complexity and the vertical gray gradient complexity. The maximum and minimum of the degree of gradient complexity mutation; and

The to-be-detected area acquisition unit is configured to perform the calculation based on the columns corresponding to the maximum and minimum values of the horizontal gray gradient complexity mutation, and the rows corresponding to the maximum and minimum values of the vertical gray gradient complexity mutation. The partial image is cut to obtain the area to be detected.
A system for acquiring the weight of a counterweight, wherein the acquiring system includes:

The counterweight weight detection system according to any one of claims 12-20, which is used to detect the counterweight weight of the first counterweight and the counterweight weight of the second counterweight; and

The total counterweight weight obtaining device is configured to obtain the total counterweight weight of the counterweight based on the counterweight weight of the first counterweight and the second counterweight.
The system for obtaining the weight of a counterweight according to claim 21, wherein the obtaining system further comprises:

An acquisition device for acquiring images of the first counterweight and the second counterweight;

The assigning device is used to, after the detection system detects the weight of the first counterweight, and the image collected by the collection device shows that the second counterweight is installed on the positioning pin, based on the In the column corresponding to the maximum value of the vertical gradient mutation of the image of the first weight block and the row corresponding to the maximum value of the horizontal gradient mutation, the pixel of the image of the first weight block is assigned a value of 0.
The system for obtaining the weight of a counterweight according to claim 22, wherein the collecting device comprises:

A camera for collecting images of the first counterweight and the second counterweight; and

The telescopic control module is used to control the telescopic and/or rotation of the camera so that the viewing angle of the camera is greater than or equal to the range of the area where the first counterweight and the second counterweight are located.
A crane, characterized in that the crane is configured with the counterweight weight obtaining system according to any one of claims 21-23.
A machine-readable storage medium, characterized in that instructions are stored on the machine-readable storage medium, and the instructions are used to make a machine execute the method for detecting the weight of a counterweight according to claims 1-9 or claims 10- 11. The method for obtaining the weight of the counterweight according to any one of the claims.