CN111382726A - Engineering work detection method and related device - Google Patents

Abstract

The application discloses an engineering operation detection method and a related device, wherein the engineering operation detection method comprises the following steps: acquiring an original image shot by a camera device on a working site, wherein the original image comprises a preset detection area; carrying out target detection on the original image, and acquiring a target area corresponding to a target object in the original image, wherein the target object is used for realizing warning; and determining whether the operation site meets the operation specification or not based on the position relation between the preset detection area and the target area. By the scheme, the engineering operation detection quality can be improved.

Description

Engineering work detection method and related device

technical field

The present application relates to the technical field of image processing, and in particular, to a method for detecting engineering work and a related device.

Background technique

In engineering operations, in order to warn the operators on the job site and avoid them from entering the dangerous area by mistake, so as to ensure the safety of the operation, warning objects such as warning signs are usually set near the dangerous area on the job site. Taking power maintenance as an example, due to the insufficiency of the "five-proof" function of the switchgear or the distraction of the operator's energy, dangerous phenomena such as the operator's mistaken entry into the live interval, the wrong switch and the wrong switch may occur. Therefore, in the switching operation or maintenance operation, the red cloth curtain is generally used to warn the operator, so that the operator can clearly distinguish the power failure maintenance panel cabinet and the adjacent live non-maintenance panel cabinet. It can be seen that warning objects such as red cloth curtains and warning signs play a very important role in ensuring operational safety.

At present, manual inspection is still used to determine whether the above-mentioned warning objects are used in a standardized manner, and therefore, there is a problem of low efficiency. In addition, limited by factors such as human resources and the energy of inspectors, manual inspections will inevitably lead to omissions. All these have reduced the inspection quality of engineering operations. In view of this, how to improve the inspection quality of engineering operations has become an urgent problem to be solved.

SUMMARY OF THE INVENTION

The main technical problem to be solved by the present application is to provide an engineering operation detection method and a related device, which can improve the engineering operation detection quality.

In order to solve the above problem, a first aspect of the present application provides an engineering work detection method, which includes: acquiring an original image captured by a camera device on a work site, wherein the original image includes a preset detection area; performing target detection on the original image, The target area corresponding to the target object in the original image is acquired, wherein the target object is used to realize the warning; based on the positional relationship between the preset detection area and the target area, it is determined whether the operation site conforms to the operation specification.

In order to solve the above problem, a second aspect of the present application provides a device for detecting engineering work, including a memory processor coupled to each other, and the processor is configured to execute program instructions stored in the memory, so as to realize the engineering work detection in the first aspect. method.

In order to solve the above problem, a third aspect of the present application provides a storage device that stores program instructions that can be run by a processor, and the program instructions are used to implement the engineering work detection method in the first aspect.

In the above scheme, the original image captured by the camera device on the job site is obtained, and the original image includes a preset detection area, so as to perform target detection on the original image, and obtain the target area in the original image corresponding to the target object used for realizing the warning, Then, based on the positional relationship between the preset detection area and the target area, it is determined whether the job site complies with the job specification, so that whether the job site complies with the job specification can be detected based on the original image captured by the camera device on the job site without relying on manual labor. By inspecting the operation site, the inspection efficiency can be improved, the probability of omissions can be reduced, and the inspection quality of the engineering operation can be improved.

Description of drawings

1 is a schematic flowchart of an embodiment of an engineering work detection method of the present application;

FIG. 2 is a schematic flowchart of an embodiment of step S12 in FIG. 1;

3 is a schematic diagram of an embodiment of an acquisition process of the first integral graph in FIG. 2;

4 is a schematic diagram of an embodiment of morphological processing in FIG. 2;

FIG. 5 is a schematic flowchart of an embodiment of step S123 in FIG. 2;

6 is a schematic diagram of a framework of an embodiment of the engineering work detection device of the present application;

FIG. 7 is a schematic frame diagram of another embodiment of the engineering action detection device of the present application;

FIG. 8 is a schematic diagram of a framework of an embodiment of a storage device of the present application.

Detailed ways

The solutions of the embodiments of the present application will be described in detail below with reference to the accompanying drawings.

In the following description, for purposes of illustration and not limitation, specific details such as specific system structures, interfaces, techniques, etc. are set forth in order to provide a thorough understanding of the present application.

The terms "system" and "network" are often used interchangeably herein. The term "and/or" in this article is only an association relationship to describe the associated objects, indicating that there can be three kinds of relationships, for example, A and/or B, it can mean that A exists alone, A and B exist at the same time, and A and B exist independently B these three cases. In addition, the character "/" in this document generally indicates that the related objects are an "or" relationship. Also, "multiple" herein means two or more than two.

Please refer to FIG. 1 . FIG. 1 is a schematic flowchart of an embodiment of the engineering operation detection method of the present application. Specifically, the following steps can be included:

Step S11: Acquire an original image captured by the camera device on the job site, wherein the original image includes a preset detection area.

In this embodiment, the camera device may include a surveillance camera such as a ball camera, a card camera, etc., which is not specifically limited in this embodiment. In practical applications, there may be multiple areas to be detected on the job site. In this case, in order to enable the camera device to automatically photograph multiple areas to be detected on the job site, the pose parameters set by the user on the camera device can be obtained. , so that multiple preset positions are set for the camera device, so that the camera device can photograph the job site at multiple preset positions, thereby covering the entire job site. Taking power maintenance as an example, the camera device can be set in the northeast corner of the power machine room, and during inspection, it is necessary to take pictures of the southeast, northwest and southwest regions of the power machine room. Therefore, three preset positions can be set for the camera device. In the process of engineering operation detection, the camera device is made to shoot respectively toward the southeast area, southwest area, and northwest area of the power machine room, so as to cover the operation site.

In this embodiment, the preset detection area is an area preset by the user for engineering work detection. Specifically, the preset detection area can be set as a rectangular area, and the user can set the coordinates of the rectangular area in the image when setting. In a specific implementation scenario, when the imaging device is set with multiple preset positions, multiple preset detection areas may also be set, and correspond to the preset positions one-to-one. Still taking the power maintenance as an example, when the camera devices are respectively equipped with the above three preset positions, so that the camera devices can be respectively photographed towards the southeast, southwest and northwest areas of the power room during the inspection process of the project. The live areas are set for the above three preset positions, so that it can be determined whether the power maintenance site conforms to the operation specification based on the positional relationship between the target object obtained by subsequent detection and the preset live areas.

In addition, similar settings can also be performed in other engineering applications such as construction engineering and communication engineering, which will not be illustrated one by one in this embodiment.

Step S12: Perform target detection on the original image, and obtain a target area corresponding to the target object in the original image.

In this embodiment, the target object is used to realize the warning. Taking power maintenance as an example, the target object may be a red cloth curtain. In other engineering operations, the target object may also be other warning objects. For example, in a construction project, the target object may also be a warning line, etc.; in a communication project, the target object may also be a warning slogan, etc. No more examples.

In this embodiment, the specific method of target detection may be a detection method based on a neural network. For example, training a preset neural network with a training image marked with a target object to obtain a trained preset neural network, and then using the trained preset neural network to detect the original image to obtain a target corresponding to the target object area. Alternatively, the specific method of target detection can also be a detection method based on traditional image analysis. For example, the color features of each pixel in the original image are analyzed to obtain pixels similar to the color features of the target object, and then these pixels are analyzed. The point is subjected to noise reduction and other processing, and finally the minimum circumscribed rectangle of the pixel point after the noise reduction and other processing is used as the target area corresponding to the target object. There is no specific limitation in this embodiment.

Step S13: Based on the positional relationship between the preset detection area and the target area, determine whether the job site complies with the job specification.

In an implementation scenario, when there is an overlapping area between the preset detection area and the target area, it can be considered that a target object is set in the preset detection area, and it is determined that the job site complies with the job specification; otherwise, it is determined that the job site does not meet the job specification. specification.

In another implementation scenario, when the preset detection area completely includes the target area, it can be considered that a target object is set in the preset detection area, and it is determined that the job site conforms to the operation specification; otherwise, it is determined that the operation site does not conform to the operation specification.

In yet another implementation scenario, only when the IoU (Intersectionover Union, IoU) between the preset detection area and the target area is greater than a preset intersection over union ratio threshold (for example: 0.5) can it be considered that the preset detection area is set If there is a target object, it is determined that the job site conforms to the operation specification; otherwise, it is determined that the operation site does not conform to the operation specification.

In a specific implementation scenario, when it is determined that the job site does not meet the job specification, preset alarm information can also be output. In addition, when it is determined that the work site complies with the work specification, preset safety information may be output, or no information may be output. The above-mentioned preset alarm information and preset security information may include, but are not limited to, text, sound, image, etc., which will not be exemplified one by one in this embodiment. In addition, when the target detection is performed on the original image, but the target area corresponding to the target object is not detected, the preset alarm information can be directly output, so as to realize the warning that the operation site does not meet the operation specification.

In the above scheme, the original image captured by the camera device on the job site is obtained, and the original image includes a preset detection area, so as to perform target detection on the original image, and obtain the target area in the original image corresponding to the target object used for realizing the warning, Then, based on the positional relationship between the preset detection area and the target area, it is determined whether the job site complies with the job specification, so that whether the job site complies with the job specification can be detected based on the original image captured by the camera device on the job site without relying on manual labor. By inspecting the operation site, the inspection efficiency can be improved, the probability of omissions can be reduced, and the inspection quality of the engineering operation can be improved.

Please refer to FIG. 2 , which is a schematic flowchart of an embodiment of step S12 in FIG. 1 . Specifically include the following steps:

Step S121: Using a preset threshold value related to the color feature of the target object, perform threshold segmentation on the original image to obtain an image to be detected.

In a specific implementation scenario, taking power maintenance as an example, and the target object is a red cloth curtain, a preset threshold related to the color characteristics of the red cloth curtain can be used to perform threshold segmentation on the original image to obtain an image to be detected. In other application scenarios, it can be deduced by analogy, and this embodiment will not give examples one by one here.

In addition, since the RGB (Red Green Blue, red green blue) color space is very different from the perception of human eyes, in order to make the definition of color distance conform to human visual characteristics, in this embodiment, before performing threshold segmentation, the original image can also be The color space is mapped to the HSV (Hue Saturation Value, Hue/Saturation/Brightness) color space. In a specific implementation scenario, when the color space of the original image is the RGB color space, the color space of the original image can be mapped to the HSV color space by the following formula:

In the above formula, (R, G, B) represents the R channel pixel value, G channel pixel value, and B channel pixel value of a certain pixel in the original image.

In addition, further, the above-mentioned (H, S, V) may also be mapped to an interval of 0 to 255, which is not repeated in this embodiment. In a specific implementation scenario, the above-mentioned preset thresholds may include: a preset H channel threshold interval (for example: 156-180), a preset S channel threshold interval (for example: 80-255), and a preset V channel threshold interval (For example: 50 to 255), in other implementation scenarios, the above value may be other values, which are not specifically limited in this embodiment. Using the above preset thresholds, it is possible to sequentially determine whether each pixel in the original image mapped to the HSV color space satisfies the following conditions: whether the pixel value of the H channel is within the preset H channel threshold range, and whether the pixel value of the S channel is within the preset Within the S channel threshold range, and whether the V channel pixel value is within the preset V channel threshold range, if so, set the pixel value of the corresponding pixel in the image to be detected as the first pixel value (for example: 255), otherwise, Set the pixel value of the corresponding pixel in the image to be detected as the second pixel value (for example: 0). Specifically, the pixel value of the pixel point related to the color feature of the target object in the image to be detected is the first pixel value, and the pixel value of the pixel point unrelated to the color feature of the target object is the second pixel value.

Step S122: Count the sum of the pixel values of all pixel points in the upper left area of each pixel point in the image to be detected, as the pixel value of the corresponding pixel point in the first integral image corresponding to the image to be detected.

Please refer to FIG. 3 , which is a schematic diagram of an embodiment of an acquisition process of the first integral graph in FIG. 2 . As shown in Figure 3, P ₁ is the image to be detected, P ₁ (i, j) is the pixel value of the pixel at the i-th row and the j-th column of the to-be-detected image, Q ₁ is the first integral image, Q ₁ ( i,j) is the pixel value of the pixel located in the i-th row and the j-th column of the first integral image. The first integral image can be obtained based on the image to be detected by the following formula:

Step S123: Perform morphological processing on the image to be detected by using the preset structural element and the first integral image to obtain a target area corresponding to the target object in the original image.

The preset structural element in this embodiment is a two-dimensional matrix, the value of the matrix element is 1, and the size may be 15*15, or 13*13, or 11*11, or 9*9. The embodiments are not specifically limited herein.

In one implementation scenario, the image to be processed can be directly morphologically processed by using preset structural elements. For example, when using a preset structural element to perform corrosion morphological processing on an image to be processed, the atoms (ie, the center point) of the preset structural element can be used to traverse the target pixel point in the image to be processed whose pixel value is the first pixel value. Assume that the pixel value of other pixels in the size area of the structural element is the same as the pixel value of the target pixel, that is, the pixel value of other pixels is the first pixel value, then the pixel value of the target pixel is retained, otherwise, the target pixel value is The pixel value of the pixel point is set to the second pixel value. Please refer to FIG. 4. FIG. 4 is a schematic diagram of an embodiment of morphological processing in FIG. 2. Specifically, FIG. 4 is a schematic diagram of an embodiment of etching morphological processing using preset structural elements. As shown in FIG. Suppose the size of the structural element is k*k, the size of the image to be detected is w*h, and the image to be detected after the corrosion morphological processing is shown in Figure 4, where the slashed shaded pixels indicate that the pixel value is the first A target pixel of one pixel value, as shown in Figure 4, undergoes corrosion morphological processing, which can eliminate burrs in the image to be processed. However, the above-mentioned algorithm complexity of directly using the preset structural elements to perform corrosion morphological processing on the image to be processed is O(w*h*k*k), the complexity is high, and the processing load and resource overhead are high.

In another implementation scenario, in order to reduce algorithm complexity, reduce processing load, and reduce resource overhead, the above-mentioned first integral image may be used to perform morphological processing on the image to be processed. Specifically, please refer to FIG. 5 , which is a schematic flowchart of an embodiment of step S123 in FIG. 2 , which may specifically include:

Step S1231 : perform corrosion morphological processing on the image to be detected by using the preset structural element and the first integral image to obtain a first processed image.

Specifically, the first target pixel with the pixel value of the first pixel value in the image to be detected can be determined, and the second target pixel corresponding to the first target pixel in the first integral image can be determined, based on the second target pixel and the pre- Set the size of the element structure, determine the third target pixel in the first integral image, and then use the pixel value of the third target pixel to calculate the sum of the pixel values of all pixels within the size range around the first target pixel , if the quotient of the calculated sum of pixel values and the size is the first pixel value, it can be considered that the first target pixel is not a burr/noise, then the pixel value of the first target pixel is kept as the first pixel value, If the quotient of the sum of the pixel values and the size is not the first pixel value, it can be considered that the first target pixel is a burr/noise, and the pixel value of the first target pixel is reset to the second pixel value.

确定第一个第三目标像素点，其像素值为

表示像素点

左上角区域内所有像素点的像素值之和，再确定另外三个第三目标像素点

故第一目标像素点(i,j)周围尺寸大小(k*k)范围内所有像素点的像素值之和可以表示为：In a specific implementation scenario, please continue to refer to FIG. 3. As shown in FIG. ₃ , in the image to be detected P1, the pixel point (i, j) is the first target pixel point, correspondingly, the first integral image Q In ₁ , the pixel point (i, j) is the second target pixel point corresponding to the first target pixel point, and the thick black frame area is the size (k*k) area of the preset structural element, in order to calculate the image to be detected. The sum of the pixel values of the pixel points in the size area of the preset structural element in P ₁ can be calculated by using the first integral image Q _1. During the calculation, the size of the preset structural element can be used first to determine the first integral For the third target pixel point in the image Q ₁ , firstly, the pixel point in the lower right corner of the size area of the preset structural element of the first integral image Q ₁ can be

Determine the first and third target pixel, whose pixel value is

Represents a pixel

The sum of the pixel values of all pixels in the upper left area, and then determine the other three third target pixels

Therefore, the sum of the pixel values of all pixels in the range of the size (k*k) around the first target pixel (i, j) can be expressed as:

)，若商为第一像素值，则说明待检测图像P₁在第一目标像素点(i,j)尺寸大小(k*k)范围内所有像素点的像素值均为第一像素值，第一目标像素点(i,j)可以认为并非毛刺/噪点，此时可以保持待检测图像P₁中第一目标像素点(i,j)的像素值不变，否则，说明第一目标像素点(i,j)为毛刺/噪点，并将待检测图像P₁中第一目标像素点(i,j)的像素值重置为第二像素值。After calculating the sum S of pixel values of all pixels in the range of size (k*k) around the first target pixel (i, j), the sum S of pixel values and the size (k*k) can be further calculated. quotient (ie

), if the quotient is the first pixel value, it means that the pixel values of all the pixel points of the image to be detected P1 in the range of the size (k*k) of the _first target pixel point (i, j) are the first pixel value, The first target pixel point (i, j) can be considered as not a burr/noise point, at this time, the pixel value of the first target pixel point (i, j) in the image P ₁ to be detected can be kept unchanged, otherwise, it is explained that the first target pixel point Point (i, j) is a burr/noise point, and the pixel value of the first target pixel point (i, j) in the image P ₁ to be detected is reset to the second pixel value.

By using the integral graph for corrosion morphology processing, the algorithm complexity can be reduced to O(w*h), and the algorithm complexity is significantly reduced.

Step S1232: Count the sum of the pixel values of all pixel points in the upper left area of each pixel point in the first processed image as the pixel value of the corresponding pixel point in the second integral image corresponding to the first processed image.

In one implementation scenario, after the image to be processed is subjected to erosion morphological processing, dilation morphological processing may be further continued. Similar to the above steps, all pixels in the upper left area of each pixel in the first processed image may be counted The sum of the pixel values of the points is taken as the pixel value of the corresponding pixel point in the second integral image corresponding to the first processed image. For details, reference may be made to the above-mentioned relevant steps in this embodiment, which will not be repeated here.

Step S1233 : performing dilation morphological processing on the first processed image by using the preset structural element and the second integral image to obtain a second processed image.

Specifically, each pixel in the first processed image can be used as the fourth target pixel in turn, and the fifth target pixel corresponding to the fourth target pixel in the second integral image can be determined, based on the fifth target pixel and The size of the preset structural element is determined, the sixth target pixel in the second integral image is determined, and the pixel value of the sixth target pixel is used to calculate the sum of the pixel values of all the pixels within the size range around the fourth target pixel. and, if the quotient of the sum of the pixel values and the size is the second pixel value, set the pixel value of the fourth target pixel to the second pixel value, if the quotient of the sum of the pixel values and the size is not the second pixel value, the pixel value of the fourth target pixel is set to the first pixel value. Wherein, based on the size of the fifth target pixel point and the preset structural element, the sixth target pixel point in the second integral image is determined, and the pixel value of the sixth target pixel point is used to calculate the size around the fourth target pixel point For the specific manner of the sum of the pixel values of all the pixel points within the range, reference may be made to the foregoing step S1231 in this embodiment, which will not be repeated here. Different from erosion morphological processing, the purpose of dilation morphological processing is to expand edges and fill holes, so as long as there are pixels with a pixel value of the first pixel value within the size range of the preset structural element, that is, as long as the pixel value The quotient of the sum and the size is not the second pixel value, then the pixel value of the fourth target pixel is set to the first pixel value, otherwise, if the pixel value is the first pixel value within the size range of the preset structural element A pixel of one pixel value, that is, if the quotient of the sum of the pixel values and the size is the second pixel value, then the pixel value of the fourth target pixel is set to the second pixel value. In this way, the edge can be significantly expanded, and fill the voids inside.

Step S1234: Determine the pixel value of each pixel in the second processed image as a target connected domain of the first pixel value.

After the to-be-processed image is subjected to corrosion morphological processing and dilation morphological processing, a second processed image is obtained. At this time, if the pixel value of a certain pixel in the second processed image and the pixel value of its neighboring pixels are the first If the pixel value is one pixel, the pixel and its neighboring pixels are divided into the same connected domain, and all the pixels in the second processed image are traversed to obtain at least one target connected domain.

Step S1235: Obtain the minimum circumscribed rectangle of the target connected domain as the target area.

Take the minimum enclosing rectangle of the target connected domain as the target area.

Different from the foregoing embodiments, the original image is subjected to threshold segmentation by using a preset threshold related to the color characteristics of the target object to obtain the image to be detected, and the pixel values of all pixels in the upper left area of each pixel in the image to be detected are counted. The sum is taken as the pixel value of the corresponding pixel point in the first integral image corresponding to the image to be detected, so that the image to be detected is morphologically processed by using the preset structural element and the first integral image to obtain the corresponding pixel value of the target object in the original image. The target area can reduce the complexity of the algorithm and shorten the processing time, thereby speeding up the speed of target detection, which is conducive to real-time detection of whether the engineering operation is standardized, and timely alarm when it is not standardized.

Please refer to FIG. 6 . FIG. 6 is a schematic diagram of a framework of an embodiment of the engineering operation detection device 60 of the present application. The engineering work detection device 60 includes an acquisition module 61, a detection module 62 and a determination module 63. The acquisition module 61 is used to acquire the original image captured by the camera device on the work site, wherein the original image includes a preset detection area, and the detection module 62 is used for Perform target detection on the original image, and obtain the target area corresponding to the target object in the original image, wherein the target object is used to realize the warning, and the determination module 63 is used to determine the job site based on the positional relationship between the preset detection area and the target area Compliance with work specifications.

In the above scheme, the original image captured by the camera device on the job site is obtained, and the original image includes a preset detection area, so as to perform target detection on the original image, and obtain the target area in the original image corresponding to the target object used for realizing the warning, Then, based on the positional relationship between the preset detection area and the target area, it is determined whether the job site complies with the job specification, so that whether the job site complies with the job specification can be detected based on the original image captured by the camera device on the job site without relying on manual labor. By inspecting the operation site, the inspection efficiency can be improved, the probability of omissions can be reduced, and the inspection quality of the engineering operation can be improved.

In some embodiments, the detection module 62 includes a threshold segmentation sub-module for performing threshold segmentation on the original image by using a preset threshold related to the color feature of the target object to obtain an image to be detected, and the detection module 62 further includes an integral statistics sub-module The module is used to count the sum of the pixel values of all pixels in the upper left area of each pixel in the image to be detected, as the pixel value of the corresponding pixel in the first integral image corresponding to the image to be detected. The detection module 62 also includes a form The morphological processing sub-module is used to perform morphological processing on the image to be detected by using the preset structural element and the first integral image to obtain the target area corresponding to the target object in the original image.

Different from the foregoing embodiments, the original image is subjected to threshold segmentation by using a preset threshold related to the color characteristics of the target object to obtain the image to be detected, and the pixel values of all pixels in the upper left area of each pixel in the image to be detected are counted. The sum is taken as the pixel value of the corresponding pixel point in the first integral image corresponding to the image to be detected, so that the image to be detected is morphologically processed by using the preset structural element and the first integral image to obtain the corresponding pixel value of the target object in the original image. The target area can reduce the complexity of the algorithm and shorten the processing time, thereby speeding up the speed of target detection, which is conducive to real-time detection of whether the engineering operation is standardized, and timely alarm when it is not standardized.

In some embodiments, the pixel value of the pixel point related to the color feature of the target object in the image to be detected is the first pixel value, and the pixel value of the pixel point unrelated to the color feature of the target object is the second pixel value. The processing sub-module includes a corrosion processing unit, which is used to perform corrosion morphological processing on the image to be detected by using the preset structural element and the first integral image to obtain the first processed image, and the integral statistics sub-module is also used to count each pixel in the first processed image. The sum of the pixel values of all pixel points in the upper left area of the point is taken as the pixel value of the corresponding pixel point in the second integral image corresponding to the first processed image, and the morphological processing sub-module also includes an expansion processing unit for using the preset The structural element and the second integral image perform dilation morphological processing on the first processed image to obtain a second processed image. The morphological processing sub-module further includes a connected domain determination unit, which is used to determine the pixel value of each pixel in the second processed image. is the target connected domain of the first pixel value, and the morphological processing sub-module further includes a target area determination unit, which is used for obtaining the minimum circumscribed rectangle of the target connected domain as the target area.

In some embodiments, the corrosion processing unit is specifically configured to determine a first target pixel with a pixel value of a first pixel value in the image to be detected, and to determine a second target pixel corresponding to the first target pixel in the first integral image point, based on the size of the second target pixel point and the preset element structure, determine the third target pixel point in the first integral image, and use the pixel value of the third target pixel point to calculate the size range around the first target pixel point The sum of the pixel values of all the pixels within, if the quotient of the sum of the pixel values and the size is the first pixel value, the pixel value of the first target pixel is kept as the first pixel value, if the sum of the pixel values and the size If the size quotient is not the first pixel value, the pixel value of the first target pixel is reset to the second pixel value.

In some embodiments, the expansion processing unit is specifically configured to sequentially take each pixel in the first processed image as the fourth target pixel, and determine the fifth target pixel corresponding to the fourth target pixel in the second integral image , based on the size of the fifth target pixel point and the preset structural element, determine the sixth target pixel point in the second integral image, and use the pixel value of the sixth target pixel point to calculate the size range around the fourth target pixel point The sum of the pixel values of all pixels, if the quotient of the sum of the pixel values and the size is the second pixel value, then the pixel value of the fourth target pixel is set to the second pixel value, if the sum of the pixel values and the size If the quotient is not the second pixel value, the pixel value of the fourth target pixel is set as the first pixel value.

In some embodiments, the detection module 62 further includes a color space mapping sub-module for mapping the color space of the original image to the HSV color space, and the preset thresholds include: a preset H channel threshold interval, a preset S channel threshold interval, and Preset the V channel threshold interval, the threshold segmentation sub-module includes a judgment unit, used to sequentially determine whether each pixel in the original image mapped to the HSV color space satisfies the following conditions: the H channel pixel value is within the preset H channel threshold interval, And the pixel value of the S channel is within the preset S channel threshold value range, and the V channel pixel value is within the preset V channel threshold value range, the threshold segmentation sub-module further includes a pixel value setting unit, for when the judgment unit judges that the condition is satisfied, the The pixel value of the corresponding pixel in the image to be detected is set as the first pixel value, and the pixel value setting unit is further configured to set the pixel value of the corresponding pixel in the image to be detected as the second pixel value when the judgment unit determines that the condition is not satisfied .

In some embodiments, the determination module 63 includes a coincidence area judging submodule for judging whether there is a coincidence area between the preset detection area and the target area, and the determination module 63 further includes a job determination submodule for judging the submodule in the coincidence area. When the module judges that there is an overlapping area, it determines that the job site complies with the operation specification, and the operation determination sub-module is also used to determine that the operation site does not meet the operation specification when the overlapping area judging sub-module judges that there is no overlapping area.

In some embodiments, the determination module 63 further includes an information output sub-module, configured to output preset alarm information when the job determination sub-module determines that the job site does not meet the job specification, and the information output sub-module is further configured to output the information in the job determination sub-module When it is determined that the job site complies with the job specification, preset safety information is output.

In some embodiments, the engineering work detection apparatus 60 further includes a setting module for acquiring pose parameters set by the user on the camera device, wherein the pose parameters are used to control the camera device to photograph the work site.

Please refer to FIG. 7 . FIG. 7 is a schematic diagram of a framework of an embodiment of the engineering operation detection device 70 of the present application. The engineering work detection device 70 may include a mutually coupled memory 71 and a processor 72; the processor 72 is configured to execute program instructions stored in the memory 71, so as to implement the steps in any of the foregoing engineering work detection method embodiments.

Specifically, the processor 72 is used to control itself and the memory 71 to implement the steps in any of the above-mentioned embodiments of the engineering work detection method. The processor 72 may also be referred to as a CPU (Central Processing Unit, central processing unit). The processor 72 may be an integrated circuit chip with signal processing capability. The processor 72 may also be a general-purpose processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable Logic devices, discrete gate or transistor logic devices, discrete hardware components. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like. In addition, the processor 72 may be implemented jointly by a plurality of integrated circuit chips.

In this embodiment, the processor 72 is configured to acquire the original image captured by the camera device on the job site, wherein the original image includes a preset detection area, and the processor 72 is further configured to perform target detection on the original image, and obtain the The target area corresponding to the target object, wherein the target object is used to realize the warning, and the processor 72 is further configured to determine whether the job site conforms to the job specification based on the positional relationship between the preset detection area and the target area.

In the above scheme, the original image captured by the camera device on the job site is obtained, and the original image includes a preset detection area, so as to perform target detection on the original image, and obtain the target area in the original image corresponding to the target object used for realizing the warning, Then, based on the positional relationship between the preset detection area and the target area, it is determined whether the job site complies with the job specification, so that whether the job site complies with the job specification can be detected based on the original image captured by the camera device on the job site without relying on manual labor. By inspecting the operation site, the inspection efficiency can be improved, the probability of omissions can be reduced, and the inspection quality of the engineering operation can be improved.

In some embodiments, the processor 72 is further configured to perform threshold segmentation on the original image by using a preset threshold related to the color feature of the target object to obtain an image to be detected, and the processor 72 is further configured to count each pixel in the image to be detected The sum of the pixel values of all the pixel points in the upper left area of the point is used as the pixel value of the corresponding pixel point in the first integral image corresponding to the image to be detected, and the processor 72 is also used to use the preset structural element and the first integral image to treat Morphological processing is performed on the detected image to obtain the target area corresponding to the target object in the original image.

Different from the foregoing embodiments, the original image is subjected to threshold segmentation by using a preset threshold related to the color characteristics of the target object to obtain the image to be detected, and the pixel values of all pixels in the upper left area of each pixel in the image to be detected are counted. The sum is taken as the pixel value of the corresponding pixel point in the first integral image corresponding to the image to be detected, so that the image to be detected is morphologically processed by using the preset structural element and the first integral image to obtain the corresponding pixel value of the target object in the original image. The target area can reduce the complexity of the algorithm and shorten the processing time, thereby speeding up the speed of target detection, which is conducive to real-time detection of whether the engineering operation is standardized, and timely alarm when it is not standardized.

In some embodiments, the pixel value of the pixel point related to the color feature of the target object in the image to be detected is the first pixel value, the pixel value of the pixel point unrelated to the color feature of the target object is the second pixel value, and the processor 72 is also used to perform corrosion morphological processing on the image to be detected by using the preset structural element and the first integral image to obtain a first processed image, and the processor 72 is also used to count all the pixels in the upper left area of each pixel in the first processed image The sum of the pixel values of the points, as the pixel value of the corresponding pixel point in the second integral image corresponding to the first processed image, the processor 72 is further configured to expand the first processed image by using the preset structural element and the second integral image Morphological processing to obtain a second processed image, the processor 72 is also used to determine the target connected domain whose pixel value of each pixel in the second processed image is the first pixel value, and the processor 72 is also used to obtain the minimum value of the target connected domain. The enclosing rectangle is used as the target area.

In some embodiments, the processor 72 is further configured to determine a first target pixel whose pixel value is the first pixel value in the image to be detected, and determine a second target pixel corresponding to the first target pixel in the first integral image point, the processor 72 is further configured to determine the third target pixel point in the first integral image based on the size of the second target pixel point and the preset element structure, and the processor 72 is further configured to utilize the pixels of the third target pixel point value, calculate the sum of the pixel values of all the pixel points in the size range around the first target pixel point, the processor 72 is further configured to calculate the first target pixel value when the quotient of the sum of the pixel values and the size is the first pixel value The pixel value of the point remains the first pixel value, and the processor 72 is further configured to reset the pixel value of the first target pixel point to the second pixel when the quotient of the sum of the pixel values and the size is not the first pixel value value.

In some embodiments, the processor 72 is further configured to sequentially use each pixel in the first processed image as the fourth target pixel, and determine a fifth target pixel corresponding to the fourth target pixel in the second integral image , the processor 72 is further configured to determine the sixth target pixel in the second integral image based on the size of the fifth target pixel and the preset structural element, and the processor 72 is further configured to utilize the pixel value of the sixth target pixel , calculate the sum of the pixel values of all the pixel points in the size range around the fourth target pixel point, and the processor 72 is further configured to calculate the fourth target pixel point when the quotient of the sum of the pixel values and the size is the second pixel value The pixel value of the pixel is set as the second pixel value, and the processor 72 is further configured to set the pixel value of the fourth target pixel as the first pixel value when the quotient of the sum of the pixel values and the size is not the second pixel value.

In some embodiments, the processor 72 is further configured to map the color space of the original image to the HSV color space, and the preset thresholds include: a preset H channel threshold interval, a preset S channel threshold interval, and a preset V channel threshold interval, The processor 72 is further configured to sequentially determine whether each pixel in the original image mapped to the HSV color space satisfies the following conditions: the pixel value of the H channel is within the preset H channel threshold interval, and the pixel value of the S channel is within the preset S channel threshold range, and the V channel pixel value is within the preset V channel threshold value range, the processor 72 is further configured to set the pixel value of the corresponding pixel point in the image to be detected as the first pixel value when the condition is met, and the processor 72 is further When the condition is not satisfied, the pixel value of the corresponding pixel in the image to be detected is set as the second pixel value.

In some embodiments, the processor 72 is further configured to determine whether there is an overlapping area between the preset detection area and the target area, and when the determination is yes, the processor 72 is further configured to determine that the job site complies with the job specification, and the processor 72 is further configured to It is used to determine that the work site does not meet the work specification when the judgment is no.

In some embodiments, the engineering work detection device 70 further includes a human-computer interaction circuit for outputting preset alarm information when the processor 72 determines that the work site does not meet the work specification. The human-computer interaction circuit is further configured to output preset safety information when the processor 72 determines that the job site conforms to the job specification.

In some embodiments, the processor 72 is further configured to control the human-computer interaction circuit to obtain pose parameters set by the user on the camera device, wherein the pose parameters are used to control the camera device to photograph the job site.

In some embodiments, the engineering work detection device 70 further includes a camera device, which is used for photographing the work site to obtain original images.

Please refer to FIG. 8 , which is a schematic diagram of a framework of an embodiment of a storage device 80 of the present application. The storage device 80 stores program instructions 81 that can be executed by the processor, and the program instructions 81 are used to implement the steps in the foregoing embodiments of the method for detecting any engineering work.

The above solution can detect whether the job site complies with the operation specifications based on the original image captured by the camera device on the job site, without relying on manual inspection of the job site, thereby improving the detection efficiency, reducing the probability of omissions, and improving engineering operations. Check quality.

In the several embodiments provided in this application, it should be understood that the disclosed method and apparatus may be implemented in other manners. For example, the apparatus implementations described above are only illustrative, for example, the division of modules or units is only a logical function division, and there may be other divisions in actual implementation, for example, multiple units or components may be combined or Can be integrated into another system, or some features can be ignored, or not implemented. On the other hand, the shown or discussed mutual coupling or direct coupling or communication connection may be through some interfaces, indirect coupling or communication connection of devices or units, which may be in electrical, mechanical or other forms.

Units described as separate components may or may not be physically separated, and components shown as units may or may not be physical units, that is, may be located in one place, or may be distributed to multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution in this implementation manner.

In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically alone, or two or more units may be integrated into one unit. The above-mentioned integrated units may be implemented in the form of hardware, or may be implemented in the form of software functional units.

The integrated unit, if implemented as a software functional unit and sold or used as a stand-alone product, may be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the present application can be embodied in the form of software products in essence, or the parts that contribute to the prior art, or all or part of the technical solutions, and the computer software products are stored in a storage medium , including several instructions to make a computer device (which may be a personal computer, a server, or a network device, etc.) or a processor (processor) to execute all or part of the steps of the methods of the various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, Read-Only Memory (ROM, Read-Only Memory), Random Access Memory (RAM, Random Access Memory), magnetic disk or optical disk and other media that can store program codes .

Claims (12)

1. An engineering operation detection method is characterized by comprising the following steps:

acquiring an original image shot by a camera device on a working site, wherein the original image comprises a preset detection area;

performing target detection on the original image, and acquiring a target area corresponding to a target object in the original image, wherein the target object is used for realizing warning;

and determining whether the operation site meets the operation specification or not based on the position relation between the preset detection area and the target area.

2. The project work detection method according to claim 1, wherein the performing the target detection on the original image and acquiring the target area corresponding to the target object in the original image comprises:

performing threshold segmentation on the original image by using a preset threshold related to the color feature of the target object to obtain an image to be detected;

counting the sum of pixel values of all pixel points in the upper left area of each pixel point in the image to be detected, and taking the sum as the pixel value of the corresponding pixel point in the first integral image corresponding to the image to be detected;

and performing morphological processing on the image to be detected by using a preset structural element and the first integral image to obtain a target area corresponding to the target object in the original image.

3. The engineering work detection method according to claim 2, wherein a pixel value of a pixel point related to the color feature of the target object in the image to be detected is a first pixel value, and a pixel value of a pixel point unrelated to the color feature of the target object is a second pixel value;

the morphological processing of the image to be detected by using a preset structural element and the first integral image to obtain a target area corresponding to the target object in the original image comprises the following steps:

carrying out corrosion morphological processing on the image to be detected by utilizing the preset structural element and the first integral image to obtain a first processed image;

counting the sum of pixel values of all pixel points in the upper left area of each pixel point in the first processed image, and taking the sum as the pixel value of the corresponding pixel point in the second integral image corresponding to the first processed image;

performing dilation morphological processing on the first processed image by using the preset structural element and the second integral image to obtain a second processed image;

determining the pixel value of each pixel point in the second processed image as a target connected domain of the first pixel value;

and acquiring the minimum circumscribed rectangle of the target connected domain as the target area.

4. The engineering work detection method according to claim 3, wherein the performing corrosion morphological processing on the image to be detected by using the preset structural element and the first integral image to obtain a first processed image comprises:

determining a first target pixel point of which the pixel value in the image to be detected is the first pixel value, and determining a second target pixel point corresponding to the first target pixel point in the first integral image;

determining a third target pixel point in the first integral image based on the second target pixel point and the size of the preset element structure;

calculating the sum of the pixel values of all the pixels in the size range around the first target pixel point by using the pixel value of the third target pixel point;

if the quotient of the sum of the pixel values and the size is the first pixel value, keeping the pixel value of the first target pixel point as the first pixel value;

if the quotient of the sum of the pixel values and the size is not the first pixel value, the pixel value of the first target pixel point is reset to the second pixel value.

5. The engineering work detection method according to claim 3, wherein the performing the dilation morphology on the first processed image using the preset structural element and the second integrated image to obtain a second processed image comprises:

sequentially taking each pixel point in the first processed image as a fourth target pixel point, and determining a fifth target pixel point corresponding to the fourth target pixel point in the second integral image;

determining a sixth target pixel point in the second integral image based on the size of the fifth target pixel point and the size of the preset structural element;

calculating the sum of the pixel values of all the pixels in the size range around the fourth target pixel point by using the pixel value of the sixth target pixel point;

setting the pixel value of the fourth target pixel as the second pixel value if the quotient of the sum of the pixel values and the size is the second pixel value;

and if the quotient of the sum of the pixel values and the size is not the second pixel value, setting the pixel value of the fourth target pixel point as the first pixel value.

6. The method for detecting engineering works according to claim 2, wherein before the threshold segmentation is performed on the original image by using the preset threshold related to the color feature of the target object to obtain the image to be detected, the method further comprises:

mapping the color space of the original image to an HSV color space;

the preset threshold includes: presetting an H channel threshold interval, an S channel threshold interval and a V channel threshold interval; the threshold segmentation is carried out on the original image by using a preset threshold related to the color feature of the target object to obtain an image to be detected, and the method comprises the following steps:

whether each pixel point in the original image mapped to the HSV color space meets the following conditions is sequentially judged: the H channel pixel value is within the preset H channel threshold interval, the S channel pixel value is within the preset S channel threshold interval, and the V channel pixel value is within the preset V channel threshold interval;

if so, setting the pixel value of the corresponding pixel point in the image to be detected as a first pixel value;

and if not, setting the pixel value of the corresponding pixel point in the image to be detected as a second pixel value.

7. The project work detection method according to claim 1, wherein the determining whether the work site meets a work specification based on the positional relationship between the preset detection area and the target area comprises:

judging whether a coincidence region exists between the preset detection region and the target region;

if so, determining that the operation site meets the operation specification;

if not, determining that the operation site does not meet the operation specification.

8. The project work detection method of claim 7, wherein after said determining that said work site is not compliant with a work specification, said method further comprises:

outputting preset alarm information;

and/or, after determining that the job site meets a job specification, the method further comprises:

preset security information is output, or no information is output.

9. The work order detection method according to claim 1, wherein before the obtaining of the original image taken of the work site by the imaging device, the method comprises:

and acquiring pose parameters set by a user on the camera device, wherein the pose parameters are used for controlling the camera device to shoot the operation site.

10. An engineering work detection device, comprising mutually coupled memory processors for executing program instructions stored in the memories to implement the engineering work detection method according to any one of claims 1 to 9.

11. The work detection apparatus of claim 10, further comprising an imaging device for capturing an original image of the work site.

12. A storage device storing program instructions executable by a processor to implement the method of any one of claims 1 to 9.

Images