CN111382726B - Engineering operation detection method and related device - Google Patents

Abstract

The application discloses an engineering operation detection method and a related device, wherein the engineering operation detection method comprises the following steps: acquiring an original image shot by an imaging device on a working site, wherein the original image comprises a preset detection area; performing target detection on the original image to obtain a target area corresponding to a target object in the original image, wherein the target object is used for realizing warning; and determining whether the operation field accords with the operation specification or not based on the position relation between the preset detection area and the target area. By means of the scheme, the engineering operation detection quality can be improved.

Description

Engineering operation detection method and related device

technical field

The present application relates to the technical field of image processing, in particular to an engineering operation detection method and a related device.

Background technique

In engineering operations, in order to warn workers on the job site and prevent workers from straying into dangerous areas to ensure job safety, warning objects such as warning signs are usually set up near the dangerous areas on the job site. Taking electric power maintenance as an example, due to factors such as the incomplete function of the "five preventions" of the switchgear or the distraction of the operators, dangerous phenomena such as operators entering the live compartment by mistake, opening and closing switches by mistake may occur. Therefore, during switching operations or maintenance operations, red curtains are generally used to warn operators, so that operators can clearly distinguish between power-off maintenance screen cabinets and adjacent live non-maintenance screen cabinets. It can be seen that warning objects such as red curtains and warning slogans play a very important role in ensuring operation safety.

At present, manual inspection is still used to determine whether the above-mentioned warnings are used in a standardized manner, so there is a problem of low efficiency. In addition, limited by factors such as human resources and the energy of inspectors, manual inspections will inevitably lead to omissions. All of these have reduced the quality of engineering work inspection. In view of this, how to improve the quality of engineering operation inspection has become an urgent problem to be solved.

Contents of the invention

The technical problem mainly solved by this application is to provide an engineering operation detection method and a related device, which can improve the quality of engineering operation detection.

In order to solve the above problems, the first aspect of the present application provides an engineering operation detection method, including: obtaining the original image taken by the camera device on the operation site, wherein the original image contains a preset detection area; performing target detection on the original image, Obtain the target area corresponding to the target object in the original image, wherein the target object is used to realize the warning; based on the positional relationship between the preset detection area and the target area, determine whether the job site conforms to the job specification.

In order to solve the above problems, the second aspect of the present application provides an engineering operation detection device, which includes a memory processor coupled to each other, and the processor is used to execute the program instructions stored in the memory, so as to realize the engineering operation detection in the above first aspect method.

In order to solve the above problems, the third aspect of the present application provides a storage device, which stores program instructions that can be executed by a processor, and the program instructions are used to implement the engineering operation detection method in the first aspect above.

In the above solution, by obtaining the original image taken by the camera device on the job site, and the original image contains a preset detection area, the target detection is performed on the original image, and the target area corresponding to the target object used to realize the warning is obtained in the original image, Then, based on the positional relationship between the preset detection area and the target area, it is determined whether the job site complies with the job specification, so that whether the job site meets the job specification can be detected based on the original image captured by the camera device, without relying on manual work. Check the job site, so as to improve the detection efficiency, reduce the probability of omissions, and improve the quality of engineering operation inspection.

Description of drawings

Fig. 1 is a schematic flow chart of an embodiment of the engineering operation detection method of the present application;

Fig. 2 is a schematic flow chart of an embodiment of step S12 in Fig. 1;

Fig. 3 is a schematic diagram of an embodiment of the acquisition process of the first integral map in Fig. 2;

Fig. 4 is a schematic diagram of an embodiment of morphological processing in Fig. 2;

Fig. 5 is a schematic flow chart of an embodiment of step S123 in Fig. 2;

Fig. 6 is a schematic frame diagram of an embodiment of the engineering operation detection device of the present application;

Fig. 7 is a schematic frame diagram of another embodiment of the engineering function detection device of the present application;

FIG. 8 is a schematic frame diagram of an embodiment of a storage device of the present application.

Detailed ways

The solutions of the embodiments of the present application will be described in detail below in conjunction with the accompanying drawings.

In the following description, for purposes of illustration rather than limitation, specific details, such as specific system architectures, interfaces, and techniques, are set forth in order to provide a thorough understanding of the present application.

The terms "system" and "network" are often used interchangeably herein. The term "and/or" in this article is just an association relationship describing associated objects, which means that there can be three relationships, for example, A and/or B can mean: A exists alone, A and B exist simultaneously, and there exists alone B these three situations. In addition, the character "/" in this article generally indicates that the contextual objects are an "or" relationship. In addition, "many" herein means two or more than two.

Please refer to FIG. 1 . FIG. 1 is a schematic flowchart of an embodiment of an engineering operation detection method of the present application. Specifically, the following steps may be included:

Step S11: Obtain the original image taken by the camera device on the job site, wherein the original image includes a preset detection area.

In this embodiment, the imaging device may include surveillance cameras such as ball cameras and card cameras, which are not specifically limited in this embodiment. In practical applications, there may be multiple areas that need to be detected on the job site. In this case, in order for the camera device to automatically capture multiple areas that need to be detected in the job site, the pose parameters set by the user for the camera device can be obtained , so as to set multiple preset positions for the imaging device, so that the imaging device can shoot the job site at the multiple preset positions, thereby covering the entire job site. Taking electric power maintenance as an example, the camera device can be set in the northeast corner of the power machine room, and during detection, it is necessary to take pictures of the southeast, northwest, and southwest areas of the power machine room. Therefore, three preset positions can be set for the camera device. During the detection process of the engineering operation, the imaging device can respectively shoot towards the southeast area, the southwest area, and the northwest area of the electric machine room, thereby being able to cover the operation site.

In this embodiment, the preset detection area is an area preset by the user that requires engineering work detection. Specifically, the preset detection area can be set as a rectangular area, and the coordinates of the rectangular area in the image can be set when the user sets. In a specific implementation scenario, when the imaging device is provided with multiple preset positions, there may also be multiple preset detection areas, which correspond to the preset positions one by one. Still taking electric power maintenance as an example, when the camera device is respectively provided with the above three preset positions, so that the camera device can shoot towards the southeast area, southwest area, and northwest area of the electric machine room during the engineering operation inspection process, it can be respectively The electrified area is set for the above three preset positions, so that based on the positional relationship between the target object obtained in subsequent detection and the pre-set electrified area, it can be determined whether the electric power maintenance site complies with the operation specification.

In addition, similar settings can also be performed in other engineering applications such as construction engineering and communication engineering, and this embodiment will not give examples one by one here.

Step S12: Perform target detection on the original image, and obtain the target area corresponding to the target object in the original image.

In this embodiment, the target object is used to realize the warning. Taking electric power maintenance as an example, the target object may be a red curtain. In other engineering operations, the target object can also be other warning objects, for example, in construction engineering, the target object can also be a warning line, etc.; in communication engineering, the target object can also be a warning slogan, etc., the present embodiment here No more examples.

In this embodiment, the specific manner of target detection may be a detection manner based on a neural network. For example, the preset neural network is trained by using the training images labeled with the target object to obtain the trained preset neural network, and then the trained preset neural network is used to detect the original image to obtain the target corresponding to the target object area. Alternatively, the specific method of target detection can also be a detection method based on traditional image analysis, for example, analyze the color characteristics of each pixel in the original image to obtain pixels similar to the color characteristics of the target object, and then analyze these pixels Points are subjected to noise reduction and other processing, and finally the minimum circumscribed rectangle of the pixel points after noise reduction and other processing is used as the target area corresponding to the target object. This embodiment does not make specific limitations here.

Step S13: Based on the positional relationship between the preset detection area and the target area, determine whether the job site complies with the job specification.

In an implementation scenario, when there is an overlapping area between the preset detection area and the target area, it can be considered that the target object is set in the preset detection area, and it is determined that the job site meets the job specifications; otherwise, it is determined that the job site does not meet the job specifications. specification.

In another implementation scenario, when the preset detection area completely includes the target area, it can be considered that the target object is set in the preset detection area, and then it is determined that the job site complies with the job specification; otherwise, it is determined that the job site does not meet the job specification.

In yet another implementation scenario, when the intersection over union (IoU) between the preset detection area and the target area is greater than a preset intersection over union threshold (for example: 0.5), it can be considered that the preset detection area is set If there is a target object, it is determined that the operation site conforms to the operation specification; otherwise, it is determined that the operation site does not meet the operation specification.

In a specific implementation scenario, when it is determined that the job site does not meet the job specifications, preset alarm information can also be output. In addition, when it is determined that the job site conforms to the job specifications, preset safety information may also be output, or no information may be output. The above-mentioned preset alarm information and preset security information may include but not limited to: text, sound, image, etc., and this embodiment will not give examples one by one here. In addition, when the target detection is performed on the original image, but the target area corresponding to the target object is not detected, the preset alarm information can be directly output, so as to realize the warning that the job site does not meet the job specifications.

In the above solution, by obtaining the original image taken by the camera device on the job site, and the original image contains a preset detection area, the target detection is performed on the original image, and the target area corresponding to the target object used to realize the warning is obtained in the original image, Then, based on the positional relationship between the preset detection area and the target area, it is determined whether the job site complies with the job specification, so that whether the job site meets the job specification can be detected based on the original image captured by the camera device, without relying on manual work. Check the job site, so as to improve the detection efficiency, reduce the probability of omissions, and improve the quality of engineering operation inspection.

Please refer to FIG. 2 . FIG. 2 is a schematic flowchart of an embodiment of step S12 in FIG. 1 . Specifically include the following steps:

Step S121: Using a preset threshold related to the color feature of the target object, perform threshold segmentation on the original image to obtain an image to be detected.

In a specific implementation scenario, taking electric power maintenance as an example, and the target object is red cloth, the preset threshold value related to the color characteristics of red cloth can be used to perform threshold segmentation on the original image to obtain the image to be detected. In other application scenarios, it can be deduced by analogy, and this embodiment will not give examples one by one here.

In addition, since the RGB (Red Green Blue, red, green, blue) color space is very different from human perception, in order to make the definition of color distance conform to human visual characteristics, this embodiment can also divide the original image into The color space is mapped to the HSV (Hue Saturation Value, Hue/Saturation/Brightness) color space. In a specific implementation scenario, when the color space of the original image is the RGB color space, the color space of the original image can be mapped to the HSV color space by the following formula:

In the above formula, (R, G, B) represents the R channel pixel value, the G channel pixel value, and the B channel pixel value of a certain pixel in the original image.

In addition, further, the above-mentioned (H, S, V) may also be mapped to an interval of 0-255, which will not be repeated in this embodiment. In a specific implementation scenario, the above-mentioned preset thresholds may include: a preset H-channel threshold range (for example: 156-180), a preset S-channel threshold range (for example: 80-255), and a preset V-channel threshold range (eg: 50-255), in other implementation scenarios, the above values may be other values, which are not specifically limited in this embodiment. Using the above preset thresholds, it is possible to sequentially judge whether each pixel in the original image mapped to the HSV color space satisfies the following conditions: whether the H channel pixel value is within the preset H channel threshold range, and whether the S channel pixel value is within the preset In the S channel threshold interval, and whether the V channel pixel value is in the preset V channel threshold interval, if satisfied, the pixel value of the corresponding pixel point in the image to be detected is set to the first pixel value (for example: 255), otherwise, Set the pixel value of the corresponding pixel point in the image to be detected as the second pixel value (for example: 0). Specifically, the pixel values of the pixel points related to the color feature of the target object in the image to be detected have the first pixel value, and the pixel values of the pixel points not related to the color feature of the target object have the second pixel value.

Step S122: Count the sum of pixel values of all pixels in the upper left area of each pixel in the image to be detected as the pixel value of the corresponding pixel in the first integrated image corresponding to the image to be detected.

Please refer to FIG. 3 in conjunction with FIG. 3 , which is a schematic diagram of an embodiment of an acquisition process of the first integral map in FIG. 2 . As shown in Figure 3, P ₁ is the image to be detected, P ₁ (i, j) is the pixel value of the pixel located in the i-th row and the j-th column of the image to be detected, Q ₁ is the first integrated image, Q ₁ ( i, j) is the pixel value of the pixel located in the i-th row and j-th column of the first integral image. Through the following formula, the first integral image can be obtained based on the image to be detected:

Step S123: Perform morphological processing on the image to be detected by using the preset structural elements and the first integrated image to obtain the target area corresponding to the target object in the original image.

The preset structural element in this embodiment is a two-dimensional matrix, the value of its matrix element is 1, and its size can be 15*15, or 13*13, or 11*11, or 9*9. The embodiments are not specifically limited here.

In an implementation scenario, the image to be processed can be directly morphologically processed by using the preset structural elements. For example, when using the preset structural element to perform corrosion morphology processing on the image to be processed, the atom (ie, the central point) of the preset structural element can be used to traverse the target pixel point whose pixel value is the first pixel value in the image to be processed. Assume that the pixel values of other pixels in the size area of the structural element are the same as the pixel values of the target pixel, that is, the pixel values of other pixels are all the first pixel value, then keep the pixel value of the target pixel, otherwise, set the target pixel value to The pixel value of the pixel point is set as the second pixel value. Please refer to FIG. 4 in combination. FIG. 4 is a schematic diagram of an embodiment of morphological processing in FIG. 2. Specifically, FIG. 4 is a schematic diagram of an embodiment of corrosion morphological processing using preset structural elements. As shown in FIG. Suppose the size of the structural element is k*k, the size of the image to be detected is w*h, the image to be detected after corrosion morphology processing is shown in Fig. The target pixel of one pixel value, as shown in FIG. 4 , undergoes corrosion morphology processing, which can eliminate burrs in the image to be processed. However, the complexity of the algorithm for performing corrosion morphology processing on the image to be processed by directly using the preset structural elements is O(w*h*k*k), which is high in complexity, processing load and resource overhead.

In another implementation scenario, in order to reduce algorithm complexity, reduce processing load, and reduce resource overhead, the above-mentioned first integral image may be used to perform morphological processing on the image to be processed. Specifically, please refer to FIG. 5. FIG. 5 is a schematic flowchart of an embodiment of step S123 in FIG. 2, which may specifically include:

Step S1231: Perform corrosion morphology processing on the image to be detected by using the preset structural elements and the first integrated image to obtain a first processed image.

Specifically, it is possible to determine the first target pixel point whose pixel value is the first pixel value in the image to be detected, and determine the second target pixel point corresponding to the first target pixel point in the first integral image, based on the second target pixel point and the predicted Set the size of the element structure, determine the third target pixel in the first integral image, and then use the pixel value of the third target pixel to calculate the sum of the pixel values of all pixels within the size range around the first target pixel , if the quotient of the sum of the calculated pixel values and the size is the first pixel value, it can be considered that the first target pixel point is not a glitch/noise point, and the pixel value of the first target pixel point is kept as the first pixel value, If the quotient of the sum of the pixel values and the size is not the first pixel value, then the first target pixel point can be considered as glitch/noise, and the pixel value of the first target pixel point is reset to the second pixel value.

In a specific implementation scenario, please continue to refer to Figure 3, as shown in Figure 3, in the image to be detected P ₁ , the pixel point (i, j) is the first target pixel point, correspondingly, the first integrated image Q In ₁ , the pixel point (i, j) is the second target pixel point corresponding to the first target pixel point, and the thick black frame area is the size (k*k) area of the preset structural element. In order to calculate the image to be detected The sum of the pixel values of the pixel points in the size area of the preset structural elements in _P1 can be calculated by using the first integral image _Q1 . When calculating, the first integral can be determined by using the size of the preset structural elements For the third target pixel in image _Q1 , firstly, the pixel in the lower right corner of the size area of the preset structural element of the first integral image _Q1 can be Determine the first and third target pixel, whose pixel value is represent pixels /> The sum of the pixel values of all pixels in the upper left corner area, and then determine the other three third target pixel points /> Therefore, the sum of the pixel values of all pixels within the size range (k*k) around the first target pixel (i, j) can be expressed as:

After calculating the sum S of the pixel values of all pixels within the size (k*k) range around the first target pixel (i, j), the sum S of pixel values and the size (k*k) can be further calculated business (i.e. ), if the quotient is the first pixel value, it means that the pixel values of the image to be detected P ₁ in the size (k*k) range of the first target pixel point (i, j) are all pixel values of the first pixel value, The first target pixel point (i, j) can be considered as not a burr/noise point, at this time, the pixel value of the first target pixel point (i, j) in the _image to be detected can be kept unchanged; otherwise, the first target pixel point The point (i, j) is a burr/noise point, and the pixel value of the first target pixel point (i, j) in the image to be detected _P1 is reset to the second pixel value.

By using the integral graph for corrosion morphology processing, the algorithm complexity can be reduced to O(w*h), and the algorithm complexity is significantly reduced.

Step S1232: Count the sum of pixel values of all pixels in the upper left area of each pixel in the first processed image as the pixel value of the corresponding pixel in the second integrated image corresponding to the first processed image.

In one implementation scenario, after subjecting the image to be processed to corrosion morphology processing, dilation morphology processing can be further continued. Similar to the above steps, all pixels in the upper left area of each pixel point in the first processed image can be counted The sum of the pixel values of the points is used as the pixel value of the corresponding pixel point in the second integrated image corresponding to the first processed image. For details, reference may be made to the above-mentioned relevant steps in this embodiment, and details are not repeated here.

Step S1233: Perform dilated morphological processing on the first processed image by using the preset structural elements and the second integral image to obtain a second processed image.

Specifically, each pixel point in the first processed image may be sequentially used as the fourth target pixel point, and the fifth target pixel point corresponding to the fourth target pixel point in the second integral image may be determined, based on the fifth target pixel point and Preset the size of the structural element, determine the sixth target pixel in the second integral image, and use the pixel value of the sixth target pixel to calculate the pixel value of all pixels within the size range around the fourth target pixel And, if the quotient of the sum of the pixel values and the size is the second pixel value, set the pixel value of the fourth target pixel point as the second pixel value, if the quotient of the sum of the pixel values and the size is not the second pixel value, then set the pixel value of the fourth target pixel point as the first pixel value. Wherein, based on the fifth target pixel and the size of the preset structural element, determine the sixth target pixel in the second integrated image, and use the pixel value of the sixth target pixel to calculate the size around the fourth target pixel For the specific manner of summing the pixel values of all pixel points within the range, reference may be made to the above-mentioned step S1231 in this embodiment, which will not be repeated here. Different from corrosion morphology processing, the purpose of dilation morphology processing is to expand the edge and fill the hole, so as long as there is a pixel point with the first pixel value within the size range of the preset structural element, that is, as long as the pixel value If the quotient of the sum and the size is not the second pixel value, set the pixel value of the fourth target pixel point as the first pixel value; otherwise, if all the pixel values within the size range of the preset structural element are the second pixel value A pixel point with a pixel value, that is, if the quotient of the sum of the pixel values and the size is the second pixel value, then set the pixel value of the fourth target pixel point as the second pixel value. In this way, the edge can be significantly expanded, And fill the internal cavity.

Step S1234: Determine the target connected domain of the first pixel value of the pixel value of each pixel point in the second processed image.

After the image to be processed undergoes corrosion morphology processing and expansion morphology processing, the second processing image is obtained. At this time, if the pixel value of a certain pixel point in the second processing image and the pixel value of its neighboring A pixel value, divide the pixel point and its neighboring pixels into the same connected domain, traverse all the pixel points in the second processed image, and at least one target connected domain can be obtained.

Step S1235: Obtain the smallest circumscribed rectangle of the target connected domain as the target area.

The smallest bounding rectangle of the target connected domain is used as the target area.

Different from the foregoing embodiments, the original image is thresholded by using the preset threshold related to the color feature of the target object to obtain the image to be detected, and the pixel values of all pixels in the upper left area of each pixel in the image to be detected are counted The sum is used as the pixel value of the corresponding pixel in the first integral image corresponding to the image to be detected, so that the image to be detected is subjected to morphological processing by using the preset structural elements and the first integral image, and the pixel corresponding to the target object in the original image is obtained. The target area can reduce the complexity of the algorithm, shorten the processing time, thereby speeding up the speed of target detection, which is conducive to real-time detection of whether the engineering operation is standardized, and timely alarm when it is not standardized.

Please refer to FIG. 6 . FIG. 6 is a schematic frame diagram of an embodiment of an engineering operation detection device 60 of the present application. The engineering operation detection device 60 includes an acquisition module 61, a detection module 62 and a determination module 63. The acquisition module 61 is used to acquire the original image taken by the camera device on the job site, wherein the original image contains a preset detection area, and the detection module 62 is used for Perform target detection on the original image, and obtain the target area corresponding to the target object in the original image, wherein the target object is used to realize warning, and the determining module 63 is used to determine the position of the job site based on the positional relationship between the preset detection area and the target area. Is it in compliance with job specifications.

In the above solution, by obtaining the original image taken by the camera device on the job site, and the original image contains a preset detection area, the target detection is performed on the original image, and the target area corresponding to the target object used to realize the warning is obtained in the original image, Then, based on the positional relationship between the preset detection area and the target area, it is determined whether the job site complies with the job specification, so that whether the job site meets the job specification can be detected based on the original image captured by the camera device, without relying on manual work. Check the job site, so as to improve the detection efficiency, reduce the probability of omissions, and improve the quality of engineering operation inspection.

In some embodiments, the detection module 62 includes a threshold segmentation sub-module, which is used to perform threshold segmentation on the original image to obtain the image to be detected by using a preset threshold related to the color feature of the target object. The detection module 62 also includes an integral statistics sub-module module, for counting the sum of the pixel values of all pixels in the upper left area of each pixel in the image to be detected, as the pixel value of the corresponding pixel in the first integrated image corresponding to the image to be detected, the detection module 62 also includes a form The morphological processing sub-module is used to perform morphological processing on the image to be detected by using the preset structural elements and the first integral image, so as to obtain the target area corresponding to the target object in the original image.

Different from the foregoing embodiments, the original image is thresholded by using the preset threshold related to the color feature of the target object to obtain the image to be detected, and the pixel values of all pixels in the upper left area of each pixel in the image to be detected are counted The sum is used as the pixel value of the corresponding pixel in the first integral image corresponding to the image to be detected, so that the image to be detected is subjected to morphological processing by using the preset structural elements and the first integral image, and the pixel corresponding to the target object in the original image is obtained. The target area can reduce the complexity of the algorithm, shorten the processing time, thereby speeding up the speed of target detection, which is conducive to real-time detection of whether the engineering operation is standardized, and timely alarm when it is not standardized.

In some embodiments, the pixel value of the pixel point related to the color feature of the target object in the image to be detected has the first pixel value, and the pixel value of the pixel point irrelevant to the color feature of the target object is the second pixel value, and the morphology The processing sub-module includes a corrosion processing unit, which is used to perform corrosion morphological processing on the image to be detected by using the preset structural elements and the first integral image to obtain the first processed image, and the integral statistics sub-module is also used to count each pixel in the first processed image The sum of the pixel values of all pixels in the upper left area of the point is used as the pixel value of the corresponding pixel in the second integral image corresponding to the first processed image. The morphology processing sub-module also includes an expansion processing unit for using the preset Structural elements and the second integral image perform expansion morphological processing on the first processed image to obtain the second processed image, and the morphological processing sub-module also includes a connected domain determination unit for determining the pixel value of each pixel point in the second processed image is the target connected domain of the first pixel value, and the morphological processing submodule further includes a target area determining unit, configured to obtain a minimum circumscribed rectangle of the target connected domain as the target area.

In some embodiments, the erosion processing unit is specifically configured to determine a first target pixel in the image to be detected whose pixel value is the first pixel value, and determine a second target pixel corresponding to the first target pixel in the first integrated image point, based on the second target pixel and the size of the preset element structure, determine the third target pixel in the first integral image, and use the pixel value of the third target pixel to calculate the size range around the first target pixel The sum of the pixel values of all pixels within, if the quotient of the sum of the pixel values and the size is the first pixel value, then keep the pixel value of the first target pixel as the first pixel value, if the sum of the pixel values and the size If the quotient of the sizes is not the first pixel value, the pixel value of the first target pixel point is reset to the second pixel value.

In some embodiments, the dilation processing unit is specifically configured to sequentially use each pixel point in the first processed image as the fourth target pixel point, and determine a fifth target pixel point corresponding to the fourth target pixel point in the second integrated image , based on the fifth target pixel and the size of the preset structural element, determine the sixth target pixel in the second integrated image, and use the pixel value of the sixth target pixel to calculate the size range of the fourth target pixel The sum of the pixel values of all pixels, if the quotient of the sum of the pixel values and the size is the second pixel value, then set the pixel value of the fourth target pixel as the second pixel value, if the sum of the pixel values and the size is not the second pixel value, then set the pixel value of the fourth target pixel point as the first pixel value.

In some embodiments, the detection module 62 also includes a color space mapping sub-module for mapping the color space of the original image to the HSV color space, and the preset thresholds include: a preset H channel threshold interval, a preset S channel threshold interval and The preset V channel threshold interval, the threshold segmentation submodule includes a judging unit for sequentially judging whether each pixel in the original image mapped to the HSV color space satisfies the following conditions: the H channel pixel value is within the preset H channel threshold interval, And the S channel pixel value is within the preset S channel threshold interval, and the V channel pixel value is within the preset V channel threshold interval, the threshold segmentation submodule also includes a pixel value setting unit, which is used to set the The pixel value of the corresponding pixel in the image to be detected is set as the first pixel value, and the pixel value setting unit is also used to set the pixel value of the corresponding pixel in the image to be detected as the second pixel value when the judgment unit judges that the condition is not satisfied .

In some embodiments, the determining module 63 includes an overlapping area judging sub-module for judging whether there is an overlapping area between the preset detection area and the target area, and the determining module 63 also includes a job determining sub-module for judging the overlapping area. When the module judges that there is an overlapping area, it determines that the operation site conforms to the operation specification, and the operation determination sub-module is also used to determine that the operation site does not meet the operation specification when the overlapping area judgment sub-module determines that there is no overlapping area.

In some embodiments, the determination module 63 also includes an information output submodule, which is used to output preset alarm information when the job determination submodule determines that the job site does not meet the operation specifications. When it is determined that the job site conforms to the job specifications, the preset safety information is output.

In some embodiments, the engineering operation detection device 60 further includes a setting module, configured to obtain pose parameters set by the user for the camera device, wherein the pose parameters are used to control the camera device to take pictures of the job site.

Please refer to FIG. 7 . FIG. 7 is a schematic frame diagram of an embodiment of an engineering operation detection device 70 of the present application. The engineering operation detection device 70 may include a memory 71 and a processor 72 coupled to each other; the processor 72 is configured to execute the program instructions stored in the memory 71 to implement the steps in any of the above embodiments of the engineering operation detection method.

Specifically, the processor 72 is used to control itself and the memory 71 to implement the steps in any of the above embodiments of the engineering operation detection method. The processor 72 may also be referred to as a CPU (Central Processing Unit, central processing unit). The processor 72 may be an integrated circuit chip with signal processing capability. The processor 72 can also be a general-purpose processor, a digital signal processor (Digital Signal Processor, DSP), an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), a field-programmable gate array (Field-Programmable GateArray, FPGA) or other programmable Logic devices, discrete gate or transistor logic devices, discrete hardware components. A general-purpose processor may be a microprocessor, or the processor may be any conventional processor, or the like. In addition, the processor 72 may be jointly implemented by multiple integrated circuit chips.

In this embodiment, the processor 72 is used to obtain the original image taken by the imaging device on the job site, wherein the original image contains a preset detection area, and the processor 72 is also used to perform target detection on the original image, and obtain the original image corresponding to The target area corresponding to the target object, wherein the target object is used to realize the warning, and the processor 72 is also used to determine whether the job site conforms to the job specification based on the positional relationship between the preset detection area and the target area.

In the above solution, by obtaining the original image taken by the camera device on the job site, and the original image contains a preset detection area, the target detection is performed on the original image, and the target area corresponding to the target object used to realize the warning is obtained in the original image, Then, based on the positional relationship between the preset detection area and the target area, it is determined whether the job site complies with the job specification, so that whether the job site meets the job specification can be detected based on the original image captured by the camera device, without relying on manual work. Check the job site, so as to improve the detection efficiency, reduce the probability of omissions, and improve the quality of engineering operation inspection.

In some embodiments, the processor 72 is also used to perform threshold segmentation on the original image by using a preset threshold related to the color feature of the target object to obtain the image to be detected, and the processor 72 is also used to count each pixel in the image to be detected The sum of the pixel values of all pixels in the upper left area of the point is used as the pixel value of the corresponding pixel in the first integral image corresponding to the image to be detected, and the processor 72 is also used to use the preset structural elements and the first integral image to treat The detection image is subjected to morphological processing to obtain the target area corresponding to the target object in the original image.

Different from the foregoing embodiments, the original image is thresholded by using the preset threshold related to the color feature of the target object to obtain the image to be detected, and the pixel values of all pixels in the upper left area of each pixel in the image to be detected are counted The sum is used as the pixel value of the corresponding pixel in the first integral image corresponding to the image to be detected, so that the image to be detected is subjected to morphological processing by using the preset structural elements and the first integral image, and the pixel corresponding to the target object in the original image is obtained. The target area can reduce the complexity of the algorithm, shorten the processing time, thereby speeding up the speed of target detection, which is conducive to real-time detection of whether the engineering operation is standardized, and timely alarm when it is not standardized.

In some embodiments, the pixel value of the pixel point related to the color feature of the target object in the image to be detected has a first pixel value, and the pixel value of a pixel point unrelated to the color feature of the target object has a second pixel value, and the processor 72 is also used to perform corrosion morphology processing on the image to be detected by using the preset structural elements and the first integral image to obtain the first processed image, and the processor 72 is also used to count all pixels in the upper left area of each pixel in the first processed image The sum of the pixel values of the points is used as the pixel value of the corresponding pixel point in the second integrated image corresponding to the first processed image, and the processor 72 is also used to dilate the first processed image by using the preset structural elements and the second integrated image Morphological processing to obtain a second processed image, the processor 72 is also used to determine the target connected domain of the first pixel value of the pixel value of each pixel point in the second processed image, and the processor 72 is also used to obtain the minimum value of the target connected domain The bounding rectangle, as the target area.

In some embodiments, the processor 72 is further configured to determine a first target pixel whose pixel value is the first pixel value in the image to be detected, and determine a second target pixel corresponding to the first target pixel in the first integral image point, the processor 72 is also used to determine the third target pixel point in the first integrated image based on the second target pixel point and the size of the preset element structure, and the processor 72 is also used to use the pixel of the third target pixel point value, calculating the sum of the pixel values of all pixels within the size range around the first target pixel, and the processor 72 is also used to calculate the first target pixel when the quotient of the sum of pixel values and the size is the first pixel value The pixel value of the point remains as the first pixel value, and the processor 72 is also used for resetting the pixel value of the first target pixel point to the second pixel when the quotient of the sum of the pixel values and the size is not the first pixel value value.

In some embodiments, the processor 72 is further configured to sequentially use each pixel point in the first processed image as the fourth target pixel point, and determine the fifth target pixel point corresponding to the fourth target pixel point in the second integrated image , the processor 72 is also used to determine the sixth target pixel in the second integrated image based on the fifth target pixel and the size of the preset structural element, and the processor 72 is also used to use the pixel value of the sixth target pixel , calculate the sum of the pixel values of all pixels within the size range around the fourth target pixel, and the processor 72 is also used to convert the fourth target pixel The pixel value of is set as the second pixel value, and the processor 72 is further configured to set the pixel value of the fourth target pixel point as the first pixel value when the quotient of the sum of the pixel values and the size is not the second pixel value.

In some embodiments, the processor 72 is further configured to map the color space of the original image to the HSV color space, and the preset thresholds include: a preset H-channel threshold interval, a preset S-channel threshold interval, and a preset V-channel threshold interval, The processor 72 is also used to sequentially determine whether each pixel in the original image mapped to the HSV color space satisfies the following conditions: the H channel pixel value is within the preset H channel threshold interval, and the S channel pixel value is within the preset S channel threshold value. interval, and the V channel pixel value is within the preset V channel threshold interval, the processor 72 is also used to set the pixel value of the corresponding pixel in the image to be detected as the first pixel value when the condition is met, and the processor 72 is also configured to It is used to set the pixel value of the corresponding pixel point in the image to be detected as the second pixel value when the condition is not satisfied.

In some embodiments, the processor 72 is further configured to determine whether there is an overlapping area between the preset detection area and the target area, and the processor 72 is further configured to determine that the job site complies with the job specification when the judgment is yes, and the processor 72 is also configured to It is used to determine that the job site does not meet the job specifications when the judgment is negative.

In some embodiments, the engineering operation detection device 70 further includes a human-computer interaction circuit, configured to output preset alarm information when the processor 72 determines that the operation site does not meet the operation specifications. The human-computer interaction circuit is also used to output preset safety information when the processor 72 determines that the job site complies with job specifications.

In some embodiments, the processor 72 is also used to control the human-computer interaction circuit to obtain the pose parameters set by the user for the camera device, wherein the pose parameters are used to control the camera device to take pictures of the job site.

In some embodiments, the engineering operation detection device 70 also includes an imaging device, which is used to capture the original image of the operation site.

Please refer to FIG. 8 . FIG. 8 is a schematic frame diagram of an embodiment of a storage device 80 of the present application. The storage device 80 stores program instructions 81 that can be executed by the processor, and the program instructions 81 are used to implement the steps in any of the above-mentioned embodiments of the engineering operation detection method.

The above solution can detect whether the operation site conforms to the operation specification based on the original image taken by the camera device on the operation site, without relying on manual inspection of the operation site, thereby improving detection efficiency, reducing the probability of omissions, and improving engineering operations. Check quality.

In the several embodiments provided in this application, it should be understood that the disclosed methods and devices may be implemented in other ways. For example, the device implementations described above are only illustrative. For example, the division of modules or units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or May be integrated into another system, or some features may be ignored, or not implemented. In another point, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces, and the indirect coupling or communication connection of devices or units may be in electrical, mechanical or other forms.

A unit described as a separate component may or may not be physically separated, and a component displayed as a unit may or may not be a physical unit, that is, it may be located in one place, or may be distributed to multiple network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, each unit may exist separately physically, or two or more units may be integrated into one unit. The above-mentioned integrated units can be implemented in the form of hardware or in the form of software functional units.

If the integrated unit is realized in the form of a software function unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or part of the contribution to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium , including several instructions to make a computer device (which may be a personal computer, a server, or a network device, etc.) or a processor (processor) execute all or part of the steps of the methods in various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), magnetic disk or optical disk and other media that can store program codes. .

Claims (11)

1. An engineering operation detection method is characterized by comprising the following steps:

acquiring an original image shot by an imaging device on a working site, wherein the original image comprises a preset detection area;

performing threshold segmentation on the original image by using a preset threshold related to the color characteristics of the target object to obtain an image to be detected; the target object is used for realizing warning;

counting the sum of pixel values of all pixel points in the upper left area of each pixel point in the image to be detected, and taking the sum as the pixel value of the corresponding pixel point in the first integral image corresponding to the image to be detected;

carrying out morphological processing on the image to be detected by using a preset structural element and the first integral image to obtain a target area corresponding to the target object in the original image;

and determining whether the operation site meets an operation specification or not based on the position relation between the preset detection area and the target area.

2. The engineering job detection method according to claim 1, wherein a pixel value of a pixel point related to a color feature of the target object in the image to be detected is a first pixel value, and a pixel value of a pixel point unrelated to the color feature of the target object is a second pixel value;

The step of performing morphological processing on the image to be detected by using a preset structural element and the first integral image to obtain a target area corresponding to the target object in the original image, wherein the step of obtaining the target area comprises the following steps:

carrying out corrosion morphological processing on the image to be detected by utilizing the preset structural elements and the first integral image to obtain a first processed image;

counting the sum of pixel values of all pixel points in the upper left area of each pixel point in the first processing image, and taking the sum as the pixel value of the corresponding pixel point in the second integral image corresponding to the first processing image;

performing expansion morphological processing on the first processed image by using the preset structural elements and the second integral image to obtain a second processed image;

determining the pixel value of each pixel point in the second processed image as a target connected domain of the first pixel value;

and acquiring the minimum circumscribed rectangle of the target connected domain as the target area.

3. The method for detecting engineering operations according to claim 2, wherein performing the corrosive morphological processing on the image to be detected using the preset structural element and the first integral image to obtain a first processed image includes:

Determining a first target pixel point with a pixel value of the first pixel value in the image to be detected, and determining a second target pixel point corresponding to the first target pixel point in the first integral image;

determining a third target pixel point in the first integral image based on the second target pixel point and the size of the preset structural element;

calculating the sum of pixel values of all pixel points in the size range around the first target pixel point by using the pixel value of the third target pixel point;

if the quotient of the sum of the pixel values and the size is the first pixel value, the pixel value of the first target pixel point is kept to be the first pixel value;

and if the quotient of the sum of the pixel values and the size is not the first pixel value, resetting the pixel value of the first target pixel point to be the second pixel value.

4. The engineering job inspection method of claim 2, wherein performing the dilation morphological processing on the first processed image using the preset structural element and the second integral image to obtain a second processed image comprises:

Sequentially taking each pixel point in the first processed image as a fourth target pixel point, and determining a fifth target pixel point corresponding to the fourth target pixel point in the second integral image;

determining a sixth target pixel point in the second integral image based on the fifth target pixel point and the size of the preset structural element;

calculating the sum of pixel values of all pixel points in the size range around the fourth target pixel point by using the pixel value of the sixth target pixel point;

setting the pixel value of the fourth target pixel point to be the second pixel value if the quotient of the sum of the pixel values and the size is the second pixel value;

and if the quotient of the sum of the pixel values and the size is not the second pixel value, setting the pixel value of the fourth target pixel point as the first pixel value.

5. The method for detecting engineering operations according to claim 1, wherein the method further comprises, before performing threshold segmentation on the original image by using a preset threshold related to the color feature of the target object to obtain an image to be detected:

Mapping a color space of the original image to an HSV color space;

the preset threshold value comprises the following steps: presetting an H channel threshold interval, an S channel threshold interval and a V channel threshold interval; the threshold segmentation is performed on the original image by using a preset threshold related to the color characteristic of the target object, and the obtaining of the image to be detected includes:

sequentially judging whether each pixel point in the original image mapped to the HSV color space meets the following conditions: the H-channel pixel value is in the preset H-channel threshold interval, the S-channel pixel value is in the preset S-channel threshold interval, and the V-channel pixel value is in the preset V-channel threshold interval;

if yes, setting the pixel value of the corresponding pixel point in the image to be detected as a first pixel value;

if not, setting the pixel value of the corresponding pixel point in the image to be detected as a second pixel value.

6. The engineering job detection method according to claim 1, wherein the determining whether the job site meets a job specification based on the positional relationship between the preset detection area and the target area comprises:

judging whether a coincidence region exists between the preset detection region and the target region;

If yes, determining that the operation site meets the operation specification;

if not, determining that the operation field does not accord with the operation specification.

7. The engineering job detection method of claim 6, wherein after the determining that the job site is not in compliance with a job specification, the method further comprises:

outputting preset alarm information;

and/or, after the determining that the job site meets the job specification, the method further comprises:

outputting preset safety information or not outputting any information.

8. The engineering work detection method according to claim 1, wherein before the original image of the work site taken by the image pickup device is obtained, the method includes:

and acquiring pose parameters set by a user on the image pickup device, wherein the pose parameters are used for controlling the image pickup device to shoot the operation site.

9. An engineering job inspection apparatus comprising memory processors coupled to each other, the processors configured to execute program instructions stored in the memory to implement the engineering job inspection method of any one of claims 1 to 8.

10. The construction work detection device according to claim 9, further comprising an imaging device for capturing an original image of the work site.

11. A storage device storing program instructions executable by a processor for implementing the engineering job detection method according to any one of claims 1 to 8.