CN117197050A

CN117197050A - Method for judging abnormal situation of well lid and computing equipment

Info

Publication number: CN117197050A
Application number: CN202311044279.9A
Authority: CN
Inventors: 赵景程; 熊超; 牛昕宇
Original assignee: Shandong Industry Research Kunyun Artificial Intelligence Research Institute Co ltd
Current assignee: Shandong Industry Research Kunyun Artificial Intelligence Research Institute Co ltd
Priority date: 2023-08-18
Filing date: 2023-08-18
Publication date: 2023-12-08

Abstract

The invention provides a method and computing equipment for judging abnormal conditions of a well lid, and relates to the technical fields of image processing and traffic safety. The method comprises the following steps: and presetting an initialization circular area according to an image from the image pickup device, wherein the initialization circular area is a wellhead area in the image. And acquiring a frame image of the real-time video stream from a camera device, wherein the camera device is used for monitoring the road surface well cover in real time. And carrying out well lid segmentation on the frame image to obtain a well lid region. The initialization circular region is superimposed to the frame image. And judging whether the well lid is closed or not according to the initialization circular area and the well lid area. By utilizing the method provided by the invention, the real-time monitoring and early warning of the abnormal situation of the well lid can be realized, the accuracy of the existing detection technology is improved, and meanwhile, the equipment cost and the manual installation cost are reduced.

Description

Method for judging abnormal situation of well lid and computing equipment

Technical Field

The invention relates to the technical field of image processing and traffic safety, in particular to a method and computing equipment for judging abnormal conditions of a well lid.

Background

With the acceleration of the urban process, the construction of the urban underground system is also becoming more and more perfect. Well covers are also becoming an important component of underground systems, as are their safety and stability. The well cover is used as an important facility for urban road safety, has very wide functions and types, and is very important for daily maintenance and management. The types of manhole covers generally include manhole covers, rainwater manhole covers, fire manhole covers, cable manhole covers, and the like. The well cover can cover various forms of underground pipelines, equipment, channels and the like, and effectively avoids the mistaken stepping or falling of pedestrians and vehicles, thereby ensuring the safety of the pedestrians and the vehicles.

However, because the use frequency of the well lid is higher, and the influence of environmental factors is added, the damage and the loss phenomenon of the well lid occur sometimes, and great potential safety hazards are brought to urban traffic and civilian life. Therefore, the research and implementation of the well lid monitoring solution and the measure are particularly important.

Traditional well lid monitoring scheme is mainly artifical inspection, inspects the state of inspection shaft well lid through inspection personnel, and discovery problem is in time maintained or is changed. The scheme has the advantages of low cost and simple operation, but has obvious defects, namely low monitoring efficiency, incapability of realizing real-time monitoring and the conditions of artificial negligence and omission. Along with the technical development, intelligent monitoring scheme also gradually adopts, carries out real-time supervision and data acquisition to the state of well lid through installation sensor and monitoring facilities, through data analysis and processing, in time discovers the damage and the loss condition of well lid. The scheme has the advantages of high monitoring efficiency, capability of realizing real-time monitoring and early warning, high cost and need of maintenance and management by professional technicians.

Therefore, a technical scheme is needed, so that the abnormal situation of the well lid can be accurately detected, and the cost is saved.

Disclosure of Invention

The invention aims to provide a method and computing equipment for judging the abnormal situation of a well lid, and solves the potential safety hazards of pedestrians and vehicles caused by the abnormal situation of the well lid on a road surface.

According to an aspect of the present invention, there is provided a method for judging an abnormal situation of a manhole cover, comprising:

presetting an initialization circular area according to an image from a camera device, wherein the initialization circular area is a wellhead area in the image;

acquiring a frame image of a real-time video stream from a camera device, wherein the camera device is used for monitoring a road manhole cover in real time;

performing well lid segmentation on the frame image to obtain a well lid region;

superimposing the initialization circular region to the frame image;

and judging whether the well lid is closed or not according to the initialization circular area and the well lid area.

According to some embodiments, the method further comprises;

determining a longest cutting line for dividing the initialization circular area parallel to a first coordinate axis in the frame image as a first cutting line;

determining a longest cutting line parallel to a first coordinate axis in the frame image for dividing the well lid area as a second cutting line;

calculating Euclidean distances between two endpoints of the first secant and two endpoints of the second secant;

and if the Euclidean distance is larger than a preset threshold value, judging that the well lid is not closed and giving an alarm.

According to some embodiments, determining whether the manhole cover is closed according to the initializing the circular region and the manhole cover region includes:

the intersection ratio Iou of the initialized circular area and the manhole cover area is calculated.

According to some embodiments, judging whether the manhole cover is closed according to the initializing circular area and the manhole cover area, further comprising:

when the intersection ratio is 1, judging that the well lid is closed;

dividing the broken area in the well cover area, and counting the pixel area of the broken area;

and when the pixel area of the damaged area is larger than a preset damage threshold value, sending out a maintenance alarm.

and when the intersection ratio is 0 and the minimum distance between the initialized circular area and the well lid area is larger than a preset displacement threshold value, judging that the well lid is displaced greatly and sending out a well lid displacement alarm.

and when the intersection ratio is more than 0 and less than 1 or the intersection ratio is 0 and the minimum distance between the initialized circular area and the well lid area is less than or equal to a preset displacement threshold value, judging that the well lid is not closed, and carrying out classification detection on the well lid area.

According to some example embodiments, classifying the manhole cover area includes:

performing image classification on the well lid area by using a pre-trained neural network model;

according to the result of image classification, the well lid is little shifted, the well lid is opened, the well lid edge is damaged, the well lid is protruding, or the well lid is sunken, sends corresponding early warning.

According to some example embodiments, the method further comprises setting an algorithm identification region for the frame image.

Determining a circumscribed rectangle of the initialization circular region on the frame image;

expanding the circumscribed rectangle according to a preset proportion;

taking the area corresponding to the expanded circumscribed rectangle as the algorithm identification area;

and filling the image area outside the algorithm identification area into any color or filling the pixel average value, the maximum value, the minimum value, the mode or the median of all the pixel points outside the algorithm identification area.

According to some example embodiments, the method further comprises:

the focal length, position and angle of the image pickup device are fixed, and the overlooking angle is 60-90 degrees.

According to another aspect of the invention there is provided a computing device comprising a processor, and a memory storing a computer program which, when executed by the processor, causes the processor to perform the method of any of the above.

According to another aspect of the invention there is provided a non-transitory computer readable storage medium having stored thereon computer readable instructions which, when executed by a processor, cause the processor to perform the method of any of the above.

According to the embodiment of the invention, the image from the camera device is acquired, and whether the well lid is abnormal or not is judged by identifying the well mouth area and the well lid area. According to the technical scheme provided by the embodiment of the invention, the monocular camera can be adopted, and the equipment and manual installation cost is greatly saved on the basis of accurately detecting the occurrence of the abnormal situation of the well lid through image acquisition and relation judgment.

According to some embodiments, the abnormal condition of the well lid is judged through the cross-correlation ratio, and then the pre-trained neural network model is utilized to classify the damage type, and corresponding early warning is sent out. Compared with the existing manual inspection, sensor detection and other modes, the technical accuracy of the detection and judgment of the abnormal well lid condition is higher, the automatic inspection well lid detection device has a classification alarm function, and can comprehensively and specifically detect the abnormal well lid condition to avoid potential safety hazards and traffic safety risks caused by aging and damage.

According to some embodiments, the well lid is further subjected to damage identification and maintenance prompt, so that potential safety hazard elimination is ensured, and reliability is improved. In addition, through classifying judgment and alarm to the well lid condition, the selection of follow-up actual manual maintenance mode of being convenient for is favorable to the maintenance of well lid facility, reduces maintenance cost, improves maintenance efficiency.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the invention as claimed.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings that are required to be used in the description of the embodiments will be briefly described below.

Fig. 1 shows an application scenario of an image pickup apparatus according to an example embodiment in judging an abnormal situation of a manhole cover.

FIG. 2 illustrates a flow chart of a method for determining an anomaly of a manhole cover, according to an example embodiment.

FIG. 3 illustrates a flow chart of a method of setting an algorithm identification area according to an example embodiment.

FIG. 4 illustrates a flow chart of a method of determining an anomaly of a manhole cover in accordance with an example embodiment.

Fig. 5 shows a schematic diagram of a camera calibration process according to an example embodiment.

Fig. 6 shows a schematic diagram of a solution for determining an anomaly of a manhole cover according to an example embodiment.

FIG. 7 illustrates a block diagram of a computing device according to an example embodiment.

Detailed Description

Example embodiments will now be described more fully with reference to the accompanying drawings. However, the exemplary embodiments can be embodied in many forms and should not be construed as limited to the embodiments set forth herein; rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the concept of the example embodiments to those skilled in the art. The same reference numerals in the drawings denote the same or similar parts, and thus a repetitive description thereof will be omitted.

Furthermore, the described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments. In the following description, numerous specific details are provided to give a thorough understanding of embodiments of the invention. One skilled in the relevant art will recognize, however, that the invention may be practiced without one or more of the specific details, or with other methods, components, devices, steps, etc. In other instances, well-known methods, devices, implementations, or operations are not shown or described in detail to avoid obscuring aspects of the invention.

The block diagrams depicted in the figures are merely functional entities and do not necessarily correspond to physically separate entities. That is, the functional entities may be implemented in software, or in one or more hardware modules or integrated circuits, or in different networks and/or processor devices and/or microcontroller devices.

The flow diagrams depicted in the figures are exemplary only, and do not necessarily include all of the elements and operations/steps, nor must they be performed in the order described. For example, some operations/steps may be decomposed, and some operations/steps may be combined or partially combined, so that the order of actual execution may be changed according to actual situations.

It will be understood that, although the terms first, second, third, etc. may be used herein to describe various components, these components should not be limited by these terms. These terms are used to distinguish one element from another element. Accordingly, a first component discussed below could be termed a second component without departing from the teachings of the present inventive concept. As used herein, the term "and/or" includes any one of the associated listed items and all combinations of one or more.

The user information (including but not limited to user equipment information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, presented data, etc.) related to the present invention are information and data authorized by the user or fully authorized by each party, and the collection, use and processing of related data is required to comply with the relevant laws and regulations and standards of the relevant country and region, and is provided with corresponding operation entries for the user to select authorization or rejection.

Those skilled in the art will appreciate that the drawings are schematic representations of example embodiments and that the modules or flows in the drawings are not necessarily required to practice the invention and therefore should not be taken to limit the scope of the invention.

With the acceleration of urban progress, the use frequency of the well cover is higher and higher, and the well cover becomes one of important facilities for urban road safety. Because the well lid is greatly influenced by environmental factors, the damage and the loss of the well lid occur sometimes, and potential safety hazards are brought to urban traffic and citizens. Therefore, it is important to adopt a high-efficiency accurate well lid monitoring scheme and measures. Therefore, the invention provides a method for judging the abnormal situation of the well lid, and according to some example embodiments, the real-time image of the road surface well lid acquired by the image pick-up device can effectively judge whether the well lid is closed or not, whether the well lid is damaged or not and the like through image analysis and comparison calculation. And the type of the specific abnormal condition is analyzed according to the image, and an alarm of the corresponding type of maintenance is sent out, so that real-time accurate monitoring is facilitated, and maintenance alarm prompt is timely carried out.

Before describing embodiments of the present invention, some terms or concepts related to the embodiments of the present invention are explained.

Calibrating internal parameters: camera internal parameter calibration refers to determining internal parameters of a camera, such as focal length, principal point coordinates, distortion parameters and the like.

Internal reference matrix: the internal reference matrix reflects the properties of the cameras themselves, which are different and require calibration to know the parameters.

Distortion coefficient: the distortion coefficient refers to the degree of bending or twisting of a line or corner point in an image due to factors such as the characteristics of a lens of the camera when the camera collects the image. The distortion coefficient is a parameter mainly describing the degree of lens distortion in an imaging optical system and is generally expressed by a polynomial function.

Distortion correction: distortion correction is a mapping that projects distorted pixels onto corrected pixel locations.

Loss function: is a function that maps the value of a random event or its related random variable to a non-negative real number to represent the "risk" or "loss" of the random event.

Exemplary embodiments of the present invention are described below with reference to the accompanying drawings.

Referring to fig. 1, the camera device is a monocular camera, and is disposed on a fixed building such as a utility pole, a street lamp, etc. at one side of a road. When the fixed position is selected, the monocular camera faces the wellhead area on the detection road section, and in order to reduce errors, the shooting angle is ensured to be a large overlook angle, namely, the overlook angle is 60-90 degrees. Each monocular camera shoots a well lid area with a certain range, the whole road traffic is monitored by a plurality of monocular cameras together, the whole area to be detected is ensured to be monitored in a full coverage mode, and no monitoring blind area exists.

According to the embodiment of the invention, the monocular camera is adopted to collect video frame images of the well lid area, so that abnormal condition alarm prompt is realized. Compared with other well lid detection means, the well lid detection device is simpler in composition and low in installation and maintenance cost.

According to an example embodiment, the manhole cover is monitored by a camera device.

Referring to fig. 2, in S201, an initialization circular area, which is a wellhead area in an image, is preset according to the image from an image pickup device.

According to an example embodiment, the focal length, position and angle of the image capturing device are fixed, and the image capturing device is arranged at a proper position, so that the detection angle is proper. And setting an initialization circular area according to an initial image picture, wherein the initialization circular area is a wellhead area in the image. And setting parameters of an initialized circular area through a software platform algorithm parameter configuration interface, wherein the initialized circular area is used as a basis for judging the position of a wellhead of a pavement and is used as a reference for judging the abnormal condition of a subsequent well cover.

In S203, a frame image of a real-time video stream from a camera device for real-time monitoring of a manhole cover is acquired.

According to an exemplary embodiment, as described above, after the initialization circular area is set, the camera starts the monitoring operation of the manhole cover, and a real-time image of the camera is acquired.

And in S205, performing well lid segmentation on the frame image to obtain a well lid region.

According to an example embodiment, the acquired real-time image of the road surface manhole cover is segmented by a segmentation algorithm for the manhole cover in the current frame of the real-time video stream, so that an independent manhole cover area in the current frame is obtained, and subsequent calculation is facilitated.

According to further embodiments, the real-time image may be processed using HRNet methods to segment the manhole cover area of the frame image.

The HRNET (High-Resolution Net) architecture consists of parallel High-Resolution to low-Resolution subnets, and repeated information exchange (multi-scale fusion) is performed between the multi-Resolution subnets. The horizontal and vertical directions correspond to the depth of the network and the scale of the feature map, respectively. Some image segmentation networks are built by concatenating high resolution sub-networks, each sub-network forming a stage, consisting of a series of convolutions, with a downsampling layer between adjacent sub-networks to halve resolution.

According to further embodiments, the real-time image may be processed using the deeplbv 3+ method to segment the well lid area of the frame image. According to some embodiments, DEEPLABV3+ semantic segmentation can be divided into semantic segmentation (Semantic Segmentation) and instance segmentation (Instance Segmentation), and the biggest characteristic is that cavity convolution is introduced, so that the receptive field is enlarged under the condition of no loss of information, and each convolution output contains a larger range of information. The feature points are extracted across pixels, which is beneficial to extracting multi-scale information.

In S207, the initialization circular region is superimposed to the frame image.

According to an example embodiment, in the algorithmic identification region, in an initial state, the initialization circular region is coincident with a wellhead region of the manhole cover. The initialization circular area is overlapped into the frame image, so that the position of the well mouth can be clarified in the subsequent real-time frame image, the identification error caused by the damage of the well mouth is prevented, and the accuracy of well lid identification is improved.

And S209, judging whether the well cover is closed or not according to the initialized circular area and the well cover area.

According to some embodiments, the initializing circular area and the manhole cover area may determine whether the manhole cover is closed according to a euclidean distance of a longest cutting line between the initializing circular area and the manhole cover area. And determining the longest cutting line of the initialization circular area, which is parallel to the first coordinate axis, in the frame image as a first cutting line, and determining the longest cutting line of the well lid area, which is parallel to the first coordinate axis, in the frame image as a second cutting line. And calculating Euclidean distance between two endpoints of the first secant and two endpoints of the second secant, and if the Euclidean distance is larger than a preset threshold value, judging that the well lid is not closed and giving an alarm.

According to some embodiments, the initialization circular area and the manhole cover area can also be judged whether the manhole cover is closed according to the intersection ratio Iou of the two image areas.

According to some embodiments, calculating an intersection ratio of the well lid area and a known well hole area, and judging whether the well lid state is displaced according to the intersection ratio value is as follows; and when the intersection ratio is 1, judging that the well lid is closed. And dividing the broken area inside the well cover in the well cover area, counting the pixel area of the broken area, and sending out maintenance alarm when the pixel area of the broken area is larger than a preset broken threshold value.

According to some embodiments, calculating an intersection ratio of the well lid area and a known well hole area, and determining whether the well lid state is displaced according to the intersection ratio value further comprises; and when the intersection ratio is 0 and the minimum distance between the initialized circular area and the well lid area is larger than a preset displacement threshold value, judging that the well lid is displaced greatly and sending out a well lid displacement alarm. And when the intersection ratio is more than 0 and less than 1 or the intersection ratio is 0 and the minimum distance between the initialized circular area and the well lid area is less than or equal to a preset displacement threshold value, judging that the well lid is not closed, and carrying out classification detection on the well lid area.

According to some embodiments, the classification of the manhole cover area may be detected using a pre-trained neural network model. The classification of the images is summarized as follows: the well lid is little to shift, the well lid is opened, well lid edge damage, well lid arch, or well lid is sunken these five kinds of corresponding circumstances, send corresponding early warning suggestion.

Referring to fig. 3, an algorithm identification area is set according to the acquired image of the manhole cover.

At S301, a circumscribed rectangle of the initialization circular region is determined on the frame image.

According to some embodiments, the determination of the initialized circular area may be implemented by a software platform algorithm parameter configuration interface, which transmits parameters to the camera device.

In S303, the circumscribed rectangle is expanded by a predetermined ratio.

According to some embodiments, the image expansion ratio may be set directly by a user, and the setting is made in configuring algorithm parameters, and the image expansion ratio is implemented in the process that the software platform inputs the algorithm parameter configuration file into the image capturing device.

And S305, taking the area corresponding to the expanded circumscribed rectangle as the algorithm identification area.

According to an example embodiment, image processing is performed in an algorithmic recognition area where the image capturing device captures an image.

At S307, the image area outside the algorithm recognition area is filled with any one color or the pixel average value, the maximum value, the minimum value, the mode, or the median of all the pixel points of the area outside the algorithm recognition area is filled.

According to the example embodiment, in the algorithm identification area, the algorithm identification initialization circular area is overlapped with the wellhead area in the real-time monitoring image, the initialization circular area is overlapped in the frame image, the wellhead position can be clarified, the identification error caused by wellhead damage is prevented, and the accuracy of manhole cover identification is improved. .

The initialization circular area is an important reference for judging the abnormal condition of the subsequent well lid.

According to the embodiment, according to the position relation between the initial circular area or the wellhead area and the well lid area, the abnormal situation of the well lid can be judged by calculating the Euclidean distance between the longest cutting lines corresponding to the area.

Referring to fig. 4, an intersection ratio Iou of the initialization circular area and the manhole cover area is calculated, and whether the manhole cover is closed is determined by the intersection ratio Iou.

At S401, an intersection ratio Iou of the initialization circular area and the manhole cover area is calculated.

According to an example embodiment, according to the real-time image acquired by the image pickup device, the well lid in the current frame of the real-time video stream is segmented by a segmentation algorithm, so as to obtain an individual well lid region in the current frame, and the intersection ratio Iou of the initialized circular region and the well lid region is calculated.

In S403, it is determined whether or not the Iou is equal to 1.

When the cross ratio Iou =1 is true, the process goes to S407. Otherwise, the process goes to S415.

In S407, it is determined that the manhole cover is in a closed state.

And S409, after judging that the well cover is in a closed state, dividing the broken area inside the well cover in the well cover area, and counting the pixel area S of the broken area.

In S411, it is determined whether the breakage area is greater than a preset breakage threshold.

When the damage area S is greater than a preset damage threshold, at S413, a maintenance alarm is issued at the monitoring device end.

When the overlap ratio is not equal to 1, at S415, further judgment is made.

At S415, it is determined whether the intersection ratio Iou is equal to 0 and the distance between the initialization circular area and the well lid area is greater than a preset displacement threshold, and when it is determined that the intersection ratio Iou is true, at S417, it is determined that the well lid is displaced greatly. And further at S419, a well lid displacement alarm is issued.

At S415, it is determined whether the intersection ratio Iou is equal to 0 and the minimum distance between the initialization circular area and the manhole cover area is greater than a preset displacement threshold, and when it is determined as false, at S421, the next determination is made.

In S421, it is determined whether the intersection ratio Iou is greater than 0 and less than 1 or the Iou is 0, and the minimum distance between the initialization circular area and the manhole cover area is less than or equal to a preset displacement threshold, and when it is determined that the intersection ratio Iou is true, in S423, it is determined that the manhole cover is closed.

After judging the state of the well lid being closed, in S425, the image is subjected to classification type detection.

According to some embodiments, the image classification detection classifies the well lid area by using a pre-trained neural network model;

according to the result of the image classification, the manhole cover is slightly shifted, the manhole cover is opened, the edge of the manhole cover is damaged, the manhole cover is convex or the manhole cover is concave, and the corresponding type of early warning is sent out in S427.

According to some embodiments, detection of the target object may be performed using a pre-trained YOLOV7 neural network model, but the invention is not limited thereto.

YOLO is an object detection algorithm that provides a solution to many real-life computer vision problems. It will be readily appreciated that other methods for image classification detection may be employed by those skilled in the art, such as by means of RCNN, fast RCNN, mask RCNN, and the like.

According to some embodiments, the target object may be classified using a pre-trained Vi sion Transformer model, but the invention is not limited thereto.

Vi sion Transformer (ViT model), given an h×w×c image and a block size P, an image can be divided into N p×p×c blocks, n=h×w/(p×p). After obtaining the block, the block is converted into a D-dimensional feature vector by using linear transformation, and then the D-dimensional feature vector is added with a position coding vector, and a classification flag bit [ class ] is also added before the sequence ViT.

The input sequence is passed in ViT and then sorted using the final output characteristics of the class flag bit. ViT is mainly composed of MSA (Multi-head self-attention) and MLP (two-layer fully connected network using GELU activation function), with LayerNorm and residual connection added before MSA and MLP.

The loss function of the pre-trained neural network model is as follows:

wherein y is _i,j A label indicating that the ith sample belongs to the jth class, p _i,j Representing the probability that the ith sample is predicted as the jth class, n is the number of pictures per lot, and C is the number of classes.

FIG. 5 shows a schematic diagram of a camera calibration process according to an example embodiment

In S501, an image pickup device is set to collect an image.

The camera device is a monocular camera, the focal length, the position and the angle of the camera device are fixed, and the road surface well lid is monitored and the image is acquired in real time.

In S503 and S505, internal reference calibration is performed.

And performing internal parameter calibration on the image pickup device so as to obtain an internal parameter matrix and a distortion coefficient.

At S507, the configuration file is stored.

And storing the internal reference calibration into a configuration file.

In S509, distortion correction is performed.

And carrying out distortion correction on the picture acquired by the monocular camera.

At S511, a picture after distortion correction is obtained.

And storing the picture after the distortion correction into a configuration file.

Referring to fig. 6, the image pickup device is first fixedly installed, its focal length, position and angle are fixed, and the angle is a large angle in a plan view, i.e., a plan view angle of 60 to 90 degrees.

According to some embodiments, after the camera is installed, the camera is connected to a corresponding computer terminal device through a wired or wireless network, a video stream frame picture is obtained by the camera device, and the computer terminal is used for image processing and data processing.

Setting initial parameters of the image pickup device, initializing the image pickup device, and identifying a circular area in a monitoring area, wherein the initialized circular area is a wellhead area in the frame image.

And performing internal parameter calibration on the image pickup equipment so as to obtain an internal parameter matrix and distortion coefficients, and storing the internal parameter matrix and the distortion coefficients into a configuration file.

And acquiring frame images of real-time video streams from a camera device, wherein the camera device is used for monitoring the road surface well cover in real time.

And acquiring an internal reference matrix and a distortion coefficient of the image pickup device, and carrying out distortion correction on the image to obtain a picture after the distortion correction.

And setting an algorithm identification area according to the acquired road surface well lid image. And determining the circumscribed rectangle of the initialization circular area on the frame image. And expanding the circumscribed rectangle according to a preset proportion. And taking the area corresponding to the expanded circumscribed rectangle as the algorithm identification area. And filling the image area outside the algorithm identification area into any color or filling the pixel average value, the maximum value, the minimum value, the mode or the median of all the pixel points outside the algorithm identification area.

According to some embodiments, the determination of the initialized circular area may be implemented by a software platform algorithm parameter configuration interface, which transmits parameters to the camera device. The image expansion proportion can be set by a user directly, is set in the configuration algorithm parameters, and is realized in the process that the software platform inputs the algorithm parameter configuration file into the camera device.

According to an example embodiment, the algorithm identification area is predetermined, and the area may be set automatically or manually.

And dividing the acquired real-time image of the road surface well lid in the current frame of the real-time video stream by a dividing algorithm to obtain an independent well lid area in the current frame, so that the subsequent calculation is convenient.

According to some embodiments, the segmentation algorithm of the well lid region of the frame image is selected in a plurality of ways. For example, HRNet method processing or deeplbv 3+ method may be used, and specific advantages and disadvantages are described in detail above and are not repeated here.

And overlapping the initialization circular area to the frame image, wherein in the algorithm identification area, the initialization circular area is overlapped to the frame image, so that the position of a well mouth can be clarified in the subsequent real-time frame image, the identification error caused by the damage of the well mouth is prevented, and the accuracy of well lid identification is improved.

Judging whether the well lid is closed or not according to the initialization circular area and the well lid area. The specific way for judging whether the well lid is closed or not can be as follows: whether the well lid is closed or not is judged by the initialization circular area and the well lid area according to the Euclidean distance of the longest secant between the initialization circular area and the well lid area. And determining the longest cutting line of the initialization circular area, which is parallel to the first coordinate axis, in the frame image as a first cutting line, and determining the longest cutting line of the well lid area, which is parallel to the first coordinate axis, in the frame image as a second cutting line. And calculating Euclidean distance between two endpoints of the first secant and two endpoints of the second secant, and if the Euclidean distance is larger than a preset threshold value, judging that the well lid is not closed and giving an alarm.

According to some embodiments, the method of judging whether the manhole cover is closed according to the Euclidean distance of the longest secant between the initialized circular area and the manhole cover area can be applied to a road section with manual inspection, the worker performs regular maintenance, and the camera system monitors sudden conditions such as sudden manhole cover displacement, loss and the like and makes up for the defect of real-time aspect of manual monitoring.

And the initialization circular area and the well lid area judge whether the well lid is closed or not. The specific mode for judging whether the well lid is closed or not can also be as follows:

and judging and calculating the intersection ratio Iou of the well lid region and the well head region, and calculating the intersection ratio Iou of the initialization circular region and the well lid region according to the intersection ratio.

And when the intersection ratio is 1, judging that the well lid is closed. And dividing the broken area inside the well cover in the well cover area, counting the pixel area of the broken area, and sending out maintenance alarm when the pixel area of the broken area is larger than a preset broken threshold value.

And when the intersection ratio is more than 0 and less than 1 or the intersection ratio is 0 and the minimum distance between the initialized circular area and the well lid area is less than or equal to a preset displacement threshold value, judging that the well lid is not closed, and carrying out classification detection on the well lid area. And classifying the image of the well lid area by using a pre-trained neural network model. According to the result of image classification, the well lid is little shifted, the well lid is opened, the well lid edge is damaged, the well lid is protruding, or the well lid is sunken, sends corresponding early warning.

According to some embodiments, the method for judging the abnormal condition of the well lid by the cross-ratio can completely replace manual work, perform threshold judgment on various conditions such as aging, damage and the like of the well lid, and then send out early warning under maintenance or corresponding conditions. The detection and judgment of the abnormal condition of the well lid are higher than the accuracy of the prior art.

According to some embodiments, the invention uses image processing technology to monitor the abnormal situation of the well lid in real time, and does not need to additionally install devices such as a sensor, a signal generator and the like. And (5) monitoring the complete road section of the equipment in part. The method can directly utilize the installed camera equipment to set programs in the computer terminal, fully utilize the existing equipment, and greatly reduce the equipment cost required by well lid monitoring and the labor cost required by equipment installation.

FIG. 7 illustrates a block diagram of a computing device according to an example embodiment of the invention.

As shown in fig. 7, computing device 30 includes processor 12 and memory 14. Computing device 30 may also include a bus 22, a network interface 16, and an I/O interface 18. The processor 12, memory 14, network interface 16, and I/O interface 18 may communicate with each other via a bus 22.

The processor 12 may include one or more general purpose CPUs (Central Processing Unit, processors), microprocessors, or application specific integrated circuits, etc. for executing relevant program instructions. According to some embodiments, computing device 30 may also include a high performance display adapter (GPU) 20 that accelerates processor 12.

Memory 14 may include machine-system-readable media in the form of volatile memory, such as Random Access Memory (RAM), read Only Memory (ROM), and/or cache memory. Memory 14 is used to store one or more programs including instructions as well as data. The processor 12 may read instructions stored in the memory 14 to perform the methods according to embodiments of the invention described above.

Computing device 30 may also communicate with one or more networks through network interface 16. The network interface 16 may be a wireless network interface.

Bus 22 may be a bus including an address bus, a data bus, a control bus, etc. Bus 22 provides a path for exchanging information between the components.

It should be noted that, in the implementation, the computing device 30 may further include other components necessary to achieve normal operation. Furthermore, it will be understood by those skilled in the art that the above-described apparatus may include only the components necessary to implement the embodiments of the present description, and not all the components shown in the drawings.

The present invention also provides a computer readable storage medium having stored thereon a computer program which when executed by a processor performs the steps of the above method. The computer readable storage medium may include, but is not limited to, any type of disk including floppy disks, optical disks, DVDs, CD-ROMs, micro-drives, and magneto-optical disks, ROM, RAM, EPROM, EEPROM, DRAM, VRAM, flash memory devices, magnetic or optical cards, nanosystems (including molecular memory ICs), network storage devices, cloud storage devices, or any type of media or device suitable for storing instructions and/or data.

Embodiments of the present invention also provide a computer program product comprising a non-transitory computer readable storage medium storing a computer program operable to cause a computer to perform part or all of the steps of any one of the methods described in the method embodiments above.

It will be clear to a person skilled in the art that the solution according to the invention can be implemented by means of software and/or hardware. "Unit" and "module" in this specification refer to software and/or hardware capable of performing a specific function, either alone or in combination with other components, where the hardware may be, for example, a field programmable gate array, an integrated circuit, or the like.

It should be noted that, for simplicity of description, the foregoing method embodiments are all described as a series of acts, but it should be understood by those skilled in the art that the present invention is not limited by the order of acts described, as some steps may be performed in other orders or concurrently in accordance with the present invention. Further, those skilled in the art will also appreciate that the embodiments described in the specification are all preferred embodiments, and that the acts and modules referred to are not necessarily required for the present invention.

In the foregoing embodiments, the descriptions of the embodiments are emphasized, and for parts of one embodiment that are not described in detail, reference may be made to related descriptions of other embodiments.

In the several embodiments provided by the present invention, it should be understood that the disclosed apparatus may be implemented in other manners. For example, the apparatus embodiments described above are merely illustrative, such as a division of units, merely a division of logic functions, and there may be additional divisions in actual implementation, such as multiple units or components may be combined or integrated into another system, or some features may be omitted, or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed with each other may be through some service interface, device or unit indirect coupling or communication connection, electrical or otherwise.

The units described as separate units may or may not be physically separate, and units shown as units may or may not be physical units, may be located in one place, or may be distributed over a plurality of network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

In addition, each functional unit in the embodiments of the present invention may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit. The integrated units may be implemented in hardware or in software functional units.

The integrated units, if implemented in the form of software functional units and sold or used as stand-alone products, may be stored in a computer readable memory. Based on this understanding, the technical solution of the present invention may be embodied essentially or in a part contributing to the prior art or in whole or in part in the form of a software product stored in a memory, comprising several instructions for causing a computer device (which may be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the method of the various embodiments of the present invention.

The exemplary embodiments of the present invention have been particularly shown and described above. It is to be understood that this invention is not limited to the precise arrangements, instrumentalities and instrumentalities described herein; on the contrary, the invention is intended to cover various modifications and equivalent arrangements included within the spirit and scope of the appended claims.

Claims

1. The method for judging the abnormal condition of the well lid is characterized by comprising the following steps of:

acquiring a frame image of a real-time video stream from the camera device, wherein the camera device is used for monitoring a road well lid in real time;

superimposing the initialization circular region to the frame image;

2. The method of claim 1, wherein determining whether the manhole cover is closed based on the initialization circular area and the manhole cover area comprises:

3. The method of claim 1, wherein determining whether the manhole cover is closed based on the initialization circular area and the manhole cover area comprises:

and calculating the intersection ratio of the initialized circular area and the well lid area.

4. The method of claim 3, wherein determining whether the manhole cover is closed based on the initialization circular area and the manhole cover area, further comprises;

when the intersection ratio is 1, judging that the well lid is closed;

5. The method of claim 3, wherein determining whether the manhole cover is closed based on the initialization circular area and the manhole cover area further comprises:

6. The method of claim 3, wherein determining whether the manhole cover is closed based on the initialization circular area and the manhole cover area further comprises:

7. The method of claim 6, wherein classifying the manhole cover area comprises:

8. The method of claim 1, further comprising setting an algorithm identification region for the frame image:

expanding the circumscribed rectangle according to a preset proportion;

9. The method of claim 1, wherein the focal length, position and angle of the imaging device are fixed and the top view angle is 60-90 degrees.

10. A computing device, comprising:

a processor; and

memory storing a computer program which, when executed by the processor, implements the method according to any of claims 1-9.