CN112270668B

CN112270668B - Suspended cable detection method and system and electronic equipment

Info

Publication number: CN112270668B
Application number: CN202011231919.3A
Authority: CN
Inventors: 王斌
Original assignee: Weihai Shiyi Electronics Co ltd
Current assignee: Weihai Shiyi Electronics Co.,Ltd.
Priority date: 2020-11-06
Filing date: 2020-11-06
Publication date: 2021-09-21
Anticipated expiration: 2040-11-06
Also published as: CN112270668A

Abstract

The application discloses a suspended cable detection method and system based on shallow low-dimensional shape characteristics and electronic equipment. The method comprises the following steps: acquiring a street view image containing cables; dividing the street view image into M × N divided images, wherein each divided image has the same size; passing the M × N segmented images through convolution layers of P layers to obtain M × N feature maps, wherein P is a positive integer greater than 1 and less than or equal to 3; pooling each of the M × N feature maps by row or column average to obtain M × N feature vectors; and inputting the M multiplied by N feature vectors into a deep neural network to obtain a classification result of the street view image, wherein the classification result is used for indicating whether the hanging state of the cable is normal or not. In this way, the suspended cable is detected by combining the shallow convolutional layer and the deep neural network and applying proper image segmentation processing, and an accurate classification result can be obtained.

Description

Suspended cable detection method and system and electronic equipment

Technical Field

The present application relates to the field of artificial intelligence technology, and more particularly, to a suspended cable detection method, system and electronic device based on shallow low-dimensional shape characteristics.

Background

The intelligent city effectively fuses information technology and advanced city operation service concepts, and provides a more convenient, efficient and flexible innovative service mode for public management for the city by carrying out digital network management on the geography, resources, environment, economy and the like of the city.

In urban management, the 'no main' cable is hung on the roadside, which is a very serious potential safety hazard. It is known that the majority of the cable without main cable is the cable pulled by private (for example, some citizens can pull small pot cover cable for convenience), and the transverse peduncles of the cables are in the air, which not only affect the appearance of the city, but also are very serious safety hazards. However, the supervision of the cable hanging condition is difficult to be carried out because: the labor cost is high, and the supervision efficiency is not high.

The deep learning, especially the development of the neural network, provides a new solution for the supervision of urban suspended cables.

Disclosure of Invention

The present application is proposed to solve the above-mentioned technical problems. The embodiment of the application provides a suspended cable detection method, a system and electronic equipment based on shallow low-dimensional shape features, wherein the suspended cable detection method, the system and the electronic equipment are combined with a shallow convolutional layer and a deep neural network, proper image segmentation processing is applied to detect the suspended cable, and accurate classification results can be obtained.

According to one aspect of the application, a suspended cable detection method based on shallow layer low-dimensional shape features is provided, and comprises the following steps:

acquiring a street view image containing cables;

dividing the street view image into M × N divided images, wherein each divided image has the same size, and M and N are positive integers greater than 1 and less than or equal to 4;

passing the M × N segmented images through convolution layers of P layers to obtain M × N feature maps, wherein P is a positive integer greater than 1 and less than or equal to 3;

pooling each of the M × N feature maps by row or column average to obtain M × N feature vectors; and

inputting the M multiplied by N feature vectors into a deep neural network to obtain a classification result of the street view image, wherein the classification result is used for indicating whether the hanging state of the cable is normal or not.

In the method for detecting a suspended cable based on the shallow low-dimensional shape feature, dividing the street view image into M × N divided images includes:

determining the panoramic degree of the street view image;

determining M and N specific numbers based on the panoramic degree of the street view image; and

dividing the street view image into the determined specific number of M × N divided images.

In the method for detecting a suspended cable based on a shallow low-dimensional shape feature, determining the panoramic degree of the street view image includes:

determining a reference distance unit in the street view image;

determining a first depth of field and a second depth of field corresponding to a first reference object and a second reference object in the streetscape image based on the reference distance unit;

determining a distance between the first reference object and the second reference object in the street view image based on the reference distance unit; and

determining a degree of panoramaging of the street view image based on the distance, the first depth of field, and the second depth of field.

In the method for detecting a suspended cable based on shallow low-dimensional shape features, the method for obtaining M × N feature maps by passing the M × N divided images through a convolution layer of P layers includes:

determining the number of model layers of the deep neural network;

determining the number of convolutional layers based on the number of model layers of the deep neural network; and

the M × N segmented images are passed through the determined number of convolution layers to obtain M × N feature maps.

In the method for detecting a suspended cable based on shallow low-dimensional shape features, the inputting the M × N feature vectors into a deep neural network to obtain a classification result of the street view image includes:

cascading the M × N feature vectors to obtain cascaded feature vectors;

inputting the concatenated feature vectors into the deep neural network to obtain classified feature vectors; and

and obtaining a classification result of the street view image by using a classification function based on the classification feature vector.

calculating a weighted sum of the M × N eigenvectors by row or column to obtain M or N weighted eigenvectors;

cascading the M or N weighted feature vectors to obtain cascaded feature vectors;

In the method for detecting the suspended cable based on the shallow low-dimensional shape feature, in the training process, the convolution layer of the P layer and the parameters of the deep neural network are synchronously updated in each iteration.

In the method for detecting the suspended cable based on the shallow low-dimensional shape feature, in the training process, the parameters of the convolution layer of the P layer are updated firstly in each iteration, and then the parameters of the deep neural network are updated.

According to another aspect of the present application, there is provided a suspended cable detection system based on shallow low dimensional shape features, comprising:

the street view image acquisition unit is used for acquiring a street view image containing cables;

an image dividing unit configured to divide the street view image obtained by the street view image obtaining unit into M × N divided images, where each divided image has the same size, and M and N are positive integers greater than 1 and less than or equal to 4;

a feature map generation unit configured to pass the M × N segmented images obtained by the image segmentation unit through a convolution layer of P layers to obtain M × N feature maps, where P is a positive integer greater than 1 and less than or equal to 3;

a feature vector generation unit, configured to pool each feature map of the M × N feature maps obtained by the feature map generation unit by a row or column average value to obtain M × N feature vectors; and

and the classification unit is used for inputting the M multiplied by N feature vectors acquired by the feature vector generation unit into a deep neural network to acquire a classification result of the street view image, and the classification result is used for indicating whether the hanging state of the cable is normal or not.

In the above system for detecting a suspended cable based on shallow low-dimensional shape features, the image segmentation unit includes:

a panoramic degree determining subunit, configured to determine a panoramic degree of the street view image;

a number determination subunit configured to determine M and N specific numbers based on the degree of panoralization of the street view image obtained by the panoralization determination subunit; and

a dividing subunit configured to divide the street view image into a specific number of M × N divided images determined by the number determining subunit.

In the above suspended cable detection system based on the shallow low-dimensional shape feature, the panoramic degree determining subunit is further configured to:

determining a reference distance unit in the street view image;

In the above-described suspended cable detection system based on the shallow low-dimensional shape feature, the feature map generation unit includes:

the model layer number determining subunit is used for determining the model layer number of the deep neural network;

the convolution layer number determining subunit is used for determining the number of the convolution layers based on the model layer number of the deep neural network determined by the model layer number determining subunit; and

and the feature map generation subunit is used for enabling the M multiplied by N divided images to pass through the convolution layers with the number determined by the convolution layer number determination subunit so as to obtain M multiplied by N feature maps.

In the above system for detecting a suspended cable based on shallow low-dimensional shape features, the classification unit is further configured to: cascading the M × N feature vectors to obtain cascaded feature vectors; inputting the concatenated feature vectors into the deep neural network to obtain classified feature vectors; and obtaining a classification result of the street view image by using a classification function based on the classification feature vector.

In the above system for detecting a suspended cable based on shallow low-dimensional shape features, the classification unit is further configured to: calculating a weighted sum of the M × N eigenvectors by row or column to obtain M or N weighted eigenvectors; cascading the M or N weighted feature vectors to obtain cascaded feature vectors; inputting the concatenated feature vectors into the deep neural network to obtain classified feature vectors; and obtaining a classification result of the street view image by using a classification function based on the classification feature vector.

According to still another aspect of the present application, there is provided an electronic apparatus including: a processor; and a memory having stored therein computer program instructions which, when executed by the processor, cause the processor to perform the method of catenary cable detection based on shallow low-dimensional shape features as described above.

According to yet another aspect of the present application, there is provided a computer readable medium having stored thereon computer program instructions which, when executed by a processor, cause the processor to perform the method of catenary cable detection based on shallow low-dimensional shape features as described above.

Compared with the prior art, the suspended cable detection method, system and electronic equipment based on the shallow low-dimensional shape features combine the shallow convolutional layer and the deep neural network, apply proper image segmentation processing to detect the suspended cable, and can obtain accurate classification results.

Drawings

The above and other objects, features and advantages of the present application will become more apparent by describing in more detail embodiments of the present application with reference to the attached drawings. The accompanying drawings are included to provide a further understanding of the embodiments of the application and are incorporated in and constitute a part of this specification, illustrate embodiments of the application and together with the description serve to explain the principles of the application. In the drawings, like reference numbers generally represent like parts or steps.

Fig. 1 illustrates an application scenario of a suspended cable detection method based on shallow low-dimensional shape features according to an embodiment of the present application.

Fig. 2 illustrates a model architecture diagram of a suspended cable detection method based on shallow low-dimensional shape features according to an embodiment of the present application.

FIG. 3 illustrates a flow chart of a suspended cable detection method based on shallow low-dimensional shape features according to an embodiment of the application.

Fig. 4 illustrates a flowchart of dividing the street view image into M × N segmented images in the suspended cable detection method based on the shallow low-dimensional shape feature according to the embodiment of the present application.

Fig. 5 is a flowchart illustrating a process of determining a panorama of the street view image in a suspended cable detection method based on a shallow low-dimensional shape feature according to an embodiment of the present application.

FIG. 6 illustrates a block diagram of a suspended cable detection system based on shallow low-dimensional shape features according to an embodiment of the application.

FIG. 7 illustrates a block diagram of an image segmentation unit in a suspended cable detection system based on shallow low-dimensional shape features according to an embodiment of the application.

Fig. 8 illustrates a block diagram of a feature map generation unit in a suspended cable detection system based on shallow low-dimensional shape features according to an embodiment of the application.

FIG. 9 illustrates a block diagram of an electronic device in accordance with an embodiment of the present application.

Detailed Description

Hereinafter, example embodiments according to the present application will be described in detail with reference to the accompanying drawings. It should be understood that the described embodiments are only some embodiments of the present application and not all embodiments of the present application, and that the present application is not limited by the example embodiments described herein.

Overview of a scene

As mentioned above, in urban management, hanging a roadside with a "no main" cable is a serious safety hazard. Most of the cable without the main cable is a private cable, and the transverse peduncles of the cables are in the air, so that the appearance of the city is influenced, and the potential safety hazard is very serious. However, the supervision of the cable hanging condition is difficult to be carried out due to high labor cost and low efficiency.

Specifically, the detection as to whether the state of the suspended cable is normal or not may be performed by image classification, that is, classifying images into an image in which the state of the suspended cable is normal and an image in which the state of the suspended cable is not normal.

The inventors of the present application found that if features are mined and classified based on a conventional image morphological model or simply by a deep neural network, it is difficult to obtain good classification results because the state of a suspended cable is normally related to the morphology of the cable itself, but because the cable usually appears as a line in the image and each part of the line has a different curvature, e.g., some parts are straight and some parts are slightly curved, this morphology is easily confused with other objects in the image, e.g., the edges of other objects, etc.

Moreover, since in the current street view image, the cable may be mixed with other objects, rather than simply existing in a line in the clear sky, its characteristics are not obvious with respect to the entire image. That is, since image classification does not perform semantic segmentation of images at a pixel level, but performs classification based on only semantic features of images, it is not easy to make accurate classification of image elements such as a messenger having the above features.

In view of the above technical problems, the inventor of the present application has the following basic concepts: considering that the convolutional layer of the shallow layer can extract low-dimensional shape features in the image, for example, the convolutional layer of the first layer can extract features such as edges and corners in the image, and the convolutional layers of the second layer and the third layer can also extract some low-dimensional shape features of the image. Furthermore, by combining the convolutional layer with the deep neural network and applying appropriate image segmentation processing to detect the suspended cable, the detection of the suspended cable can be realized by the relatively simple deep neural network for classification without adopting a complicated semantic segmentation technology.

More specifically, the inventors of the present application found that: the messenger has some integrity in one dimension of length or width and some integrity in the other dimension with respect to the image as a whole. For example, a cable across a street in an image of a street view taken along the street may only exist with the top half of the image and not with the bottom half, and thus, may be partial in the width dimension, i.e., each image taken in the width dimension has a portion of the cable, but may have integrity in the height dimension, i.e., each image taken in the height dimension may or may not contain the entirety of the cable. Therefore, by dividing the image into a certain number of divided images and extracting low-dimensional shape features with the convolution layer, such features of the suspended cable can be utilized. Furthermore, the segmented image is subjected to feature vector fusion by extracting shallow low-dimensional shape features through a shallow convolutional layer, and classification is performed on the basis of a deep neural network, so that an accurate classification result can be obtained.

Based on this, the application provides a suspended cable detection method based on shallow layer low-dimensional shape characteristics, which includes: acquiring a street view image containing cables; dividing the street view image into M × N divided images, wherein each divided image has the same size, and M and N are positive integers greater than 1 and less than or equal to 4; passing the M × N segmented images through convolution layers of P layers to obtain M × N feature maps, wherein P is a positive integer greater than 1 and less than or equal to 3; pooling each of the M × N feature maps by row or column average to obtain M × N feature vectors; and inputting the M multiplied by N feature vectors into a deep neural network to obtain a classification result of the street view image, wherein the classification result is used for indicating whether the hanging state of the cable is normal or not. In this way, the suspended cable is detected by combining the shallow convolutional layer and the deep neural network and applying proper image segmentation processing, and an accurate classification result can be obtained.

As shown in fig. 1, in the application scenario, a street view image including a cable is acquired by a camera (e.g., C as illustrated in fig. 1), and then the street view image is input into a server (e.g., S as illustrated in fig. 1) deployed with a suspended cable detection algorithm based on a shallow low-dimensional shape feature, wherein the server is configured to classify the street view image to generate a detection result of whether a suspended state of the cable is normal.

Having described the general principles of the present application, various non-limiting embodiments of the present application will now be described with reference to the accompanying drawings.

Exemplary method

FIG. 2 illustrates a flow chart of a suspended cable detection method based on shallow low-dimensional shape features according to an embodiment of the application. As shown in fig. 2, a suspended cable detection method based on shallow low-dimensional shape features according to an embodiment of the present application includes: s110, obtaining a street view image containing cables; s120, dividing the street view image into M multiplied by N divided images, wherein each divided image has the same size, and M and N are positive integers which are more than 1 and less than or equal to 4; s130, enabling the M multiplied by N divided images to pass through a convolution layer of P layers to obtain M multiplied by N characteristic maps, wherein P is a positive integer which is larger than 1 and smaller than or equal to 3; s140, pooling each feature map in the M × N feature maps according to the average value of the rows or the columns to obtain M × N feature vectors; and S150, inputting the M multiplied by N feature vectors into a deep neural network to obtain a classification result of the street view image, wherein the classification result is used for indicating whether the hanging state of the cable is normal or not.

Fig. 3 illustrates a model architecture diagram of a suspended cable detection method based on shallow low-dimensional shape features according to an embodiment of the present application. As shown in fig. 3, a model architecture of a suspended cable detection method based on shallow low-dimensional shape features according to an embodiment of the present application includes: a P-layer convolutional layer (e.g., CNN as illustrated in fig. 3) for dividing the street view image into M × N divided images (Fs as illustrated in fig. 3) having the same size before entering the P-layer convolutional layer for convolution processing the M × N divided images to obtain M × N feature maps (e.g., Ff as illustrated in fig. 3) and a deep neural network (e.g., DN as illustrated in fig. 3); then, each of the M × N feature maps is pooled by row or column average to generate M × N feature vectors (for example, Vf as illustrated in fig. 3), wherein the deep neural network is configured to perform a classification process on the M × N feature vectors to obtain a classification result of the street view image, and the classification result is used to indicate whether a hanging state of the cable is normal.

In step S110, a street view image including a cable is acquired. As described above, if street view images containing cables are classified based on a conventional image morphology model or simply by deep neural network mining features, it is difficult to obtain good classification results because the state of a suspended cable is normally related to the morphology of the cable itself, but since the cable usually appears as a line in an image and each part of the line has different curvatures, e.g., some parts are straight and some parts are slightly curved, such morphology is easily confused with other objects in the image, e.g., the edges of other objects, etc. In addition, in the street view image, the cable may be mixed with other objects, and may not be present in a clear sky in a line form, which is not obvious with respect to the overall characteristics of the image.

However, in street view images, the drop wire has its unique image characteristics: the messenger has some integrity in one dimension of length or width and some integrity in the other dimension with respect to the image as a whole. For example, a cable across a street in an image of a street view taken along the street may only exist in the top half of the image and not in the bottom half, and thus may be partial in the width dimension, i.e., each image taken in the width dimension has a portion of the cable, but may have integrity in the height dimension, i.e., each image taken in the height dimension may or may not contain the entirety of the cable. Accordingly, by using these image characteristics, the detection accuracy of whether the state of the suspended cable is normal can be improved.

In step S120, the street view image is divided into M × N divided images, where each divided image has the same size, and M and N are positive integers greater than 1 and less than or equal to 4. The reason why the street view image is subjected to the image segmentation processing is to fully utilize the distribution characteristic of the suspended cable relative to the whole image, that is, the suspended cable has a partial characteristic in one dimension of the length or the width and a whole distribution characteristic in the other dimension relative to the whole image.

In the embodiment of the present application, each of the divided images has the same size, that is, the street view image is divided into M × N divided images on average. In addition, M and N are positive integers greater than 1 and less than or equal to 4, that is, M and N take any value of {2,3,4}, and for example, in a specific implementation, the street view image may be divided into 4 (2 × 2) divided images of the same size.

In a specific example of the present application, the process of dividing the street view image into M × N divided images first includes: and determining the panoramic degree of the street view image. Here, the degree of panorama formation means a degree of the street view image between the distant view image and the near view image, and the higher the degree of panorama formation is, the more parts of the image need to be divided, and the lower the degree of panorama formation is, the less parts of the image can be divided.

It should be noted that even a panoramic image is not suitable for segmenting the image too finely, the maximum number of M and N in the present application is 4, that is, the street view image is divided into 16 segmented images of 4 × 4, so as to ensure that the cable as a single semantic object maintains semantic integrity in the segmented images.

In this example of the present application, the process of determining the degree of panorama of the street view image first includes: determining a reference distance unit in the street view image, wherein the reference distance unit can be set as a preset height size or the height of other objects in the street view image, such as the height of a pedestrian, the height of a house and the like; then, determining a first depth of field and a second depth of field corresponding to a first reference object and a second reference object in the street view image based on the reference distance unit, where the first reference object and the second reference object are objects included in the street view image, and preferably, objects included in most street view images, such as houses, pedestrians, utility poles, and the like; then, determining a distance between the first reference object and the second reference object in the street view image based on the reference distance unit; then, a degree of panoramaging of the street view image is determined based on the distance, the first depth of field, and the second depth of field.

For example, in a specific example, the reference distance unit is the height of a pedestrian or a house, the first reference object and the second reference object are the pedestrian and the house, and accordingly, if the distance between the pedestrian and the house in the street view image is close, the depth difference is large, which indicates that the degree of panorama is low, and if the distance between the pedestrian and the house in the street view image is far, the depth difference is small, which indicates that the degree of panorama is high.

Fig. 5 is a flowchart illustrating a process of determining a panorama of the street view image in a suspended cable detection method based on a shallow low-dimensional shape feature according to an embodiment of the present application. As shown in fig. 5, determining the panoramic degree of the street view image includes: s310, determining a reference distance unit in the street view image; s320, determining a first depth of field and a second depth of field corresponding to a first reference object and a second reference object in the streetscape image based on the reference distance unit; s330, determining the distance between the first reference object and the second reference object in the street view image based on the reference distance unit; and S340, determining the panoramic degree of the streetscape image based on the distance, the first depth of field and the second depth of field.

Further, after determining the degree of panoralization, determining M and N specific numbers based on the degree of panoralization of the street view image;

then, the street view image is divided into the determined specific number of M × N divided images.

Fig. 4 illustrates a flowchart of dividing the street view image into M × N segmented images in the suspended cable detection method based on the shallow low-dimensional shape feature according to the embodiment of the present application. As shown in fig. 4, dividing the street view image into M × N divided images includes: s210, determining the panoramic degree of the street view image; s220, determining M and N specific numbers based on the panoramic degree of the street view image; and S230, dividing the street view image into the determined specific number of M × N divided images.

In step S130, the M × N divided images are passed through a convolution layer of P layers to obtain M × N feature maps, where P is a positive integer greater than 1 and less than or equal to 3. That is, low-dimensional shape features are extracted with convolutional layers by dividing an image into a certain number of divided images.

It is noted that the convolutional layers have a relatively small number of layers, and therefore, can extract low-dimensional shape features in the image, for example, the convolutional layer of the first layer can extract features such as edges, corners, etc. in the image, and the convolutional layers of the second and third layers can also extract some low-dimensional shape features of the image.

In a specific implementation, the number of convolutional layers is determined based on the number of model layers of the subsequent deep neural network, and the specific relationship is as follows: the number of layers of the model of the deep neural network is large, the number of layers of the convolutional layers is small, the shape features are prevented from being over-fitted, the number of layers of the model of the deep neural network is small, the number of layers of the convolutional layers is large, and the extracted low-dimensional shape features can be richer. For example, when the number of model layers of the deep neural network is 30, one convolutional layer is used, and when the number of model layers of the deep neural network is 50, 2 convolutional layers are used.

In one example of the present application, passing the M × N segmented images through convolution layers of P layers to obtain M × N feature maps includes: determining the number of model layers of the deep neural network; determining the number of convolutional layers based on the number of model layers of the deep neural network; and passing the M × N segmented images through the determined number of convolution layers to obtain M × N feature maps.

In step S140, each of the M × N feature maps is pooled by row or column average to obtain M × N feature vectors. That is, each of the M × N feature maps is pooled by rows or columns to obtain M × N feature vectors, although it will be understood by those skilled in the art that other pooling manners, such as maximum pooling, may also be used to obtain the M × N feature vectors. In step S150, the M × N feature vectors are input to a deep neural network to obtain a classification result of the street view image, where the classification result is used to indicate whether the hanging state of the cable is normal.

In an example of the present application, inputting the M × N feature vectors into a deep neural network to obtain a classification result of the street view image, includes: cascading the M × N feature vectors to obtain cascaded feature vectors; inputting the concatenated feature vectors into the deep neural network to obtain classified feature vectors; and obtaining a classification result of the street view image by using a classification function based on the classification feature vector.

It should be understood that, by cascading the M × N feature vectors, different feature vectors can be respectively calculated by using the same node of the deep neural network to a certain extent in the process of obtaining the feature vectors through the deep neural network, so that mining of relevance among the feature vectors (for example, representing the same suspended cable) is ensured, and relevance among the different feature vectors can also be considered, so that accuracy of classification is improved.

In another example of the present application, inputting the M × N feature vectors into a deep neural network to obtain a classification result of the streetscape image, includes: calculating a weighted sum of the M × N eigenvectors by row or column to obtain M or N weighted eigenvectors; cascading the M or N weighted feature vectors to obtain cascaded feature vectors; inputting the concatenated feature vectors into the deep neural network to obtain classified feature vectors; and obtaining a classification result of the street view image by using a classification function based on the classification feature vector.

It should be understood that by calculating the weighted sum of the M × N feature vectors according to rows or columns, information in different feature vectors can be fused in the process of obtaining the feature vectors through the deep neural network, so that the method is beneficial to mining association patterns among the feature vectors, and is beneficial to mining features of different feature vectors as a whole, thereby improving the accuracy of classification. That is, the weighting process is employed in consideration of the correlation of features between the respective divided images within one row or one column of the image, such as a cable extending from the left to the right.

Furthermore, such weighting does not destroy the locality of the shape in a dimension that results from segmenting the image, since the shape features of the previously segmented image have already been extracted by the convolutional layer, where the weighting is at the level of the high-dimensional features representing the shape, rather than at the level of the low-dimensional features representing the shallow shape.

Meanwhile, after weighting processing, the weighting feature vectors are cascaded, different weighting feature vectors can be respectively calculated by the same node of the deep neural network to a certain extent in the process of obtaining the feature vectors through the deep neural network, so that the mining of the relevance among the weighting feature vectors is ensured, the relevance among the weighting feature vectors can be considered, and the classification accuracy is improved.

In particular, in the present embodiment, the classification function is a Softmax classification function, i.e., a binary classification function. That is, in the embodiment of the present application, the tag of the suspended cable is set so that the state of the suspended cable is normal, and the state of the suspended cable is not normal.

In summary, the suspended cable detection method based on the shallow low-dimensional shape features according to the embodiment of the present application is clarified, and the suspended cable detection method is performed by combining the shallow convolutional layer and the deep neural network and applying appropriate image segmentation processing, so that an accurate classification result can be obtained.

It is worth mentioning that, in the embodiment of the present application, the shallow convolutional layer and the deep neural network are obtained by training through a training street view image having a label indicating whether a suspended cable is normal or not.

It is worth noting that, in the training process, the parameters of the convolution layer of the P layer and the deep neural network are synchronously updated in each iteration, that is, in each iteration, the parameters of the convolution layer and the deep neural network are synchronously updated according to each iteration, so as to ensure the synchronicity of the parameter updating of the convolution layer and the deep neural network, and improve the training efficiency.

Alternatively, in another training scheme, the parameters of the convolutional layer of the P layers are updated first in each iteration, and then the parameters of the deep neural network are updated, that is, the parameters of the convolutional layer are updated first, and then the parameters of the deep neural network are updated later, so that the transfer of the low-dimensional shallow shape features in the deep neural network can be promoted. That is, since the number of layers of the convolutional layer is updated first, the low-dimensional shallow shape feature extracted by the updated convolutional layer can be further applied to the parameter update of the deep neural network, thereby facilitating the transfer of the low-dimensional shallow shape feature in the deep neural network and improving the training effect.

Exemplary devices

As shown in fig. 6, a suspended cable detection system 600 based on shallow low-dimensional shape features according to an embodiment of the present application includes: a street view image acquisition unit 610 for acquiring a street view image including a cable; an image dividing unit 620 configured to divide the street view image obtained by the street view image obtaining unit 610 into M × N divided images, where each divided image has the same size, and M and N are positive integers greater than 1 and less than or equal to 4; a feature map generation unit 630, configured to pass the M × N segmented images obtained by the image segmentation unit 620 through a convolution layer of P layers to obtain M × N feature maps, where P is a positive integer greater than 1 and less than or equal to 3; a feature vector generating unit 640, configured to pool, by row or column average, each of the M × N feature maps obtained by the feature map generating unit 630 to obtain M × N feature vectors; and a classification unit 650 configured to input the M × N feature vectors acquired by the feature vector generation unit 640 into a deep neural network to obtain a classification result of the street view image, where the classification result is used to indicate whether a hanging state of the cable is normal.

In one example, in the above-mentioned suspended cable detection system 600, as shown in fig. 7, the image segmentation unit 620 includes: a panorama degree determining subunit 621, configured to determine a panorama degree of the street view image; a number determination subunit 622 configured to determine M and N specific numbers based on the degree of panoralization of the street view image obtained by the panoralization determination subunit; and a dividing subunit 623 configured to divide the street view image into a specific number of M × N divided images determined by the number determining subunit 622.

In one example, in the above-mentioned suspended cable detection system 600, the panoramic degree determining subunit 621 is further configured to: determining a reference distance unit in the street view image; determining a first depth of field and a second depth of field corresponding to a first reference object and a second reference object in the streetscape image based on the reference distance unit; determining a distance between the first reference object and the second reference object in the street view image based on the reference distance unit; and determining a degree of panoramaging of the street view image based on the distance, the first depth of field, and the second depth of field.

In one example, in the suspended cable detection system 600, as shown in fig. 8, the feature map generation unit 630 includes: a model layer number determining subunit 631 configured to determine the model layer number of the deep neural network; a convolution layer number determining subunit 632, configured to determine the number of convolution layers based on the model layer number of the deep neural network determined by the model layer number determining subunit 631; and a feature map generation subunit 633, configured to pass the M × N segmented images through the convolution layers of the number determined by the convolution layer number determination subunit 632 to obtain M × N feature maps.

In one example, in the above-mentioned drop cable detection system 600, the sorting unit 650 is further configured to: cascading the M × N feature vectors to obtain cascaded feature vectors; inputting the concatenated feature vectors into the deep neural network to obtain classified feature vectors; and obtaining a classification result of the street view image by using a classification function based on the classification feature vector.

In one example, in the above-mentioned drop cable detection system 600, the sorting unit 650 is further configured to: calculating a weighted sum of the M × N eigenvectors by row or column to obtain M or N weighted eigenvectors; cascading the M or N weighted feature vectors to obtain cascaded feature vectors; inputting the concatenated feature vectors into the deep neural network to obtain classified feature vectors; and obtaining a classification result of the street view image by using a classification function based on the classification feature vector.

Here, it will be understood by those skilled in the art that the specific functions and operations of the respective units and modules in the above-described suspended cable detection system 600 have been described in detail in the above description of the suspended cable detection method based on the shallow low-dimensional shape feature with reference to fig. 1 to 5, and thus, a repetitive description thereof will be omitted.

As described above, the suspended cable detection system 600 according to the embodiment of the present application may be implemented in various terminal devices, such as a server for suspended cable monitoring and the like. In one example, the drop cable detection system 600 according to embodiments of the present application may be integrated into a terminal device as one software module and/or hardware module. For example, the drop cable detection system 600 may be a software module in the operating system of the terminal device, or may be an application developed for the terminal device; of course, the drop cable detection system 600 could equally be one of many hardware modules of the terminal device.

Alternatively, in another example, the drop cable detection system 600 and the terminal device may be separate devices, and the drop cable detection system 600 may be connected to the terminal device through a wired and/or wireless network and transmit the interaction information according to an agreed data format.

Exemplary electronic device

Next, an electronic apparatus according to an embodiment of the present application is described with reference to fig. 9.

As shown in fig. 9, the electronic device 10 includes one or more processors 11 and memory 12.

The processor 11 may be a Central Processing Unit (CPU) or other form of processing unit having data processing capabilities and/or instruction execution capabilities, and may control other components in the electronic device 10 to perform desired functions.

Memory 12 may include one or more computer program products that may include various forms of computer-readable storage media, such as volatile memory and/or non-volatile memory. The volatile memory may include, for example, Random Access Memory (RAM), cache memory (cache), and/or the like. The non-volatile memory may include, for example, Read Only Memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium and executed by processor 11 to implement the functions of the shallow low-dimensional shape feature-based catenary cable detection method of the various embodiments of the present application described above and/or other desired functions. Various contents such as a detection result, a street view image, etc. may also be stored in the computer-readable storage medium.

In one example, the electronic device 10 may further include: an input device 13 and an output device 14, which are interconnected by a bus system and/or other form of connection mechanism (not shown).

The input device 13 may include, for example, a keyboard, a mouse, and the like.

The output device 14 can output various information including the detection result to the outside. The output devices 14 may include, for example, a display, speakers, a printer, and a communication network and its connected remote output devices, among others.

Of course, for simplicity, only some of the components of the electronic device 10 relevant to the present application are shown in fig. 9, and components such as buses, input/output interfaces, and the like are omitted. In addition, the electronic device 10 may include any other suitable components depending on the particular application.

Exemplary computer program product and computer-readable storage Medium

In addition to the above-described methods and apparatus, embodiments of the present application may also be a computer program product comprising computer program instructions that, when executed by a processor, cause the processor to perform the steps in the functions in the method for suspended cable detection based on shallow low-dimensional shape features according to various embodiments of the present application described in the "exemplary methods" section above of this specification.

The computer program product may be written with program code for performing the operations of embodiments of the present application in any combination of one or more programming languages, including an object oriented programming language such as Java, C + + or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computing device, partly on the user's device, as a stand-alone software package, partly on the user's computing device and partly on a remote computing device, or entirely on the remote computing device or server.

Furthermore, embodiments of the present application may also be a computer-readable storage medium having stored thereon computer program instructions that, when executed by a processor, cause the processor to perform steps in functions in a method for suspended cable detection based on shallow low-dimensional shape features according to various embodiments of the present application described in the "exemplary methods" section above in this specification.

The computer-readable storage medium may take any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may include, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or a combination of any of the foregoing. More specific examples (a non-exhaustive list) of the readable storage medium include: an electrical connection having one or more wires, a portable disk, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

The foregoing describes the general principles of the present application in conjunction with specific embodiments, however, it is noted that the advantages, effects, etc. mentioned in the present application are merely examples and are not limiting, and they should not be considered essential to the various embodiments of the present application. Furthermore, the foregoing disclosure of specific details is for the purpose of illustration and description and is not intended to be limiting, since the foregoing disclosure is not intended to be exhaustive or to limit the disclosure to the precise details disclosed.

The block diagrams of devices, apparatuses, systems referred to in this application are only given as illustrative examples and are not intended to require or imply that the connections, arrangements, configurations, etc. must be made in the manner shown in the block diagrams. These devices, apparatuses, devices, systems may be connected, arranged, configured in any manner, as will be appreciated by those skilled in the art. Words such as "including," "comprising," "having," and the like are open-ended words that mean "including, but not limited to," and are used interchangeably therewith. The words "or" and "as used herein mean, and are used interchangeably with, the word" and/or, "unless the context clearly dictates otherwise. The word "such as" is used herein to mean, and is used interchangeably with, the phrase "such as but not limited to".

It should also be noted that in the devices, apparatuses, and methods of the present application, the components or steps may be decomposed and/or recombined. These decompositions and/or recombinations are to be considered as equivalents of the present application.

The previous description of the disclosed aspects is provided to enable any person skilled in the art to make or use the present application. Various modifications to these aspects will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other aspects without departing from the scope of the application. Thus, the present application is not intended to be limited to the aspects shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

The foregoing description has been presented for purposes of illustration and description. Furthermore, the description is not intended to limit embodiments of the application to the form disclosed herein. While a number of example aspects and embodiments have been discussed above, those of skill in the art will recognize certain variations, modifications, alterations, additions and sub-combinations thereof.

Claims

1. A suspended cable detection method based on shallow layer low-dimensional shape features is characterized by comprising the following steps:

acquiring a street view image containing cables;

dividing the street view image into M × N segmented images, including: determining the panoramic degree of the street view image; determining M and N specific numbers based on the panoramic degree of the street view image; and dividing the street view image into the determined specific number of M × N divided images, wherein each divided image has the same size, and M and N are positive integers greater than 1 and less than or equal to 4;

inputting the M multiplied by N feature vectors into a deep neural network to obtain a classification result of the street view image, wherein the classification result is used for indicating whether the hanging state of the cable is normal or not;

wherein determining the panoramic degree of the street view image comprises: determining a reference distance unit in the street view image; determining a first depth of field and a second depth of field corresponding to a first reference object and a second reference object in the streetscape image based on the reference distance unit; determining a distance between the first reference object and the second reference object in the street view image based on the reference distance unit; and determining a degree of panoramaging of the street view image based on the distance, the first depth of field, and the second depth of field.

2. The suspended cable detection method based on shallow low-dimensional shape features of claim 1, wherein the M x N segmented images are passed through a convolution layer of P layers to obtain M x N feature maps, and the method comprises the following steps:

determining the number of model layers of the deep neural network;

3. The method for detecting the suspended cable based on the shallow-layer low-dimensional shape feature of claim 1, wherein the inputting the M x N feature vectors into a deep neural network to obtain the classification result of the street view image comprises:

cascading the M × N feature vectors to obtain cascaded feature vectors;

4. The method for detecting the suspended cable based on the shallow-layer low-dimensional shape feature of claim 1, wherein the inputting the M x N feature vectors into a deep neural network to obtain the classification result of the street view image comprises:

5. The shallow low-dimensional shape feature-based suspended cable detection method of claim 1, wherein parameters of the convolutional layer of the P-layer and the deep neural network are updated synchronously in each iteration during training.

6. The suspended cable detection method based on shallow low-dimensional shape features of claim 1, wherein in the training process, parameters of the convolution layer of the P layer are updated firstly in each iteration, and then parameters of the deep neural network are updated.

7. A suspended cable detection system based on shallow low dimensional shape features, comprising:

an image dividing unit configured to divide the street view image obtained by the street view image obtaining unit into M × N divided images, wherein the dividing of the street view image into M × N divided images includes: determining the panoramic degree of the street view image; determining M and N specific numbers based on the panoramic degree of the street view image; and dividing the street view image into the determined specific number of M × N divided images, wherein each divided image has the same size, and M and N are positive integers greater than 1 and less than or equal to 4;

8. An electronic device, comprising:

a processor; and

a memory having stored therein computer program instructions which, when executed by the processor, cause the processor to perform the method of suspended cable detection based on shallow low dimensional shape features of any of claims 1-6.