CN114399660A

CN114399660A - Fault type determination method and device, electronic equipment and storage medium

Info

Publication number: CN114399660A
Application number: CN202111625009.8A
Authority: CN
Inventors: 赵波; 马斌斌; 张建
Original assignee: Beijing Baidu Netcom Science and Technology Co Ltd
Current assignee: Beijing Baidu Netcom Science and Technology Co Ltd
Priority date: 2021-12-28
Filing date: 2021-12-28
Publication date: 2022-04-26

Abstract

The disclosure provides a fault type determination method and device, electronic equipment, a readable storage medium and a computer program product, and relates to the field of computer vision. The specific implementation scheme is as follows: in the process of extracting the features of an image to be detected, adjusting the region weights of different image regions to obtain the image features of the image to be detected, wherein the image to be detected is an image comprising a target object; and determining a target fault type corresponding to the target object by using the image characteristics. The scheme can obtain the image characteristics of the image to be detected including the target object, and determine the target fault type corresponding to the target object by utilizing the image characteristics. Therefore, the automatic detection of the fault type of the target object is realized, and the detection efficiency of the fault type is further improved.

Description

Fault type determination method and device, electronic equipment and storage medium

Technical Field

The present disclosure relates to the field of artificial intelligence, and in particular to computer vision and image processing techniques, which are particularly applicable to scenes such as image processing.

Background

With the rapid development of internet technology and computer technology, more and more terminal devices are being manufactured to implement various functions. Meanwhile, a large amount of operation guarantee equipment of the terminal equipment is also produced and manufactured, and is used for providing operation and maintenance guarantee for the terminal equipment so as to guarantee normal operation of the terminal equipment. In order to ensure the reliability of the operation support equipment during the operation of the operation support equipment, the operation support equipment needs to be periodically subjected to fault detection.

Disclosure of Invention

The present disclosure provides a method, an apparatus, an electronic device, a readable storage medium, and a computer program product for determining a fault type to improve detection efficiency of fault detection.

According to an aspect of the present disclosure, there is provided a method of determining a type of a fault, which may include the steps of:

in the process of extracting the features of the image to be detected, adjusting the region weights of different image regions to obtain the image features of the image to be detected, wherein the image to be detected is an image comprising a target object;

and determining a target fault type corresponding to the target object by using the image characteristics.

According to a second aspect of the present disclosure, there is provided an apparatus for determining a type of a fault, the apparatus may include:

the image feature extraction unit is used for adjusting the region weights of different image regions in the process of extracting the features of the image to be detected to obtain the image features of the image to be detected, wherein the image to be detected is an image comprising a target object;

and the target fault type determining unit is used for determining a target fault type corresponding to the target object by using the image characteristics.

According to another aspect of the present disclosure, there is provided an electronic device including:

at least one processor; and

a memory communicatively coupled to the at least one processor; wherein the content of the first and second substances,

the memory stores instructions executable by the at least one processor to enable the at least one processor to perform a method according to any one of the embodiments of the present disclosure.

According to another aspect of the present disclosure, there is provided a non-transitory computer readable storage medium having stored thereon computer instructions for causing a computer to perform a method in any of the embodiments of the present disclosure.

According to another aspect of the present disclosure, there is provided a computer program product comprising computer programs/instructions, characterized in that the computer programs/instructions, when executed by a processor, implement the method in any of the embodiments of the present disclosure.

The technology disclosed by the invention can be used for obtaining the image characteristics of the image to be detected comprising the target object and determining the target fault type corresponding to the target object by utilizing the image characteristics. Therefore, the automatic detection of the fault type of the target object is realized, and the detection efficiency of the fault type is further improved.

In addition, the fault type of the target object is automatically detected, so that operation and maintenance personnel do not need to manually detect the fault of the target object, the safety of the operation and maintenance personnel is guaranteed, and the labor cost of fault detection can be reduced.

It should be understood that the statements in this section do not necessarily identify key or critical features of the embodiments of the present disclosure, nor do they limit the scope of the present disclosure. Other features of the present disclosure will become apparent from the following description.

Drawings

The drawings are included to provide a better understanding of the present solution and are not to be construed as limiting the present disclosure. Wherein:

fig. 1 is a flow chart of a method of determining a fault type provided by an embodiment of the present disclosure;

FIG. 2 is a flow chart of a method of obtaining image features provided in an embodiment of the present disclosure;

fig. 3 is a flowchart of a method for determining a target fault type provided in an embodiment of the present disclosure;

FIG. 4 is a flow chart of a probability determination method provided in an embodiment of the present disclosure;

FIG. 5 is a flow chart of another probability determination method provided in embodiments of the present disclosure;

FIG. 6 is a schematic diagram of a failure type determination process provided in an embodiment of the present disclosure;

FIG. 7 is a schematic diagram of a fault type determination apparatus provided in an embodiment of the present disclosure;

fig. 8 is a schematic view of an electronic device provided in an embodiment of the present disclosure.

Detailed Description

Exemplary embodiments of the present disclosure are described below with reference to the accompanying drawings, in which various details of the embodiments of the disclosure are included to assist understanding, and which are to be considered as merely exemplary. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the present disclosure. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.

In order to ensure the reliability of the operation support equipment during the operation of the operation support equipment, the operation support equipment needs to be periodically subjected to fault detection. Use the battery that is used for terminal equipment such as guarantee server can normal operating under the power failure condition as an example, at the operation in-process of battery, because increase when the operation is long, the inside chemical reaction of battery probably is no longer stable, and the discharge valve fracture that the battery often can appear, utmost point post acid leakage, trouble such as copper tablet corruption to seriously influence the reliability of battery power supply. Therefore, it is necessary to detect a failure of the battery.

However, the conventional method for performing periodic fault detection on the operation support equipment generally includes: and carrying out artificial fault detection by operation and maintenance personnel. The fault detection mode is low in fault detection efficiency, and potential safety hazards caused by faults of the operation guarantee equipment may exist in the detection process of the operation guarantee equipment.

In order to solve the above problem, the present disclosure provides a method for determining a fault type, and specifically, refer to fig. 1, which is a flowchart of a method for determining a fault type according to an embodiment of the present disclosure. The method may comprise the steps of:

step S101: in the process of extracting the features of the image to be detected, adjusting the region weights of different image regions to obtain the image features of the image to be detected, wherein the image to be detected is an image including a target object.

Step S102: and determining a target fault type corresponding to the target object by using the image characteristics.

The method for determining the fault type provided in the embodiment of the disclosure can obtain the image characteristics of the image to be detected including the target object, and determine the target fault type corresponding to the target object by using the image characteristics. Therefore, the automatic detection of the fault type of the target object is realized, and the detection efficiency of the fault type is further improved.

In the embodiment of the present disclosure, the image to be detected is an image acquired for the target object. For example, an image of the target object captured by the drone or an image of the target object captured by an image capturing device preset in a specified area.

The region weight is a weight perceived by the visual field of different image regions in the image feature extraction process.

The target object is generally an operation guarantee device of the terminal device, and includes but is not limited to a storage battery, a solar panel and a lithium battery. The storage battery is a battery which can realize recharging through reversible chemical reaction, and mainly refers to a lead-acid storage battery.

It should be noted that, in the embodiment of the present disclosure, the operation support device is not specifically limited, and the operation support device may also be other devices besides the storage battery, for example: power supply stations, power supply lines, etc.

The following describes in detail the implementation procedure of the method for determining the type of failure, taking the target object as a storage battery as an example. The implementation procedure of the method for determining the type of failure of the support equipment other than the storage battery is the same as that of the method for determining the type of failure when the target object is the storage battery.

Before feature extraction is carried out on the image to be detected, image preprocessing can be carried out on the image to be detected so as to enrich image information of the image, reduce pixels of the image and reduce workload when subsequent image processing is carried out on the image to be detected. The image preprocessing includes, but is not limited to, image rotation processing, image cropping processing, image increment processing, image pixel reduction processing, and the like.

The failure type is a type divided in advance for a failure that may occur in the target object. Taking the battery as an example, the types of battery failures generally include, but are not limited to, exhaust valve breakage, pole tracking, and copper plate corrosion. In addition, different target objects have different corresponding fault types.

In the embodiment of the present disclosure, a specific implementation manner of adjusting the region weights of different image regions to obtain the image features of the image to be detected is shown in fig. 2, and fig. 2 is a flowchart of a method for obtaining the image features provided in the embodiment of the present disclosure. The image feature obtaining method comprises the following steps:

step S201: and performing first-time feature extraction on the image to be detected, and reducing the region weight of other image regions in the first-time feature extraction process to obtain a first feature map of the image to be detected, wherein the other image regions are image regions except a target image region in the image to be detected, and the target image region is an image region corresponding to the region to be detected of the target object in the image to be detected.

Step S202: and performing secondary feature extraction on the first feature map, and increasing the region weight of the target image region in the secondary feature extraction process to obtain a second feature map of the image to be detected.

Step S203: and determining the second feature map as the image feature.

In the embodiment of the present disclosure, the second feature map can be focused on the target image region by reducing the region weight of the other image region in the first feature extraction process and increasing the region weight of the target image region in the second feature extraction process. Therefore, when the second feature map is determined as an image feature, the feature of the region to be detected can be highlighted in the image feature. Based on the image characteristics, the target fault type corresponding to the target object can be accurately determined.

The region to be detected is a region preset for the target object. For example, the area is preset according to the prior value of the operation and maintenance personnel. Specifically, taking the target object as the battery as an example, the types of battery failures generally include, but are not limited to, exhaust valve breakage, pole acid leakage, and copper plate corrosion. At this time, the preset region to be detected for the storage battery includes, but is not limited to, the region where the exhaust valve is located, the region where the pole is located, and the region where the copper plate is located.

In addition, in the embodiment of the present disclosure, the image features may also be obtained in the following manner: firstly, inputting an image to be detected into a trained region extraction model to obtain a target image region, wherein the region extraction model is a pre-trained model based on an image sample and a corresponding labeled image region. And then, carrying out feature extraction on the target image area to obtain the target image feature of the target image area. Finally, the target image feature is determined as the image feature.

The target image features of the target image area are obtained as the image features, the pertinence of the image features can be stronger, the existing interference features are fewer, and therefore the accuracy of the determined target fault type can be improved.

In the embodiment of the present disclosure, the specific implementation process of performing the first feature extraction on the image to be detected and reducing the region weights of other image regions in the first feature extraction process is as follows: and performing first convolution pooling on the image to be detected, and adjusting the weight value of a first convolution kernel in the first convolution pooling, wherein the first convolution kernel is a convolution kernel for reducing the region weights of other image regions.

In the first convolution pooling process, the weight value of the first convolution kernel is adjusted, so that the first convolution kernel can reduce the region weight of other image regions, and the obtained first feature map can highlight the features of the region to be detected.

In the embodiment of the present disclosure, the specific implementation process of performing the second feature extraction on the first feature map and increasing the region weight of the target image region in the second feature extraction process is as follows: and performing second convolution pooling on the first feature map, and adjusting the weight value of a second convolution kernel in the second convolution pooling, wherein the second convolution kernel is a convolution kernel for increasing the area weight of the target image area.

In the second convolution pooling process, the second convolution kernel can increase the region weights of other image regions by adjusting the weight value of the second convolution kernel, so that the obtained second feature map can further highlight the features of the region to be detected.

In the embodiment of the present disclosure, specific steps of determining a target fault type corresponding to a target object are shown in fig. 3, and fig. 3 is a flowchart of a method for determining a target fault type provided in the embodiment of the present disclosure. The determination of the target fault type comprises the following steps:

step S301: and acquiring a first probability of the target object corresponding to the candidate fault type aiming at the image characteristics by utilizing the first fault classification model.

Step S302: and obtaining a second probability of the target object corresponding to the candidate fault type aiming at the image characteristics by utilizing a second fault classification model.

Step S303: and determining the target probability of the candidate fault type corresponding to the target object by using the first probability and the second probability.

Step S304: and determining a target fault type in the candidate fault types according to the target probability, wherein the fault classification model is a model which is trained in advance based on the image feature sample and the corresponding labeled probability.

And respectively obtaining a first probability and a second probability aiming at different image characteristics, and determining the target probability of the candidate fault type corresponding to the target object based on the first probability and the second probability. Based on the target probability, a target fault type is determined among the candidate fault types. Therefore, contradictions between under-fitting and over-fitting of the fault classification model can be balanced, and accuracy of the target fault type is improved.

It should be noted that, the embodiment of the present disclosure may also determine the target fault type by only one fault classification model of the first fault classification model or the second fault classification model.

Specifically, taking the first fault classification model as an example, the step of determining the target fault type based on the first fault classification model is as follows: firstly, acquiring a first probability of a target object corresponding to a candidate fault type aiming at image characteristics by using a first fault classification model; determining the first probability as a target probability; and determining a target fault type in the candidate fault types according to the target probability.

In the embodiment of the present disclosure, the fault classification model is generally a Softmax multi-classification model constructed based on a S regression function (Softmax).

In the embodiment of the present disclosure, the determining step of the first probability is shown in fig. 4, and fig. 4 is a flowchart of a probability determining method provided in the embodiment of the present disclosure. The probability determination method comprises the following steps:

step S401: and performing feature extraction on the second feature map at a plurality of preset scales to obtain third feature maps corresponding to the image to be detected at a plurality of scales respectively.

Step S402: and fusing the plurality of third characteristic graphs to obtain a fourth characteristic graph of the image to be detected.

Step S403: and inputting the fourth feature map into the first fault classification model to obtain a first probability.

And after the third feature maps corresponding to the image to be detected in a plurality of scales are obtained, fusing the plurality of third feature maps to obtain a fourth feature map of the image to be detected. Therefore, the fourth feature map can be ensured to perform good feature representation on the image to be detected on the basis of compressing the feature dimension number and reducing the calculation amount of subsequent data processing. The process achieves the effect of guaranteeing the accuracy of fault detection while improving the detection efficiency of fault detection.

The implementation mode of extracting a plurality of preset scales of the second feature graph and fusing a plurality of third feature graphs comprises the following steps: and realizing multi-scale extraction and feature fusion of features based on a residual error series network (inclusion-ResNet, inclusion). In this case, the predetermined dimensions are 1 × 1, 3 × 3, and 5 × 5.

Taking the target object as an example of the storage battery, the fourth feature map is input to the first fault classification model, and the obtained first probability may be as follows: the failure type is that the probability of the exhaust valve breaking is 0.34, the probability of the pole creeping acid is 0.36, the probability of the copper plate corrosion is 0.2, and the probability of the failure type being that no failure occurs is 0.1.

In the embodiment of the present disclosure, the determining step of the second probability is shown in fig. 5, and fig. 5 is a flowchart of another probability determining method provided in the embodiment of the present disclosure. The probability determination method comprises the following steps:

step S501: and performing feature extraction on the fourth feature map at a plurality of preset scales to obtain fifth feature maps corresponding to the image to be detected at a plurality of scales respectively.

Step S502: and fusing the plurality of fifth feature maps to obtain a sixth feature map of the image to be detected.

Step S503: and inputting the sixth feature map into the second fault classification model to obtain a second probability.

And after the fifth feature maps corresponding to the image to be detected in a plurality of scales are obtained, fusing the fifth feature maps to obtain a sixth feature map of the image to be detected. Therefore, the sixth feature map can be ensured to perform good feature representation on the image to be detected on the basis of further compressing the feature dimension number and reducing the calculation amount of subsequent data processing. The above process achieves the effect of ensuring the accuracy of fault detection while further improving the detection efficiency of fault detection.

The implementation mode of extracting the features of the fourth feature map with a plurality of preset scales and fusing the fifth feature maps is as follows: and realizing multi-scale extraction and feature fusion of features based on a residual error series network (inclusion-ResNet). In this case, the predetermined dimensions are 1 × 1, 3 × 3, and 5 × 5.

Taking the target object as the storage battery as an example, the fifth feature map is input to the second fault classification model, and the obtained second probability may be: the failure type is that the probability of the exhaust valve breaking is 0.5, the probability of the pole creeping acid is 0.3, the probability of the copper plate corrosion is 0.18, and the probability of the failure type being that no failure occurs is 0.02.

In an embodiment of the present disclosure, the determining the target probability using the first probability and the second probability includes: and adding the first probability and the second probability to obtain the target probability. Therefore, the target probability has a better probability distinguishing effect on the probability of the candidate fault type corresponding to the target object, and the accuracy of the target fault type can be improved.

Specifically, for the first probability, the probability that the fault type is the fracture of the exhaust valve in the first probability is 0.34, and the probability that the fault type is the polar column acid-climbing is 0.36. Since the probability that the fault type is the exhaust valve fracture is closer to the probability that the fault type is the polar column acid leakage, if the polar column acid leakage is taken as the target fault type, the target fault type may be wrong.

And adding the first probability and the second probability to obtain the following target probability: the probability of the failure type being that the exhaust valve is broken is 0.84, the probability of the failure type being that the pole climbs acid is 0.66, the probability of the failure type being that the copper plate is corroded is 0.38, and the probability of the failure type being that no failure occurs is 0.12. At this time, according to the target probability, the target fault type determined in the candidate fault types is the exhaust valve fracture.

The method for determining the fault type provided in the embodiment of the present disclosure is specifically implemented when the region weight is adjusted by adjusting the weight value of the convolution kernel, the multi-scale extraction and feature fusion of the features are implemented by the inclusion network, and the target probability is determined by the Softmax multi-classification model, as shown in fig. 6. Fig. 6 is a schematic diagram of a process for determining a fault type provided in an embodiment of the present disclosure. The residual error series network 1 is used for extracting features of a plurality of preset scales from the second feature map to obtain third feature maps corresponding to the image to be detected in a plurality of scales respectively; fusing the plurality of third feature maps to obtain an inclusion network of a fourth feature map' of the image to be detected; the residual error series network 2 is used for' carrying out feature extraction on the fourth feature map at a plurality of preset scales to obtain fifth feature maps corresponding to the image to be detected at a plurality of scales respectively; fusing the plurality of fifth feature maps to obtain an inclusion network of a sixth feature map' of the image to be detected; the Softmax multi-classification model 1 is a first fault classification model; softmax multi-classification model 2 is the second fault classification model. The determination of the type of fault shown in fig. 6 is as follows:

firstly, image preprocessing work such as image rotation processing, image cutting processing, image increment processing, image pixel reduction processing and the like is performed on an image to be processed, and a gray image corresponding to the image to be processed is obtained.

Second, the gray image is subjected to a first convolution pooling process to obtain a first feature map. In the first convolution pooling, the weight value of the first convolution kernel needs to be adjusted to reduce the area weight of other image areas except the image area where the storage battery is located in the image to be detected, so that the first characteristic image shows the image characteristics of the image area where the storage battery is located in a highlighted mode.

Thirdly, the first feature map is subjected to convolution pooling for the second time to obtain a second feature map. In the second convolution pooling, the weight value of the second convolution kernel needs to be adjusted to increase the area weight of the image area where the storage battery is located in the image to be detected, so that the second characteristic diagram further highlights the image characteristics of the image area where the storage battery is located.

Fourthly, inputting the second feature map into the residual error series network 1 to obtain a fourth feature map, and inputting the fourth feature map into the residual error series network 2 and the Softmax multi-classification model 1 respectively.

Fifthly, after obtaining the fourth feature map, the Softmax multi-classification model 1 outputs the first probability for the fourth feature map. For example: the failure type is that the probability of the exhaust valve breaking is 0.34, the probability of the pole creeping acid is 0.36, the probability of the copper plate corrosion is 0.2, and the probability of the failure type being that no failure occurs is 0.1.

Sixthly, after obtaining the fourth feature map, the residual series network 2 outputs the sixth feature map for the fourth feature map, and inputs the sixth feature map into the Softmax multi-classification model 2.

Seventhly, after obtaining the sixth feature map, the Softmax multi-classification model 2 outputs the second probability for the sixth feature map. For example, the failure type is that the probability of the exhaust valve breaking is 0.5, the probability of the failure type being that the pole climbs acid is 0.3, the probability of the failure type being that the brass plate corrodes is 0.18, and the probability of the failure type being that no failure occurs is 0.02.

And eighthly, after the first probability and the second probability are obtained, correspondingly adding the first probability and the second probability to obtain a target probability. For example: the probability of the failure type being that the exhaust valve is broken is 0.84, the probability of the failure type being that the pole climbs acid is 0.66, the probability of the failure type being that the copper plate is corroded is 0.38, and the probability of the failure type being that no failure occurs is 0.12.

And ninthly, selecting the target probability with the maximum probability value, and determining the candidate fault type corresponding to the target probability with the maximum probability value as the target fault type. That is, the target failure type is determined as the exhaust valve breakage.

Tenth, the target fault type is output and displayed.

Because the operation and maintenance guarantee is provided for terminals such as servers, a large number of storage batteries are often needed, and the storage batteries can release harmful gases and leak electricity under the power-on state, so that potential safety hazards are easily caused. Therefore, the battery has a strong demand for automated detection of the type of failure. Therefore, the target object in the fault type determination method provided in the embodiment of the present disclosure is often a storage battery. At this time, the step of determining the target fault type corresponding to the target object is as follows: firstly, a storage battery fault type corresponding to the storage battery is determined, and then the storage battery fault type is determined as a target fault type.

As shown in fig. 7, an embodiment of the present disclosure provides a failure type determination apparatus, including:

the image feature extraction unit 701 is configured to, in the process of performing feature extraction on an image to be detected, adjust region weights of different image regions to obtain image features of the image to be detected, where the image to be detected is an image including a target object;

and a target fault type determining unit 702, configured to determine a target fault type corresponding to the target object by using the image feature.

In one embodiment, the image feature extraction unit 701 may further include:

the first-time feature extraction subunit is used for performing first-time feature extraction on the image to be detected, reducing the region weight of other image regions in the first-time feature extraction process, and obtaining a first feature map of the image to be detected, wherein the other image regions are image regions except a target image region in the image to be detected, and the target image region is an image region corresponding to the region to be detected of a target object in the image to be detected;

the second-time feature extraction subunit is used for performing second-time feature extraction on the first feature map, and increasing the region weight of the target image region in the second-time feature extraction process to obtain a second feature map of the image to be detected;

and the image characteristic determining subunit is used for determining the second characteristic map as the image characteristic.

In one embodiment, the first-time feature extraction subunit may further include:

and the first convolution pooling processing subunit is used for performing first convolution pooling processing on the image to be detected and adjusting a weight value of a first convolution kernel in the first convolution pooling, wherein the first convolution kernel is a convolution kernel used for reducing the area weights of other image areas.

In an embodiment, the second-time feature extraction subunit may further include:

and the second convolution pooling processing subunit is used for performing second convolution pooling processing on the first feature map, and adjusting the weight value of a second convolution kernel in the second convolution pooling, wherein the second convolution kernel is a convolution kernel used for increasing the area weight of the target image area.

In an embodiment, the target fault type determining unit 702 may further include:

the first probability obtaining subunit is used for obtaining a first probability of the candidate fault type corresponding to the target object aiming at the image characteristics by using the first fault classification model;

the second probability obtaining subunit is used for obtaining a second probability of the candidate fault type corresponding to the target object aiming at the image characteristics by using a second fault classification model;

the target probability obtaining subunit is used for determining a target probability of the candidate fault type corresponding to the target object by using the first probability and the second probability;

the target fault type determining subunit is used for determining a target fault type in the candidate fault types according to the target probability;

the fault classification model is a model pre-trained based on the image feature samples and the corresponding labeled probabilities.

In one embodiment, the first probability obtaining subunit may further include:

the third feature map obtaining subunit is used for performing feature extraction on the second feature map at a plurality of preset scales to obtain third feature maps corresponding to the image to be detected at a plurality of scales respectively;

the fourth feature map obtaining subunit is configured to fuse the plurality of third feature maps to obtain a fourth feature map of the image to be detected;

and the first fault classification model subunit is used for inputting the fourth feature map into the first fault classification model to obtain a first probability.

In one embodiment, the second probability obtaining subunit may further include:

the fifth feature map obtaining subunit is configured to perform feature extraction on the fourth feature map at multiple preset scales to obtain fifth feature maps corresponding to the image to be detected at multiple scales respectively;

a sixth feature map obtaining subunit, configured to fuse the multiple fifth feature maps to obtain a sixth feature map of the image to be detected;

and the second fault classification model subunit is used for inputting the sixth feature map into the second fault classification model to obtain a second probability.

In one embodiment, the target fault type determining subunit may further include:

and the target probability calculating subunit is used for adding the first probability and the second probability to obtain a target probability.

In an embodiment, in a case that the target object includes a storage battery, the target fault type determining unit 702 may further include:

the storage battery fault type determining subunit is used for determining a storage battery fault type corresponding to the storage battery;

and the fault type determining subunit is used for determining the storage battery fault type as a target fault type.

In the technical scheme of the disclosure, the acquisition, storage, application and the like of the personal information of the related user all accord with the regulations of related laws and regulations, and do not violate the good customs of the public order.

According to an embodiment of the present disclosure, the present disclosure also provides an electronic device and a readable storage medium.

FIG. 8 illustrates a schematic block diagram of an example electronic device 700 that can be used to implement embodiments of the present disclosure. Electronic devices are intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be examples only, and are not meant to limit implementations of the disclosure described and/or claimed herein.

As shown in fig. 8, the apparatus 800 includes a computing unit 801 that can perform various appropriate actions and processes according to a computer program stored in a Read Only Memory (ROM)802 or a computer program loaded from a storage unit 808 into a Random Access Memory (RAM) 803. In the RAM803, various programs and data required for the operation of the device 800 can also be stored. The calculation unit 801, the ROM 802, and the RAM803 are connected to each other by a bus 804. An input/output (I/O) interface 805 is also connected to bus 804.

A number of components in the device 800 are connected to the I/O interface 805, including: an input unit 806, such as a keyboard, a mouse, or the like; an output unit 807 such as various types of displays, speakers, and the like; a storage unit 808, such as a magnetic disk, optical disk, or the like; and a communication unit 809 such as a network card, modem, wireless communication transceiver, etc. The communication unit 809 allows the device 800 to exchange information/data with other devices via a computer network such as the internet and/or various telecommunication networks.

Computing unit 801 may be a variety of general and/or special purpose processing components with processing and computing capabilities. Some examples of the computing unit 801 include, but are not limited to, a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), various dedicated Artificial Intelligence (AI) computing chips, various computing units running machine learning model algorithms, a Digital Signal Processor (DSP), and any suitable processor, controller, microcontroller, and the like. The calculation unit 801 executes the respective methods and processes described above, such as the determination method of the type of failure. For example, in some embodiments, the method of determining the type of fault may be implemented as a computer software program tangibly embodied in a machine-readable medium, such as storage unit 808. In some embodiments, part or all of the computer program can be loaded and/or installed onto device 800 via ROM 802 and/or communications unit 809. When the computer program is loaded into the RAM803 and executed by the computing unit 801, one or more steps of the method of determining the type of fault described above may be performed. Alternatively, in other embodiments, the computing unit 801 may be configured to perform the method of determining the type of failure by any other suitable means (e.g., by means of firmware).

Various implementations of the systems and techniques described here above may be implemented in digital electronic circuitry, integrated circuitry, Field Programmable Gate Arrays (FPGAs), Application Specific Integrated Circuits (ASICs), Application Specific Standard Products (ASSPs), system on a chip (SOCs), Complex Programmable Logic Devices (CPLDs), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs that are executable and/or interpretable on a programmable system including at least one programmable processor, which may be special or general purpose, receiving data and instructions from, and transmitting data and instructions to, a storage system, at least one input device, and at least one output device.

Program code for implementing the methods of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable fault type determination device, such that the program codes, when executed by the processor or controller, cause the functions/operations specified in the flowchart and/or block diagram to be performed. The program code may execute entirely on the machine, partly on the machine, as a stand-alone software package partly on the machine and partly on a remote machine or entirely on the remote machine or server.

In the context of this disclosure, a machine-readable medium may be a tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and a pointing device (e.g., a mouse or a trackball) by which a user can provide input to the computer. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic, speech, or tactile input.

The systems and techniques described here can be implemented in a computing system that includes a back-end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back-end, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local Area Networks (LANs), Wide Area Networks (WANs), and the Internet.

The computer system may include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server may be a cloud server, a server of a distributed system, or a server with a combined blockchain.

It should be understood that various forms of the flows shown above may be used, with steps reordered, added, or deleted. For example, the steps described in the present disclosure may be executed in parallel, sequentially, or in different orders, as long as the desired results of the technical solutions disclosed in the present disclosure can be achieved, and the present disclosure is not limited herein.

The above detailed description should not be construed as limiting the scope of the disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may be made in accordance with design requirements and other factors. Any modification, equivalent replacement, and improvement made within the spirit and principle of the present disclosure should be included in the scope of protection of the present disclosure.

Claims

1. A method of determining a type of fault, comprising:

in the process of extracting the features of an image to be detected, adjusting the region weights of different image regions to obtain the image features of the image to be detected, wherein the image to be detected is an image comprising a target object;

2. The method according to claim 1, wherein the adjusting the region weights of the different image regions to obtain the image features of the image to be detected comprises:

performing first-time feature extraction on the image to be detected, and reducing the region weight of other image regions in the first-time feature extraction process to obtain a first feature map of the image to be detected, wherein the other image regions are image regions except a target image region in the image to be detected, and the target image region is an image region corresponding to the region to be detected of the target object in the image to be detected;

performing second feature extraction on the first feature map, and increasing the region weight of the target image region in the second feature extraction process to obtain a second feature map of the image to be detected;

determining the second feature map as the image feature.

3. The method according to claim 2, wherein the performing the first feature extraction on the image to be detected and reducing the region weight of other image regions in the first feature extraction process comprises:

and performing first convolution pooling on the image to be detected, and adjusting the weight value of a first convolution kernel in the first convolution pooling, wherein the first convolution kernel is a convolution kernel for reducing the area weight of other image areas.

4. The method of claim 2 or 3, wherein said performing a second feature extraction on said first feature map and increasing said region weight of said target image region in said second feature extraction process comprises:

and performing a second convolution pooling process on the first feature map, and adjusting a weight value of a second convolution kernel in the second convolution pooling, wherein the second convolution kernel is a convolution kernel for increasing the area weight of the target image area.

5. The method according to claim 2 or 3, wherein the determining of the target fault type corresponding to the target object comprises:

obtaining a first probability of the target object corresponding to the candidate fault type aiming at the image characteristics by utilizing a first fault classification model;

obtaining a second probability of the target object corresponding to the candidate fault type aiming at the image characteristics by utilizing a second fault classification model;

determining a target probability of the target object corresponding to the candidate fault type by using the first probability and the second probability;

determining the target fault type in the candidate fault types according to the target probability;

the fault classification model is a model pre-trained based on image feature samples and corresponding labeled probabilities.

6. The method of claim 5, wherein the determining of the first probability step comprises:

extracting the features of the second feature map at a plurality of preset scales to obtain third feature maps corresponding to the image to be detected at a plurality of scales respectively;

fusing the plurality of third feature maps to obtain a fourth feature map of the image to be detected;

and inputting the fourth feature map into the first fault classification model to obtain the first probability.

7. The method of claim 6, wherein the determining of the second probability comprises:

extracting the features of the fourth feature map at a plurality of preset scales to obtain fifth feature maps corresponding to the image to be detected at a plurality of scales respectively;

fusing the fifth feature maps to obtain a sixth feature map of the image to be detected;

and inputting the sixth feature map into the second fault classification model to obtain the second probability.

8. The method of claim 5, wherein said determining a target probability that the target object corresponds to the candidate fault type using the first probability and the second probability comprises:

and adding the first probability and the second probability to obtain the target probability.

9. The method of claim 1, wherein, in the case that the target object includes a battery, the determining a target fault type corresponding to the target object includes:

determining a storage battery fault type corresponding to the storage battery;

and determining the storage battery fault type as the target fault type.

10. A fault type determination apparatus comprising:

the image feature extraction unit is used for adjusting the region weights of different image regions in the process of extracting the features of an image to be detected to obtain the image features of the image to be detected, wherein the image to be detected is an image comprising a target object;

11. The apparatus of claim 10, wherein the image feature extraction unit comprises:

a first-time feature extraction subunit, configured to perform first-time feature extraction on the image to be detected, and reduce the region weight of other image regions in the first-time feature extraction process to obtain a first feature map of the image to be detected, where the other image regions are image regions of the image to be detected except for a target image region, and the target image region is an image region corresponding to a region to be detected of the target object in the image to be detected;

the second-time feature extraction subunit is used for performing second-time feature extraction on the first feature map, and increasing the area weight of the target image area in the second-time feature extraction process to obtain a second feature map of the image to be detected;

and the image feature determining subunit is used for determining the second feature map as the image feature.

12. The apparatus of claim 11, wherein the first-time feature extraction subunit comprises:

and the first convolution pooling processing subunit is configured to perform first convolution pooling processing on the image to be detected, and adjust a weight value of a first convolution kernel in the first convolution pooling, where the first convolution kernel is a convolution kernel used to reduce the area weight of the other image areas.

13. The apparatus according to claim 11 or 12, wherein the second-time feature extraction subunit comprises:

a second convolution pooling processing subunit, configured to perform a second convolution pooling process on the first feature map, and adjust a weight value of a second convolution kernel in the second convolution pooling, where the second convolution kernel is a convolution kernel used to increase the area weight of the target image area.

14. The apparatus according to claim 11 or 12, wherein the target fault type determination unit comprises:

a first probability obtaining subunit, configured to obtain, for the image feature, a first probability of the target object corresponding to the candidate fault type by using a first fault classification model;

a second probability obtaining subunit, configured to obtain, for the image feature, a second probability that the target object corresponds to the candidate fault type by using a second fault classification model;

a target probability obtaining subunit, configured to determine, by using the first probability and the second probability, a target probability that the target object corresponds to the candidate fault type;

a target fault type determining subunit, configured to determine the target fault type from the candidate fault types according to the target probability;

15. The apparatus of claim 14, wherein the first probability obtaining subunit comprises:

a third feature map obtaining subunit, configured to perform feature extraction on the second feature map at multiple preset scales, and obtain third feature maps corresponding to the to-be-detected image at multiple scales respectively;

a fourth feature map obtaining subunit, configured to fuse the multiple third feature maps to obtain a fourth feature map of the image to be detected;

and the first fault classification model subunit is used for inputting the fourth feature map into the first fault classification model to obtain the first probability.

16. The apparatus of claim 15, wherein the second probability obtaining subunit comprises:

a fifth feature map obtaining subunit, configured to perform feature extraction on the fourth feature map at multiple preset scales, and obtain fifth feature maps corresponding to the to-be-detected image at multiple scales respectively;

a sixth feature map obtaining subunit, configured to fuse the fifth feature maps to obtain a sixth feature map of the image to be detected;

and the second fault classification model subunit is configured to input the sixth feature map to the second fault classification model, so as to obtain the second probability.

17. The apparatus of claim 14, wherein the target fault type determination subunit comprises:

and the target probability calculating subunit is used for adding the first probability and the second probability to obtain the target probability.

18. The apparatus according to claim 10, wherein in a case where the target object includes a storage battery, the target failure type determination unit includes:

and the fault type determining subunit is used for determining the storage battery fault type as the target fault type.

19. An electronic device, comprising:

at least one processor; and

the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of any one of claims 1 to 9.

20. A non-transitory computer readable storage medium having stored thereon computer instructions for causing a computer to perform the method of any one of claims 1 to 9.

21. A computer program product comprising computer programs/instructions, wherein the computer programs/instructions, when executed by a processor, implement the steps of the method of claims 1 to 9.