CN111259969A - Failure reporting identification method, device, server and medium - Google Patents
Failure reporting identification method, device, server and medium Download PDFInfo
- Publication number
- CN111259969A CN111259969A CN202010057729.8A CN202010057729A CN111259969A CN 111259969 A CN111259969 A CN 111259969A CN 202010057729 A CN202010057729 A CN 202010057729A CN 111259969 A CN111259969 A CN 111259969A
- Authority
- CN
- China
- Prior art keywords
- fault
- picture
- vehicle
- maintenance
- type
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 41
- 238000012423 maintenance Methods 0.000 claims abstract description 161
- 238000013136 deep learning model Methods 0.000 claims abstract description 29
- 238000004458 analytical method Methods 0.000 claims abstract description 17
- 230000008439 repair process Effects 0.000 claims description 30
- 238000004590 computer program Methods 0.000 claims description 10
- 238000012545 processing Methods 0.000 description 7
- 238000005516 engineering process Methods 0.000 description 6
- 238000013507 mapping Methods 0.000 description 5
- 238000010586 diagram Methods 0.000 description 4
- 230000000694 effects Effects 0.000 description 3
- 238000004364 calculation method Methods 0.000 description 2
- 238000013527 convolutional neural network Methods 0.000 description 2
- 230000002354 daily effect Effects 0.000 description 2
- 230000007547 defect Effects 0.000 description 2
- 230000002950 deficient Effects 0.000 description 2
- 230000008030 elimination Effects 0.000 description 2
- 238000003379 elimination reaction Methods 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000000007 visual effect Effects 0.000 description 2
- 238000003491 array Methods 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000003203 everyday effect Effects 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 229920000642 polymer Polymers 0.000 description 1
- 230000008569 process Effects 0.000 description 1
- 238000009877 rendering Methods 0.000 description 1
- 230000035807 sensation Effects 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 238000010897 surface acoustic wave method Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/20—Administration of product repair or maintenance
-
- G06Q50/40—
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/74—Image or video pattern matching; Proximity measures in feature spaces
- G06V10/75—Organisation of the matching processes, e.g. simultaneous or sequential comparisons of image or video features; Coarse-fine approaches, e.g. multi-scale approaches; using context analysis; Selection of dictionaries
- G06V10/751—Comparing pixel values or logical combinations thereof, or feature values having positional relevance, e.g. template matching
Abstract
The invention provides a fault reporting identification method, a fault reporting identification device, a server and a medium, which are applied to the server, wherein the fault reporting identification method comprises the following steps: acquiring fault reporting information uploaded by user side equipment; the fault reporting information comprises fault vehicle information about a fault vehicle, a fault picture comprising a fault area and a specified fault part selected by a user; carrying out picture characteristic analysis on the fault picture to obtain a picture fault part corresponding to the fault picture; when the picture fault part is matched with the specified fault part, dividing the fault picture into a plurality of candidate image areas according to a preset picture dividing rule; and inputting the candidate image areas into a deep learning model corresponding to the fault part of the picture so as to acquire a fault type corresponding to the fault picture. The invention can automatically analyze and identify the fault reporting picture uploaded by the user, improve the auditing efficiency and accuracy of fault reporting of the user, improve the vehicle operation and maintenance efficiency and improve the vehicle using experience of the user.
Description
Technical Field
The invention relates to the field of intelligent management of vehicles, in particular to a fault reporting identification method, a fault reporting identification device, a server and a medium.
Background
The sharing of the single bicycle refers to that an enterprise provides bicycle sharing service in a campus, a subway station, a bus station, a residential area, a commercial area, a public service area and the like, and the sharing mode is a time-sharing rental mode. In the current society, with the rapid development of the internet, the sharing bicycle is rapidly popularized in cities, and great convenience is brought to people for short-distance traveling. As a shared product of urban residents, the shared bicycle is inevitably worn and damaged by others in use, which causes poor experience to users who use the shared bicycle next time, so that the fault type of the damaged shared bicycle is accurately judged in time, and the maintenance is carried out in time, which is a very necessary thing.
At present, the shared bicycle needing to be maintained is positioned mainly by means of manual auditing of fault vehicle information uploaded by a user and daily patrol and maintenance of operation and maintenance personnel, the efficiency of the two modes is too low, and no method is available for timely auditing and processing the fault information of the shared bicycle reported by the user generated every day, so that a damaged shared bicycle cannot be timely positioned and maintained, and a new challenge is provided for a shared bicycle enterprise.
Accordingly, those skilled in the art have endeavored to develop a technique capable of automatically auditing a vehicle failure.
Disclosure of Invention
In view of the above defects in the prior art, the technical problem to be solved by the present invention is to improve the efficiency and accuracy of auditing the user fault report information.
In order to achieve the above object, the present invention provides an error reporting identification method, which is applied in a server, and the error reporting identification method includes: acquiring fault reporting information uploaded by user side equipment; the fault reporting information comprises fault vehicle information about a fault vehicle, a fault picture comprising a fault area and a specified fault part selected by a user; carrying out picture characteristic analysis on the fault picture to obtain a picture fault part corresponding to the fault picture; when the picture fault part is matched with the specified fault part, dividing the fault picture into a plurality of candidate image areas according to a preset picture dividing rule; and inputting the candidate image areas into a deep learning model corresponding to the picture fault part to acquire a fault type corresponding to the fault picture.
In some embodiments, the step of performing picture characteristic analysis on the fault picture to obtain a picture fault location corresponding to the fault picture further includes: pre-storing a plurality of pictures about respective portions of a vehicle; and carrying out picture comparison operation on the fault picture according to the plurality of pictures of all parts of the vehicle so as to obtain a picture fault part matched with the fault picture.
In some embodiments, the step of inputting the plurality of candidate image regions into the depth learning model corresponding to the picture failure part to obtain a specific failure type corresponding to the picture failure part further includes: and inputting the candidate image regions into a deep learning model corresponding to the picture fault part, and classifying and regressing the candidate image regions to obtain a fault type corresponding to the picture fault part.
In some embodiments, the server further stores maintenance suggestions corresponding to the fault types in advance, wherein after the fault types are obtained, the fault types, the maintenance suggestions corresponding to the fault types, and the fault vehicle information are sent to a specified operation and maintenance end device.
In some embodiments, obtaining an optimal operation and maintenance person according to a maintenance task of each operation and maintenance person and/or a distance between an operation and maintenance end device of each operation and maintenance person and the faulty vehicle; and sending a maintenance suggestion corresponding to the fault type and the fault vehicle information to operation and maintenance end equipment of the optimal operation and maintenance personnel.
In some embodiments, the faulty vehicle information includes: the number of the fault single vehicle and the vehicle position information of the fault vehicle; and the operation and maintenance end equipment generates a maintenance route according to the position information of the operation and maintenance end equipment and the received vehicle position information of the fault vehicle.
In some embodiments, the repair recommendation includes at least one of: a repair instruction corresponding to the failure type, a repair tool corresponding to the failure type, and a type and number of replacement spare parts corresponding to the failure type.
In some embodiments, when the distance between the faulty vehicle and the operation and maintenance end device corresponding to each operation and maintenance person is greater than a preset distance threshold or the fault difficulty coefficient corresponding to the fault type is greater than a preset difficulty coefficient threshold, the repair suggestion is to abandon field repair or postpone repair.
In some embodiments, when the server receives a maintenance end picture including the fault region sent by the operation and maintenance end device, the maintenance end picture is subjected to picture region division and then sent to a specified deep learning model, so as to determine whether the fault of the fault region is eliminated.
In some embodiments, when the picture failure location does not match the specified failure location, it is determined that the failure reporting information uploaded by the user end device is invalid information, and the failure reporting identification method is ended.
In order to achieve the above object, the present invention further provides an error reporting identification device, which is applied in a server, and the error reporting identification device includes: the information acquisition module is used for acquiring fault reporting information uploaded by user side equipment; the fault reporting information comprises fault vehicle information about a fault vehicle, a fault picture comprising a fault area and a specified fault part selected by a user; the picture analysis module is used for carrying out picture characteristic analysis on the fault picture so as to obtain a picture fault part corresponding to the fault picture; the image dividing module is used for dividing the fault image into a plurality of candidate image areas according to a preset image dividing rule when the image fault part is matched with the specified fault part; and the fault type judging module is used for inputting the candidate image areas into a deep learning model corresponding to the fault part of the picture so as to acquire a fault type corresponding to the fault picture.
To achieve the above object, the present invention also provides a computer-readable storage medium on which a computer program is stored, which when executed by a processor, implements the failure recognition method as described in any one of the above.
In order to achieve the above object, the present invention also provides a server, including: a processor and a memory; the memory is used for storing a computer program, and the processor is used for executing the computer program stored by the memory to enable the server to execute the fault identification method.
The invention provides a fault reporting identification method, a fault reporting identification device, a server and a medium, which are applied to the server, wherein the fault reporting identification method comprises the following steps: acquiring fault reporting information uploaded by user side equipment; the fault reporting information comprises fault vehicle information about a fault vehicle, a fault picture comprising a fault area and a specified fault part selected by a user; carrying out picture characteristic analysis on the fault picture to obtain a picture fault part corresponding to the fault picture; when the picture fault part is matched with the specified fault part, dividing the fault picture into a plurality of candidate image areas according to a preset picture dividing rule; and inputting the candidate image areas into a deep learning model corresponding to the picture fault part to acquire a fault type corresponding to the fault picture. The invention can automatically analyze and identify the fault reporting picture uploaded by the user, improve the auditing efficiency and accuracy of fault reporting of the user, improve the vehicle operation and maintenance efficiency and improve the vehicle using experience of the user.
The conception, the specific structure and the technical effects of the present invention will be further described with reference to the accompanying drawings to fully understand the objects, the features and the effects of the present invention.
Drawings
Fig. 1 is a flow chart illustrating a fault reporting identification method according to an embodiment of the invention.
Fig. 2 is a schematic interface display diagram of a client device according to an embodiment of the invention.
Fig. 3 is a schematic view of an obstacle indicating device according to an embodiment of the present invention.
Fig. 4 is a schematic diagram of a server according to an embodiment of the present invention.
Detailed Description
The embodiments of the present invention are described below with reference to specific embodiments, and other advantages and effects of the present invention will be easily understood by those skilled in the art from the disclosure of the present specification. The invention is capable of other and different embodiments and of being practiced or of being carried out in various ways, and its several details are capable of modification in various respects, all without departing from the spirit and scope of the present invention. It is to be noted that the features in the following embodiments and examples may be combined with each other without conflict.
It should be noted that the drawings provided in the following embodiments are only for illustrating the basic idea of the present invention, and the drawings only show the components related to the present invention rather than the number, shape and size of the components in actual implementation, and the type, quantity and proportion of the components in actual implementation may be changed freely, and the layout of the components may be more complicated.
Some exemplary embodiments of the invention have been described for illustrative purposes, and it is to be understood that the invention may be practiced otherwise than as specifically described.
Currently, the positioning of a shared bicycle needing maintenance mainly depends on manual auditing of fault vehicle information uploaded by a user and daily patrol and maintenance of operation and maintenance personnel, invalid fault report information uploaded by the user cannot be accurately audited, the operation and maintenance personnel are likely to be interfered by the invalid fault information reported by the user, manpower and financial resources are wasted, the specific fault type of the reported fault cannot be effectively acquired currently, maintenance suggestions cannot be provided for the operation and maintenance personnel in advance, and the efficiency of vehicle operation and maintenance can be reduced.
Please refer to fig. 1, which is a flowchart illustrating a fault reporting identification method according to an embodiment of the present invention. The fault reporting identification method is applied to a server and comprises the following steps:
s11: acquiring fault reporting information uploaded by user side equipment; the fault reporting information comprises fault vehicle information about a fault vehicle, a fault picture comprising a fault area and a specified fault part selected by a user; the faulty vehicle may be a shared bicycle or a shared electric vehicle, etc. The user end device is, for example, a portable intelligent data processing device such as a smart phone, a tablet computer or a smart watch. The user end device runs a designated APP, and further comprises a touch screen, wherein the touch screen accepts input of a user based on touch sensation and/or tactile contact. The touch screen forms a touch sensitive surface that accepts user input. The touch screen and touch screen controller (along with any associated modules and/or sets of instructions in memory) detect contact on the touch screen (and any movement or breaking of the touch) and transform the detected contact into an interaction with a multimedia sample file (such as a picture file or video file) object displayed on the touch screen.
In one exemplary embodiment, the point of contact between the touch screen and the user corresponds to one or more fingers of the user. The touch screen may use LCD (liquid crystal display) technology or LPD (light emitting polymer display) technology, but in other embodiments other display technologies may be used.
Touch screens and touch screen controllers may detect contact and movement or breaking thereof using any of a number of touch sensitive technologies, including but not limited to capacitive, resistive, infrared, and surface acoustic wave technologies, as well as other proximity sensor arrays, or other technologies for determining one or more points of contact with a touch screen.
The touch screen displays visual output from the portable device, while the touch sensitive panel does not provide visual output. The touch screen may have a resolution of greater than 100 dpi. In one exemplary embodiment, the touch screen may have a resolution of approximately 168 dpi. The user may contact the touch screen using any suitable object or accessory, such as a stylus, finger, or the like.
The application icons displayed on the touch display screen are, for example, icons corresponding to the following applications: a mapping application, a rendering application, a word processing application, a website creation application, a disc editing application, a spreadsheet application, a gaming application, a telephone application, a video conferencing application, an email application, an instant messaging application, a fitness support application, a photo management application, a digital camera application, a digital video camera application, a web browsing application, a digital music player application, and/or a digital video player application.
After the user can touch a fault reporting interface in a display interface (shown in fig. 2) of the APP, the user can scan a two-dimensional code of a vehicle body of a fault vehicle through a camera device carried by the user equipment according to an interface prompt to acquire fault vehicle information of the fault vehicle. The faulty vehicle information includes, for example, a faulty bicycle number and vehicle position information of the faulty vehicle. And the user can prompt to open a camera device carried by the user terminal equipment to shoot the picture including the fault area through an interface, and the fault vehicle information and the fault picture including the fault area are uploaded to the server. And a plurality of fault parts can be provided in the APP interface operated by the user side equipment for the user to select, and after the user selects one fault part by touch control, the specified fault part is sent to the server. The selectable fault positions comprise nine positions of a cushion, a vehicle head, pedals, a handlebar, a brake, a mudguard, a chain, a two-dimensional code and a vehicle lock.
S12: carrying out picture characteristic analysis on the fault picture to obtain a picture fault part corresponding to the fault picture;
in a specific application, the server may store a plurality of pictures about each part of the vehicle in advance; and performing picture comparison operation on the fault picture according to the plurality of pictures of all parts of the vehicle to acquire a picture fault part matched with the fault picture. For example, the server stores images of vehicle fronts of vehicles of multiple types and at various angles in advance, and when the failure image is an image including a vehicle front, the failure image can be matched with the characteristics of the vehicle front stored in the server, and it can be determined that the image seal part included in the failure image is the vehicle front.
S13: when the picture fault part is matched with the specified fault part, dividing the fault picture into a plurality of candidate image areas according to a preset picture dividing rule; the candidate image area may or may not include a defective area.
In some embodiments, when the picture failure location is not matched with the specified failure location, it is determined that the failure reporting information uploaded by the user equipment is invalid information, and the failure reporting identification method is ended.
S14: and inputting the candidate image areas into a deep learning model corresponding to the picture fault part to acquire a fault type corresponding to the fault picture.
Wherein, in some embodiments, the server is pre-trained and stores deep learning models for each vehicle part. The deep learning model may optionally use a deep convolutional neural network.
The step of inputting the candidate image regions into the deep learning model corresponding to the picture failure part to obtain a specific failure type corresponding to the picture failure part further includes: and inputting the candidate image regions into a deep learning model corresponding to the picture fault part, and classifying and regressing the candidate image regions to obtain a fault type corresponding to the picture fault part. In a specific application, regression is performed on the candidate image regions, and the fault region can be further positioned. For example, the position of each rectangular region on the image is represented by [ x, y, w, h ], where x is the abscissa point of the upper left corner of the region, y is the ordinate of the upper left corner of the region, w is the width of the region, and h is the height of the region, and the four values of each region are adjusted through a defined depth learning model to further locate the fault region. Taking the abscissa x of the upper left corner as an example, the index adjustment function in the deep learning model is f (x), and the adjusted abscissa of the upper left corner is f (x).
By classifying the candidate image regions, a specific failure type about the failure part can be obtained. For example, through the feature processing of deep learning, w × h candidate region is extracted as an N-dimensional feature vector feature, a classification function is defined in the deep learning model, wherein the input is the feature vector of the region, the output is the probability that the region is of each fault type, it is assumed that there are 3 classes output, and when the output result is [0.9, 0.6, 0.2], the region is classified as a first class fault.
For example, the fault part is a vehicle lock, and the classification and regression of the candidate image regions can further determine whether the vehicle lock cannot be opened or is privately locked. For example, the fault location is a vehicle chain, and by classifying and regressing the plurality of candidate image regions, it is possible to further determine whether the chain is broken or loose.
In a specific application, the server also stores maintenance suggestions corresponding to the fault types in advance, wherein after the fault types are obtained, the fault types, the maintenance suggestions corresponding to the fault types and the fault vehicle information are sent to specified operation and maintenance end equipment. The operation and maintenance end device is an electronic device held by an operation and maintenance person, and in a specific embodiment, the operation and maintenance end device may be an electronic device such as a desktop computer, a tablet computer, a smart phone or a smart watch. The server can send the fault type, the maintenance suggestion corresponding to the fault type and the fault vehicle information to the operation and maintenance end equipment through a wireless module carried by the server. The wireless module is, for example, a 2G module, a 3G module, a 4G module or a 5G module. The faulty vehicle information includes, for example, a faulty bicycle number and vehicle position information of the faulty vehicle. The operation and maintenance end equipment can also generate a maintenance route according to the position information of the operation and maintenance end equipment and the received vehicle position information of the fault vehicle, and the maintenance route is identified in a map displayed on the operation and maintenance end equipment so as to provide route guidance for operation and maintenance personnel to find the fault vehicle.
In specific application, the optimal operation and maintenance personnel can be obtained according to the maintenance tasks of the operation and maintenance personnel and/or the distance between the operation and maintenance end equipment of the operation and maintenance personnel and the fault vehicle; and sending a maintenance suggestion corresponding to the fault type and the fault vehicle information to operation and maintenance end equipment of the optimal operation and maintenance personnel. For example, the operation and maintenance personnel with the least maintenance task are the optimal operation and maintenance personnel. Or the operation and maintenance personnel closest to the fault vehicle are the optimal operation and maintenance personnel. Or, the calculated proportions of the two optimal operation and maintenance personnel about the maintenance task and the distance between the operation and maintenance end equipment of each operation and maintenance personnel and the fault vehicle are preset, and the optimal operation and maintenance personnel are obtained according to the joint calculation of the two optimal operation and maintenance personnel.
In some embodiments, a one-to-one mapping table between a fault type and the maintenance suggestion may be pre-established in the server, and after the fault type is obtained, the fault type is compared with the mapping table to obtain the maintenance suggestion corresponding to the fault type. The maintenance suggestion can be displayed on a display screen of the operation and maintenance end equipment in a text or picture mode. In some embodiments, the repair recommendation includes at least one of: a repair instruction corresponding to the failure type, a repair tool corresponding to the failure type, and a type and number of replacement spare parts corresponding to the failure type. In a specific application, the server can also traverse pre-stored inventory information about a warehouse in advance to judge the type and the number of the replacement spare parts which can be provided for the fault vehicle. For example, if the current failure location is a chain, the failure type is a chain break, and the server determines that there is a replaceable chain in the inventory by traversing the inventory information about the warehouse pipe stored in advance, the repair tool corresponding to the failure type may include a wrench, the type of the replacement spare part corresponding to the failure type is a chain, and the number of the replacement spare parts is one.
In some embodiments, when the distance between the faulty vehicle and the operation and maintenance end equipment corresponding to each operation and maintenance person is greater than a preset distance threshold, the maintenance recommendation is to abandon field maintenance or postpone maintenance. The server can calculate the distance between the fault vehicle and each operation and maintenance end device according to the vehicle position information of the fault vehicle and the position information of each operation and maintenance end device. When the distance between the fault vehicle and the operation and maintenance end equipment corresponding to each operation and maintenance personnel is larger than the preset distance threshold value, the maintenance time cost is higher, and a strategy such as postponed maintenance can be implemented.
For another example, when the fault difficulty coefficient corresponding to the fault type is greater than the preset difficulty coefficient threshold, the repair suggestion is to abandon field repair or postpone repair. The difficulty coefficient corresponding to each fault type can be prestored in the server, and when the fault type of the current fault vehicle is received, the difficulty coefficient corresponding to the fault type is obtained, and the difficulty coefficient is compared with a preset difficulty coefficient threshold value. For example, the fault type is a vehicle head fracture, the difficulty coefficient corresponding to the fault type is greater than the difficulty coefficient threshold, the repair suggestion may be that field repair is abandoned, and an instruction for pulling the fault vehicle back to a warehouse may be sent to an operation and maintenance end device closest to the fault vehicle.
In some embodiments, when the server receives a maintenance end picture including the fault region sent by the operation and maintenance end device, the maintenance end picture is subjected to picture region division and then sent to a specified deep learning model, so as to determine whether the fault of the fault region is eliminated. For example, the server receives the maintenance end picture, judges that a fault part corresponding to the maintenance end picture is a chain according to picture characteristic analysis, divides the picture region of the maintenance end picture, sends the picture region divided picture to a deep learning model corresponding to the chain, judges that the fault of the fault region is eliminated when the chain is judged to be intact after passing through the deep learning model, and makes a corresponding fault elimination record.
Referring to fig. 3, a schematic composition diagram of an obstacle indicating device according to an embodiment of the invention is shown. The fault reporting identification device 1 is applied to a server, and the fault reporting identification device 1 comprises: the system comprises an information acquisition module 11, a picture analysis module 12, a picture division module 13 and a fault type judgment module 14.
The information acquisition module 11 acquires fault reporting information uploaded by user equipment; the fault reporting information comprises fault vehicle information about a fault vehicle, a fault picture comprising a fault area and a specified fault part selected by a user; the faulty vehicle may be a shared bicycle or a shared electric vehicle, etc. The user end device is, for example, a portable intelligent data processing device such as a smart phone, a tablet computer or a smart watch. The method comprises the steps that the user end equipment runs an appointed APP, the user end equipment further comprises a touch screen, and after the user can touch a fault reporting interface in a display interface (shown in figure 2) of the APP, according to interface prompt, a camera device carried by the user end equipment scans a two-dimensional code of a vehicle body of a fault vehicle to acquire fault vehicle information of the fault vehicle. The faulty vehicle information includes, for example, a faulty bicycle number and vehicle position information of the faulty vehicle. And the user can prompt to open a camera device carried by the user terminal equipment to shoot the picture including the fault area through an interface, and the fault vehicle information and the fault picture including the fault area are uploaded to the server. And a plurality of fault parts can be provided in the APP interface operated by the user side equipment for the user to select, and after the user selects one fault part by touch control, the specified fault part is sent to the server. The selectable fault positions comprise nine positions of a cushion, a vehicle head, pedals, a handlebar, a brake, a mudguard, a chain, a two-dimensional code and a vehicle lock.
The picture analysis module 12 is configured to perform picture feature analysis on the fault picture to obtain a picture fault portion corresponding to the fault picture; in a specific application, the server may store a plurality of pictures about each part of the vehicle in advance; and performing picture comparison operation on the fault picture according to the plurality of pictures of all parts of the vehicle to acquire a picture fault part matched with the fault picture. For example, the server stores images of vehicle fronts of vehicles of multiple types and at various angles in advance, and when the failure image is an image including a vehicle front, the failure image can be matched with the characteristics of the vehicle front stored in the server, and it can be determined that the image seal part included in the failure image is the vehicle front.
The picture dividing module 13 is configured to divide the faulty picture into a plurality of candidate image areas according to a preset picture dividing rule when the faulty picture is matched with the specified faulty part; the candidate image area may or may not include a defective area. In some embodiments, when the picture failure location is not matched with the specified failure location, it is determined that the failure reporting information uploaded by the user equipment is invalid information, and the operation of the failure reporting identification device is ended.
The failure type determining module 14 is configured to input the candidate image regions into a deep learning model corresponding to the failure portion of the picture, so as to obtain a failure type corresponding to the failure picture.
Wherein, in some embodiments, the server is pre-trained and stores deep learning models for each vehicle part. The deep learning model may optionally use a deep convolutional neural network.
The failure type determining module 14 is further configured to input the candidate image regions into a deep learning model corresponding to the image failure portion, and classify and regress the candidate image regions to obtain a failure type corresponding to the image failure portion. In a specific application, regression is performed on the candidate image regions, and the fault region can be further positioned. By classifying the candidate image regions, a specific failure type about the failure part can be obtained.
For example, the fault part is a vehicle lock, and the classification and regression of the candidate image regions can further determine whether the vehicle lock cannot be opened or is privately locked. For example, the fault location is a vehicle chain, and by classifying and regressing the plurality of candidate image regions, it is possible to further determine whether the chain is broken or loose.
In a specific application, the server also stores maintenance suggestions corresponding to the fault types in advance, wherein after the fault types are obtained, the fault types, the maintenance suggestions corresponding to the fault types and the fault vehicle information are sent to specified operation and maintenance end equipment. The operation and maintenance end device is an electronic device held by an operation and maintenance person, and in a specific embodiment, the operation and maintenance end device may be an electronic device such as a desktop computer, a tablet computer, a smart phone or a smart watch. The server can send the fault type, the maintenance suggestion corresponding to the fault type and the fault vehicle information to the operation and maintenance end equipment through a wireless module carried by the server. The wireless module is, for example, a 2G module, a 3G module, a 4G module or a 5G module. The faulty vehicle information includes, for example, a faulty bicycle number and vehicle position information of the faulty vehicle. The operation and maintenance end equipment can also generate a maintenance route according to the position information of the operation and maintenance end equipment and the received vehicle position information of the fault vehicle, and the maintenance route is identified in a map displayed on the operation and maintenance end equipment so as to provide route guidance for operation and maintenance personnel to find the fault vehicle.
In specific application, the optimal operation and maintenance personnel can be obtained according to the maintenance tasks of the operation and maintenance personnel and/or the distance between the operation and maintenance end equipment of the operation and maintenance personnel and the fault vehicle; and sending a maintenance suggestion corresponding to the fault type and the fault vehicle information to operation and maintenance end equipment of the optimal operation and maintenance personnel. For example, the operation and maintenance personnel with the least maintenance task are the optimal operation and maintenance personnel. Or the operation and maintenance personnel closest to the fault vehicle are the optimal operation and maintenance personnel. Or, the calculated proportions of the two optimal operation and maintenance personnel about the maintenance task and the distance between the operation and maintenance end equipment of each operation and maintenance personnel and the fault vehicle are preset, and the optimal operation and maintenance personnel are obtained according to the joint calculation of the two optimal operation and maintenance personnel.
In some embodiments, a one-to-one mapping table between a fault type and the maintenance suggestion may be pre-established in the server, and after the fault type is obtained, the fault type is compared with the mapping table to obtain the maintenance suggestion corresponding to the fault type. The maintenance suggestion can be displayed on a display screen of the operation and maintenance end equipment in a text or picture mode. In some embodiments, the repair recommendation includes at least one of: a repair instruction corresponding to the failure type, a repair tool corresponding to the failure type, and a type and number of replacement spare parts corresponding to the failure type. In a specific application, the server can also traverse pre-stored inventory information about a warehouse in advance to judge the type and the number of the replacement spare parts which can be provided for the fault vehicle. For example, if the current failure location is a chain, the failure type is a chain break, and the server determines that there is a replaceable chain in the inventory by traversing the inventory information about the warehouse pipe stored in advance, the repair tool corresponding to the failure type may include a wrench, the type of the replacement spare part corresponding to the failure type is a chain, and the number of the replacement spare parts is one.
In some embodiments, when the distance between the faulty vehicle and the operation and maintenance end equipment corresponding to each operation and maintenance person is greater than a preset distance threshold, the maintenance recommendation is to abandon field maintenance or postpone maintenance. The server can calculate the distance between the fault vehicle and each operation and maintenance end device according to the vehicle position information of the fault vehicle and the position information of each operation and maintenance end device. When the distance between the fault vehicle and the operation and maintenance end equipment corresponding to each operation and maintenance personnel is larger than the preset distance threshold value, the maintenance time cost is higher, and a strategy such as postponed maintenance can be implemented.
For another example, when the fault difficulty coefficient corresponding to the fault type is greater than the preset difficulty coefficient threshold, the repair suggestion is to abandon field repair or postpone repair. The difficulty coefficient corresponding to each fault type can be prestored in the server, and when the fault type of the current fault vehicle is received, the difficulty coefficient corresponding to the fault type is obtained, and the difficulty coefficient is compared with a preset difficulty coefficient threshold value. For example, the fault type is a vehicle head fracture, the difficulty coefficient corresponding to the fault type is greater than the difficulty coefficient threshold, the repair suggestion may be that field repair is abandoned, and an instruction for pulling the fault vehicle back to a warehouse may be sent to an operation and maintenance end device closest to the fault vehicle.
In some embodiments, when the server receives a maintenance end picture including the fault region sent by the operation and maintenance end device, the maintenance end picture is subjected to picture region division and then sent to a specified deep learning model, so as to determine whether the fault of the fault region is eliminated. For example, the server receives the maintenance end picture, judges that a fault part corresponding to the maintenance end picture is a chain according to picture characteristic analysis, divides the picture region of the maintenance end picture, sends the picture region divided picture to a deep learning model corresponding to the chain, judges that the fault of the fault region is eliminated when the chain is judged to be intact after passing through the deep learning model, and makes a corresponding fault elimination record.
In an embodiment of the present invention, a computer-readable storage medium is further provided, on which a computer program is stored, and the computer program, when executed by a processor, implements the failure recognition method, and the failure recognition method is described with reference to fig. 1 and the related description related to fig. 1.
Referring to fig. 4, a schematic composition diagram of a server according to an embodiment of the invention is shown. The server 2 includes: a processor 21 and a memory 22;
the memory 22 is used for storing a computer program, and the processor 21 is used for executing the computer program stored in the memory 22, so as to make the server 2 execute the failure recognition method shown in fig. 1.
The memory 22 includes at least one type of readable storage medium, which includes a flash memory, a hard disk, a multimedia card, a card-type memory (e.g., SD or DX memory, etc.), a Random Access Memory (RAM), a Static Random Access Memory (SRAM), a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), a programmable read-only memory (PROM), a magnetic memory, a magnetic disk, an optical disk, etc. In some embodiments, the storage 22 may be an internal storage unit of the server 2, such as a hard disk or a memory of the server 2. In other embodiments, the memory 22 may also be an external storage device of the server 2, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card), or the like, provided on the application server 2. Of course, the memory 22 may also include both an internal storage unit of the server 2 and an external storage device thereof. In this embodiment, the memory 22 is generally used for storing an operating system installed in the server 2 and various types of application software, such as program codes of the failure recognition method. The memory 22 may also be used to temporarily store various types of data that have been output or are to be output.
The processor 21 may be a Central Processing Unit (CPU), controller, microcontroller, microprocessor, or other data Processing chip in some embodiments. The processor 21 is typically used to control the overall operation of the server 2. In this embodiment, the processor 21 is configured to execute the program code stored in the memory 22 or process data, for example, execute the failure recognition method.
In summary, the present invention provides an error reporting identification method, an apparatus, a medium, and a server, where the error reporting identification method is applied in the server, and the error reporting identification method includes: acquiring fault reporting information uploaded by user side equipment; the fault reporting information comprises fault vehicle information about a fault vehicle, a fault picture comprising a fault area and a specified fault part selected by a user; carrying out picture characteristic analysis on the fault picture to obtain a picture fault part corresponding to the fault picture; when the picture fault part is matched with the specified fault part, dividing the fault picture into a plurality of candidate image areas according to a preset picture dividing rule; and inputting the candidate image areas into a deep learning model corresponding to the picture fault part to acquire a fault type corresponding to the fault picture. The invention can check the fault report of the user timely and accurately so as to improve the maintenance efficiency of the fault vehicle. Therefore, the invention effectively overcomes various defects in the prior art and has high industrial utilization value.
The foregoing embodiments are merely illustrative of the principles and utilities of the present invention and are not intended to limit the invention. Any person skilled in the art can modify or change the above-mentioned embodiments without departing from the spirit and scope of the present invention. Accordingly, it is intended that all equivalent modifications or changes which can be made by those skilled in the art without departing from the spirit and technical spirit of the present invention be covered by the claims of the present invention.
Claims (13)
1. The fault reporting identification method is applied to a server and comprises the following steps:
acquiring fault reporting information uploaded by user side equipment; the fault reporting information comprises fault vehicle information about a fault vehicle, a fault picture comprising a fault area and a specified fault part selected by a user;
carrying out picture characteristic analysis on the fault picture to obtain a picture fault part corresponding to the fault picture;
when the picture fault part is matched with the specified fault part, dividing the fault picture into a plurality of candidate image areas according to a preset picture dividing rule;
and inputting the candidate image areas into a deep learning model corresponding to the picture fault part to acquire a fault type corresponding to the fault picture.
2. The fault reporting identification method according to claim 1, wherein the step of performing picture feature analysis on the fault picture to obtain a picture fault portion corresponding to the fault picture further comprises:
pre-storing a plurality of pictures about respective portions of a vehicle;
and carrying out picture comparison operation on the fault picture according to the plurality of pictures of all parts of the vehicle so as to obtain a picture fault part matched with the fault picture.
3. The fault recognition method of claim 1, wherein the step of inputting the candidate image regions into the deep learning model corresponding to the picture fault location to obtain a specific fault type corresponding to the picture fault location further comprises: and inputting the candidate image regions into a deep learning model corresponding to the picture fault part, and classifying and regressing the candidate image regions to obtain a fault type corresponding to the picture fault part.
4. The fault reporting identification method according to claim 1, wherein a maintenance recommendation corresponding to each fault type is further stored in the server in advance, and after the fault type is obtained, the fault type, the maintenance recommendation corresponding to the fault type, and the faulty vehicle information are sent to a specified operation and maintenance end device.
5. The fault reporting identification method of claim 4, wherein the optimal operation and maintenance personnel are obtained according to the maintenance task of each operation and maintenance personnel and/or the distance between the operation and maintenance end equipment of each operation and maintenance personnel and the faulty vehicle; and sending a maintenance suggestion corresponding to the fault type and the fault vehicle information to operation and maintenance end equipment of the optimal operation and maintenance personnel.
6. The failure recognition method of claim 4, wherein the faulty vehicle information includes: the number of the fault single vehicle and the vehicle position information of the fault vehicle; and the operation and maintenance end equipment generates a maintenance route according to the position information of the operation and maintenance end equipment and the received vehicle position information of the fault vehicle.
7. The fault identification method of claim 4, wherein the repair recommendation includes at least one of: a repair instruction corresponding to the failure type, a repair tool corresponding to the failure type, and a type and number of replacement spare parts corresponding to the failure type.
8. The fault reporting identification method of claim 4, wherein when the distance between the faulty vehicle and the operation and maintenance end equipment corresponding to each operation and maintenance person is greater than a preset distance threshold or the fault difficulty coefficient corresponding to the fault type is greater than a preset difficulty coefficient threshold, the repair suggestion is to abandon field repair or postpone repair.
9. The fault reporting identification method of claim 4, wherein when the server receives a maintenance end picture including the fault region sent by the operation and maintenance end device, the server divides the maintenance end picture into picture regions and sends the picture regions to a specified deep learning model to determine whether the fault of the fault region is eliminated.
10. The method according to claim 1, wherein when the picture failure location does not match the specified failure location, determining that the failure information uploaded by the user end device is invalid information, and ending the failure recognition method.
11. The utility model provides a report fault recognition device which characterized in that, is applied to in the server, report fault recognition device includes:
the information acquisition module is used for acquiring fault reporting information uploaded by user side equipment; the fault reporting information comprises fault vehicle information about a fault vehicle, a fault picture comprising a fault area and a specified fault part selected by a user;
the picture analysis module is used for carrying out picture characteristic analysis on the fault picture so as to obtain a picture fault part corresponding to the fault picture;
the image dividing module is used for dividing the fault image into a plurality of candidate image areas according to a preset image dividing rule when the image fault part is matched with the specified fault part;
and the fault type judging module is used for inputting the candidate image areas into a deep learning model corresponding to the fault part of the picture so as to acquire a fault type corresponding to the fault picture.
12. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out a fault identification method according to any one of claims 1 to 10.
13. A server, comprising: a processor and a memory;
the memory is configured to store a computer program, and the processor is configured to execute the computer program stored by the memory to cause the server to perform the failure recognition method according to any one of claims 1 to 10.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010057729.8A CN111259969A (en) | 2020-01-19 | 2020-01-19 | Failure reporting identification method, device, server and medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010057729.8A CN111259969A (en) | 2020-01-19 | 2020-01-19 | Failure reporting identification method, device, server and medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111259969A true CN111259969A (en) | 2020-06-09 |
Family
ID=70944129
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010057729.8A Pending CN111259969A (en) | 2020-01-19 | 2020-01-19 | Failure reporting identification method, device, server and medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111259969A (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111830937A (en) * | 2020-07-23 | 2020-10-27 | 上海钧正网络科技有限公司 | Vehicle fault identification model construction and identification method and device and management system |
| CN112130180A (en) * | 2020-09-18 | 2020-12-25 | 上海钧正网络科技有限公司 | Method, device and server for detecting fault of satellite positioning equipment of shared equipment |
| CN113128357A (en) * | 2021-03-29 | 2021-07-16 | 东风柳州汽车有限公司 | Vehicle fault processing method and system |
| CN113689014A (en) * | 2021-09-01 | 2021-11-23 | 宁波喵走科技有限公司 | Shared electric vehicle maintenance management method, device, storage medium and device |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106226050A (en) * | 2016-07-15 | 2016-12-14 | 北京航空航天大学 | A kind of TFDS fault automatic identifying method |
| CN107392218A (en) * | 2017-04-11 | 2017-11-24 | 阿里巴巴集团控股有限公司 | A kind of car damage identification method based on image, device and electronic equipment |
| CN108256654A (en) * | 2018-01-02 | 2018-07-06 | 中电长城(长沙)信息技术有限公司 | A kind of equipment report barrier and fault diagnosis method and system |
| CN108734702A (en) * | 2018-04-26 | 2018-11-02 | 平安科技（深圳）有限公司 | Vehicle damages determination method, server and storage medium |
| CN108985343A (en) * | 2018-06-22 | 2018-12-11 | 深源恒际科技有限公司 | Automobile damage detecting method and system based on deep neural network |
| CN110276458A (en) * | 2019-06-25 | 2019-09-24 | 秒针信息技术有限公司 | Processing method, device, storage medium and the electronic device of vehicle trouble |
| CN110569696A (en) * | 2018-08-31 | 2019-12-13 | 阿里巴巴集团控股有限公司 | Neural network system, method and apparatus for vehicle component identification |
-
2020
- 2020-01-19 CN CN202010057729.8A patent/CN111259969A/en active Pending
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106226050A (en) * | 2016-07-15 | 2016-12-14 | 北京航空航天大学 | A kind of TFDS fault automatic identifying method |
| CN107392218A (en) * | 2017-04-11 | 2017-11-24 | 阿里巴巴集团控股有限公司 | A kind of car damage identification method based on image, device and electronic equipment |
| CN108256654A (en) * | 2018-01-02 | 2018-07-06 | 中电长城(长沙)信息技术有限公司 | A kind of equipment report barrier and fault diagnosis method and system |
| CN108734702A (en) * | 2018-04-26 | 2018-11-02 | 平安科技（深圳）有限公司 | Vehicle damages determination method, server and storage medium |
| CN108985343A (en) * | 2018-06-22 | 2018-12-11 | 深源恒际科技有限公司 | Automobile damage detecting method and system based on deep neural network |
| CN110569696A (en) * | 2018-08-31 | 2019-12-13 | 阿里巴巴集团控股有限公司 | Neural network system, method and apparatus for vehicle component identification |
| CN110276458A (en) * | 2019-06-25 | 2019-09-24 | 秒针信息技术有限公司 | Processing method, device, storage medium and the electronic device of vehicle trouble |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111830937A (en) * | 2020-07-23 | 2020-10-27 | 上海钧正网络科技有限公司 | Vehicle fault identification model construction and identification method and device and management system |
| CN111830937B (en) * | 2020-07-23 | 2021-07-02 | 上海钧正网络科技有限公司 | Vehicle fault identification model construction and identification method and device and management system |
| CN112130180A (en) * | 2020-09-18 | 2020-12-25 | 上海钧正网络科技有限公司 | Method, device and server for detecting fault of satellite positioning equipment of shared equipment |
| CN112130180B (en) * | 2020-09-18 | 2023-10-27 | 上海钧正网络科技有限公司 | Method, device and server for detecting faults of satellite positioning equipment of shared equipment |
| CN113128357A (en) * | 2021-03-29 | 2021-07-16 | 东风柳州汽车有限公司 | Vehicle fault processing method and system |
| CN113689014A (en) * | 2021-09-01 | 2021-11-23 | 宁波喵走科技有限公司 | Shared electric vehicle maintenance management method, device, storage medium and device |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| AU2018250370B2 (en) | Weakly supervised model for object detection | |
| CN111259969A (en) | Failure reporting identification method, device, server and medium | |
| US10223345B2 (en) | Interactively predicting fields in a form | |
| EP2662831B1 (en) | Comparing virtual and real images of a shopping planogram | |
| US8196066B1 (en) | Collaborative gesture-based input language | |
| EP2572295A1 (en) | System and method for web page segmentation using adaptive threshold computation | |
| US20180285965A1 (en) | Multi-dimensional font space mapping and presentation | |
| CN116168038B (en) | Image reproduction detection method and device, electronic equipment and storage medium | |
| CN114550051A (en) | Vehicle loss detection method and device, computer equipment and storage medium | |
| CN113673500A (en) | Certificate image recognition method and device, electronic equipment and storage medium | |
| CN114639087A (en) | Traffic sign detection method and device | |
| US20240095782A1 (en) | Artificial intelligence machine learning system for classifying images and producing a predetermined visual output | |
| CN108369647B (en) | Image-based quality control | |
| US20100082662A1 (en) | Information Retrieval System User Interface | |
| CN111062388A (en) | Advertisement character recognition method, system, medium and device based on deep learning | |
| CN108446693B (en) | Marking method, system, equipment and storage medium of target to be identified | |
| US9230366B1 (en) | Identification of dynamic objects based on depth data | |
| CN115186240A (en) | Social network user alignment method, device and medium based on relevance information | |
| CN112084364A (en) | Object analysis method, local image search method, device, and storage medium | |
| KR102555733B1 (en) | Object management for improving machine learning performance, control method thereof | |
| Sabek | A Performance Study of Different Deep Learning Architectures For Detecting Construction Equipment in Sites | |
| CN113970992A (en) | Image labeling method, device and system, storage medium and electronic device | |
| CN116777711A (en) | Digital city management system and method | |
| CN114549817A (en) | Seal detection method and device, computer equipment and storage medium | |
| CN114779986A (en) | System and method for navigating pages of a digital map |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20200609 |