TWI713995B - Interactive processing method, device, equipment, client device and electronic equipment for vehicle damage assessment - Google Patents

Abstract

The embodiments of this specification disclose an interactive processing method, device, processing equipment and client for vehicle damage assessment. Using the provided implementation plan, the user opens the terminal loss assessment application, starts the AR-combined shooting window to view the vehicle, and guides and feedbacks the user according to the actual vehicle position, angle and other information, and the user does not need other complicated operations such as taking photos and videos , According to the shooting guide information to complete the damage identification, and then quickly realize the damage assessment and claim settlement. In the embodiment solutions provided in this specification, the user does not need professional fixed-loss image shooting skills and complicated shooting operation steps, and the cost of the fixed-loss processing is lower. Combining with the guided shooting of AR can further improve the service experience of the user's loss-fixing service.

Description

Interactive processing method, device, equipment, client device and electronic equipment for vehicle damage assessment

The solution of the embodiment of this specification belongs to the technical field of computer terminal insurance business data processing, and in particular relates to an interactive processing method, device, processing device and client for vehicle damage assessment.

Motor vehicle insurance, namely car insurance (or car insurance for short), refers to a type of commercial insurance that compensates for personal injury or property damage caused by natural disasters or accidents. With the development of the economy, the number of motor vehicles continues to increase. At present, auto insurance has become one of the largest types of property insurance in China.

When an insured vehicle is involved in a traffic accident, the insurance company usually conducts an on-site survey first, takes pictures to obtain damage assessment images, and then conducts damage assessment. Vehicle damage assessment involves various technologies and benefits such as follow-up maintenance and evaluation, and is a very important process in the entire auto insurance service. With the development of technology and the business development needs of rapid loss assessment and claims settlement, when the current vehicle accidents are processed for damage assessment, the owner of the accident vehicle can use the car damage photos taken by mobile phones or other terminal equipment on the spot, and the images taken by the users will be uploaded to The insurance company is used to determine the vehicle damage, and then determine the maintenance plan, evaluate the damage and so on. Due to the loss of Certain shooting requirements. Due to the lack of knowledge of auto insurance or the limitation of shooting technology, the image information captured by mobile phones on site often includes a large number of non-compliant photos and video files. When an invalid fixed-loss image is collected, the user needs to take another shot, or even has lost the shooting opportunity, which affects the efficiency of the loss-fixing processing and the user's loss-fixing service experience.

Therefore, there is an urgent need in the industry for a more convenient and faster vehicle damage treatment solution.

The purpose of the embodiments of this specification is to provide an interactive processing method, device, processing equipment and client for vehicle damage assessment, which can use AR (Augmented Reality) to shoot interactive guidance, so that users can carry out the vehicle by themselves, quickly and conveniently. Loss determination, improve the processing efficiency of vehicle loss determination, and improve the interactive experience of user loss determination.

The interactive processing method, device, processing equipment and client for vehicle damage assessment provided by the embodiments of this specification are implemented in the following manner: an interactive processing method for vehicle damage assessment, the method includes: acquiring the characteristics of the vehicle through a shooting window Data; constructing an augmented reality space model of the vehicle according to the characteristic data, the augmented reality space model is displayed in the shooting window, and matching the actual space position of the vehicle in the shooting window is achieved; Performing damage recognition guidance in the shooting window based on the augmented reality space model, the damage recognition guidance including displaying shooting guide information determined based on image information obtained from the shooting window; The result information of damage identification is displayed in the shooting window.

An interactive processing device for vehicle damage assessment. The device includes: a feature acquisition module for acquiring feature data of a vehicle through a shooting window; an AR processing module for constructing an augmented reality of the vehicle based on the feature data An environment space model, where the augmented reality space model is displayed in the shooting window and is matched with the actual space position of the vehicle in the shooting window; a shooting guidance module is used to be based on the augmented reality space The model performs damage recognition guidance in the shooting window, and the damage recognition guidance includes displaying the shooting guide information determined based on the image information obtained from the shooting window; the result display module is used to display the shooting guide information in the shooting window. Display the result information of damage identification in.

An interactive processing device for vehicle damage assessment, comprising a processor and a memory for storing executable instructions of the processor. When the processor executes the instructions, it realizes: acquiring characteristic data of the vehicle through a shooting window; and according to the characteristic data Constructing an augmented reality space model of the vehicle, displaying the augmented reality space model in the shooting window, and matching the actual space position of the vehicle in the shooting window; based on the augmented reality The space model performs damage recognition guidance in the shooting window, and the damage recognition guidance includes displaying shooting guidance information determined based on the image information obtained from the shooting window; The result information of damage identification is displayed in the shooting window.

A client includes a processor and a memory for storing executable instructions of the processor. When the processor executes the instructions, it realizes: acquiring characteristic data of a vehicle through a photographing window; constructing the characteristic data of the vehicle according to the characteristic data An augmented reality space model, where the augmented reality space model is displayed in the shooting window and is matched with the actual space position of the vehicle in the shooting window; based on the augmented reality space model in the The damage recognition guidance is performed in the shooting window, and the damage recognition guidance includes displaying shooting guide information determined based on the image information obtained from the shooting window; displaying the result information of the damage recognition in the shooting window.

An electronic device includes a display screen, a processor, and a memory storing executable instructions of the processor. The processor implements the method steps described in any embodiment of the specification when the processor executes the instructions.

The interactive processing method, device, processing equipment, and client for vehicle damage assessment provided by the embodiments of this specification can use AR to instantly interact with the user in the video shooting window of the terminal to perform the interactive processing of damage identification, and guide the user to take pictures that meet the specifications. The damage recognition result can be instantly fed back in the shooting window of the terminal. Using the solution of the embodiment of this manual, the user opens the terminal loss assessment application, starts the shooting window combined with AR to view the vehicle, and guides and feedbacks the user according to the actual vehicle position, angle and other information. The user does not need other photos and videos, etc. Complicated operation, shooting according to the shooting guide information can complete damage identification, and then quickly realize damage assessment and settlement. In the example solutions provided in this specification, users can Without the need for professional fixed-loss image shooting skills and complicated shooting operation steps, the cost of fixed-loss processing is lower, and combined with AR guided shooting can further improve the service experience of users' loss-fixing services.

10: client

102: processor

104: Storage

106: Transmission module

201: Feature Acquisition Module

202: AR processing module

203: shooting guide module

204: Results display module

In order to more clearly illustrate the technical solutions in the embodiments of this specification or the prior art, the following will briefly introduce the drawings that need to be used in the embodiments or the description of the prior art. Obviously, the drawings in the following description are only the present For some of the embodiments described in the specification, for those with ordinary knowledge in the relevant technical field, other schemes can be obtained based on these schemes without creative labor.

Figure 1 is a schematic diagram of a processing flow of the method embodiment described in this specification; Figure 2 is a schematic diagram of an application scenario of AR model matching in vehicle loss assessment interaction provided in this specification; Figure 3 is another of the methods provided in this specification Figure 4 is a schematic diagram of an implementation scenario of another embodiment of the method provided in this specification; Figure 5 is a schematic diagram of an implementation scenario of another embodiment of the method provided in this specification; Figure 6 is A schematic diagram of the method flow of another embodiment of the method provided in this specification; FIG. 7 is a method of another embodiment of the method provided in this specification Schematic flow diagram; Figure 8 is a hardware structural block diagram of a client terminal that uses a method or device embodiment of the present invention for interactive processing of vehicle damage assessment; Figure 9 is a module of an embodiment of an interactive processing device for vehicle damage assessment provided in this specification Structural schematic diagram; FIG. 10 is a schematic structural diagram of an embodiment of an electronic device provided in this specification.

In order to enable persons with ordinary knowledge in the technical field to better understand the technical solutions in this specification, the following will clearly and completely describe the technical solutions in the embodiments of this specification in conjunction with the drawings in the embodiments of this specification. Obviously, The described embodiments are only a part of the embodiments in this specification, rather than all the embodiments. Based on one or more embodiments in this specification, all other embodiments obtained by a person with ordinary knowledge in the technical field without creative work shall fall within the protection scope of the embodiments of this specification.

An implementation solution provided in this specification can be applied to a client/server system architecture. The client can include terminal equipment with shooting functions used by personnel at the scene of the car damage (which can be the owner of the car in the accident, or the personnel of the insurance company or other personnel who perform damage assessment), such as smart phones, tablet computers, Smart wearable devices, dedicated fixed-loss terminals, etc. The client can have a communication module, and can communicate with a remote server to realize data transmission with the server. Said Servo The server can include the server on the side of the insurance company or the server on the side of the loss assessment service party. In other implementation scenarios, it can also include the servers of other service parties, such as those with a communication link with the server of the loss assessment service party. Terminals of parts suppliers, terminals of vehicle repair shops, etc. The server may include a single computer device, a server cluster composed of multiple servers, or a server of a distributed system. The client side can send the image data collected on-site to the server in real time, and the server side performs damage identification, maintenance plan formulation, maintenance cost calculation, etc., such as the fixed-loss server identifying damaged parts and After the degree of damage, the repair cost can be confirmed to the server of the repair shop and the amount of the claim can be confirmed to the server of the insurance company. The loss determination server will feed back the information of the claim amount given by the insurance company and the repair cost of the repair shop to the client. The implementation of processing on the server side, damage identification and other processing are performed by the server side, and the processing speed is usually higher than that on the client side, which can reduce the processing pressure of the client and increase the speed of damage recognition. Of course, this specification does not exclude that all or part of the above-mentioned processing in other embodiments is implemented by the client side, such as real-time damage detection and identification on the client side.

The following takes a specific application scenario of a mobile phone client as an example to describe the implementation of this specification. Specifically, FIG. 1 is a schematic flowchart of an embodiment of the interactive processing method for vehicle damage assessment provided in this specification. Although this specification provides method operation steps or device structures as shown in the following embodiments or drawings, the method or device may include more or fewer operation steps after partial combination based on conventional or no creative labor. Or module unit. Logically there is no necessary causal step or In the structure, the execution sequence of these steps or the module structure of the device is not limited to the execution sequence or module structure shown in the embodiments or drawings of this specification. When the described method or module structure is used in an actual device, server or terminal product application, it can be executed sequentially or in parallel according to the method or module structure shown in the embodiment or the diagram (for example, parallel processor Or multi-threaded processing environment, even including distributed processing, server cluster implementation environment). Of course, the description of the following embodiments does not limit other expandable technical solutions based on this specification. For example, in other implementation scenarios. A specific embodiment is shown in Fig. 1. In an embodiment of an interactive processing method for vehicle damage assessment provided in this specification, the method may include:

S0: Obtain the characteristic data of the vehicle through the shooting window.

In this embodiment, the client on the user side may be a smart phone, and the smart phone may have a shooting function. The user can open the mobile phone application that implements the implementation of this specification at the scene of a vehicle accident to take a view of the scene of the vehicle accident. After the client opens the application, the shooting window can be displayed on the display screen of the client, and the vehicle can be photographed through the shooting window to obtain the characteristic data of the vehicle. The shooting window may be a video shooting window, and the image information obtained through the camera integrated with the client can be displayed in the shooting window. The specific interface structure of the shooting window and related information displayed can be customized.

The feature data can be specifically set according to data processing requirements such as vehicle recognition, environment recognition, and image recognition. Generally, the feature data may include data information of each component of the identified vehicle. To construct 3D coordinate information, build an augmented reality space model of the vehicle (AR space model, a data representation method, the outline of the subject). Of course, the feature data may also include other data such as the brand, model, color, outline, and unique identification code of the vehicle.

S2: Construct an augmented reality space model of the vehicle according to the characteristic data, the augmented reality space model is displayed in the shooting window, and matching with the actual space position of the vehicle in the shooting window is achieved.

The augmented reality AR usually refers to a technical solution that calculates the position and angle of the camera image in real time and adds corresponding images, videos, and 3D models. This solution can put the virtual world on the screen ( Superimposed on) the real world and can interact. The enhanced information space model constructed by using the feature data in the embodiment of this specification may be the contour information of the vehicle, which may be specifically based on the acquired vehicle model, shooting angle, and tire position, roof position, front face position, and front size of the vehicle. Multiple characteristic data such as lamp position, tail lamp position, front and rear window positions, etc. construct the outline of the vehicle. The outline may include a data model based on 3D coordinates, and the outline contains corresponding 3D coordinate information. Then the constructed outline can be displayed in the shooting window. Of course, this specification does not exclude that the augmented reality space model described in other embodiments may also include other model forms or other model information added above the outline.

The AR model can be matched with the real vehicle position during the shooting time, such as matching the constructed 3D contour with the real vehicle contour position. In the specific matching process, the viewfinder direction can be guided, and the user can move the shooting direction or angle through the guide, and then combine the constructed outline with the shooting The contours of the real vehicle are aligned. As shown in Fig. 2, Fig. 2 is a schematic diagram of an application scenario of AR model matching in vehicle loss assessment interaction provided in this specification.

The embodiments of this specification combined with augmented reality technology not only display the real information of the vehicle captured by the user's actual client, but also display the constructed augmented reality space model information of the vehicle at the same time. The two kinds of information complement and overlap each other. , Can provide a better loss assessment service experience.

S4: Perform damage recognition guidance in the shooting window based on the augmented reality space model, the damage recognition guidance including displaying the shooting guide information determined based on the image information obtained from the shooting window.

The shooting window combined with the AR space model can more intuitively display the on-site situation of the vehicle, and can effectively perform damage assessment and shooting guidance of the vehicle damage location. The client may perform damage recognition guidance in an AR scene, and the damage recognition guidance may specifically include displaying shooting guidance information determined based on the image information obtained from the shooting window. The client can obtain the image information obtained under the AR scene in the shooting window, analyze and calculate the obtained image information, and determine what kind of shooting guide information needs to be displayed in the shooting window according to the analysis result. For example, the current position of the vehicle in the shooting window is far away, and the user may be prompted to approach the shooting in the shooting window. If the shooting position is to the left and the rear of the vehicle cannot be shot, the shooting guide information can be displayed, prompting the user to shift the shooting angle to the right. The data and information specifically processed by the damage identification guide and what kind of shooting guide information is displayed under what conditions can be preset corresponding strategies or rules, which will not be described one by one in this embodiment.

A specific damage identification guide of the method provided in this manual In an embodiment, the damage recognition guide may include: S40: Identify whether there is a suspected damage in the captured image; S42: If so, take the image based on the coordinate information of the vehicle and the suspected damage in the shooting window A matching calculation is required, and shooting guide information is determined according to the calculation result; S44: the shooting guide information is displayed in the shooting window.

In this embodiment, if a suspected damage is found in the vehicle through the image recognition algorithm, the coordinate information of the suspected damage in the actual space position of the vehicle can be calculated, and then the image shooting requirements of the suspected damage can be compared to determine What kind of operation is required for the user. According to the result of the matching calculation, the shooting guide information to be displayed is determined. For example, if a scratch on the rear fender of a vehicle is captured, and the scratch needs to be photographed from the front and along the direction of the scratch, but according to the coordinate information, it is calculated that the user is shooting at an angle of 45 degrees and the distance is scratched. The mark is far away. At this time, the user can be prompted to approach the location of the scratch, and the user is prompted to take pictures in front and along the direction of the scratch. The shooting guide information can be adjusted in real time according to the current viewfinder. For example, if the user is already close to the scratch position and meets the shooting requirements, the shooting guide information prompting the user to approach the scratch position at this time can no longer be displayed. The suspected damage can be identified by the client or server side.

The specific shooting guide information and shooting conditions that need to be displayed during shooting can be set accordingly according to the interactive design of the loss determination or the needs of the loss determination processing. In an embodiment provided in this specification, the shooting guide information may include at least one of the following: adjusting the shooting direction; Adjust the shooting angle; adjust the shooting distance; adjust the shooting light; the suspected location of the suspected damage.

The suspected damage may include damages that may be preliminarily identified, or damages that have not been calculated and confirmed by a designated damage recognition system/algorithm. Accordingly, the location area of the suspected damage may be referred to as a suspected location.

An example of shooting guidance is shown in Figure 3. Users can use real-time shooting guidance information to more conveniently and efficiently perform loss determination processing. The user shoots according to the shooting guide information, without professional shooting skills or tedious shooting operations, and the user experience is better. The above embodiment describes the shooting guide information displayed through text. In an expandable embodiment, the shooting guide information can also include display methods such as images, voices, animations, vibrations, etc. The current shooting screen is displayed through arrows or voice prompts. Aim at an area.

S6: Display the result information of damage identification in the shooting window.

The damage identification guided interactive method is used for damage shooting. The image data obtained by shooting can be further processed by the client or server, such as whether there is damage detection, damage type identification, damage component identification, maintenance costs Calculation, damage assessment and processing of nuclear damage, etc. The above processing can be attributed to the result information of damage recognition based on the AR interactive scene. One or more result information of damage recognition can be displayed in the shooting window of the client, and the user can view it in real time. In a specific embodiment, the result information of the damage recognition may include the damage location, damaged parts, maintenance plan, and repair plan determined based on the image information obtained by the damage recognition guide. At least one of the maintenance costs.

An example is shown in Figure 4. The damage recognition result information can be displayed to the user in the video interface of fixed damage shooting, and multiple damage recognition results can be displayed at the same time. For example, when the bumper and left rear fender are recognized as damaged If both of them are in the current shooting window, the result information of the damage identification of both can be displayed at the corresponding position at the same time.

Fig. 5 is a schematic diagram of an implementation scenario of another embodiment of the method provided in this specification. As shown in FIG. 5, if the result information of the target damage being identified and processed in the current shooting window has not been determined, the processing progress of the target damage can be displayed. Real-time display of target damage processing progress can further increase the user's interactive experience of damage determination. Therefore, in another embodiment of the method, before displaying the damage recognition result information of the target damage, the method may further include:

S8: Display the processing progress of the target damage.

Fig. 6 is a schematic diagram of a method flow of another embodiment of the method provided in this specification. In some embodiments, the interface window displaying the processing progress and the interface window displaying the result information may be the same interface window used, or the interface window at the same location. Of course, you can also use different interface windows.

In another embodiment, the interface window displaying the result information or processing progress can be adjusted in size adaptively according to the displayed information content, and the window position can also be moved and tracked according to the current shooting angle or shooting position. Therefore, as shown in FIG. 7, in another embodiment of the method provided in this specification, the method may further include:

S10: Displaying at least one interface window of the guide prompt information, result information, and processing progress can be tracked and changed accordingly based on the image change in the shooting window.

The tracking change may include the aforementioned position tracking, window size adjustment, or color and contour changes. For example, when the user moves and changes the shooting angle, if the damaged part A always exists in the shooting window, the result information of the damaged part A can always be displayed in the shooting window according to the user's shooting.

It should be noted that the real-time described in the above embodiments may include sending, receiving, or displaying immediately after obtaining or determining a certain data information. Those with ordinary knowledge in the technical field can understand that after caching or expected calculation, Sending, receiving or displaying after the waiting time can still fall within the real-time definition. The images described in the embodiments of this specification may include video, and the video may be regarded as a continuous image collection.

In addition, the images captured in the solutions of the embodiments of this specification can be stored in the local client or uploaded to the remote server in real time. After storing some data on the local client to prevent tampering or uploading to the server for storage, it can effectively prevent the loss of damage data from being tampered with, or insurance fraud by embezzling other data that is not the image of the accident. Therefore, the embodiments of the present specification can also improve the data security of the loss determination processing and the reliability of the loss determination results.

In the above-mentioned embodiment, the client or server side may use a pre-built or real-time damage recognition algorithm to recognize the image captured by the client. The damage recognition algorithm may include a damage recognition algorithm trained and constructed using a variety of training models, such as the deep neural network Faster R-CNN, which can be transparent After marking a large number of pictures of the damaged area in advance, a deep neural network is trained to give the range of the damaged area for the pictures of the vehicle's various directions and lighting conditions.

The above embodiments describe the implementation manners for the user to perform the interactive processing on the mobile phone client. It should be noted that the methods described in the embodiments of the present specification can be used in multiple processing devices, and in implementation scenarios including clients and servers.

The various embodiments of the above method in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referred to each other, and each embodiment focuses on the difference from other embodiments. For related details, please refer to the partial description of the method embodiment.

The method embodiments provided in the embodiments of the present invention can be executed in a mobile terminal, a PC terminal, a dedicated loss-making terminal, a server, or a similar computing device. Taking running on a mobile terminal as an example, FIG. 8 is a hardware structural block diagram of a client terminal that applies an interactive processing of vehicle damage assessment according to the method or device embodiment of the present invention. As shown in FIG. 8, the client 10 may include one or more (only one is shown in the figure) processor 102 (the processor 102 may include, but is not limited to, a processing device such as a microprocessor MCU or a programmable logic device FPGA. ), a memory 104 for storing data, and a transmission module 106 for communication. Those with ordinary knowledge in the technical field can understand that the structure shown in FIG. 8 is only for illustration and does not limit the structure of the above electronic device. For example, the client 10 may also include more or fewer components than those shown in FIG. 8, for example, may also include other processing hardware, such as a GPU (Graphics Processing Unit, graphics processing unit), or have the same components as those shown in FIG. Show the difference Configuration.

The memory 104 can be used to store software programs and modules of application software, such as the program instructions/modules corresponding to the search method in the embodiment of this specification. The processor 102 runs the software programs and modules stored in the memory 104, In this way, various functional applications and data processing are executed, that is, the processing method for displaying the contents of the navigation interactive interface is realized. The storage 104 may include a high-speed random storage, and may also include a non-volatile storage, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state storage. In some examples, the storage 104 may further include storage remotely disposed relative to the processor 102, and these remote storages may be connected to the client 10 via a network. Examples of the aforementioned networks include, but are not limited to, the Internet, corporate intranet, local area network, mobile communication network, and combinations thereof.

The transmission module 106 is used to receive or send data via a network. The above-mentioned specific examples of the network may include a wireless network provided by the communication provider of the computer terminal 10. In one example, the transmission module 106 includes a network interface controller (NIC), which can be connected to other network devices through a base station to communicate with the Internet. In one example, the transmission module 106 may be a radio frequency (RF) module, which is used to communicate with the Internet in a wireless manner.

Based on the above-mentioned image object positioning method, this specification also provides an interactive processing device for vehicle damage assessment. The described devices may include systems (including distributed systems), software (applications), modules, components, servers, clients, etc. that use the methods described in the embodiments of this specification, combined with necessary hardware-implemented equipment Device. Based on the same innovative idea, this description The processing device in an embodiment provided in the book is as described in the following embodiment. Since the implementation scheme of the device to solve the problem is similar to the method, the implementation of the specific processing device in the embodiment of this specification can refer to the implementation of the foregoing method, and the repetition will not be repeated. Although the device described in the following embodiments is preferably implemented by software, the implementation of hardware or a combination of software and hardware is also possible and conceived. Specifically, as shown in FIG. 9, FIG. 9 is a schematic diagram of the module structure of an embodiment of an interactive processing device for vehicle damage assessment provided in this specification. Specifically, it may include: a feature acquisition module 201, which can be used to acquire through a shooting window The characteristic data of the vehicle; the AR processing module 202 can be used to construct an augmented reality space model of the vehicle according to the characteristic data, the augmented reality space model is displayed in the shooting window, and the The actual space position of the vehicle in the shooting window is matched; the shooting guidance module 203 can be used to perform damage recognition guidance in the shooting window based on the augmented reality space model, and the damage recognition guidance includes displaying based on The shooting guide information determined from the image information obtained in the shooting window; the result display module 204 can be used to display the result information of damage identification in the shooting window.

It should be noted that, according to the description of the related method embodiment, the device described in the foregoing embodiment may also include other implementation manners, and a module that displays the processing progress. For specific implementation manners, reference may be made to the description of the method embodiments, which will not be repeated here.

The device model identification method provided by the embodiments of this specification can be implemented in a computer by a processor executing corresponding program instructions, such as using the c++/java language of the windows/Linux operating system to be implemented on the PC/server side, or other such as android , The necessary hardware implementation of the application programming language corresponding to the iOS system, or the implementation of processing logic based on quantum computers, etc. Specifically, the interactive processing device for vehicle damage assessment provided in this specification implements the above method. The processing device may include a processor and a memory for storing executable instructions of the processor. The processor executes all The instruction is realized: acquiring characteristic data of the vehicle through a shooting window; constructing an augmented reality space model of the vehicle according to the characteristic data, displaying the augmented reality space model in the shooting window, and realizing and Matching the actual space position of the vehicle in the shooting window; performing damage recognition guidance in the shooting window based on the augmented reality space model, and the damage recognition guidance includes displaying images based on the images obtained from the shooting window. The shooting guide information determined by the image information; the result information of the damage identification is displayed in the shooting window.

Based on the description of the foregoing method embodiment, in another embodiment of the processing device, when the processor executes the injury recognition guidance, the processor realizes: identifying whether there is a suspected injury in the captured image; if so, according to the shooting window The coordinate information of the vehicle and the image shooting requirements of the suspected damage are matched and calculated, and shooting guide information is determined according to the calculation result; the shooting guide information is displayed in the shooting window.

Based on the foregoing description of the method embodiment, in another embodiment of the processing device, the shooting guide information includes at least one of the following: adjusting the shooting direction; adjusting the shooting angle; adjusting the shooting distance; and the suspected location of the suspected damage.

Based on the description of the foregoing method embodiment, in another embodiment of the processing device, the result information of the damage recognition includes the damage location, damage component, repair plan, and repair cost determined based on the image information obtained by the damage recognition guide At least one of.

Based on the description of the foregoing method embodiments, in another embodiment of the processing device, before displaying the damage recognition result information of the target damage, the processor further executes: displaying the processing progress of the target damage.

Based on the foregoing description of the method embodiment, in another embodiment of the processing device, the processor further executes: displaying at least one interface window of the guide prompt information, result information, and processing progress based on the image change in the shooting window Follow the changes accordingly.

It should be noted that the processing device described in the foregoing embodiment may also include other extensible implementation manners according to the description of the related method embodiment. For specific implementation manners, reference may be made to the description of the method embodiments, which will not be repeated here.

The above instructions can be stored in a variety of computer-readable storage media. So The computer-readable storage medium may include a physical device for storing information, and the information may be digitized and then stored in a medium using electric, magnetic, or optical methods. The computer-readable storage medium described in this embodiment may include: devices that use electrical energy to store information, such as various types of storage, such as RAM, ROM, etc.; devices that use magnetic energy to store information, such as hard disks, floppy disks, Magnetic tapes, magnetic core storage, bubble storage, flash drives; devices that use optical methods to store information, such as CDs or DVDs. Of course, there are other ways of readable storage media, such as quantum storage, graphene storage, etc. The instructions in the device or server or client or system described in the embodiments of this specification are the same as those described above.

The above-mentioned method or device embodiment can be applied to a client on the user side, such as a smart phone. Therefore, this specification provides a client, including a processor and a memory for storing executable instructions of the processor, which is realized when the processor executes the instructions: Based on the foregoing, an embodiment of the specification also provides an electronic device , Including a display screen, a processor, and a storage for storing processor executable instructions. FIG. 10 is a schematic structural diagram of an embodiment of an electronic device provided in this specification. When the processor executes the instructions, the method steps described in any embodiment of this specification can be implemented.

The various embodiments of the device, client, electronic equipment, etc. in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on the description of other embodiments. The difference between the cases. In particular, for the hardware + program embodiment, since it is basically similar to the method embodiment, the description is relatively simple and related For details, please refer to the description of the method embodiment.

The foregoing describes specific embodiments of this specification. Other embodiments are within the scope of the attached patent application. In some cases, the actions or steps described in the scope of the patent application may be performed in a different order from the embodiment and still achieve desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or sequential order shown in order to achieve the desired result. In some embodiments, multiplexing and parallel processing are also possible or may be advantageous.

Although the present invention provides method operation steps as described in the embodiments or flowcharts, conventional or uninvented labor may include more or fewer operation steps. The sequence of steps listed in the embodiments is only one way of the execution sequence of many steps, and does not represent the only execution sequence. When the actual device or client product is executed, it can be executed sequentially or in parallel (for example, in a parallel processor or multi-threaded environment) according to the methods shown in the embodiments or drawings.

Although the contents of the embodiments of this specification mention AR technology, shooting guidance information display, shooting guidance for interacting with users, using deep neural networks to initially identify damage locations, etc., data acquisition, position arrangement, interaction, calculation, judgment, etc. However, the embodiments of this specification are not limited to complying with industry communication standards, standard image data processing protocols, communication protocols and standard data models/templates or the conditions described in the embodiments of this specification. Certain industry standards or implementations described in custom methods or examples with slight modifications can also achieve the same, equivalent or similar implementation effects of the foregoing examples, or predictable implementation effects after modification. The examples obtained by applying these modified or deformed data acquisition, storage, judgment, processing methods, etc., can still fall within the scope of the optional implementation scheme of this specification.

In the 1990s, the improvement of a technology can be clearly distinguished between hardware improvements (for example, improvements in the circuit structure of diodes, transistors, switches, etc.) or software improvements (improvements in method and process) . However, with the development of technology, the improvement of many methods and processes of today can be regarded as a direct improvement of the hardware circuit structure. Designers almost always get the corresponding hardware circuit structure by programming the improved method flow into the hardware circuit. Therefore, it cannot be said that the improvement of a method flow cannot be realized by the hardware entity module. For example, a Programmable Logic Device (PLD) (such as a Field Programmable Gate Array (FPGA)) is such an integrated circuit whose logic function is determined by the user programming the device. It is programmed by the designer to "integrate" a digital system on a PLD without requiring the chip manufacturer to design and manufacture a dedicated integrated circuit chip. Moreover, nowadays, instead of manually making integrated circuit chips, this programming is mostly realized by using "logic compiler" software, which is similar to the software compiler used in program development and writing. The source code before compilation must also be written in a specific programming language, which is called Hardware Description Language (HDL), and there is not only one type of HDL, but many types, such as ABEL (Advanced Boolean). Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, RHDL (Ruby Hardware Description Language), etc. The most commonly used is VHDL (Very-High-Speed Integrated Circuit Hardware Description Language) and Verilog. Those with general knowledge in the technical field should also be aware that only need to use the above-mentioned hardware description languages to program the method flow in a little logic and program it into the integrated circuit, then it is easy to get the logic method flow. Hardware circuit.

The controller can be implemented in any suitable manner. For example, the controller can be a microprocessor or a processor, and a computer readable program code (such as software or firmware) that can be executed by the (micro) processor. Media, logic gates, switches, application specific integrated circuits (ASIC), programmable logic controllers and embedded microcontrollers. Examples of controllers include but are not limited to the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicone Labs C8051F320, the storage controller can also be implemented as part of the storage control logic. Those with ordinary knowledge in the technical field also know that in addition to implementing the controller in purely computer-readable code, it is entirely possible to make the controller use logic gates, switches, dedicated integrated circuits, and Programmable logic controllers and embedded microcontrollers can achieve the same function. Therefore, such a controller can be regarded as a hardware component, and the devices included in it for realizing various functions can also be regarded as a structure within the hardware component. or Furthermore, the device for realizing various functions can be regarded as both a software module for realizing the method and a structure within a hardware component.

The systems, devices, modules, or units explained in the above embodiments may be implemented by computer chips or entities, or implemented by products with certain functions. A typical implementation device is a computer. Specifically, the computer can be, for example, a personal computer, a laptop computer, a vehicle-mounted human-computer interaction device, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, Tablets, wearable devices, or any combination of these devices.

Although the embodiments of this specification provide method operation steps as described in the embodiments or flowcharts, conventional or non-inventive means may include more or fewer operation steps. The sequence of steps listed in the embodiments is only one way of the execution sequence of many steps, and does not represent the only execution sequence. When an actual device or terminal product is executed, it can be executed sequentially or in parallel according to the method shown in the embodiment or the drawings (for example, a parallel processor or multi-threaded processing environment, or even a distributed data processing environment). The terms "include", "include" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, product, or device that includes a series of elements includes not only those elements, but also other elements that are not explicitly listed. Elements, or also include elements inherent to such processes, methods, products, or equipment. If there are no more restrictions, it does not exclude that there are other identical or equivalent elements in the process, method, product, or device including the elements.

For the convenience of description, the above devices are divided into various modes according to their functions. The groups are described separately. Of course, when implementing the embodiments of this specification, the functions of each module can be implemented in the same or multiple software and/or hardware, or the modules that implement the same function can be composed of multiple sub-modules or sub-units. Realization etc. The device embodiments described above are only illustrative. For example, the division of the units is only a logical function division, and there may be other divisions in actual implementation. For example, multiple units or components can be combined or integrated into Another system, or some features can be ignored, or not implemented. In addition, the displayed or discussed mutual coupling or direct coupling or communication connection may be indirect coupling or communication connection through some interfaces, devices or units, and may be in electrical, mechanical or other forms.

Those with ordinary knowledge in the technical field also know that in addition to implementing the controller in purely computer-readable code, it is entirely possible to make the controller use logic gates, switches, dedicated integrated circuits, and Programmable logic controllers and embedded microcontrollers can achieve the same function. Therefore, such a controller can be regarded as a hardware component, and the devices included in the controller for realizing various functions can also be regarded as a structure within the hardware component. Or even, the device for realizing various functions can be regarded as both a software module for realizing the method and a structure in a hardware component.

The present invention is described with reference to flowcharts and/or block diagrams of methods, equipment (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and/or block in the flowchart and/or block diagram, and the combination of processes and/or blocks in the flowchart and/or block diagram can be realized by computer program instructions. These computer program instructions can be provided to general-purpose computers, The processor of a dedicated computer, embedded processor, or other programmable data processing equipment is used to generate a machine so that the instructions executed by the processor of the computer or other programmable data processing equipment are used to implement a process or Multiple processes and/or block diagrams A device with functions specified in one block or multiple blocks.

These computer program instructions can also be stored in a computer-readable storage that can guide a computer or other programmable data processing equipment to work in a specific manner, so that the instructions stored in the computer-readable storage produce a manufactured product including the instruction device , The instruction device realizes the functions specified in one process or multiple processes in the flowchart and/or one block or multiple blocks in the block diagram.

These computer program instructions can also be loaded on a computer or other programmable data processing equipment, so that a series of operation steps are executed on the computer or other programmable equipment to generate computer-implemented processing, so that the computer or other programmable equipment The instructions executed above provide steps for implementing functions specified in one or more processes in the flowchart and/or one or more blocks in the block diagram.

In a typical configuration, the computing device includes one or more processors (CPU), input/output interfaces, network interfaces, and memory.

Memory may include non-permanent storage in computer-readable media, random access memory (RAM) and/or non-volatile memory, such as read-only memory (ROM) or flash memory ( flash RAM). Memory is an example of computer-readable media.

Computer-readable media include permanent and non-permanent, mobile and non-mobile media, and information storage can be realized by any method or technology. Information can So it is computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), and other types of random access memory (RAM) , Read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, read-only CD-ROM, digital multi-function Optical discs (DVD) or other optical storage, magnetic cassettes, magnetic tape storage or other magnetic storage devices or any other non-transmission media can be used to store information that can be accessed by computing devices. According to the definition in this article, computer-readable media does not include transitory media, such as modulated data signals and carrier waves.

Those skilled in the art should understand that the embodiments of this specification can be provided as methods, systems or computer program products. Therefore, the embodiments of this specification may adopt the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Moreover, the embodiments of the present specification may adopt computer program products implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes. form.

The embodiments of this specification may be described in the general context of computer-executable instructions executed by a computer, such as a program module. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The embodiments of this specification can also be practiced in a distributed computing environment. In these distributed computing environments, Remote processing equipment connected via a communication network to perform tasks. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.

The various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the part of the description of the method embodiment. In the description of this specification, descriptions with reference to the terms "one embodiment", "some embodiments", "examples", "specific examples", or "some examples" etc. mean specific features described in conjunction with the embodiment or example , Structure, materials or features are included in at least one embodiment or example of the embodiments of this specification. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the described specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner. In addition, if there is no conflict with each other, a person with ordinary knowledge in the relevant technical field can combine and combine the different embodiments or examples and the features of the different embodiments or examples described in this specification.

The above descriptions are only examples of the embodiments of this specification, and are not used to limit the embodiments of this specification. For those with ordinary knowledge in the technical field, the embodiments of this specification can have various modifications and changes. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the embodiment of this specification shall be included in the scope of the claims of the embodiment of this specification within.

Claims (15)

An interactive processing method for vehicle damage assessment, the method comprising: acquiring characteristic data of the vehicle through a shooting window; constructing an augmented reality space model of the vehicle according to the characteristic data, and the augmented reality space model is displayed in In the shooting window, matching with the actual space position of the vehicle in the shooting window is achieved; damage recognition guidance is performed in the shooting window based on the augmented reality space model, and the damage recognition guidance includes displaying based on The shooting guide information determined from the image information obtained from the shooting window includes the suspected damage found on the vehicle through the image recognition algorithm, and the coordinate information of the suspected damage in the actual space position of the vehicle is calculated, based on the suspected damage The coordinate information of the damage relative to the position of the user and the image shooting requirements of the suspected damage, determine the operation that needs to be performed by the user, and then determine the shooting guide information that needs to be displayed; and display the damage recognition in the shooting window Result information. According to the method of claim 1, the damage recognition guide includes: identifying whether there is a suspected damage in the captured image; if so, shooting the suspected damaged image according to the coordinate information of the vehicle in the shooting window A matching calculation is required, and shooting guide information is determined according to the calculation result; and the shooting guide information is displayed in the shooting window. According to the method of claim 2, the shooting guide information includes at least one of the following: adjusting the shooting direction; adjusting the shooting angle; adjusting the shooting distance; adjusting the shooting light; and the suspected location of the suspected damage. According to the method of claim 1, the result information of the damage recognition includes at least one of a damage location, a damaged component, a repair plan, and a repair cost determined based on the image information obtained by the damage recognition guide. According to the method of claim 4, before displaying the damage recognition result information of the target damage, the method further includes: displaying the processing progress of the target damage. According to the method according to claim 1 or 4, the method further includes: displaying at least one interface window among the guide prompt information, result information, and processing progress to perform corresponding tracking changes based on image changes in the shooting window. An interactive processing device for vehicle damage assessment, the device comprising: a feature acquisition module for acquiring feature data of the vehicle through a shooting window; The AR processing module is used to construct an augmented reality space model of the vehicle according to the characteristic data, the augmented reality space model is displayed in the shooting window, and is used to realize the relationship with the vehicle in the shooting window Real-world position matching; a shooting guide module for performing damage recognition guidance in the shooting window based on the augmented reality space model, the damage recognition guide including displaying images based on the shooting window The shooting guide information determined by the image information, including the suspected damage found on the field vehicle through the image recognition algorithm, the calculation of the coordinate information of the suspected damage in the actual space position of the vehicle, based on the coordinate information of the suspected damage relative to the position of the user And the image shooting requirements of the suspected damage, determine the operation that needs to be performed by the user, and then determine the shooting guide information that needs to be displayed; and the result display module for displaying the result of the damage recognition in the shooting window News. An interactive processing device for vehicle damage assessment, comprising a processor and a memory for storing executable instructions of the processor. When the processor executes the instructions, it realizes: acquiring characteristic data of the vehicle through a shooting window; and according to the characteristic data Constructing an augmented reality space model of the vehicle, displaying the augmented reality space model in the shooting window, and matching the actual space position of the vehicle in the shooting window; based on the augmented reality The space model performs damage recognition guidance in the shooting window, and the damage recognition guidance includes displaying based on the shooting The shooting guide information determined by the image information obtained in the camera window includes the suspected damage found on the field vehicle through the image recognition algorithm, and the calculation of the coordinate information of the suspected damage in the actual space position of the vehicle, based on the coordinate of the suspected damage Information relative to the location of the user and the image shooting requirements of the suspected damage, determine the operation that needs to be performed by the user, and then determine the shooting guide information that needs to be displayed; and display the result information of the damage recognition in the shooting window . For the processing device according to claim 8, when the processor executes the injury recognition guidance, the processor realizes: identifying whether there is a suspected injury in the captured image; if so, according to the coordinate information of the vehicle and the location in the shooting window. The shooting of the suspected damaged image requires matching calculation, and the shooting guide information is determined according to the calculation result; and the shooting guide information is displayed in the shooting window. According to the processing device of claim 9, the shooting guide information includes at least one of the following: adjusting the shooting direction; adjusting the shooting angle; adjusting the shooting distance; adjusting the shooting light; and the suspected location of the suspected damage. The processing device according to claim 8, wherein the result information of the damage recognition includes at least one of a damage location, a damaged component, a repair plan, and a repair cost determined based on the image information obtained by the damage recognition guide. According to the processing device of claim 11, before displaying the damage recognition result information of the target damage, the processor further executes: displaying the processing progress of the target damage. For the processing device according to claim 8 or 12, the processor further executes: displaying at least one interface window among the guide prompt information, result information, and processing progress to perform corresponding tracking changes based on image changes in the shooting window. A client device for a user side includes a processor and a memory for storing executable instructions of the processor. When the processor executes the instructions, it realizes: acquiring characteristic data of a vehicle through a photographing window; The data constructs an augmented reality space model of the vehicle, the augmented reality space model is displayed in the shooting window, and is matched with the actual space position of the vehicle in the shooting window; based on the augmented reality The environment space model performs damage recognition guidance in the shooting window, and the damage recognition guidance includes displaying the shooting guide information determined based on the image information obtained from the shooting window, including through The image recognition algorithm finds the suspected damage in the vehicle, calculates the coordinate information of the suspected damage in the actual space position of the vehicle, based on the coordinate information of the suspected damage relative to the user's position and the image shooting requirements of the suspected damage , Determine the operation that needs to be performed by the user, and then determine the shooting guide information that needs to be displayed; and display the result information of the damage recognition in the shooting window. An electronic device, comprising a display screen combined with augmented reality shooting guidance, a processor, and a memory storing executable instructions of the processor, and the processor implements any one of request items 1 to 6 when executing the instructions Method steps.

Images