CN113361426A - Vehicle loss assessment image acquisition method, medium, device and electronic equipment - Google Patents
Vehicle loss assessment image acquisition method, medium, device and electronic equipment Download PDFInfo
- Publication number
- CN113361426A CN113361426A CN202110653719.5A CN202110653719A CN113361426A CN 113361426 A CN113361426 A CN 113361426A CN 202110653719 A CN202110653719 A CN 202110653719A CN 113361426 A CN113361426 A CN 113361426A
- Authority
- CN
- China
- Prior art keywords
- image
- key frame
- damaged
- vehicle
- images
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 56
- 230000008859 change Effects 0.000 claims abstract description 20
- 238000000605 extraction Methods 0.000 claims description 7
- 230000008569 process Effects 0.000 abstract description 8
- 230000008901 benefit Effects 0.000 abstract description 3
- 238000012545 processing Methods 0.000 description 15
- 238000013527 convolutional neural network Methods 0.000 description 7
- 238000012216 screening Methods 0.000 description 7
- 230000006870 function Effects 0.000 description 6
- 238000005070 sampling Methods 0.000 description 6
- 230000003287 optical effect Effects 0.000 description 4
- 238000004364 calculation method Methods 0.000 description 3
- 238000004891 communication Methods 0.000 description 3
- 238000004590 computer program Methods 0.000 description 3
- 238000010586 diagram Methods 0.000 description 3
- 206010039203 Road traffic accident Diseases 0.000 description 2
- 238000013528 artificial neural network Methods 0.000 description 2
- 230000006835 compression Effects 0.000 description 2
- 238000007906 compression Methods 0.000 description 2
- 238000001514 detection method Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 238000004422 calculation algorithm Methods 0.000 description 1
- 230000001413 cellular effect Effects 0.000 description 1
- 239000007795 chemical reaction product Substances 0.000 description 1
- 230000003247 decreasing effect Effects 0.000 description 1
- 230000004069 differentiation Effects 0.000 description 1
- 230000001771 impaired effect Effects 0.000 description 1
- 238000003064 k means clustering Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000007781 pre-processing Methods 0.000 description 1
- 238000013139 quantization Methods 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 238000012549 training Methods 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
Images
Abstract
The embodiment of the invention provides a vehicle damage assessment image acquisition method, a vehicle damage assessment image acquisition device and electronic equipment, wherein the method comprises the following steps: acquiring shot video data; extracting a plurality of key frame images from the video data according to the display content change in the video data; detecting the extracted key frame images, and identifying key frame images comprising damaged areas of the vehicle; and selecting a damage assessment image of the vehicle from the key frame images comprising the damaged area of the vehicle according to the image attributes. By extracting and identifying the key frame images in the shot video, the method provided by the invention has the advantages that the acquisition efficiency of the vehicle loss assessment images is obviously improved, the consumption of computing resources in the identification process is reduced, and better experience is brought to users.
Description
Technical Field
The embodiment of the invention relates to the field of vehicle damage assessment, in particular to a vehicle damage assessment image acquisition method, medium, device and electronic equipment.
Background
This section is intended to provide a background or context to the embodiments of the invention that are recited in the claims. The description herein is not admitted to be prior art by inclusion in this section.
After a vehicle traffic accident occurs, an insurance company needs a plurality of loss assessment images to assess the loss of an emergent vehicle and archive the emergent data.
At present, the image of vehicle damage is usually obtained by shooting on site by an operator, and then vehicle damage assessment is carried out according to the shot picture on site. The image requirement of vehicle damage assessment needs to be able to clearly reflect information such as a damaged specific part, a damaged part, a damage type, a damage degree and the like of a vehicle, which usually requires a photographer to have knowledge related to professional vehicle damage assessment to photograph and acquire an image meeting the requirement of damage assessment processing, which obviously requires relatively large manpower training and experience cost of damage assessment processing. Particularly, in the case where the vehicle needs to be evacuated or moved as soon as possible after a vehicle traffic accident, it takes a long time for the insurance company operator to arrive at the accident site. And if the owner user takes a picture actively or in advance under the requirement of the operating personnel of the insurance company to obtain some original damage assessment images, the damage assessment images obtained by taking the picture of the owner user often do not meet the processing requirement of the damage assessment images due to non-professional personnel. In addition, images obtained by field shooting by operators often need to be exported from shooting equipment again in a later stage to be manually screened, and needed damage assessment images are determined, so that the same needs to consume larger manpower and time, and the acquisition efficiency of the damage assessment images needed by final damage assessment processing is further reduced.
Disclosure of Invention
For this reason, there is a high necessity for an improved vehicle damage assessment image acquisition method for obtaining an image that can be used for vehicle damage assessment quickly and efficiently.
In a first aspect of an embodiment of the present invention, a vehicle damage assessment image acquisition method includes: acquiring shot video data; extracting a plurality of key frame images from the video data according to the display content change in the video data; detecting the extracted key frame images, and identifying key frame images comprising damaged areas of the vehicle; and selecting a damage assessment image of the vehicle from the key frame images comprising the damaged area of the vehicle according to the image attributes.
Optionally, the extracting a plurality of key frame images from the video data according to the display content change in the video data includes: identifying video data of a plurality of shots based on color and/or spatial structure variation of the display content; a plurality of key frame images are extracted from the video data of each shot.
Optionally, the extracting the plurality of key frame images from the video data of each shot comprises: and selecting a frame image with the maximum entropy value in the video data of each shot as a key frame image of the shot.
Optionally, the selecting a damage assessment image of the vehicle from the key frame images including the damaged area of the vehicle according to the image attributes includes: classifying the key frame images based on the attributes of the detected damaged regions, and determining a candidate image classification set of the damage assessment images; and selecting the loss assessment image of the vehicle from the candidate image classification set according to the definition and/or the shooting angle.
Optionally, the candidate image classification set includes: a set of images of damaged details of the damaged region and a set of images of a vehicle component to which the damaged region belongs.
Optionally, the classifying the key frame image based on the detected attribute of the damaged region includes: judging whether the area ratio of the damaged area in the key frame image is larger than a first preset threshold value; when the area ratio of the damaged area in the belonging key frame image is larger than a first preset threshold, determining that the current key frame image belongs to the candidate image classification set of the damage assessment image; or judging whether the ratio of the abscissa span of the damaged area to the length of the key frame image to which the damaged area belongs is larger than a second preset threshold value, and/or whether the ratio of the ordinate span of the damaged area to the height of the key frame image to which the damaged area belongs is larger than a third preset threshold value; when the ratio of the abscissa span of the damaged area to the length of the key frame image to which the damaged area belongs is larger than a second preset threshold value, and/or the ratio of the ordinate span of the damaged area to the height of the key frame image to which the damaged area belongs is larger than a third preset threshold value, determining that the current key frame image belongs to the candidate image classification set of the damage assessment image; or selecting a plurality of key frame images which are sorted in descending order according to the area of the damaged area and are ranked at the front from the key frame images of the same damaged area as a candidate image classification set of the damage-assessment image, or selecting the key frame images which belong to a fourth preset threshold value after being sorted in descending order according to the area of the damaged area as a candidate image classification set of the damage-assessment image.
Optionally, the classifying the keyframe images based on the detected attributes of the damaged area, determining a candidate image classification set of the damaged images, and selecting the damaged images of the vehicle from the candidate image classification set according to definition and/or shooting angle includes: and judging whether an empty set exists in the image set of the damaged details of the damaged area and the image set of the vehicle component to which the damaged area belongs or whether a damaged area indicated by a key frame image in the damaged detail image set is missing, and generating a retaking instruction for indicating to retake the corresponding video when at least one empty set exists in the image set of the damaged details of the damaged area and the image set of the vehicle component to which the damaged area belongs or the damaged area indicated by the key frame image in the damaged detail image set is missing.
According to a second aspect, an embodiment of the present invention provides a vehicle damage assessment image acquisition apparatus, including: the video data acquisition module is used for acquiring shot video data; the key frame extraction module is used for extracting a plurality of key frame images from the video data according to the display content change in the video data; the damaged image identification module is used for detecting the extracted key frame images and identifying the key frame images comprising the damaged area of the vehicle; and the damage assessment image acquisition module is used for selecting a damage assessment image of the vehicle from the key frame images comprising the damaged area of the vehicle according to the image attribute.
In a third aspect of embodiments of the present invention, there is provided a computer-readable storage medium storing program code, which when executed by a processor, implements a method as in one of the above embodiments.
In a fourth aspect of embodiments of the present invention, there is provided an electronic device comprising a processor and a storage medium storing program code that, when executed by the processor, implements a method as in one of the above embodiments.
According to the method, the medium, the device and the electronic equipment for acquiring the vehicle damage assessment image, the key frame image in the shot video can be extracted and identified, and each frame in the video does not need to be identified, so that the acquisition efficiency of the vehicle damage assessment image is remarkably improved, the consumption of computing resources in the identification process is reduced, and better experience is brought to users.
Drawings
The above and other objects, features and advantages of exemplary embodiments of the present invention will become readily apparent from the following detailed description read in conjunction with the accompanying drawings. Several embodiments of the invention are illustrated by way of example, and not by way of limitation, in the figures of the accompanying drawings and in which:
FIG. 1 schematically illustrates an application scenario in accordance with various embodiments of the present invention;
FIG. 2 schematically shows a flow chart of a vehicle damage assessment image acquisition method according to an embodiment of the present invention;
FIG. 3 schematically illustrates yet another application scenario in accordance with various embodiments of the present invention;
fig. 4 schematically shows a structural view of a vehicle damage assessment image acquisition apparatus according to an embodiment of the present invention;
FIG. 5 schematically illustrates a schematic diagram of a computer-readable storage medium provided in accordance with an embodiment of the present invention;
FIG. 6 schematically illustrates a schematic diagram of an electronic device provided in accordance with an embodiment of the invention;
in the drawings, the same or corresponding reference numerals indicate the same or corresponding parts.
Detailed Description
The principles and spirit of the present invention will be described with reference to a number of exemplary embodiments. It is understood that these embodiments are given solely for the purpose of enabling those skilled in the art to better understand and to practice the invention, and are not intended to limit the scope of the invention in any way. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.
As will be appreciated by one skilled in the art, embodiments of the present invention may be embodied as a system, apparatus, device, method, or computer program product. Accordingly, the present disclosure may be embodied in the form of: entirely hardware, entirely software (including firmware, resident software, micro-code, etc.), or a combination of hardware and software.
According to the embodiment of the invention, a method, a medium, a device and an electronic device for acquiring a vehicle damage assessment image are provided.
In this context, it is to be understood that the terminology so involved
Frame (Frame): each frame of the basic unit of video can be regarded as a still picture, and each frame contains specific semantic information.
Lens (Shot): a shot is typically a continuous picture taken at a time by a camera, consisting of a set of consecutive frames.
Moreover, any number of elements in the drawings are by way of example and not by way of limitation, and any nomenclature is used solely for differentiation and not by way of limitation.
The principles and spirit of the present invention are explained in detail below with reference to several representative embodiments of the invention.
Specifically, referring to fig. 1 first, fig. 1 is a schematic view of an application scene of a vehicle loss assessment image acquisition method of the present invention, in fig. 1, a user may take a picture of an accident vehicle through a terminal device a, after the user finishes taking the picture, the terminal device a uploads the taken video data to a server, and the server processes the video data according to the vehicle loss assessment image acquisition method of the present invention to obtain loss assessment images of a plurality of accident vehicles, and may issue the loss assessment images to corresponding terminal devices a so as to perform vehicle loss assessment. The above application scenarios are only examples, in an actual application process, the server may have multiple stages, that is, the receiving server receives video data sent by the terminal device and sends the received video data to the processing server, and the processing server processes the received video data according to the vehicle damage assessment image obtaining method of the present invention to obtain a plurality of vehicle damage assessment images of accidents, and then sends the vehicle damage assessment images to the damage assessment server or the damage assessment terminal or the terminal device a so as to perform vehicle damage assessment.
In the following, a method for vehicle damage assessment image acquisition according to an exemplary embodiment of the present invention is described with reference to fig. 2 in conjunction with the application scenario of fig. 1. It should be noted that the above application scenarios are merely illustrated for the convenience of understanding the spirit and principles of the present invention, and the embodiments of the present invention are not limited in this respect. Rather, embodiments of the present invention may be applied to any scenario where applicable.
Fig. 2 is a schematic flow chart of an embodiment of a method for acquiring a vehicle damage assessment image according to the present invention. Although the present invention provides the method operation steps or apparatus structures as shown in the following embodiments or figures, more or less operation steps or module units after partial combination may be included in the method or apparatus based on conventional or non-inventive labor. In the case of steps or structures which do not logically have the necessary cause and effect relationship, the execution order of the steps or the block structure of the apparatus is not limited to the execution order or the block structure shown in the embodiment or the drawings of the present invention. When the described method or module structure is applied to a device, a server or an end product in practice, the method or module structure according to the embodiment or the figures may be executed sequentially or in parallel (for example, in a parallel processor or multi-thread processing environment, or even in an implementation environment including distributed processing and server clustering).
For clarity, the following embodiments are described in an implementation scenario in which a specific photographer takes a video through a mobile terminal, and a server processes the taken video data to obtain a damage assessment image. The photographer can be insurance company operating personnel, and the photographer carries out video shooting to the impaired vehicle for handheld mobile terminal. The mobile terminal can comprise a mobile phone, a tablet computer or other general or special equipment with a video shooting function and a data communication function. However, those skilled in the art can understand that the substantial spirit of the scheme can be applied to other implementation scenarios for obtaining the vehicle damage assessment image, for example, a photographer can also be an owner user, or refer to fig. 3, and the video data is directly processed at the mobile terminal side and the damage assessment image is obtained after the mobile terminal is shot.
In a specific embodiment, as shown in fig. 2, in an embodiment of a method for acquiring a damage assessment image of a vehicle according to the present invention, the method may include:
step S201, acquiring shot video data;
in some embodiments of the present invention, video data captured by the client may be transmitted to the server in real time, so as to facilitate fast processing by the server. In other embodiments, the video may be transmitted to the server after the client finishes shooting. If the mobile terminal used by the photographer does not have network connection currently, video shooting can be performed first, and the mobile cellular data or WLAN (wireless local area network) or proprietary network is connected and then transmitted. Of course, even in the case where the client can perform normal data communication with the server, the captured video data can be asynchronously transmitted to the server.
In the present embodiment, the captured video data obtained by capturing the damaged area of the vehicle by the photographer may be one video segment or a plurality of video segments. For example, multiple pieces of shot video data generated by shooting the same damaged area at different angles and distances for multiple times, or shot video data of each damaged area are obtained by respectively shooting different damaged areas. Of course, in some implementation scenarios, a complete shot may be taken around each damaged area of the damaged vehicle to obtain a relatively long video segment.
After the captured video data is obtained, the server may identify some vehicle damage assessment images from the video data, in a conventional manner, the server needs to determine each frame of image in the video data, determine which of the images can be used as vehicle damage assessment images, and actually there are a large number of images with the same or similar content in the video, and if each frame in the video is identified, it is very resource-consuming, therefore, in this embodiment, after the video data is obtained, step S202 is executed,
s202, extracting a plurality of key frame images from the video data according to the display content change in the video data.
For the extraction of the key frame image, a sampling-based method, a shot boundary-based method, a clustering-based method, and the like can be adopted.
In one implementation of this embodiment, a sampling-based method is used to extract the key frame, the video data is sampled at equal time intervals, and the image obtained by sampling is used as the key frame image.
The time interval sampling based method does not consider the change of video content, so that the key frames containing related content can not be extracted from the video with shorter time, and the extracted key frames of the longer video with single content have larger redundancy.
In consideration of the fact that when a video is shot in an application scene of vehicle damage, the video is generally shot by a non-professional user, the shot video is not subjected to later clipping processing, frequent frame skipping (shot cut) does not exist, that is, long shots with single content exist in the acquired video data, and a sampling-based method is adopted to extract key frames with large redundancy, so that in one embodiment of the embodiment, a plurality of key frame images can be extracted from the video data based on display content change in the video data, and the video data of each shot can be identified from the video data; in this embodiment, the video data is divided based on shot changes, and then key frames are extracted from the video data of each shot, considering that the picture transformation of the video data is smooth, when the video data is divided based on the shots, shot gradual change detection needs to be performed.
In consideration of the fact that a large number of frame images without shot boundaries exist in video data and the frame images without shot boundaries do not help shot gradual change detection, in one embodiment of the present embodiment, a method based on preprocessing is adopted, wherein a video is preprocessed firstly, video segments possibly including the video boundaries are extracted, a large number of video segments not possibly including the video shot boundaries are excluded, then, images in the extracted video segments are analyzed, shot boundaries in the video segments are calculated, and the amount of calculation for detecting the shot boundaries is greatly reduced.
Specifically, the video data is divided into a plurality of segments, each segment includes a certain number of frames (for example, 60 frames), in this embodiment, the segments that may include video shot boundaries are referred to as video candidate segments, and in order to reduce the amount of computation, video segments that may include boundaries are selected from the video, that is, the video candidate segments.
Because the video candidate segments contain video shot boundaries, all the video candidate segments at least contain frames of two shots, and the frames in the same shot have high similarity, if the similarity of the first frame and the last frame of one video clip is higher, the two frames are in the same shot, the video clip does not contain the shot boundaries, and the video clip can be excluded, namely the video clip is not the video candidate segment. For each video clip, whether the first frame and the last frame of the video clip are in the same shot or not can be judged through a similarity threshold value.
If the video clip contains the video shot boundary, the first frame of the video shot is the starting frame and the last frame is the ending frame.
If the video candidate segment contains a shot boundary, the shot boundary divides one segment into two shots, the initial frame of the shot boundary and the first frame of the segment both belong to the previous shot, the similarity of the two frames is high, and the characteristic can be used for detecting the initial frame of the video shot boundary.
The starting frame of the shot boundary in the video candidate segment is the last frame of the previous shot, the ending frame of the shot boundary is the first frame of the next shot, and because the frame similarity between two different shots is low, the first frame of the video candidate segment and the ending frame of the shot boundary belong to different shots, the similarity between the first frame and the ending frame of the shot boundary is low, and the ending frame of the shot boundary can be detected by utilizing the characteristic.
The shot boundary divides the video into two shots, and each frame in the same video shot has higher similarity. If the video candidate segment contains a shot boundary, the shot boundary divides the video candidate segment into a front shot and a rear shot, and a first frame of the video candidate segment and an initial frame of the shot boundary belong to the front shot, so that all frames between the first frame of the video candidate segment and the initial frame of the shot boundary belong to the same shot, and the similarity of all frames in the interval is high. The first frame of the video candidate segment and each frame after the initial frame of the shot boundary belong to different shots, so that the similarity of the first frame of the video candidate segment and each frame after the initial frame of the shot boundary is lower.
For the calculation of the similarity between different frames, in this embodiment, color features are used to compare two frames, the grayscale histogram of the image is used to calculate the similarity between different frames, and there are various methods for calculating the image similarity through the grayscale histogram, for example, the comparison may be performed by calculating the euclidean distance, or when the grayscale histogram is used as a vector, the included angle may also be calculated through the subtended quantity, so as to obtain the similarity between two frames.
After determining the similarity between different frames, the shots included in the video data may be determined, that is, the video data may be divided based on the shots, and then a plurality of key frame images may be extracted from the video data of each shot.
In an implementation manner of this embodiment, a key frame image of a shot is extracted according to the length of the shot, specifically, the shot is sampled at equal time intervals, and then the image obtained by sampling is used as a key frame image.
However, since extracting the key frame images of the shot according to the length of the shot does not fully consider the change of the image content in the shot, but a large number of redundant key frame images are extracted for a long shot, in an embodiment of this embodiment, the key frame extraction is dynamically performed, that is, the key frame images are extracted according to the intensity of the change of the content of the current shot, and if the content of the current shot changes more intensely, more key frames are extracted, even if the current shot is not long. On the contrary, even if a shot is long, if the picture content is basically unchanged, less key frame images are extracted. Specifically, a frame image with the largest entropy value in the video data of each shot is selected as a key frame image of the shot.
The significance of entropy in an image is: the bit average number of the representing image gray level set describes the average information amount of an image information source, and is an estimated value of the busy degree of an image, the image with lower entropy has no too much details and changes, and the image with lower entropy only needs less compression rate to meet the limitation of the image size during compression, and the image entropy calculation formula is as follows:
wherein h iskAnd the ratio of the pixel point with the pixel value of k in all the pixel points of the image to the number of all the pixel points is represented. When h is generatedkWhen 0, the image entropy is meaningless, so an additional condition needs to be added, when h iskWhen equal to 0, let log hk＝0。
For a gradual shot, gradual change involves a plurality of continuous frames, and because each frame in a gradual change interval changes uniformly, the similarity between the first frame in a video segment and each frame after the start frame of the shot boundary shows a linear decreasing trend. For a shot with important content change, the similarity between different frames is calculated only based on color features, and it is difficult to ensure that the image contents in the same shot are highly similar, in an implementation manner of this embodiment, the shot is further divided into a plurality of sub-shots, and specifically, the video data of each sub-shot in the video data of the shot can be identified based on color and spatial structure change;
in this embodiment, not only the shot of the video data is divided based on the color features, but also each shot is further divided into a plurality of sub-shots according to the spatial structure change, specifically, a color block map of the image is calculated by color quantization (a K-means clustering algorithm may be adopted), an area histogram, a position histogram, and a variance histogram of the image are calculated from the color block map, and the spatial structure histogram of the image is obtained based on the area histogram, the variance histogram, and the position histogram.
And extracting a key frame image from the video data of each sub-shot.
In this embodiment, for the extraction of the key frame image of each shot, the key frame image is extracted from the sub-shots of the shot first, the extraction method of the key frame image is the same as that in the foregoing embodiment, and details are not repeated herein.
S203, detecting the extracted plurality of key frame images, and identifying the key frame images comprising the damaged area of the vehicle.
In this embodiment, for the extracted key frames, various models and variants based on convolutional neural networks and area suggestion networks, such as FasterR-CNN, YOLO, Mask-FCN, etc., can be used for identification. The Convolutional Neural Network (CNN) can be any CNN model, such as ResNet, inclusion, VGG, and the like, and variants thereof. Generally, a convolutional network (CNN) part in a neural network can use a mature network structure that achieves a good effect in object recognition, such as an inclusion network, a ResNet network, and other networks, such as a ResNet network, and the input of the network is a picture, and the output of the network is a plurality of damaged regions, and corresponding damaged regions and confidence levels (where the confidence level is a parameter indicating the authenticity degree of the identified damaged regions). The fast-CNN, YOLO, Mask-FCN, etc. are all deep neural networks including convolutional layers that can be used in the present embodiment.
It can be understood that after all the key frame images are identified, the output image including the damaged area may have some problems, such as failing to meet the damage assessment requirement or having repetition, that is, there may be a plurality of images including the same damaged area, and therefore, it is necessary to screen the output image including the damaged area in order to select a suitable damaged image of the vehicle, that is, to perform step S204, and select a damaged image of the vehicle from the key frame images including the damaged area of the vehicle according to the preset screening condition.
Vehicle damage assessment often requires different categories of image data, such as images of different angles of the vehicle's general appearance, images that can reveal damaged parts, close-up detail views of specific damaged areas, and the like. In the process of acquiring the damage assessment image, the extracted video key frame image can be identified, for example, whether the image is an image of a damaged vehicle, whether a vehicle component contained in the image is identified, whether one or a plurality of vehicle components are contained, whether damage is caused on the vehicle component, and the like. In one embodiment of the invention, the damage assessment images required by the vehicle damage assessment can be correspondingly divided into different categories, and other images which do not meet the requirements of the damage assessment images can be separately divided into one category. Specifically, each key frame image of the shot video can be extracted, and each key frame image is identified and classified to form a candidate image classification set of the loss assessment image.
In the invention, the determined vehicle damage assessment image can be divided into two types, one type is an image capable of showing damaged details of the damaged area, the other type is an image capable of showing a vehicle part to which the damaged area belongs,
and S204, selecting a damage assessment image of the vehicle from the key frame images comprising the damaged area of the vehicle according to the image attributes. Before the damaged image screening, the key frame images including the damaged area of the vehicle may be classified, the key frame images may be classified based on the attribute of the detected damaged area, and the candidate image classification set of the damaged image may be determined.
The image set capable of showing the damaged details of the damaged area comprises a close-up image of the damaged area, the image set capable of showing the vehicle component to which the damaged area belongs comprises a damaged component of the damaged vehicle, and at least one damaged area is arranged on the damaged component. Specifically, in the application scenario of this embodiment, the photographer may perform near-to-far (or far-to-near) shooting on the designated damaged area, and the shooting may be performed by moving or zooming the photographer. The server side can classify and identify key frame images in the shot video.
In an embodiment of the present invention, the classifying the key frame images based on the detected attributes of the damaged area may determine the key frame images in the image set capable of showing the damaged details of the damaged area by at least one of the following manners:
judging whether the area ratio of the damaged area in the key frame image is larger than a first preset threshold value;
when the area ratio of the damaged area in the belonging key frame image is larger than a first preset threshold, determining that the current key frame image belongs to the candidate image classification set of the damage assessment image; or
Judging whether the ratio of the abscissa span of the damaged area to the length of the key frame image to which the damaged area belongs is larger than a second preset threshold value and/or whether the ratio of the ordinate span of the damaged area to the height of the key frame image to which the damaged area belongs is larger than a third preset threshold value;
when the ratio of the abscissa span of the damaged area to the length of the key frame image to which the damaged area belongs is larger than a second preset threshold value, and/or the ratio of the ordinate span of the damaged area to the height of the key frame image to which the damaged area belongs is larger than a third preset threshold value, determining that the current key frame image belongs to the candidate image classification set of the damage assessment image; or
Selecting a plurality of key frame images which are sorted in descending order according to the area of the damaged area and are ranked at the front from key frame images of the same damaged area as a candidate image classification set of the damage-assessment image, or selecting key frame images which belong to a fourth preset threshold value after being sorted in descending order according to the area of the damaged area as a candidate image classification set of the damage-assessment image.
For example, according to the category, definition and the like of the loss assessment image, an image meeting a preset screening condition is selected from the candidate image classification set as the loss assessment image. The preset screening condition may be set in a customized manner, for example, in one embodiment, a plurality of images (for example, 5 or 10 images) with the highest sharpness may be respectively selected from the two types of images according to the sharpness of the images, and the images with different shooting angles are taken as the damaged images of the designated damaged area. The sharpness of the image may be calculated by calculating the damaged area and the image area where the detected vehicle component is located, for example, by using an operator based on a spatial domain (e.g., Gabor operator) or an operator based on a frequency domain (e.g., fast fourier transform). In the first type of images, it is generally necessary to ensure that all of the damaged area can be displayed after one or more images are combined, so that comprehensive information of the damaged area can be obtained.
The step of classifying the key frame images based on the detected attributes of the damaged regions, determining a candidate image classification set of the damaged images and the damaged images of the vehicles selected from the candidate image classification set according to definition and/or shooting angle comprises the following steps:
judging whether an empty set exists in the image set of the damaged details of the damaged area and the image set of the vehicle component to which the damaged area belongs or whether a damaged area indicated by a key frame image in the damaged detail image set is missing,
and when at least one empty set exists in the image set of the damaged details of the damaged area and the image set of the vehicle part to which the damaged area belongs, or the damaged area indicated by the key frame image in the damaged detail image set is missing, generating a re-shooting instruction for indicating to re-shoot the corresponding video.
Considering that in some extreme cases, the video taken by the user may have no problem, that is, the taken video completely records the damaged area of the vehicle, and may be that the acquired key frame image cannot completely cover all the damaged area of the vehicle, in this embodiment, before generating the prompt message for instructing to take the corresponding video again, the method further includes:
detecting video images in the shot video data, and identifying video images including damaged parts of the vehicle in the video images;
if the video image including the damaged part of the vehicle is not identified, generating a prompt message for instructing to shoot the corresponding video again;
if the video image comprising the damaged part of the vehicle is identified, classifying the video image based on the detected damaged part, and determining a candidate image classification set of the damaged part;
and executing the step of selecting the damage assessment image of the vehicle from the candidate image classification set according to a preset screening condition.
According to the method for acquiring the vehicle damage assessment image, the key frame image in the shot video can be extracted and identified, and each frame in the video does not need to be identified, so that the acquisition efficiency of the vehicle damage assessment image is remarkably improved, the consumption of computing resources in the identification process is reduced, and better experience is brought to users.
An embodiment of the present invention provides a vehicle damage assessment image obtaining apparatus, as shown in fig. 4, where the apparatus includes:
the video data acquisition module is used for acquiring shot video data;
the key frame extraction module is used for extracting a plurality of key frame images from the video data according to the display content change in the video data;
the damaged image identification module is used for detecting the extracted key frame images and identifying the key frame images comprising the damaged area of the vehicle;
and the damage assessment image acquisition module is used for selecting a damage assessment image of the vehicle from the key frame images comprising the damaged area of the vehicle according to the image attribute.
Exemplary Medium
Referring to fig. 5, a computer-readable storage medium is shown as an optical disc 50, on which a computer program (i.e., a program product) is stored, where the computer program, when executed by a processor, implements the steps described in the above method embodiments, for example, acquiring captured video data; extracting a plurality of key frame images from the video data; detecting the extracted key frame images, and identifying key frame images comprising damaged areas of the vehicle; selecting a damage assessment image of the vehicle from the key frame images comprising the damaged area of the vehicle according to a preset screening condition; the specific implementation of each step is not repeated here.
It should be noted that examples of the computer-readable storage medium may also include, but are not limited to, phase change memory (PRAM), Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), Read Only Memory (ROM), Electrically Erasable Programmable Read Only Memory (EEPROM), flash memory, or other optical and magnetic storage media, which are not described in detail herein.
An electronic device is provided for illustration in an embodiment of the present invention, and fig. 6 shows a block diagram of an exemplary electronic device 60 suitable for implementing an embodiment of the present invention, where the electronic device 70 may be a computer system or a server. The electronic device 60 shown in fig. 6 is only an example, and should not bring any limitation to the functions and the scope of use of the embodiment of the present invention.
As shown in fig. 6, the components of the electronic device 60 may include, but are not limited to: one or more processors or processing units 601, a system memory 602, and a bus 603 that couples various system components including the system memory 602 and the processing unit 601.
The system memory 602 may include computer system readable media in the form of volatile memory, such as Random Access Memory (RAM)6021 and/or cache memory 6022. The electronic device 60 may further include other removable/non-removable, volatile/nonvolatile computer system storage media. By way of example only, ROM6023 may be used to read from and write to non-removable, nonvolatile magnetic media (not shown in FIG. 6, but typically referred to as a "hard disk drive"). Although not shown in FIG. 6, a magnetic disk drive for reading from and writing to a removable, nonvolatile magnetic disk (e.g., a "floppy disk") and an optical disk drive for reading from or writing to a removable, nonvolatile optical disk may be provided. In these cases, each drive may be connected to the bus 603 by one or more data media interfaces. At least one program product may be included in system memory 602 with a set (e.g., at least one) of program modules configured to perform the functions of embodiments of the present invention.
A program/utility 6025 having a set (at least one) of program modules 6024 may be stored, for example, in the system memory 602, and such program modules 6024 include, but are not limited to: an operating system, one or more application programs, other program modules, and program data, each of which, or some combination thereof, may comprise an implementation of a network environment. Program modules 6024 generally carry out the functions and/or methodologies of embodiments of the invention as described herein.
The electronic device 60 may also communicate with one or more external devices 604, such as a keyboard, pointing device, display, etc. Such communication may occur via input/output (I/O) interfaces 605. Also, the electronic device 60 may communicate with one or more networks (e.g., a Local Area Network (LAN), a Wide Area Network (WAN), and/or a public network, such as the internet) via the network adapter 606. As shown in FIG. 6, the network adapter 606 communicates with other modules of the electronic device 60, such as the processing unit 601, via the bus 603. It should be appreciated that although not shown in FIG. 6, other hardware and/or software modules may be used in conjunction with electronic device 60.
The processing unit 601 executes various functional applications and data processing, for example, executing and implementing steps in the vehicle damage assessment image acquisition method, by running a program stored in the system memory 602; for example, captured video data is acquired; extracting a plurality of key frame images from the video data; detecting the extracted key frame images, and identifying key frame images comprising damaged areas of the vehicle; selecting a damage assessment image of the vehicle from the key frame images comprising the damaged area of the vehicle according to a preset screening condition; the specific implementation of each step is not repeated here.
It should be noted that although in the above detailed description several units/modules or sub-units/sub-modules of the vehicle damage assessment image acquisition apparatus are mentioned, such division is merely exemplary and not mandatory. Indeed, the features and functionality of two or more of the units/modules described above may be embodied in one unit/module according to embodiments of the invention. Conversely, the features and functions of one unit/module described above may be further divided into embodiments by a plurality of units/modules.
Moreover, while the operations of the method of the invention are depicted in the drawings in a particular order, this does not require or imply that the operations must be performed in this particular order, or that all of the illustrated operations must be performed, to achieve desirable results. Additionally or alternatively, certain steps may be omitted, multiple steps combined into one step execution, and/or one step broken down into multiple step executions.
While the spirit and principles of the invention have been described with reference to several particular embodiments, it is to be understood that the invention is not limited to the disclosed embodiments, nor is the division of aspects, which is for convenience only as the features in such aspects may not be combined to benefit. The invention is intended to cover various modifications and equivalent arrangements included within the spirit and scope of the appended claims.
Claims (10)
1. A vehicle damage assessment image acquisition method comprises the following steps:
acquiring shot video data;
extracting a plurality of key frame images from the video data according to the display content change in the video data;
detecting the extracted key frame images, and identifying key frame images comprising damaged areas of the vehicle;
and selecting a damage assessment image of the vehicle from the key frame images comprising the damaged area of the vehicle according to the image attributes.
2. The method of claim 1, wherein said extracting a plurality of key frame images from said video data based on display content changes in said video data comprises:
identifying video data of a plurality of shots based on color and/or spatial structure variation of the display content;
a plurality of key frame images are extracted from the video data of each shot.
3. The method of claim 2, wherein said extracting a plurality of key frame images from the video data for each shot comprises:
and selecting a frame image with the maximum entropy value in the video data of each shot as a key frame image of the shot.
4. The method of claim 1, wherein said selecting a damage image of the vehicle from said keyframe images comprising damaged regions of the vehicle according to image attributes comprises:
classifying the key frame images based on the attributes of the detected damaged regions, and determining a candidate image classification set of the damage assessment images;
and selecting the loss assessment image of the vehicle from the candidate image classification set according to the definition and/or the shooting angle.
5. The method of claim 4, wherein the set of candidate image classifications comprises:
a set of images of damaged details of the damaged region and a set of images of a vehicle component to which the damaged region belongs.
6. The method of claim 5, wherein the classifying the key frame image based on the attributes of the detected damaged regions comprises:
judging whether the area ratio of the damaged area in the key frame image is larger than a first preset threshold value;
when the area ratio of the damaged area in the belonging key frame image is larger than a first preset threshold, determining that the current key frame image belongs to the candidate image classification set of the damage assessment image; or
Judging whether the ratio of the abscissa span of the damaged area to the length of the key frame image to which the damaged area belongs is larger than a second preset threshold value and/or whether the ratio of the ordinate span of the damaged area to the height of the key frame image to which the damaged area belongs is larger than a third preset threshold value;
when the ratio of the abscissa span of the damaged area to the length of the key frame image to which the damaged area belongs is larger than a second preset threshold value, and/or the ratio of the ordinate span of the damaged area to the height of the key frame image to which the damaged area belongs is larger than a third preset threshold value, determining that the current key frame image belongs to the candidate image classification set of the damage assessment image; or
Selecting a plurality of key frame images which are sorted in descending order according to the area of the damaged area and are ranked at the front from key frame images of the same damaged area as a candidate image classification set of the damage-assessment image, or selecting key frame images which belong to a fourth preset threshold value after being sorted in descending order according to the area of the damaged area as a candidate image classification set of the damage-assessment image.
7. The method according to claim 5 or 6, wherein said classifying the key frame images based on the detected attributes of the damaged regions, determining between the candidate classified set of images of the damage assessment images and the damage assessment image of the vehicle selected from the candidate classified set of images in terms of sharpness and/or photographic angle comprises:
judging whether an empty set exists in the image set of the damaged details of the damaged area and the image set of the vehicle component to which the damaged area belongs or whether a damaged area indicated by a key frame image in the damaged detail image set is missing,
and when at least one empty set exists in the image set of the damaged details of the damaged area and the image set of the vehicle part to which the damaged area belongs, or the damaged area indicated by the key frame image in the damaged detail image set is missing, generating a re-shooting instruction for indicating to re-shoot the corresponding video.
8. A vehicle damage assessment image acquisition apparatus, characterized by comprising:
the video data acquisition module is used for acquiring shot video data;
the key frame extraction module is used for extracting a plurality of key frame images from the video data according to the display content change in the video data;
the damaged image identification module is used for detecting the extracted key frame images and identifying the key frame images comprising the damaged area of the vehicle;
and the damage assessment image acquisition module is used for selecting a damage assessment image of the vehicle from the key frame images comprising the damaged area of the vehicle according to the image attribute.
9. A computer-readable storage medium, characterized in that a program code is stored, which program code, when being executed by a processor, realizes the method according to one of claims 1 to 7.
10. An electronic device, comprising a processor and a storage medium storing program code which, when executed by the processor, implements the method according to one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110653719.5A CN113361426A (en) | 2021-06-11 | 2021-06-11 | Vehicle loss assessment image acquisition method, medium, device and electronic equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110653719.5A CN113361426A (en) | 2021-06-11 | 2021-06-11 | Vehicle loss assessment image acquisition method, medium, device and electronic equipment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113361426A true CN113361426A (en) | 2021-09-07 |
Family
ID=77534048
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110653719.5A Pending CN113361426A (en) | 2021-06-11 | 2021-06-11 | Vehicle loss assessment image acquisition method, medium, device and electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113361426A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114422848A (en) * | 2022-01-19 | 2022-04-29 | 腾讯科技（深圳）有限公司 | Video segmentation method and device, electronic equipment and storage medium |
Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20050002452A1 (en) * | 1999-01-29 | 2005-01-06 | Frederic Dufaux | System for selecting a keyframe to represent a video |
| US20100104261A1 (en) * | 2008-10-24 | 2010-04-29 | Zhu Liu | Brief and high-interest video summary generation |
| CN102395984A (en) * | 2009-04-14 | 2012-03-28 | 皇家飞利浦电子股份有限公司 | Key frames extraction for video content analysis |
| CN107194323A (en) * | 2017-04-28 | 2017-09-22 | 阿里巴巴集团控股有限公司 | Car damage identification image acquiring method, device, server and terminal device |
| CN110188717A (en) * | 2019-06-03 | 2019-08-30 | 中国通信建设第四工程局有限公司 | Image acquiring method and device |
| CN110427810A (en) * | 2019-06-21 | 2019-11-08 | 北京百度网讯科技有限公司 | Video damage identification method, device, shooting end and machine readable storage medium |
| CN111797707A (en) * | 2020-06-12 | 2020-10-20 | 武汉大学 | Clustering-based shot key frame extraction method |
| CN111984942A (en) * | 2020-07-23 | 2020-11-24 | 西安理工大学 | Robust video zero-watermarking method based on extremely complex exponential transformation and residual error neural network |
| CN112487241A (en) * | 2020-11-26 | 2021-03-12 | 哈尔滨工程大学 | Video abstraction method based on convolutional neural network |
| WO2021057069A1 (en) * | 2019-09-27 | 2021-04-01 | 支付宝(杭州)信息技术有限公司 | Computer-executable vehicle loss assessment method and apparatus |
-
2021
- 2021-06-11 CN CN202110653719.5A patent/CN113361426A/en active Pending
Patent Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20050002452A1 (en) * | 1999-01-29 | 2005-01-06 | Frederic Dufaux | System for selecting a keyframe to represent a video |
| US20100104261A1 (en) * | 2008-10-24 | 2010-04-29 | Zhu Liu | Brief and high-interest video summary generation |
| CN102395984A (en) * | 2009-04-14 | 2012-03-28 | 皇家飞利浦电子股份有限公司 | Key frames extraction for video content analysis |
| CN107194323A (en) * | 2017-04-28 | 2017-09-22 | 阿里巴巴集团控股有限公司 | Car damage identification image acquiring method, device, server and terminal device |
| CN110188717A (en) * | 2019-06-03 | 2019-08-30 | 中国通信建设第四工程局有限公司 | Image acquiring method and device |
| CN110427810A (en) * | 2019-06-21 | 2019-11-08 | 北京百度网讯科技有限公司 | Video damage identification method, device, shooting end and machine readable storage medium |
| WO2021057069A1 (en) * | 2019-09-27 | 2021-04-01 | 支付宝(杭州)信息技术有限公司 | Computer-executable vehicle loss assessment method and apparatus |
| CN111797707A (en) * | 2020-06-12 | 2020-10-20 | 武汉大学 | Clustering-based shot key frame extraction method |
| CN111984942A (en) * | 2020-07-23 | 2020-11-24 | 西安理工大学 | Robust video zero-watermarking method based on extremely complex exponential transformation and residual error neural network |
| CN112487241A (en) * | 2020-11-26 | 2021-03-12 | 哈尔滨工程大学 | Video abstraction method based on convolutional neural network |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114422848A (en) * | 2022-01-19 | 2022-04-29 | 腾讯科技（深圳）有限公司 | Video segmentation method and device, electronic equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109151501B (en) | Video key frame extraction method and device, terminal equipment and storage medium | |
| US8358837B2 (en) | Apparatus and methods for detecting adult videos | |
| CN102007499B (en) | Detecting facial expressions in digital images | |
| US6724933B1 (en) | Media segmentation system and related methods | |
| US7127127B2 (en) | System and method for adaptive video fast forward using scene generative models | |
| US9613258B2 (en) | Image quality assessment | |
| US8644563B2 (en) | Recognition of faces using prior behavior | |
| US8467610B2 (en) | Video summarization using sparse basis function combination | |
| US8842889B1 (en) | System and method for automatic face recognition | |
| US8467611B2 (en) | Video key-frame extraction using bi-level sparsity | |
| JP2014029732A (en) | Method for generating representation of image contents using image search and retrieval criteria | |
| JP5117670B2 (en) | Image and method for representing image group, method for comparing image or image group, method for retrieving image or image group, apparatus and system for executing each method, program, and computer-readable storage medium | |
| WO2012141655A1 (en) | In-video product annotation with web information mining | |
| CN108229289B (en) | Target retrieval method and device and electronic equipment | |
| Heng et al. | How to assess the quality of compressed surveillance videos using face recognition | |
| CN112084812A (en) | Image processing method, image processing device, computer equipment and storage medium | |
| CN113361426A (en) | Vehicle loss assessment image acquisition method, medium, device and electronic equipment | |
| KR102230559B1 (en) | Method and Apparatus for Creating Labeling Model with Data Programming | |
| CN110956093A (en) | Big data-based model identification method, device, equipment and medium | |
| CN112884866B (en) | Coloring method, device, equipment and storage medium for black-and-white video | |
| CN113158773A (en) | Training method and training device for living body detection model | |
| KR20210031444A (en) | Method and Apparatus for Creating Labeling Model with Data Programming | |
| CN114185630B (en) | Screen recording method, device, computer equipment and storage medium | |
| CN113407780B (en) | Target retrieval method, device and storage medium | |
| CN113298087B (en) | Method, system, device and medium for cold start of picture classification model |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |