WO2024061194A1 - 样本标签的获取方法和镜头失效检测模型的训练方法 - Google Patents

样本标签的获取方法和镜头失效检测模型的训练方法 Download PDF

Info

Publication number
WO2024061194A1
WO2024061194A1 PCT/CN2023/119612 CN2023119612W WO2024061194A1 WO 2024061194 A1 WO2024061194 A1 WO 2024061194A1 CN 2023119612 W CN2023119612 W CN 2023119612W WO 2024061194 A1 WO2024061194 A1 WO 2024061194A1
Authority
WO
WIPO (PCT)
Prior art keywords
model
lens
image
failure
degree
Prior art date
Application number
PCT/CN2023/119612
Other languages
English (en)
French (fr)
Inventor
邱翰
方三勇
王进
Original Assignee
虹软科技股份有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 虹软科技股份有限公司 filed Critical 虹软科技股份有限公司
Publication of WO2024061194A1 publication Critical patent/WO2024061194A1/zh

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
    • G06V10/774—Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/74—Image or video pattern matching; Proximity measures in feature spaces
    • G06V10/75—Organisation of the matching processes, e.g. simultaneous or sequential comparisons of image or video features; Coarse-fine approaches, e.g. multi-scale approaches; using context analysis; Selection of dictionaries
    • G06V10/754—Organisation of the matching processes, e.g. simultaneous or sequential comparisons of image or video features; Coarse-fine approaches, e.g. multi-scale approaches; using context analysis; Selection of dictionaries involving a deformation of the sample pattern or of the reference pattern; Elastic matching
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/50—Context or environment of the image
    • G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
    • G06V20/58—Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V2201/00—Indexing scheme relating to image or video recognition or understanding
    • G06V2201/07—Target detection

Definitions

  • This application relates to the field of computer technology.
  • the present application provides a method for acquiring sample labels, a method for training a lens failure detection model, a method for detecting lens failure, an on-board control system and a vehicle, so as to solve the problem in related technologies that the judgment of the degree of failure is relatively subjective and cannot provide an objective basis for subsequent image processing processes.
  • this application provides a method for obtaining sample labels, which method includes:
  • the lens failure degree is used as the failure degree label of the input image, wherein the input image of the pre-model and its failure degree label are used as sample data for training the lens failure detection model.
  • determining the degree of lens failure of the input image of the front-end model based on the model quality evaluation result includes:
  • determining the lens failure degree of each input image of the front model according to the numerical relationship between the plurality of model quality evaluation results includes:
  • Model quality evaluation obtained by taking the image data set of the first image quality level as input for the pre-model The results serve as reference values for evaluation;
  • the lens failure degree of the input image having the second image quality level is determined based on the difference between the evaluation reference value and the evaluation value to be confirmed.
  • the image quality corresponding to the first image quality level is greater than or equal to a preset image quality threshold.
  • determining the lens failure degree of each input image of the front-end model based on the numerical relationship between multiple model quality evaluation results includes:
  • the degree of lens failure of the input image corresponding to the image quality level is determined according to the relative size relationship between a plurality of the model quality evaluation results.
  • determining the model quality evaluation results of the front-end model includes:
  • the model quality evaluation result is jointly determined based on the recall rate and precision rate of the pre-model.
  • jointly determining the model quality evaluation result based on the recall rate and precision rate of the front-end model includes:
  • the model quality evaluation result is determined according to the numerical characteristics of the recall-precision curve.
  • the numerical characteristic of the recall-precision curve is the area between the recall-precision curve and the coordinate axis, which is recorded as the evaluation area.
  • the method before determining the model quality assessment result of the front-end model, the method further includes:
  • the pre-model is a target detection model, obtain the intersection and union ratio of the detected target in the input image;
  • intersection-to-union ratio is greater than or equal to the preset intersection-to-union ratio threshold, the model quality evaluation result of the front-end model is obtained.
  • embodiments of the present application provide a method for training a lens failure detection model.
  • the method includes:
  • the input image is used as the input data of the lens failure detection model to be trained, the failure degree label is used as the target data of the lens failure detection model, and the lens failure detection model is trained, wherein, The lens failure detection model is at least used to determine the degree of lens failure of the image to be predicted.
  • the method before training the lens failure detection model, the method further includes: obtaining the lens failure type of the input image as a failure type label;
  • the training of the lens failure detection model includes:
  • the input image is used as the input data of the lens failure detection model to be trained, the failure degree label and the failure type label are used as target data of the lens failure detection model, and the lens failure detection model is trained.
  • inventions of the present application provide a method for detecting lens failure.
  • the method includes:
  • the lens failure degree of the image to be predicted is taken as the lens failure degree of the image acquisition device, wherein the image to be predicted is obtained by the image acquisition device.
  • the method further includes:
  • the obtained results include one or more of the following:
  • the degree of lens failure and the type of lens failure are the degree of lens failure and the type of lens failure.
  • obtaining the lens failure type of the image to be predicted through the trained lens failure detection model includes:
  • the final lens failure type corresponding to the image to be predicted is determined based on the degree of influence of the multiple lens failure types on the image to be predicted.
  • embodiments of the present application provide a vehicle-mounted control system, including a memory and a processor.
  • the memory stores a program.
  • the program is read and executed by the processor, the above-mentioned first aspect to The method described in any one of the third aspects.
  • embodiments of the present application provide a vehicle, which includes:
  • An image acquisition device configured to acquire images to be predicted
  • An image processor configured to implement the method described in any one of the above first to third aspects.
  • embodiments of the present application provide a computer-readable storage medium that stores one or more programs, and the one or more programs can be executed by one or more processors, To implement the method described in any one of the above first to third aspects.
  • the method for obtaining sample labels provided in this embodiment first determines the model quality evaluation results of the front-end model, where the front-end model is used to assist in determining the failure degree label of the input image of the lens failure detection model to be trained; and then based on the model quality
  • the evaluation results determine the lens failure degree of the input image of the front-end model; finally, the lens failure degree is used as the failure degree label of the input image to train the lens failure detection model, which solves the subjective problem of judging the lens failure degree in related technologies. Provides an objective basis for the image processing process.
  • Figure 1 is a hardware structure block diagram of a terminal of the method for obtaining sample tags in this embodiment
  • Figure 2 is a flow chart of a method for obtaining sample labels according to an embodiment of the present application
  • Figure 3 is a flow chart of a method for determining the degree of lens failure according to an embodiment of the present application
  • Figure 4 is a flow chart of another method for determining the degree of lens failure according to an embodiment of the present application.
  • Figure 5 is a flow chart of a training method for a lens failure detection model according to an embodiment of the present application
  • Figure 6 is a flow chart of a method for detecting lens failure according to an embodiment of the present application.
  • Figure 7 is a structural block diagram of a device for obtaining sample tags according to an embodiment of the present application.
  • Words such as “connected”, “connected”, “coupled” and the like mentioned in this application are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect.
  • the "plurality” mentioned in this application means two or more.
  • “And/or” describes the relationship between related objects, indicating that three relationships can exist. For example, “A and/or B” can mean: A alone exists, A and B exist simultaneously, and B exists alone. Normally, the character “/” indicates that the related objects are in an “or” relationship.
  • the terms “first”, “second”, “third”, etc. involved in this application only distinguish similar objects and do not represent a specific ordering of the objects.
  • FIG. 1 is a hardware structure block diagram of the terminal for the method for obtaining sample tags in this embodiment.
  • the terminal may include one or more (only one is shown in Figure 1) processors 102 and a memory 104 for storing data, wherein the processor 102 may include but is not limited to a microprocessor MCU or a memory 104 for storing data.
  • Processing device for programming logic devices such as FPGA.
  • the terminal further includes a transmission device 106 for communication and an input and output device 108.
  • the structure shown in Figure 1 is only illustrative, and it does not limit the structure of the above-mentioned terminal.
  • the terminal may also include more or fewer components than shown in FIG. 1 , or have a different configuration than that shown in FIG. 1 .
  • the memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as the computer program corresponding to the method for obtaining sample tags in the embodiment of the present disclosure.
  • the processor 102 runs the computer program stored in the memory 104, Thereby executing various functional applications and data processing, that is, realizing the above method.
  • Memory 104 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory.
  • the memory 104 may include memory located remotely relative to the processor 102, and these remote memories may be connected to the terminal through a network. Examples of the above-mentioned networks include but are not limited to the Internet, intranets, local area networks, mobile communication networks and combinations thereof.
  • Transmission device 106 is used to receive or send data via a network.
  • the above network includes communication provision of terminals wireless network provided by the provider.
  • the transmission device 106 includes a network adapter (Network Interface Controller, NIC for short), which can be connected to other network devices through a base station to communicate with the Internet.
  • the transmission device 106 may be a radio frequency (Radio Frequency, RF for short) module, which is used to communicate with the Internet wirelessly.
  • RF Radio Frequency
  • the embodiment of the present application provides a method for obtaining sample labels.
  • the image after obtaining the sample label is used to train a lens failure detection model that determines the degree of lens failure. That is, the image after obtaining the sample label is used as an input image to be trained on the lens failure.
  • the detection model is trained, as shown in Figure 2.
  • the method of obtaining sample labels includes the following steps:
  • Step S201 Determine the model quality evaluation result of the front-end model.
  • the embodiment of the present disclosure involves two models, one is the lens failure detection model that is ultimately required, and the other is a pre-model that obtains sample labels for the lens failure detection model to be trained, where the pre-model can be any type.
  • Models for example, face recognition models, pedestrian detection models or general target detection models, etc. Since the quality of the front-end model needs to be evaluated during training or use to determine the training effect, such as the accuracy of recognition, etc., the model quality evaluation results of the front-end model can be obtained, and the failure degree label can be determined based on this. .
  • the pre-model may be a model that has initially completed training, that is, the training has been initially completed according to the training plan corresponding to the selected model.
  • the input image can be input into the pre-model for testing, so that the corresponding model recognition or classification results can be obtained; for an input image set including multiple input images, the pre-model can be used to obtain each input image After the corresponding model recognition or classification results, the model quality assessment results corresponding to the input image set can be determined.
  • the pre-model can be used to assist in determining the failure degree label of the input image of the lens failure detection model to be trained, that is, it can be used to assist in determining the corresponding failure degree label of the sample image for training the lens failure detection model.
  • the model quality evaluation result can be expressed as a numerical value.
  • the numerical value is accuracy
  • the higher the accuracy the better the training effect of the pre-model
  • the model quality evaluation results are used to represent the misjudgment rate of the front-end model
  • the higher the miscarriage rate means the worse the training effect of the front-end model.
  • the recognition accuracy of the front-end model will decrease.
  • the model quality evaluation results will change as the image quality of the input image changes. Based on this, different model quality evaluation results can be obtained by controlling the image quality of the input image.
  • the image quality is evaluated according to one or more of the following parameters:
  • the image quality is evaluated according to one or more of the following parameters:
  • the lens failure detection model in this embodiment is used to determine the degree of lens failure of an image acquisition device.
  • the image acquisition device can be a mobile terminal, such as a mobile phone, a tablet, an augmented reality (Augmented Reality, AR) device, or a virtual reality (Virtual Reality, VR) equipment, XR (Extended Reality, extended reality) equipment, MR (Mixed Reality, mixed reality) equipment, etc., or can be professional equipment such as cameras and cameras.
  • Step S202 Determine the degree of lens failure of the input image of the front model according to the model quality evaluation result.
  • the model quality evaluation result of the front-end model can reflect the image quality of the input image when the front-end model is evaluated or tested, so the model quality evaluation result has a corresponding relationship with the image quality of the input image.
  • the degree of lens failure of the image capture device is also related to the image quality of the input image. For example, if there is rain, dirt, or scratches on the lens surface of the image capture device, it will cause the captured image to deteriorate. Image quality deteriorates.
  • the model quality evaluation results obtained by pre-model evaluation using different image data sets will also be different. Therefore, the model quality evaluation results can be used to define lens failure. Degree, that is, there is a corresponding relationship between the model quality evaluation results and the degree of lens failure.
  • the model quality evaluation result when the model quality evaluation result indicates the recognition accuracy of the front model, the model quality evaluation result is negatively correlated with the degree of lens failure. That is, the higher the recognition accuracy of the front model, the higher the degree of lens failure. The lower it is, the lower the recognition accuracy of the front model and the higher the degree of lens failure.
  • the input images can be multiple types of images of different qualities simulated by computers or artificially, or can be real images of different qualities obtained through image acquisition equipment.
  • Step S203 Use the lens failure degree as the failure degree label of the input image, where the input image of the pre-model and its failure degree label are used as sample data for training the lens failure detection model.
  • the input image is used to train a lens failure detection model so that the trained lens failure detection model can be used to determine the lens failure degree of the image acquisition device during the application process.
  • the failure degree label of the input image is determined based on the model quality evaluation results of the front-end model, and the lens failure degree is defined with numerical parameters, eliminating the need for manual annotation, thus solving the problem of failure degree in related technologies.
  • the judgment is relatively subjective and cannot provide an objective basis for the subsequent image processing process.
  • Objectifying the degree of lens failure provides an objective standard for judging the degree of lens failure, which is also conducive to subsequent targeted processing of images.
  • determining the degree of lens failure of the input image includes: determining the degree of lens failure of each input image according to a numerical relationship between multiple model quality evaluation results, where the numerical relationship may be a size relationship, or It can be a proportional relationship, or it can be a more complex logarithmic relationship.
  • input image data sets of different image qualities may be sequentially input to the front-end model to obtain different model quality evaluation results. At this time, all input images of the front-end model can be divided into multiple image data sets. Multiple image data sets respectively correspond to multiple image quality levels. Multiple image quality levels respectively correspond to multiple model quality evaluation results.
  • "image data set-image quality level-model quality assessment result" corresponds one to one.
  • an evaluation reference value can be defined first based on one image data set among multiple image data sets, and the model quality evaluation results of other image data sets are compared with the evaluation reference value to obtain the lens failure degree, or The degree of lens failure can be determined based on the relative relationship between multiple model quality assessment results for multiple image data sets.
  • the model quality assessment results may be processed through a proportional relationship or a logarithmic relationship to obtain a more appropriate level division.
  • the lens failure degree corresponding to the input image can be determined more accurately.
  • lens failure types include light, clarity, shadow, dirt, occlusion, water stains, screen blur, etc.
  • the input images used for pre-model evaluation are divided according to the lens failure type
  • the input images of each lens failure type are then divided into multiple image data sets according to the image quality level.
  • Each image data set has The images have the same lens failure type and the same image quality level.
  • Each image data set is used to conduct model quality assessment of the front-end model, and the model quality assessment results corresponding to the lens failure type and the image quality level are determined. For example, all image data are divided into N types of lens failure types and M image quality levels, resulting in N*M image data sets.
  • the model quality evaluation corresponding to each data set is determined. result.
  • the lens failure types classified here are used to determine failure type labels of input images for training the lens failure detection model.
  • the number of subdivided image quality levels may be the same or different, and is not limited to specific aspects.
  • a method for determining the degree of lens failure is provided, as shown in Figure 3, including the following steps:
  • Step S301 use the model quality evaluation result obtained by using the image data set of the first image quality level as input for the pre-model as the evaluation reference value;
  • Step S302 determine the model quality evaluation result obtained by using the image data set of the second image quality level as input for the pre-model, and record it as the evaluation value to be confirmed;
  • Step S303 Determine the degree of lens failure of the input image with the second image quality level based on the difference between the evaluation reference value and the evaluation value to be confirmed.
  • the images included in the input image data set with the first image quality level all have the same degree of lens failure; the images included in the input image data set with the second image quality level all have the same degree of lens failure.
  • the lens failure degree of the input image with the second image quality level may be determined according to a preset corresponding rule between the difference and the lens failure degree. For example, every time the evaluation value to be confirmed decreases by a preset value relative to the evaluation reference value, the degree of lens failure increases by a corresponding level.
  • the second lens failure degree may be determined based on the first lens failure degree corresponding to the evaluation reference value, the difference between the evaluation reference value and the evaluation value to be confirmed, wherein the first lens failure degree is a preset reference
  • the value can be the lowest value, the highest value, or any value in between within the set range.
  • the second lens failure degree is the lens failure degree corresponding to the evaluation value to be confirmed.
  • the second difference between the first lens failure degree and the second lens failure degree is determined based on the first difference between the evaluation reference value and the evaluation value to be confirmed, and based on the second difference and the first lens failure The degree determines the degree of failure of the second lens.
  • the relationship between the first image quality level and the second image quality level is not limited.
  • the first image quality level can be the lowest image quality in all image data sets.
  • the evaluation reference value is the lowest and the first lens fails.
  • the highest degree is that the second image quality level is greater than or equal to the first image quality level.
  • the degree of failure of the second lens can be determined based on the degree of failure of the first lens based on the degree to which the evaluation value to be confirmed is higher than the evaluation reference value.
  • an image data set may be selected, and the image quality corresponding to the selected image data set may be used as the first image quality level.
  • the second image quality level may be greater than the first image quality level, or it may be is less than the first image quality level, similarly, the degree of failure of the second lens is determined based on the degree of difference between the evaluation value to be confirmed and the evaluation reference value.
  • model quality evaluation results corresponding to the other multiple image data sets are all evaluation values to be confirmed, and they need to be compared with the evaluation reference values in sequence. After comparison, confirm the corresponding degree of failure of the second lens. At this time, other multiple image data sets may correspond to different second image quality levels.
  • the image quality of the first image quality level is optimal in all image data sets, and the first lens failure degree at this time can be set to a preset value, such as 0, for other image quality levels.
  • the respective second lens failure degrees can be confirmed based on the difference between the evaluation value to be confirmed and the evaluation reference value, for example, 1, 2, 3, ..., 9 respectively. etc.
  • one model quality evaluation result is used as the evaluation reference value, and other model quality evaluation results need to be compared with the evaluation reference value before the corresponding second lens failure degree can be determined. Based on the evaluation reference value, It is determined that this embodiment can determine the actual difference between the failure degrees of multiple lenses more accurately and more objectively.
  • the image quality corresponding to the first image quality level is greater than or equal to a preset image quality threshold.
  • the image data set corresponding to the first image quality level is required to have sufficiently high image quality.
  • the image quality threshold can be image characteristics related to light, clarity, shadow, dirt, occlusion, water stains, screen blur, etc.
  • the image data set corresponding to the first image quality level is a data set with a sharpness greater than or equal to a preset sharpness threshold and a pollution degree less than or equal to the preset pollution threshold.
  • the confirmed evaluation reference value It is also optimal.
  • limiting the first image quality level can more accurately reflect the true degree of lens failure.
  • the degree of lens failure of the input image corresponding to the image quality level is determined based on the relative size relationship between multiple model quality evaluation results.
  • the multiple model quality evaluation results can be compared, and the failure degree of each lens can be determined sequentially according to the order of the comparison results.
  • the input image is divided into three image data sets A, B, and C.
  • the input image is blocked in different proportions, which are low, medium, and high in order.
  • the image quality levels of the three image data sets are In order of high, medium, and low
  • the image data sets A, B, and C are respectively input into the pre-model for evaluation, and correspondingly three different model quality evaluation results a, b, and c are obtained.
  • all inputs in the image data set A can be
  • the lens failure degree of the image is defined as "low”
  • the lens failure degree of all input images in image data set B is defined as “medium”
  • the lens failure degree of all input images in image data set C is defined as "high”.
  • the degree of lens failure is determined based on the relative size relationship between multiple model quality evaluation results, which can avoid excessive differences between the image qualities of different image data sets and the degree of lens failure exceeding the preset range, improving Scenario adaptability of the sample label acquisition method.
  • the quality of the model can be evaluated by the recall rate or the precision rate.
  • determining the model quality assessment results of the front-end model includes:
  • the model quality evaluation result is jointly determined based on the recall rate and precision rate of the pre-model.
  • the corresponding degree of lens failure can be determined based on the degree of change in precision or recall between different image data sets; or, based on the degree of change in precision or recall between different image data sets, The degree of change determines the corresponding degree of lens failure.
  • the recall rate refers to the proportion of true positive samples that the model determines to be positive samples.
  • the calculation method is as shown in Formula 1:
  • R represents the recall rate
  • TP represents the true sample
  • FN represents the false negative sample
  • the precision rate refers to the proportion of true positive samples among all the positive samples determined by the model.
  • the calculation method is as shown in Formula 2:
  • P represents the precision rate
  • FP represents the false positive sample
  • the recall rate or the precision rate can be used alone, or the front-end model can be evaluated based on the results of weighted calculations of the two.
  • the pre-model is evaluated through the recall rate or the precision rate, or the recall rate and the precision rate, which can improve the accuracy of the model quality assessment results and ensure the objectivity of the lens failure degree.
  • a recall-precision curve that is, a PR curve
  • the numerical characteristics of the recall-precision curve determine the model quality evaluation results, where the numerical characteristics can be values on the PR curve, or can be characteristic parameters calculated based on the PR curve.
  • a PR curve is obtained by plotting the recall rate as the horizontal axis and the precision rate as the vertical axis. After obtaining the PR curve, the corresponding precision rate can be found at a certain recall rate as a model quality evaluation result, or the corresponding recall rate can be found at a certain precision rate as a model quality evaluation result.
  • the numerical characteristic of the recall-precision curve is the area between the recall-precision curve and the coordinate axis, which is recorded as the evaluation area, also known as the AP (Average Precision) value. Since the evaluation area can take into account both the recall rate and the precision rate, the evaluation of the front-end model is also more comprehensive.
  • mAP mean Average Precision
  • ROC Receiver Operating Characteristic
  • the ROC curve is obtained with FPR (False positive rate, false positive rate) as the horizontal axis and TPR (True positive rate, true positive rate) as the vertical axis.
  • FPR False positive rate, false positive rate
  • TPR True positive rate, true positive rate
  • TPR is the recall rate
  • the calculation method of FPR is as follows: Formula 3 Shown:
  • TN represents true negative samples and FP represents false positive samples.
  • FIG4 is a flow chart of another method for determining the degree of lens failure according to an embodiment of the present application. As shown in FIG4 , the method includes the following steps:
  • Step S401 use the evaluation area obtained by using the image data set of the first image quality level as the input of the pre-model as the evaluation reference value; wherein, the image data set corresponding to the first image quality level is the image data set with the highest image quality among multiple image data sets. Excellent;
  • Step S402 Determine the evaluation obtained by the pre-model using the image data set of the second image quality level as input. Area, recorded as the assessed value to be confirmed;
  • Step S403 Determine the degree of lens failure of the input image with the second image quality level based on the difference between the evaluation reference value and the evaluation value to be confirmed.
  • the pre-model is evaluated through the AP value of the PR curve, which can reflect the image quality of the input image more comprehensively and comprehensively, and also makes the degree of lens failure more comprehensive and accurate.
  • the lens failure degree of the corresponding input image is determined based on the relative size relationship between the multiple AP values, where the multiple AP values respectively correspond to Multiple image quality levels.
  • the lens failure degree label corresponding to the input image is reversely calibrated based on the different model quality evaluation results obtained by the pre-model under image data sets of different image qualities. It can be seen that the model quality assessment results determined based on the front-end model can be used as a basis for determining the degree of lens failure. Therefore, the pre-model is understood as assisting in determining the failure degree label of the input image of the lens failure detection model to be trained.
  • the intersection and union ratio of the detected target in the input image is first obtained.
  • the detection targets can be other vehicles around the vehicle during autonomous driving, or pedestrians in public places, etc.
  • the Intersection over Union (IoU for short) is the ratio of the intersection and union of two bounding boxes calculated for the detection target. One of the bounding boxes is the calculated prediction box and the other is the actual annotation box. .
  • intersection ratio When the intersection ratio is greater than or equal to the preset intersection ratio threshold, the model quality evaluation results of the front model are obtained to calculate the degree of lens failure.
  • the intersection ratio is usually used to evaluate whether the target detection model is accurate in positioning the detection target. The higher the intersection ratio, the more accurate the positioning of the target detection model and the better the training effect. Therefore, it is set before determining the model quality evaluation results.
  • the intersection ratio threshold filters out poorly positioned input images, which can eliminate interfering factors other than image quality to a certain extent, allowing subsequent lens failure to more truly reflect the image quality of the input image and improve the accuracy of sample labeling. .
  • the intersection and union ratio threshold can be set according to requirements, for example, the intersection and union ratio threshold is set to 0.5.
  • this embodiment further provides a method for training a shot failure detection model, as shown in FIG5 , including the following steps:
  • Step S501 obtain the input image of the lens failure detection model to be trained and the failure degree label of the input image
  • the failure degree label can be obtained by the method of obtaining the sample label in any of the above embodiments, for example, determining the model quality evaluation result of the front model; determining the lens failure degree of the input image of the front model according to the model quality evaluation result; The lens failure degree is used as the failure degree label of the input image.
  • the front model in this embodiment is used to assist in determining the failure degree label of the input image of the lens failure detection model to be trained.
  • the input image of the front model is used as the lens failure detection model to be trained. Input images of the model to train the lens failure detection model;
  • Step S502 using the input image as input data of a lens failure detection model to be trained, using the failure degree label as target data of the lens failure detection model, and training the lens failure detection model, wherein the lens failure detection model is at least used to determine the lens failure degree of the image to be predicted.
  • the target data refers to the real or expected output that the model is expected to predict or classify, and is a set of known labels or answers for the supervised learning task.
  • a training method of the lens failure detection model is provided. Since the failure degree label of the input image is determined by the model quality evaluation result of the pre-model, the failure degree label is more objective and accurate. Based on this, Not only does it solve the problem that the judgment of the degree of failure in related technologies is relatively subjective, but it also cannot be used for subsequent drawings. The problem of providing objective basis for the image processing process can also improve the recognition accuracy of the lens failure detection model.
  • the input image of the front-end model is divided into multiple image data sets, the multiple image data sets respectively correspond to multiple image quality levels, and the multiple image quality levels Corresponding to multiple model quality assessment results respectively, at this time, the lens failure degree of each input image can be determined based on the numerical relationship between the multiple model quality assessment results. For example, the model quality evaluation result obtained when the front-end model takes the image data set of the first image quality level as input is used as the evaluation reference value; determine the result obtained when the front-end model takes the image data set of the second image quality level as input. The model quality evaluation result is recorded as the evaluation value to be confirmed; the lens failure degree of the input image with the second image quality level is determined according to the difference between the evaluation reference value and the evaluation value to be confirmed.
  • the AP value of the front model is used as the model quality evaluation result.
  • the lens failure detection model detects the lens failure type and the lens failure degree. At this time, before training the lens failure detection model, it is also necessary to obtain the lens failure type of the input image as the failure type label; therefore,
  • the training of the lens failure detection model includes: using the input image as the input data of the lens failure detection model to be trained, using the failure degree label and the failure type label as the target data of the lens failure detection model, and training the lens failure detection model.
  • the failure type labels include normal, dirty and blurred, occlusion, water stain, light, blurred screen, etc.
  • the failure type label is determined according to the lens failure type corresponding to the image data set to which the input image belongs; or, it is manually labeled.
  • the lens failure detection model may be a multi-task classification model. Before formal training, it is necessary to obtain large-scale input images and divide all input images according to lens failure type and lens failure degree. Following the same preprocessing method, the input image is cropped to the preset size. The processed input image is then input into the convolutional neural network to train the multi-task classification model.
  • the multi-task classification model in this embodiment completes the two tasks of classification and regression through one model.
  • the backbone The two branch networks of classification and regression are connected under the network. Convolutional neural network includes data layer, convolution layer, pooling layer, activation layer, fully connected layer and output layer.
  • the training method is the gradient descent method and the back propagation algorithm, and the process is as follows:
  • Step 1 Input the processed input image, failure degree label and failure type label into the training network through the data layer;
  • Step 2 The convolution layer extracts data features by setting the step size, convolution kernel size, and number of convolution kernels to obtain the feature map; the pooling layer uses the set step size and pooling size to compare the features of the previous layer The map is downsampled; the activation layer performs nonlinear changes on the downsampled feature map, where the activation function is the relu activation function;
  • Step 3 The fully connected layer connects all feature maps, and maps the feature space to the label space through linear transformation through weights.
  • the fully connected layer is followed by the relu activation function;
  • Step 4 The output layer performs classification and regression on the feature map.
  • the softmax function is used as the output layer function for classification
  • the euclidean function is used as the output layer function for failure degree regression.
  • the loss function of each attribute can be found first, and then The weighted sum of multiple attribute loss functions is used to obtain the overall loss function, thus taking each attribute into consideration, and the training goal is to minimize the overall error of all attributes.
  • the above method supports the combination of multiple different attributes.
  • the loss function for lens failure type is L_invalid_class
  • the loss function for lens failure degree is L_invalid_degree.
  • a and b are parameters that can be set during the training process in order to achieve the optimal effect of the model. Usually the value range of a and b is between 0 and 1.
  • the trained model can provide a more accurate lens failure degree for a clear lens failure type.
  • a lens failure detection method is provided, as shown in Figure 6.
  • the method includes the following steps:
  • Step S601 obtain the image to be predicted, and determine the lens failure degree of the image to be predicted through the trained lens failure detection model
  • the lens failure detection model is obtained through the training method of any of the above lens failure detection models;
  • the image to be predicted is an image collected in an actual scene, and the image to be predicted can include any target such as pedestrians, vehicles, faces, etc., or can For scene graphs that do not include specific goals;
  • Step S602 use the lens failure degree of the image to be predicted as the lens failure degree of the image acquisition device, where the image to be predicted is obtained through the image acquisition device, and the image acquisition device can be a mobile terminal, such as a mobile phone, a tablet, or AR (Augmented Reality).
  • reality virtual Reality, virtual reality
  • XR Extended Reality, extended reality
  • MR Magnetic Reality, mixed reality
  • it can be professional equipment such as cameras and cameras.
  • the degree of lens failure of the image acquisition device itself can be judged according to the image to be predicted obtained by the image acquisition device, which provides a basis for subsequent processing of the image to be predicted. Since the label of the input image is determined based on the model quality evaluation result of the front model during the training of the lens failure detection model, the degree of lens failure of the image acquisition device is also objective, which solves the problem of subjective judgment of the degree of lens failure in related technologies.
  • the method for detecting lens failure further includes:
  • the obtained results include one or more of the following:
  • the degree of lens failure and the type of lens failure are the degree of lens failure and the type of lens failure.
  • the degree of lens failure can be determined based on the image to be predicted, and the type of lens failure can be determined to obtain more lens information.
  • the lens failure type of the image to be predicted is obtained through the trained lens failure detection model.
  • the lens failure detection model uses the input image as the to-be-trained image. The input data of the lens failure detection model is used, and the failure degree label and failure type label of the input image are used as the target data of the lens failure detection model, and the lens failure detection model is trained.
  • the final lens failure type corresponding to the image to be predicted is determined according to the degree of impact of the multiple lens failure types on the image to be predicted, wherein, The degree of impact of multiple lens failure types on the image to be predicted can be determined based on the proportion of the image area corresponding to each lens failure type in the entire image to be predicted. The greater the proportion, the greater the impact.
  • the lens failure types are output in order according to their impact degree, and the area proportion of each lens failure type is output. For example, if the lens is blocked by a large area and has a small amount of water stains attached, it is classified as a failure caused by blocking factors.
  • the lens failure type and lens failure degree can be output to the user to facilitate the user to make adjustments; some examples
  • the method further includes: outputting a lens adjustment strategy according to the type of lens failure or degree of lens failure, or outputting a lens adjustment strategy based on the type of failure and the degree of lens failure.
  • the user can debug the lens accordingly according to at least one of the following information: lens failure type, lens failure degree, and lens adjustment strategy, so that the lens can work normally; in other embodiments, at least one of the information can be:
  • the lens failure type, lens failure degree and lens adjustment strategy are sent to the lens control system, and the control system makes an automatic adjustment strategy for the lens.
  • Some exemplary embodiments also include: judging whether the collected images can be used in subsequent vision algorithms based on the degree of lens failure, improving the accuracy and stability of solutions based on vision algorithms, and reducing labor costs.
  • ADAS Advanced Driving Assistance System
  • ADAS Advanced Driving Assistance System
  • dirt, occlusion, blur, blur, water Stains, etc. will have an impact on the imaging quality of ADAS lenses, causing the accuracy of vision-based related algorithms to seriously decrease, and even cause the lens to fail and become unable to work normally. Therefore, a comprehensive analysis of the lens failure scenario is conducted, and corresponding solutions are given. Solutions are particularly critical. Based on this, embodiments of the present disclosure provide a vision-based detection method for ADAS lens failure, which can effectively distinguish each type of failure scenario of ADAS lenses, improve the robustness of the vision algorithm, and improve the safety of vision-based ADAS. The method includes the following steps:
  • Step 1 Sample collection and processing
  • a video segment is obtained through actual acquisition, and a sample image is acquired based on the video segment.
  • the sample image includes a vehicle that needs to be identified.
  • lens failure types include dirt and blur, occlusion, water stains, light, blur, etc.
  • the definitions of lens failure types are as follows: 1) Dirty and blurred. Dirty refers to non-transparent sticky objects on the lens, such as mud, mud, etc.; blurred refers to loose dust attached to the lens, making the lens visible. The overall outline of the target can be seen, but the details are blurred; 2) Occlusion refers to the situation where the entire area is blocked by non-transparent objects; 3) Fragmentation refers to the situation where the image appears blurry due to camera circuit failure or lens damage. ; 4) Water stains refer to transparent liquid adhering to the surface of the lens; 5) Light refers to the brightness of the image.
  • occlusion and blurring a clear sample image can be directly obtained first, and then the occlusion or blurry screen can be simulated through a computer.
  • occlusion can be simulated by assigning pixels in some areas to 0, pixel misalignment or pixel distortion can be simulated.
  • the pixel mosaic is simulated as a blurry screen; for light, the light intensity can be adjusted during the process of collecting sample images through the lens to obtain a sample image related to the light.
  • you can adjust dirt, water stains, blur The proportion of pixels in the entire sample image and the change in light intensity simulate images under different degrees of lens failure.
  • Sample labels include failure type labels and failure degree labels.
  • the lens failure type label can be manually labeled after simulation.
  • the lens failure degree label is obtained through the following steps.
  • the front-end model is a target detection model used for vehicle detection.
  • vehicle detection is performed on the input images corresponding to each lens failure degree of each type of lens failure type in turn.
  • the quality of the target detection model is evaluated based on 1000 input images without occlusion.
  • the quality evaluation of the target detection model is carried out based on 1000 input images with a small part occluded, and the quality evaluation of the target detection model is carried out based on 1000 input images with a large part occluded.
  • the IoU is greater than or equal to 0.5
  • the AP value of each target detection model is calculated.
  • the AP value of the target detection model obtained at this time is used as an evaluation reference. value. Images in other image datasets are used as input images in turn to evaluate the quality of the target detection model. Relative to the evaluation reference value, the target detection model will fail when the AP value drops by 2%, 4%, 6%, 8%, 10%, 12%, 14%, 16%, 18%, and 20% respectively.
  • the degree is identified as 0, 1, 2, 3, 4, 5, 6, 7, 8, 9.
  • the degree of decline can be divided according to intervals. If it falls within the interval, it can be divided into the right value of the interval. For example, if it drops by 1% and falls between 0-2%, then the failure degree is divided into 1, and so on.
  • each input image will have its own failure type label and failure degree label, which can be used for subsequent lens failure detection model training.
  • the lens failure detection model in this embodiment is a multi-task classification model used to determine lens failure type and lens failure degree, and includes two branch networks: classification and regression.
  • the input image of the front-end model is used as the input image of the multi-task classification model.
  • the input image includes the failure type label and the failure degree label.
  • the feature maps of the input image are classified and regressed respectively to determine the predicted value of the input image in the lens failure type and lens failure degree as the output result.
  • classification and regression tasks are related, part of the network can be shared to simplify the multi-task classification model.
  • the classification and regression layers have their own fully connected layers and final decision output layers.
  • the respective pairs of classification and regression can be extracted. Useful features.
  • the ADAS lens acquires real-time images to be predicted, and inputs the images to be predicted into the trained lens failure detection model to obtain the lens failure type and lens failure degree of the ADAS lens.
  • ADAS can determine the lens failure type and lens failure degree based on the lens failure type and lens failure rate. The degree of failure gives corresponding lens adjustment strategies.
  • a combination of classification and regression is used to classify the failure scenarios and regress the degree of the failure scenarios, so as to effectively classify the lens failure types and lens failures in one model.
  • the failure degree label of the input image is determined based on the AP value of the target detection model, and the failure degree of the lens is defined by numerical parameters, thereby solving the problem that the judgment of the failure degree in related technologies is relatively subjective and cannot provide an objective basis for the subsequent image processing process.
  • the failure degree of the lens is objectified, so that the judgment of the failure degree of the lens has an objective standard.
  • the method for obtaining sample labels is applied to face algorithms, such as face detection, before obtaining the model quality assessment results, it is also necessary to distinguish situations where the face cannot be detected due to reasons other than failure of the lens itself, such as large angles, extremely small resolutions, etc., in order to improve the accuracy of the lens failure detection model.
  • This embodiment also provides a method for obtaining sample labels, including:
  • the lens failure degree is used as the failure degree label of the input image, wherein when training the lens failure detection model, the input image of the front model is used as the input image of the lens failure detection model.
  • a device for acquiring a sample label is also provided, and the device is used to implement any of the embodiments described in the present disclosure, and the descriptions that have been made will not be repeated.
  • the terms “module”, “unit”, “subunit”, etc. used below can implement a combination of software and/or hardware for predetermined functions.
  • the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.
  • Figure 7 is a structural block diagram of a sample label acquisition device according to an embodiment of the present application. As shown in Figure 7, the device includes a quality assessment module 71, a failure determination module 72 and a label determination module 73:
  • a quality assessment module 71 configured to determine a model quality assessment result of a pre-model
  • the failure determination module 72 is configured to determine the degree of lens failure of the input image of the front model according to the model quality evaluation result
  • the label determination module 73 is configured to use the lens failure degree as a failure degree label of the input image, wherein the input image of the pre-model is set as an input image of the lens failure detection model to the lens implementation model. Conduct training.
  • the sample label acquisition device in this embodiment determines the model quality evaluation result of the pre-model through the quality evaluation module 71, and finally determines the failure degree label of the input image through the label determination module 73, realizing the definition of the lens failure degree through numerical parameters. , it is no longer necessary to manually annotate, thereby solving the problem in related technologies that the judgment of the degree of failure is relatively subjective and cannot provide an objective basis for the subsequent image processing process.
  • the degree of lens failure is objectified, making the judgment of the degree of lens failure more efficient. It establishes objective standards and is also conducive to subsequent targeted processing of images.
  • the failure determination module 72 is further configured to determine the lens failure degree of each input image according to the numerical relationship between multiple model quality evaluation results, wherein all input images of the front-end model are divided into multiple Image data sets, multiple image data sets respectively correspond to multiple image quality levels, and multiple image quality levels respectively correspond to multiple model quality evaluation results.
  • the failure determination module 72 uses the model quality evaluation result obtained when the front-end model uses the image of the first image quality level as the input image as the evaluation reference value; determines that the front-end model uses the image of the second image quality level.
  • the model quality evaluation result obtained when inputting the image is recorded as the evaluation value to be confirmed; according to the evaluation reference
  • the difference between the value and the evaluation value to be confirmed determines the degree of lens failure for the input image having the second image quality level.
  • the image quality corresponding to the first image quality level is greater than or equal to a preset image quality threshold.
  • the quality assessment module 71 is further configured to determine the model quality assessment result according to the recall or precision of the pre-model; or, to determine the model quality assessment result according to the recall and precision of the pre-model.
  • the recall-precision curve is determined according to the recall and precision of the pre-model; and the model quality assessment result is determined according to the numerical characteristics of the recall-precision curve.
  • the numerical characteristics of the recall-precision curve are the area between the recall-precision curve and the coordinate axis, which is recorded as the assessment area.
  • each of the above modules can be a functional module or a program module, and can be implemented by software or hardware.
  • all the above-mentioned modules may be located in the same processor; or the above-mentioned modules may be located in different processors in any combination.
  • This application also provides a vehicle control system, which includes a memory and a processor.
  • the memory stores a program.
  • the program is read and executed by the processor, any method in the above embodiments is implemented.
  • the vehicle includes an image acquisition device and an image processor.
  • the image acquisition device is configured to acquire images to be predicted.
  • the image processor is configured to implement any method in the above embodiments.
  • This embodiment also provides an electronic device, including a memory and a processor.
  • the memory stores a computer program.
  • the processor is configured to run the computer program to perform the steps in any of the above method embodiments.
  • the above-mentioned electronic device further includes a transmission device and an input-output device, wherein the transmission device is connected to the above-mentioned processor, and the input-output device is connected to the above-mentioned processor.
  • the above-mentioned processor is configured to perform the following steps through a computer program:
  • S2 Determine the degree of lens failure of the input image of the front-end model according to the model quality evaluation result.
  • S3 use the lens failure degree as the failure degree label of the input image, wherein when training the lens failure detection model, use the input image of the front model as the input image of the lens failure detection model.
  • this embodiment also provides a computer-readable storage medium for implementation.
  • One or more programs are stored on the storage medium, and the one or more programs can be executed by one or more processors; when the program is executed by the processor, any one of the methods in the above embodiments is implemented.
  • the user information including but not limited to user equipment information, user personal information, etc.
  • data including but not limited to data used for analysis, stored data, displayed data, etc.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Multimedia (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • General Physics & Mathematics (AREA)
  • Physics & Mathematics (AREA)
  • Evolutionary Computation (AREA)
  • Computing Systems (AREA)
  • Medical Informatics (AREA)
  • Software Systems (AREA)
  • General Health & Medical Sciences (AREA)
  • Databases & Information Systems (AREA)
  • Health & Medical Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Image Analysis (AREA)
  • Testing, Inspecting, Measuring Of Stereoscopic Televisions And Televisions (AREA)

Abstract

一种样本标签的获取方法和镜头失效检测模型的训练方法，其中，该样本标签的获取方法包括先确定前置模型的模型质量评估结果，其中，前置模型用于辅助确定待训练镜头失效检测模型的输入图像的失效程度标签；然后根据模型质量评估结果确定前置模型的输入图像的镜头失效程度；最后将镜头失效程度作为输入图像的失效程度标签进行镜头失效检测模型的训练，解决了相关技术中对镜头失效程度的判断比较主观的问题，为图像处理过程提供了客观依据。

Description

样本标签的获取方法和镜头失效检测模型的训练方法
本申请要求于2022年09月19日提交中国专利局、申请号为202211139892.4、发明名称为“样本标签的获取方法和镜头失效检测模型的训练方法”的中国专利申请的优先权，其内容应理解为通过引用的方式并入本申请中。
技术领域
本申请涉及计算机技术领域。
背景技术
随着计算机技术的发展，对图像质量的依赖性也越来越高。但是图像采集设备的镜头常被各类污染物污染，从而导致镜头拍摄效果欠佳或无法正常拍摄，不能获得满足业务需要的有效拍摄图像，因此该情况统称为镜头失效，进一步地，通过失效镜头采集到的图像质量不佳，也会影响后续的图像处理过程。
可以理解，不同的镜头失效程度对于图像处理过程的影响不同，如车辆检测、车道线分割等等，但相关技术中，仅能通过技术人员人为对镜头的失效程度进行区分，所以对于失效程度的判断比较主观，无法为后续的图像处理过程提供客观的依据。
目前针对相关技术中对失效程度的判断比较主观，无法为后续的图像处理过程提供客观依据的问题，尚未提出有效的解决方案。
发明内容
以下是对本文详细描述的主题的概述。本概述并非是为了限制权利要求的保护范围。
本申请提供了一种样本标签的获取方法、镜头失效检测模型的训练方法、镜头失效的检测方法、车载控制系统和车辆，以解决相关技术中对失效程度的判断比较主观，无法为后续的图像处理过程提供客观依据的问题。
第一方面，本申请提供了一种样本标签的获取方法，所述方法包括：
确定前置模型的模型质量评估结果；
根据所述模型质量评估结果确定所述前置模型的输入图像的镜头失效程度；
将所述镜头失效程度作为所述输入图像的失效程度标签，其中，所述前置模型的输入图像及其失效程度标签作为训练镜头失效检测模型的样本数据。
在其中一些实施例中，所述根据所述模型质量评估结果确定所述前置模型的输入图像的镜头失效程度包括：
根据多个所述模型质量评估结果之间的数值关系，确定所述前置模型的每一个输入图像的镜头失效程度，其中，所述前置模型的所有输入图像被划分为多个图像数据集，多个图像数据集分别对应于多个图像质量级别，多个图像质量级别分别对应于多个所述模型质量评估结果。
在其中一些实施例中，所述根据多个所述模型质量评估结果之间的数值关系，确定所述前置模型的每一个输入图像的镜头失效程度包括：
将所述前置模型以第一图像质量级别的图像数据集作为输入所得到的模型质量评估 结果作为评估参考值；
确定所述前置模型以第二图像质量级别的图像数据集作为输入所得到的模型质量评估结果，记为待确认评估值；
根据所述评估参考值与所述待确认评估值之间的差异确定具有第二图像质量级别的输入图像的镜头失效程度。
在其中一些实施例中，所述第一图像质量级别对应的图像质量大于或者等于预设的图像质量阈值。
在其中一些实施例中，所述根据多个所述模型质量评估结果之间的数值关系，确定所述前置模型的每一个输入图像的镜头失效程度包括：
根据多个所述模型质量评估结果之间的相对大小关系，确定与所述图像质量级别对应的输入图像的镜头失效程度。
在其中一些实施例中，所述确定前置模型的模型质量评估结果包括：
根据所述前置模型的查全率或查准率确定所述模型质量评估结果；
或者，
根据所述前置模型的查全率和查准率共同确定所述模型质量评估结果。
在其中一些实施例中，所述根据所述前置模型的查全率和查准率共同确定所述模型质量评估结果包括：
根据所述前置模型的查全率和查准率确定查全率-查准率曲线；
根据所述查全率-查准率曲线的数值特征确定所述模型质量评估结果。
在其中一些实施例中，所述查全率-查准率曲线的数值特征为所述查全率-查准率曲线与坐标轴之间的面积，记为评估面积。
在其中一些实施例中，在所述确定前置模型的模型质量评估结果之前，所述方法还包括：
在所述前置模型为目标检测模型的情况下，获取所述输入图像中检测目标的交并比；
在所述交并比大于或者等于预设的交并比阈值的情况下，获取所述前置模型的模型质量评估结果。
第二方面，本申请实施例提供了一种镜头失效检测模型的训练方法，所述方法包括：
获取待训练的镜头失效检测模型的输入图像以及所述输入图像的失效程度标签，其中，所述失效程度标签根据第一方面中任一项所述的样本标签的获取方法得到；
将所述输入图像作为所述待训练的镜头失效检测模型的输入数据，将所述失效程度标签作为所述镜头失效检测模型的目标数据，对所述镜头失效检测模型进行训练，其中，所述镜头失效检测模型至少用于确定待预测图像的镜头失效程度。
在其中一些实施例中，在对所述镜头失效检测模型进行训练之前，还包括：获取所述输入图像的镜头失效类型作为失效类型标签；
所述对所述镜头失效检测模型进行训练包括：
将输入图像作为待训练的镜头失效检测模型的输入数据，将所述失效程度标签和所述失效类型标签作为所述镜头失效检测模型的目标数据，对所述镜头失效检测模型进行训练。
第三方面，本申请实施例提供了一种镜头失效的检测方法，所述方法包括：
获取待预测图像，通过训练好的镜头失效检测模型确定所述待预测图像的镜头失效程度，其中，所述训练好的镜头失效检测模型通过第二方面中任一项镜头失效检测模型的训练方法得到；
将所述待预测图像的镜头失效程度作为图像采集设备的镜头失效程度，其中，所述待预测图像通过所述图像采集设备得到。
在其中一些实施例中，在所述获取待预测图像之后，所述方法还包括：
通过所述训练好的镜头失效检测模型获取所述待预测图像的镜头失效类型；
根据获取结果至少执行以下步骤之一：
输出所述获取结果；
根据所述获取结果输出镜头调整策略；
其中，所述获取结果包括以下一项或多项：
所述镜头失效程度和所述镜头失效类型。
在其中一些实施例中，所述通过所述训练好的镜头失效检测模型获取所述待预测图像的镜头失效类型包括：
在所述待预测图像具有多个所述镜头失效类型的情况下，根据多个所述镜头失效类型对所述待预测图像的影响程度，确定与所述待预测图像对应的最终镜头失效类型。
第四方面，本申请实施例提供了一种车载控制系统，包括存储器和处理器，所述存储器存储有程序，所述程序在被所述处理器读取执行时，实现如上述第一方面至第三方面中任一所述的方法。
第五方面，本申请实施例提供了一种车辆，所述车辆包括：
图像采集设备，设置为获取待预测图像；
图像处理器，设置为实现如上述第一方面至第三方面中任一所述的方法。
第六方面，本申请实施例提供了一种计算机可读存储介质，所述计算机可读存储介质存储有一个或者多个程序，所述一个或者多个程序可被一个或者多个处理器执行，以实现如上述第一方面至第三方面中任一所述的方法。
本实施例中提供的样本标签的获取方法，先确定前置模型的模型质量评估结果，其中，前置模型用于辅助确定待训练镜头失效检测模型的输入图像的失效程度标签；然后根据模型质量评估结果确定前置模型的输入图像的镜头失效程度；最后将镜头失效程度作为输入图像的失效程度标签进行镜头失效检测模型的训练，解决了相关技术中对镜头失效程度的判断比较主观的问题，为图像处理过程提供了客观依据。
本申请的其它特征和优点将在随后的说明书中阐述，并且，部分地从说明书中变得显而易见，或者通过实施本申请而了解。本申请的其他优点可通过在说明书以及附图中所描述的方案来实现和获得。
在阅读并理解了附图和详细描述后，可以明白其他方面。
附图概述
附图用来提供对本申请技术方案的理解，并且构成说明书的一部分，与本申请的实施例一起用于解释本申请的技术方案，并不构成对本申请技术方案的限制。
图1是本实施例的样本标签的获取方法的终端的硬件结构框图；
图2是根据本申请实施例的一种样本标签的获取方法的流程图；
图3是根据本申请实施例的一种镜头失效程度的确定方法的流程图；
图4是根据本申请实施例的另一种镜头失效程度的确定方法的流程图；
图5是根据本申请实施例的镜头失效检测模型的训练方法的流程图；
图6是根据本申请实施例的一种镜头失效的检测方法的流程图；
图7是本申请实施例的样本标签的获取装置的结构框图。
详述
为更清楚地理解本申请的目的、技术方案和优点，下面结合附图和实施例，对本申请进行了描述和说明。
除另作定义外，本申请所涉及的技术术语或者科学术语应具有本申请所属技术领域具备一般技能的人所理解的一般含义。在本申请中的“一”、“一个”、“一种”、“该”、“这些”等类似的词并不表示数量上的限制，它们可以是单数或者复数。在本申请中所涉及的术语“包括”、“包含”、“具有”及其任何变体，其目的是涵盖不排他的包含；例如，包含一系列步骤或模块(单元)的过程、方法和系统、产品或设备并未限定于列出的步骤或模块(单元)，而可包括未列出的步骤或模块(单元)，或者可包括这些过程、方法、产品或设备固有的其他步骤或模块(单元)。在本申请中所涉及的“连接”、“相连”、“耦接”等类似的词语并不限定于物理的或机械连接，而可以包括电气连接，无论是直接连接还是间接连接。在本申请中所涉及的“多个”是指两个或两个以上。“和/或”描述关联对象的关联关系，表示可以存在三种关系，例如，“A和/或B”可以表示：单独存在A，同时存在A和B，单独存在B这三种情况。通常情况下，字符“/”表示前后关联的对象是一种“或”的关系。在本申请中所涉及的术语“第一”、“第二”、“第三”等，只是对相似对象进行区分，并不代表针对对象的特定排序。
在本公开实施例中提供的方法可以在终端、计算机或者类似的运算装置中执行。比如在终端上运行，图1是本实施例的样本标签的获取方法的终端的硬件结构框图。如图1所示，终端可以包括一个或多个(图1中仅示出一个)处理器102和用于存储数据的存储器104，其中，处理器102可以包括但不限于微处理器MCU或可编程逻辑器件FPGA等的处理装置。一些示例性实施例中，所述终端还包括用于通信的传输设备106以及输入输出设备108。本领域普通技术人员可以理解，图1所示的结构仅为示意，其并不对上述终端的结构造成限制。例如，终端还可包括比图1中所示更多或者更少的组件，或者具有与图1所示出的不同配置。
存储器104可用于存储计算机程序，例如，应用软件的软件程序以及模块，如在本公开实施例中的样本标签的获取方法对应的计算机程序，处理器102通过运行存储在存储器104内的计算机程序，从而执行各种功能应用以及数据处理，即实现上述的方法。存储器104可包括高速随机存储器，还可包括非易失性存储器，如一个或者多个磁性存储装置、闪存、或者其他非易失性固态存储器。在一些实例中，存储器104可包括相对于处理器102远程设置的存储器，这些远程存储器可以通过网络连接至终端。上述网络的实例包括但不限于互联网、企业内部网、局域网、移动通信网及其组合。
传输设备106用于经由一个网络接收或者发送数据。上述的网络包括终端的通信供应 商提供的无线网络。在一个实例中，传输设备106包括一个网络适配器(NetworkInterface Controller，简称为NIC)，其可通过基站与其他网络设备相连从而可与互联网进行通讯。在一个实例中，传输设备106可以为射频(Radio Frequency，简称为RF)模块，其用于通过无线方式与互联网进行通讯。
本申请实施例提供了一种样本标签的获取方法，获得样本标签后的图像用于训练对镜头失效程度进行判别的镜头失效检测模型，即获得样本标签后的图像作为输入图像对待训练的镜头失效检测模型进行训练，如图2所示，样本标签的获取方法包括如下步骤：
步骤S201，确定前置模型的模型质量评估结果。
本公开实施例涉及到两个模型，一个是最终需要的镜头失效检测模型，另一个是为待训练的镜头失效检测模型获取样本标签的前置模型，其中，前置模型可以为任一种类的模型，例如，人脸识别模型、行人检测模型或者通用的目标检测模型等等。由于前置模型在训练或使用的过程中，需要对其进行质量评估以确定训练效果，例如识别的准确率等等，因此可以获取前置模型的模型质量评估结果，并基于此确定失效程度标签。
本公开实施例中，前置模型可以为已初步完成训练的模型，即根据所选用模型对应的训练方案已初步完成训练。针对某一输入图像，可以将该输入图像输入前置模型进行测试，从而能够得到对应的模型识别或分类结果；针对包括多个输入图像的输入图像集，采用该前置模型得到每一个输入图像对应的模型识别或分类结果后，可以确定与该输入图像集对应的模型质量评估结果。
可以理解，所述前置模型可以用于辅助确定待训练的镜头失效检测模型的输入图像的失效程度标签，即可以用于对镜头失效检测模型进行训练的样本图像辅助确定对应的失效程度标签。
一些示例性实施例中，模型质量评估结果可以表示为一个数值，在该数值为准确度的情况下，则准确度越高，意味着前置模型的训练效果越好；一些示例性实施例中，在模型质量评估结果用于表示前置模型的误判率的情况下，误判率越高，意味着前置模型的训练效果越差。示例性地，在输入图像存在光线不好、有脏污或者花屏的情况下，图像质量较低，此时前置模型的识别准确度会下降，相反的，在输入图像清晰完整的情况下，前置模型的识别准确度会上升，因此，模型质量评估结果会随输入图像的图像质量的变化而变化。基于此，可以通过控制输入图像的图像质量，得到不同的模型质量评估结果。
一些示例性实施例中，所述图像质量根据以下一个或多个参数进行评价：
光线、清晰度、阴影、脏污、遮挡、水渍、花屏。
一些示例性实施例中，所述图像质量根据以下一个或多个参数进行评价：
分辨率、对比度、锐度、色彩准确度、噪声水平、勾勒度。
例如，分辨率越高图像质量越高，锐度越高图像质量越高，等等，用户可以根据实际需求选取进行图像质量评价的参数，不限于本公开示例的特定方面。
可以理解，上述根据上述一个或多个参数的定量的参数值或分级(分类)的参数值，可对应确定图像质量或图像质量等级。
本实施例中的镜头失效检测模型用于确定图像采集设备的镜头失效程度，该图像采集设备可以为移动终端，例如手机、平板、增强现实(Augmented Reality，AR)设备、虚拟现实(Virtual Reality，VR)设备、XR(Extended Reality，扩展现实)设备、MR(Mixed Reality，混合现实)设备等等，或可以为摄像头、相机等专业设备。
步骤S202，根据模型质量评估结果确定前置模型的输入图像的镜头失效程度。
如前所述，前置模型的模型质量评估结果可以反映前置模型在评估或测试时输入图像的图像质量，所以模型质量评估结果与输入图像的图像质量具有对应关系。另一方面，图像采集设备的镜头失效程度与输入图像的图像质量也相关，例如，在图像采集设备的镜头表面存在雨水、脏污或者被划伤等情况下，就会导致所采集的图像的图像质量下降。综上，在输入图像数据集具有不同的图像质量时，以不同的图像数据集进行前置模型评估所得到的模型质量评估结果也会存在差异，因此，模型质量评估结果可以用于定义镜头失效程度，即，模型质量评估结果与镜头失效程度具有对应关系。
一些示例性实施例中，在模型质量评估结果表示前置模型的识别准确度的情况下，则模型质量评估结果和镜头失效程度负相关，即前置模型的识别准确度越高，镜头失效程度越低，前置模型的识别准确度越低，镜头失效程度越高。
需要说明的是，输入图像可以为计算机或者人工模拟出来的多类质量不一的图像，或可以为真实的通过图像采集设备获取的质量不一的图像。
步骤S203，将镜头失效程度作为输入图像的失效程度标签，其中，所述前置模型的输入图像及其失效程度标签作为训练镜头失效检测模型样本数据。
确定输入图像的镜头失效程度后，即可将其作为输入图像的失效程度标签，该输入图像被用来训练镜头失效检测模型，以使得训练好的镜头失效检测模型能在应用过程中被用于确定图像采集设备的镜头失效程度。
通过上述步骤，基于前置模型的模型质量评估结果来确定输入图像的失效程度标签，实现了以数值参数定义镜头失效程度，不再需要通过人工进行标注，从而解决了相关技术中对失效程度的判断比较主观，无法为后续的图像处理过程提供客观依据的问题，将镜头失效程度客观化，使得镜头失效程度的评判有了客观标准，也有利于后续对图像进行针对性的处理。
一些示例性实施例中，确定输入图像的镜头失效程度包括：根据多个模型质量评估结果之间的数值关系，确定每一个输入图像的镜头失效程度，其中，该数值关系可以为大小关系，或可以为比例关系，或可以为更加复杂的对数关系。一些示例性实施例中，在训练测试或评估前置模型的过程中，可以向前置模型依次输入不同图像质量的输入图像数据集，以得到不同的模型质量评估结果。此时，前置模型的所有输入图像可以被划分为多个图像数据集，多个图像数据集分别对应于多个图像质量级别，多个图像质量等级分别对应于多个模型质量评估结果，一些示例性实施例中，“图像数据集-图像质量级别-模型质量评估结果”一一对应。
一些示例性实施例中，可以先根据多个图像数据集中的一个图像数据集，定义评估参考值，其他图像数据集的模型质量评估结果均与评估参考值进行对比，从而得到镜头失效程度，或可以根据多个图像数据集的多个模型质量评估结果之间的相对关系确定镜头失效程度。
一些示例性实施例中，在图像质量级别之间差距过大的情况下，或可以通过比例关系或者对数关系对模型质量评估结果进行处理，以得到更加合适的等级划分。
一些示例性实施例中，通过多个模型质量评估结果之间的数值关系确定每一个输入图像的镜头失效程度，可以更加准确地确定与输入图像对应的镜头失效程度。
一些示例性实施例中，在根据多个模型质量评估结果之间的数值关系确定每一个输入图像的镜头失效程度的过程中，需要每一个输入图像中的镜头失效类型为同一种，以提高 镜头失效程度划分的准确度，其中，镜头失效类型包括光线、清晰度、阴影、脏污、遮挡、水渍、花屏等。
也就是说，用于进行前置模型评估的输入图像，按镜头失效类型划分后，每个镜头失效类型的输入图像，再按图像质量等级划分为多个图像数据集，每一个图像数据集中的图像具有相同镜头失效类型和相同图像质量等级，分别采用每一个图像数据集进行前置模型的模型质量评估，确定所述镜头失效类型和所述图像质量等级所对应的模型质量评估结果。例如，全部图像数据，划分为N种镜头失效类型，M个图像质量等级，得到N*M个图像数据集，分别进行前置模型的效果评估后，确定对应于每个数据集的模型质量评估结果。一些示例性实施例中，这里划分的镜头失效类型用于确定进行对镜头失效检测模型进行训练的输入图像的失效类型标签。
需要说明的是，针对每一种镜头失效类型的图像，再划分的图像质量等级的数量可以相同，或可以不同，不限于特定的方面。
在其中一些示例性实施例中，提供了一种镜头失效程度的确定方法，如图3所示，包括如下步骤：
步骤S301，将前置模型以第一图像质量级别的图像数据集作为输入所得到的模型质量评估结果作为评估参考值；
步骤S302，确定前置模型以第二图像质量级别的图像数据集作为输入所得到的模型质量评估结果，记为待确认评估值；
步骤S303，根据评估参考值与待确认评估值之间的差异确定具有第二图像质量级别的输入图像的镜头失效程度。
可以理解，具有第一图像质量级别的输入图像数据集所包括的图像都具有相同的镜头失效程度；具有第二图像质量级别的输入图像数据集所包括的图像都具有相同的镜头失效程度。
一些示例性实施例中，可以根据预设的该差异与镜头失效程度的对应规则，确定具有第二图像质量级别的输入图像的镜头失效程度。例如，待确认评估值相对于评估参考值每下降一个预设值，镜头失效程度则上升一个对应的等级。
另一方面，或可以根据与评估参考值对应的第一镜头失效程度、评估参考值与待确认评估值之间的差异确定第二镜头失效程度，其中，第一镜头失效程度为预设的参考值，可以为设定范围内的最低值、最高值或者中间任一值，第二镜头失效程度为与待确认评估值对应的镜头失效程度。
一些示例性实施例中，根据评估参考值与待确认评估值之间的第一差异确定第一镜头失效程度与第二镜头失效程度之间的第二差异，根据第二差异以及第一镜头失效程度确定第二镜头失效程度。
需要说明的是，第一图像质量级别和第二图像质量级别的大小关系不做限定，第一图像质量级别可以为所有图像数据集中图像质量最低的，此时评估参考值最低，第一镜头失效程度最高，第二图像质量级别大于或者等于第一图像质量级别，可以根据待确认评估值比评估参考值高的程度，基于第一镜头失效程度确定第二镜头失效程度。
在其他实施例中，或可以任选一个图像数据集，将选定的图像数据集对应的图像质量作为第一图像质量级别，此时第二图像质量级别可能大于第一图像质量级别，也可能小于第一图像质量级别，同样的，根据待确认评估值与评估参考值之间差异的程度确定第二镜头失效程度。
可以理解，待确认评估值与评估参考值之间的第一差异越大，第二镜头失效程度和第一镜头失效程度之间的第二差异就越大。
在有多个图像数据集的情况下，选定一个图像数据集用于确定评估参考值，其他的多个图像数据集对应的模型质量评估结果均为待确认评估值，需要依次与评估参考值进行对比后，再确认相应的第二镜头失效程度。此时，其他的多个图像数据集可以对应不同的第二图像质量级别。
一些示例性实施例中，第一图像质量级别的图像质量在所有的图像数据集中是最优的，可以将此时的第一镜头失效程度设置为预设值，例如0，对于其他图像质量较低的第二图像质量级别的图像数据集，可以根据待确认评估值与评估参考值之间的差异，确认各自的第二镜头失效程度，例如，分别为1，2，3，……，9等等。
一些示例性实施例中，将一个模型质量评估结果作为评估参考值，其他的模型质量评估结果均需要与该评估参考值进行对比之后，才能确定相应的第二镜头失效程度，基于评估参考值的确定，本实施例可以更加准确地确定多个镜头失效程度之间实际差异，也更加客观。
一些示例性实施例中，第一图像质量级别对应的图像质量大于或者等于预设的图像质量阈值。本实施例中，需要第一图像质量级别对应的图像数据集具有足够高的图像质量，例如，图像质量阈值可以为与光线、清晰度、阴影、脏污、遮挡、水渍、花屏等图像特征相关的一个或多个参数对应的参数值。
一些示例性实施例中，第一图像质量级别对应的图像数据集为清晰度大于或者等于预设的清晰度阈值、污染程度小于或者等于预设污染阈值的数据集，此时确认的评估参考值也是最优的。一些示例性实施例中，对第一图像质量级别进行限制，可以更加准确地反映真实的镜头失效程度。
在其中一些实施例中，根据多个模型质量评估结果之间的相对大小关系，确定与图像质量级别对应的输入图像的镜头失效程度。本实施例中，可以在得到多个图像数据集所有的模型质量评估结果之后，对多个模型质量评估结果进行对比，根据对比结果的顺序依次确定每一个镜头失效程度。
例如，输入图像被划分为三个图像数据集A、B、C，三个图像数据集中，输入图像被遮挡的比例不同，依次为低、中、高，那么三个图像数据集的图像质量级别依次为高、中、低，将图像数据集A、B、C分别输入前置模型进行评估，对应得到三个不同的模型质量评估结果a、b、c。最后，对比多个模型质量评估结果之间的大小关系，假设模型质量评估结果表征前置模型的识别准确度，则有a>b>c，此时，可以将图像数据集A中所有的输入图像的镜头失效程度定义为“低”，图像数据集B中所有的输入图像的镜头失效程度定义为“中”，图像数据集C中所有的输入图像的镜头失效程度定义为“高”。
一些示例性实施例中，根据多个模型质量评估结果之间的相对大小关系确定镜头失效程度，可以避免不同图像数据集的图像质量之间差异过大，镜头失效程度超过预设的范围，提高样本标签的获取方法的场景适应性。
一些示例性实施例中，对模型进行质量评估的参数有很多，除了准确率之外，或可以通过查全率或查准率对模型的质量进行评估。
一些实施例中，所述确定前置模型的模型质量评估结果包括：
根据所述前置模型的查全率或查准率确定所述模型质量评估结果；
或者，
根据所述前置模型的查全率和查准率共同确定所述模型质量评估结果。
例如，可以根据不同图像数据集之间查准率的变化程度或查全率的变化程度，确定相应的镜头失效程度；或者，根据不同图像数据集之间查准率的变化程度和查全率的变化程度，确定相应的镜头失效程度。
其中，查全率是指真正的正样本中模型判定为正样本的比例，计算方式如公式1所示：
在公式1中，R表示查全率，TP表示真正样本，FN表示假负样本。
查准率是指对于模型判定的所有正样本中真正的正样本所占的比例，计算方式如公式2所示：
在公式2中，P表示查准率，FP表示假正样本。
一些示例性实施例中，在对前置模型进行评价的过程中，可以单独使用查全率或者查准率，或可以根据二者加权计算的结果评价前置模型。
本实施例中，通过查全率或查准率，或者，查全率和查准率评价前置模型，可以提高模型质量评估结果的准确性，确保镜头失效程度的客观性。
一些示例性实施例中，在确定前置模型的查全率和查准率之后，根据前置模型的查全率和查准率确定查全率-查准率曲线，即PR曲线，然后根据查全率-查准率曲线的数值特征确定模型质量评估结果，其中，数值特征可以为PR曲线上的值，或可以为根据PR曲线计算得到的特征参数。
一些示例性实施例中，以查全率为横轴，查准率为纵轴作图得到PR曲线，在得到PR曲线之后，可以在一个确定的查全率下，查找对应的查准率作为模型质量评估结果，或可以在一个确定的查准率下，查找对应的查全率作为模型质量评估结果。
一些示例性实施例中，查全率-查准率曲线的数值特征为查全率-查准率曲线与坐标轴之间的面积，记为评估面积，也称为AP(Average Precision)值，由于评估面积可以兼顾查全率和查准率，所以对前置模型的评价也更为全面。
在其他实施例中，或可以使用mAP(mean Average Precision)、接受者操作特征(Receiver Operating Characteristic，简称为ROC)曲线，或者ROC曲线与坐标轴之间的面积来评价前置模型。其中，ROC曲线以FPR(False positive rate，假正例率)为横轴，TPR(True positive rate，真正例率)为纵轴得到，其中，TPR为查全率，FPR的计算方式如公式3所示：
在公式3中，TN表示真负样本，FP表示假正样本。
在以AP值作为模型质量评估结果的情况下，图4是根据本申请实施例的另一种镜头失效程度的确定方法的流程图，如图4所示，该方法包括如下步骤：
步骤S401，将前置模型以第一图像质量级别的图像数据集作为输入所得到的评估面积作为评估参考值；其中，第一图像质量级别对应的图像数据集为多个图像数据集中图像质量最优的；
步骤S402，确定前置模型以第二图像质量级别的图像数据集作为输入所得到的评估 面积，记为待确认评估值；
步骤S403，根据评估参考值与待确认评估值之间的差异确定具有第二图像质量级别的输入图像的镜头失效程度。
本实施例中通过PR曲线的AP值评价前置模型，可以更加综合、全面的反应输入图像的图像质量，也使得镜头失效程度更加全面准确。
一些示例性实施例中，在确定前置模型的多个AP值之后，根据多个AP值之间的相对大小关系，确定对应的输入图像的镜头失效程度，其中，多个AP值分别对应于多个图像质量级别。
本公开实施例所提供的方案中，根据前置模型在不同图像质量的图像数据集下，所得出的模型质量评估结果的不同，反向标定对应输入图像的镜头失效程度标签。可以看到，根据前置模型所确定模型质量评估结果，可以作为确定其镜头失效程度的依据。因此，所述前置模型被理解为辅助确定待训练镜头失效检测模型的输入图像的失效程度标签。
在其中一些实施例中，在前置模型为目标检测模型的情况下，先获取输入图像中检测目标的交并比。其中，检测目标可以为自动驾驶过程中，自车周围的其他车辆，或者是公共场所中的行人等等。交并比(Intersection over Union，简称为IoU)为计算针对检测目标的两个边界框的交集和并集之比，其中一个边界框为计算出来的预测框，另一个边界框为实际的标注框。
在交并比大于或者等于预设的交并比阈值的情况下，再获取前置模型的模型质量评估结果，进行镜头失效程度的计算。交并比通常是用于评价目标检测模型对检测目标的定位是否精准，交并比越高，说明目标检测模型的定位越精确，训练效果越好，因此，在确定模型质量评估结果之前设定交并比阈值，筛除定位不佳的输入图像，可以一定程度上排除图像质量之外的干扰因素，让后续的镜头失效程度更能真实地反映输入图像的图像质量，提高样本标签的准确度。一般来说，交并比阈值可以根据需求设置，例如设定交并比阈值为0.5。
基于上述实施例中的样本标签的获取方法，本实施例中还提供了一种镜头失效检测模型的训练方法，如图5所示，包括如下步骤：
步骤S501，获取待训练的镜头失效检测模型的输入图像以及输入图像的失效程度标签；
其中，失效程度标签可以通过上述任一实施例中样本标签的获取方法得到，例如，确定前置模型的模型质量评估结果；根据模型质量评估结果确定前置模型的输入图像的镜头失效程度；将镜头失效程度作为输入图像的失效程度标签，本实施例中的前置模型用于辅助确定待训练镜头失效检测模型的输入图像的失效程度标签，前置模型的输入图像作为待训练的镜头失效检测模型的输入图像来训练镜头失效检测模型；
步骤S502，将输入图像作为待训练的镜头失效检测模型的输入数据，将失效程度标签作为镜头失效检测模型的目标数据，对镜头失效检测模型进行训练，其中，镜头失效检测模型至少用于确定待预测图像的镜头失效程度。
其中，目标数据指的是希望模型能够预测或分类的真实或期望输出，是用于监督学习任务的一组已知标签或答案。
通过上述步骤S501和步骤S502，提供了一种镜头失效检测模型的训练方法，由于输入图像的失效程度标签通过前置模型的模型质量评估结果确定，所以失效程度标签更加客观和准确，基于此，不仅解决了相关技术中对失效程度的判断比较主观，无法为后续的图 像处理过程提供客观依据的问题，还可以提高镜头失效检测模型的识别准确度。
一些示例性实施例中，在确定镜头失效程度的过程中，前置模型的输入图像被划分为多个图像数据集，多个图像数据集分别对应于多个图像质量级别，多个图像质量级别分别对应于多个模型质量评估结果，此时，可以根据多个模型质量评估结果之间的数值关系，确定每一个输入图像的镜头失效程度。例如，将前置模型以第一图像质量级别的图像数据集作为输入时所得到的模型质量评估结果作为评估参考值；确定前置模型以第二图像质量级别的图像数据集作为输入时所得到的模型质量评估结果，记为待确认评估值；根据所述评估参考值与所述待确认评估值之间的差异确定具有第二图像质量级别的输入图像的镜头失效程度。
一些示例性实施例中，将前置模型的AP值作为模型质量评估结果。
在其中一些实施例中，镜头失效检测模型检测镜头失效类型和镜头失效程度，此时在对所述镜头失效检测模型进行训练之前，还需要获取输入图像的镜头失效类型作为失效类型标签；因此，对镜头失效检测模型的训练包括：将输入图像作为待训练的镜头失效检测模型的输入数据，将失效程度标签和失效类型标签作为镜头失效检测模型的目标数据，对镜头失效检测模型进行训练。其中，失效类型标签包括正常、脏污与模糊、遮挡、水渍、光线、花屏等等。
一些示例性实施例中，所述失效类型标签根据所述输入图像归属的图像数据集对应的镜头失效类型确定；或者，人工标注。
一些示例性实施例中镜头失效检测模型可以为多任务分类模型。在正式训练前，需要先获取大规模的输入图像，将所有的输入图像按照镜头失效类型和镜头失效程度进行划分。按照相同的预处理方式，将输入图像裁剪到预设的尺寸。然后将处理好的输入图像输入到卷积神经网络进行多任务分类模型的训练，本实施例中的多任务分类模型，通过一个模型完成分类和回归两个任务，一些示例性实施例中，主干网络下连接分类和回归两个分支网络。卷积神经网络包括数据层、卷积层、池化层、激活层、全连接层和输出层。
一些示例性实施例中，训练方式为梯度下降法和反向传播算法，过程如下：
步骤1：将处理好的输入图像、失效程度标签和失效类型标签通过数据层输入到训练网络中；
步骤2：卷积层通过设置好的步长、卷积核尺寸、卷积核个数来提取数据特征，得到特征图；池化层通过设置好的步长、池化尺寸对前一层特征图进行下采样；激活层对下采样后的特征图进行非线性变化，其中，激活函数为relu激活函数；
步骤3：全连接层连接所有的特征图，通过权重将特征空间通过线性变换映射到标记空间，全连接后接relu激活函数；
步骤4：输出层是对特征图进行分类和回归，本实施例采用softmax函数作为分类的输出层函数，采用euclidean函数作为失效程度回归的输出层函数。
在训练阶段，所有的输入图像都将输入至卷积神经网络，并且通过损失函数计算输出的预测值与实际样本标签之间的差距。这个过程被称为“正向传递”。然后，根据预测值与样本标签的差异，确定卷积神经网络模型参数的误差度，对模型参数进行更新，从而进行卷积神经网络学习，这个过程被称为“反向传递”。通过调整卷积神经网络中每层的权重值，使得卷积神经网络的预测值与样本标签值之间的差距越来越小，直到卷积神经网络的预测值与样本标签值一致或保持最小差距不再变化，则最终得到所需要的卷积神经网络。
对于多任务分类模型，由于具有多个属性值，可以先求出每个属性的损失函数，再将 多个属性损失函数加权求和，得到整体的损失函数，从而考虑到了每一种属性，训练的目标是使得所有属性的整体误差最低。上述方式支持多种不同属性的组合，例如，镜头失效类型的损失函数为L_invalid_class，镜头失效程度的损失函数为L_invalid_degree，那么多任务分类模型的代价函数可以为L_All＝a×L_invalid_class+b×L_invalid_degree。其中，a和b是为了模型达到最优的效果在训练过程中可以设置的参数。通常a和b的取值范围在0～1之间。
一些示例性实施例中，由于获取了输入图像的失效类型标签和失效程度标签对镜头失效检测模型进行训练，所以训练好的模型可以针对明确的镜头失效类型给出更加准确的镜头失效程度。
一些示例性实施例中，基于上述训练好的镜头失效检测模型，给出了一种镜头失效的检测方法，如图6所示，该方法包括如下步骤：
步骤S601，获取待预测图像，通过训练好的镜头失效检测模型确定待预测图像的镜头失效程度；
其中，镜头失效检测模型通过上述任一种镜头失效检测模型的训练方法得到；待预测图像为在实际场景中采集的图像，待预测图像中可以包括行人、车辆、人脸等任何目标，或可以为不包括特定目标的场景图；
步骤S602，将待预测图像的镜头失效程度作为图像采集设备的镜头失效程度，其中，待预测图像通过图像采集设备得到，图像采集设备可以为移动终端，例如手机、平板、AR(Augmented Reality，增强现实)设备、VR(Virtual Reality，虚拟现实)设备、XR(Extended Reality，扩展现实)设备、MR(Mixed Reality，混合现实)设备等等，或可以为摄像头、相机等专业设备。
通过上述步骤S601和步骤S602，基于训练好的镜头失效检测模型，可以依据图像采集设备获取的待预测图像判断自身的镜头失效程度，为后续对待预测图像进行处理提供了依据，由于镜头失效检测模型在训练的过程中，输入图像的标签基于前置模型的模型质量评估结果确定，因此图像采集设备的镜头失效程度也是客观的，解决了相关技术中对镜头失效程度的判断比较主观的问题。
一些示例性实施例中，在所述获取待预测图像之后，所述镜头失效的检测方法还包括：
通过所述训练好的镜头失效检测模型获取所述待预测图像的镜头失效类型；
根据获取结果至少执行以下步骤之一：
输出所述获取结果；
根据所述获取结果输出镜头调整策略；
其中，所述获取结果包括以下一项或多项：
所述镜头失效程度和所述镜头失效类型。
一些示例性实施例中，可以基于待预测图像确定镜头失效程度，并确定镜头失效类型，以获得更多的镜头信息。一些示例性实施例中，在获取待预测图像之后，通过训练好的镜头失效检测模型获取待预测图像的镜头失效类型，此时，镜头失效检测模型在训练的过程中，将输入图像作为待训练的镜头失效检测模型的输入数据，将输入图像的失效程度标签和失效类型标签作为镜头失效检测模型的目标数据，对镜头失效检测模型进行训练。
在其中一些实施例中，在待预测图像具有多个镜头失效类型的情况下，根据多个镜头失效类型对待预测图像的影响程度，确定与待预测图像对应的最终镜头失效类型，其中， 多个镜头失效类型对待预测图像的影响程度可以根据每一个镜头失效类型对应的图像区域在整个待预测图像中的面积占比确定，占比越大，影响越大。在向用户输出镜头失效类型时，可以仅输出最终镜头失效类型，或可以将所有的镜头失效类型输出。一些示例性实施例中，按照镜头失效类型的影响程度按序输出，并输出每一个镜头失效类型的面积占比。例如，镜头存在大面积遮挡，又有少量水渍附着，则划分为遮挡因素导致的失效。
在其中一些实施例中，在镜头失效检测模型可以获得待预测图像的镜头失效类型和镜头失效程度的情况下，可以向用户输出镜头失效类型和镜头失效程度，便于用户做出调整；一些示例性实施例中，还包括：根据镜头失效类型或镜头失效程度输出镜头调整策略，或者失效类型和镜头失效程度输出镜头调整策略。用户可以根据以下信息中的至少一项：镜头失效类型、镜头失效程度和镜头调整策略对镜头进行相应的调试，使得镜头正常工作；在其他实施例中，或可以将信息中的至少一项：镜头失效类型、镜头失效程度和镜头调整策略送到镜头的控制系统中，由控制系统对镜头做出自动的调整策略。例如，在水渍导致的镜头失效的情况下，对镜头进行擦拭吹干即可；在脏污导致的镜头失效的情况下，则需要进行清洗擦拭吹干；在遮挡导致的失效的情况下，则需要查看镜头是否有遮挡物；在光线导致的失效的情况下，则可提示在当前的光线下，镜头成像能否适用于视觉算法；在花屏导致的失效的情况下，则需要排查线路故障或者摄像头是否有问题。一些示例性实施例中，还包括：根据镜头失效程度判断采集到的图像能否用于后续的视觉算法，提高基于视觉算法的解决方案的准确性和稳定性，降低人工成本。
以下以辅助驾驶技术的应用场景为例，给出一个实施例。
由于基于视觉的高级驾驶辅助系统(Advanced Driving Assistance System，简称为ADAS)在车载领域尤为重要，并且在实际的驾驶安全中也起到了举足轻重的作用，例如，脏污、遮挡、模糊、花屏、水渍等对ADAS镜头的成像质量会造成影响，导致基于视觉的相关算法的准确性会严重下降，甚至会镜头失效无法进行正常工作，因此对于镜头失效的场景进行全面的分析，并且给出相应的解决方案显得尤为的关键。基于此，本公开实施例提供一种基于视觉的ADAS镜头失效的检测方法，对于ADAS镜头的每一类失效场景进行有效区分，提升视觉算法的稳健性，提升基于视觉的ADAS的安全性。该方法包括如下步骤：
步骤1、样本采集和处理
通过实际采集得到视频段，基于视频段获取样本图像，一些示例性实施例中，样本图像中包括需要进行目标识别的车辆。
一些示例性实施例中，镜头失效类型包括脏污与模糊、遮挡、水渍、光线和花屏等等。镜头失效类型的定义如下：1)脏污与模糊，脏污是指非透明的粘稠物体在镜头上面，例如泥巴、泥浆等，模糊是指类似于散状灰尘附着在镜头上面，镜头可以看到目标的整体轮廓，但是细节模糊不清；2)遮挡，是指成片区域被非透明物体遮挡的情况；3)花屏，是指由于摄像头线路故障、或者镜头坏损导致成像出现花屏的情况；4)水渍，是指透明液体附着在镜头表面；5)光线，是指成像的明暗度。
对于脏污与模糊、水渍等镜头失效类型，可以先在亚克力板上进行脏污、模糊或者水渍的处理，再将处理后的亚克力板置于镜头前，对处理后的亚克力板拍摄视频从而得到不同失效类型的样本图像；对于遮挡和花屏，可以先直接获取清晰的样本图像，再通过计算机模拟遮挡或者花屏，例如，通过将部分区域的像素赋值为0模拟遮挡，通过模拟像素错位或者将像素马赛克模拟花屏；对于光线，可以在通过镜头采集样本图像的过程中调整光照强度得到与光线相关的样本图像。在上述模拟过程中，可以通过调整脏污、水渍、模糊、 像素占整个样本图像的比例以及光线的强弱变化，模拟不同镜头失效程度下的图像。在模拟结束之后，得到前置模型的输入图像。
步骤2、获取样本标签
样本标签包括失效类型标签和失效程度标签，镜头的失效类型标签可以在模拟后通过人工进行标注，镜头的失效程度标签通过以下步骤得到。
一些示例性实施例中，前置模型为用于进行车辆检测的目标检测模型。通过目标检测模型，对每一类镜头失效类型的每一个镜头失效程度对应的输入图像依次进行车辆检测，例如，对于遮挡，分别根据包括1000张没有遮挡的输入图像进行目标检测模型的质量评估，根据1000张遮挡了少部分的输入图像进行目标检测模型的质量评估，根据1000张遮挡了大部分的输入图像进行目标检测模型的质量评估。然后在IoU大于或者等于0.5的情况下，计算每次目标检测模型的AP值。
在训练时目标检测模型的输入图像在全部图像数据集中的图像质量是最优的情况下，例如，图像清晰无污染、光照足够等等，将此时得到的目标检测模型的AP值作为评估参考值。将其他图像数据集中的图像依次作为输入图像进行目标检测模型的质量评估。相对于评估参考值而言，在目标检测模型的AP值分别下降2％，4％，6％,8％，10％，12％，14％，16％，18％，20％时，将失效程度认定为0，1，2，3，4，5，6，7，8，9。
需要说明的是，下降程度可以按照区间来划分，在落在区间中的情况下，则可以划分为区间的右边值。比如下降了1％，落在0-2％之间，那么就失效程度划分为1，以次类推。
至此，每张输入图像都会拥有自己的失效类型标签和失效程度标签，用于后续的镜头失效检测模型训练。
步骤3、训练镜头失效检测模型
本实施例中的镜头失效检测模型为用于进行镜头失效类型和镜头失效程度判断的多任务分类模型，包括分类和回归两个分支网络。
在训练时，将前置模型的输入图像作为多任务分类模型的输入图像，该输入图像包括失效类型标签和失效程度标签，在经过多任务分类模型的数据层、卷积层、全连接层之后，在多任务分类模型的输出层对输入图像的特征图分别进行分类和回归，以确定输入图像在镜头失效类型和镜头失效程度的预测值作为输出结果。
本实施例中，由于分类和回归任务具有相关性，因此可以共用部分网络以简化多任务分类模型，分类和回归层有各自的全连接层和最终的决策输出层，可以提取对分类和回归各自有用的特征。
根据镜头失效类型和镜头失效程度的预测值与输入图像的样本标签的差异，调整多任务分类模型中每层网络的权重值，直到多任务分类模型的预测值与样本标签的实际值一致或保持最小差距不再变化，或者训练的迭代轮次达到预设值，结束训练，最终得到所需要的镜头失效检测模型。
4、实际应用过程
在车辆行驶过程中，ADAS镜头获取实时的待预测图像，将待预测图像输入训练好的镜头失效检测模型，即可得到ADAS镜头的镜头失效类型和镜头失效程度，ADAS可以根据镜头失效类型和镜头失效程度给出相应的镜头调整策略。
本实施例中，基于多任务分类模型，采用分类和回归相结合的方式，对失效的场景进行分类，对失效场景的程度进行回归，统一在一个模型中有效的对镜头失效类型和镜头失 效程度进行划分。并且，基于目标检测模型的AP值来确定输入图像的失效程度标签，实现了通过数值参数定义镜头失效程度，从而解决了相关技术中对失效程度的判断比较主观，无法为后续的图像处理过程提供客观依据的问题，将镜头失效程度客观化，使得镜头失效程度的评判有了客观标准。
在其他实施例中，在样本标签的获取方法应用至人脸算法的情况下，例如人脸检测，则在获取模型质量评估结果之前，还需要将并非由镜头本身失效导致人脸无法检测到的情况加以区分，例如大角度，分辨率极小等情况，以提高镜头失效检测模型的准确度。
需要说明的是，在上述流程中或者附图的流程图中示出的步骤可以在诸如一组计算机可执行指令的计算机系统中执行，并且，虽然在流程图中示出了逻辑顺序，但是在某些情况下，可以以不同于此处的顺序执行所示出或描述的步骤。
在本实施例中还提供了一种样本标签的获取方法，包括：
确定前置模型的模型质量评估结果，其中，所述前置模型用于辅助确定待训练镜头失效检测模型的输入图像的失效程度标签；
根据所述模型质量评估结果确定所述前置模型的输入图像的镜头失效程度；
将所述镜头失效程度作为所述输入图像的失效程度标签，其中，在训练所述镜头失效检测模型时，将所述前置模型的输入图像作为所述镜头失效检测模型的输入图像。
在本实施例中还提供了一种样本标签的获取装置，该装置用于实现本公开任一所述实施例，已经进行过说明的不再赘述。以下所使用的术语“模块”、“单元”、“子单元”等可以实现预定功能的软件和/或硬件的组合。尽管在以下实施例中所描述的装置较佳地以软件来实现，但是硬件，或者软件和硬件的组合的实现也是可能并被构想的。
图7是本申请实施例的样本标签的获取装置的结构框图，如图7所示，该装置包括质量评估模块71、失效确定模块72和标签确定模块73：
质量评估模块71，设置为确定前置模型的模型质量评估结果；
失效确定模块72，设置为根据所述模型质量评估结果确定所述前置模型的输入图像的镜头失效程度；
标签确定模块73，设置为将所述镜头失效程度作为所述输入图像的失效程度标签，其中，所述前置模型的输入图像被设置为作为所述镜头失效检测模型的输入图像对镜头实现模型进行训练。
本实施例中的样本标签的获取装置，通过质量评估模块71确定前置模型的模型质量评估结果，最后通过标签确定模块73来确定输入图像的失效程度标签，实现了通过数值参数定义镜头失效程度，不再需要通过人工进行标注，从而解决了相关技术中对失效程度的判断比较主观，无法为后续的图像处理过程提供客观依据的问题，将镜头失效程度客观化，使得镜头失效程度的评判有了客观标准，也有利于后续对图像进行针对性的处理。
一些示例性实施例中，失效确定模块72还设置为根据多个模型质量评估结果之间的数值关系，确定每一个输入图像的镜头失效程度，其中，前置模型的所有输入图像被划分为多个图像数据集，多个图像数据集分别对应于多个图像质量级别，多个图像质量级别分别对应于多个模型质量评估结果。
一些示例性实施例中，失效确定模块72将前置模型以第一图像质量级别的图像作为输入图像时得到的模型质量评估结果作为评估参考值；确定前置模型以第二图像质量级别的图像作为输入图像时得到的模型质量评估结果，记为待确认评估值；根据所述评估参考 值与所述待确认评估值之间的差异确定具有所述第二图像质量级别的输入图像的镜头失效程度。其中，第一图像质量级别对应的图像质量大于或者等于预设的图像质量阈值。
在其中一些实施例中，质量评估模块71还设置为根据前置模型的查全率或查准率确定模型质量评估结果；或者，根据前置模型的查全率和查准率共同确定模型质量评估结果。一些示例性实施例中，根据前置模型的查全率和查准率确定查全率-查准率曲线；根据查全率-查准率曲线的数值特征确定模型质量评估结果。一些示例性实施例中，查全率-查准率曲线的数值特征为查全率-查准率曲线与坐标轴之间的面积，记为评估面积。
需要说明的是，上述每一个模块可以是功能模块或可以是程序模块，既可以通过软件来实现，或可以通过硬件来实现。对于通过硬件来实现的模块而言，上述全部个模块可以位于同一处理器中；或者上述模块或可以按照任意组合的形式分别位于不同的处理器中。
本申请中还提供了一种车载控制系统，包括存储器和处理器，存储器存储有程序，程序在被处理器读取执行时，实现上述实施例中的任一方法。
本申请中还提供了一种车辆，该车辆包括图像采集设备和图像处理器，图像采集设备设置为采集待预测图像；图像处理器，设置为实现上述实施例中的任一方法。
在本实施例中还提供了一种电子装置，包括存储器和处理器，该存储器中存储有计算机程序，该处理器被设置为运行计算机程序以执行上述任一项方法实施例中的步骤。
一些示例性实施例中，上述电子装置还包括传输设备以及输入输出设备，其中，该传输设备和上述处理器连接，该输入输出设备和上述处理器连接。
一些示例性实施例中，上述处理器被设置为通过计算机程序执行以下步骤：
S1，确定前置模型的模型质量评估结果，其中，所述前置模型用于辅助确定待训练镜头失效检测模型的输入图像的失效程度标签。
S2，根据所述模型质量评估结果确定所述前置模型的输入图像的镜头失效程度。
S3，将所述镜头失效程度作为所述输入图像的失效程度标签，其中，在训练所述镜头失效检测模型时，将所述前置模型的输入图像作为所述镜头失效检测模型的输入图像。
需要说明的是，在本实施例中的细节示例可以参考上述实施例及可选实施方式中所描述的示例，在本实施例中不再赘述。
此外，结合上述实施例中提供的样本标签的获取方法，在本实施例中还提供一种计算机可读存储介质来实现。该存储介质上存储有一个或者多个程序，所述一个或者多个程序可被一个或者多个处理器执行；该程序被处理器执行时实现上述实施例中的任意一种方法。
应该明白的是，这里描述的实施例只是用来解释这个应用，而不是用来对它进行限定。根据本申请提供的实施例，本领域普通技术人员在不进行创造性劳动的情况下得到的所有其它实施例，均属本申请保护范围。
显然，附图只是本申请的一些例子或实施例，对本领域的普通技术人员来说，或可以根据这些附图将本申请适用于其他类似情况，但无需付出创造性劳动。另外，可以理解的是，尽管在此开发过程中所做的工作可能是复杂和漫长的，但是，对于本领域的普通技术人员来说，根据本申请披露的技术内容进行的某些设计、制造或生产等更改仅是常规的技术手段，不应被视为本申请公开的内容不足。
需要说明的是，本申请所涉及的用户信息(包括但不限于用户设备信息、用户个人信息等)和数据(包括但不限于用于分析的数据、存储的数据、展示的数据等)，均为经用户授权或者经过各方充分授权的信息和数据。
“实施例”一词在本申请中指的是结合实施例描述的具体特征、结构或特性可以包括在本申请的至少一个实施例中。该短语出现在说明书中的各个位置并不一定意味着相同的实施例，也不意味着与其它实施例相互排斥而具有独立性或可供选择。本领域的普通技术人员可以清楚或隐含地理解的是，本申请中描述的实施例在没有冲突的情况下，可以与其它实施例结合。
以上所述实施例仅表达了本申请的几种实施方式，其描述较为具体和详细，但并不能因此而理解为对专利保护范围的限制。应当指出的是，对于本领域的普通技术人员来说，在不脱离本申请构思的前提下，或可以做出若干变形和改进，这些都属于本申请的保护范围。因此，本申请的保护范围应以所附权利要求为准。

Claims (17)

  1. 一种样本标签的获取方法，包括：
    确定前置模型的模型质量评估结果；
    根据所述模型质量评估结果确定所述前置模型的输入图像的镜头失效程度；
    将所述镜头失效程度作为所述输入图像的失效程度标签，其中，所述前置模型的输入图像及其失效程度标签作为训练镜头失效检测模型的样本数据。
  2. 根据权利要求1所述的样本标签的获取方法，其中，所述根据所述模型质量评估结果确定所述前置模型的输入图像的镜头失效程度包括：
    根据多个所述模型质量评估结果之间的数值关系，确定所述前置模型的每一个输入图像的镜头失效程度，其中，所述前置模型的所有输入图像被划分为多个图像数据集，多个图像数据集分别对应于多个图像质量级别，多个图像质量级别分别对应于多个所述模型质量评估结果。
  3. 根据权利要求2所述的样本标签的获取方法，其中，所述根据多个所述模型质量评估结果之间的数值关系，确定所述前置模型的每一个输入图像的镜头失效程度包括：
    将所述前置模型以第一图像质量级别的图像数据集作为输入所得到的模型质量评估结果作为评估参考值；
    确定所述前置模型以第二图像质量级别的图像数据集作为输入所得到的模型质量评估结果，记为待确认评估值；
    根据所述评估参考值与所述待确认评估值之间的差异确定具有第二图像质量级别的输入图像的镜头失效程度。
  4. 根据权利要求3所述的样本标签的获取方法，其中，所述第一图像质量级别对应的图像质量大于或者等于预设的图像质量阈值。
  5. 根据权利要求2所述的样本标签的获取方法，其中，所述根据多个所述模型质量评估结果之间的数值关系，确定所述前置模型的每一个输入图像的镜头失效程度包括：
    根据多个所述模型质量评估结果之间的相对大小关系，确定与所述图像质量级别对应的输入图像的镜头失效程度。
  6. 根据权利要求1所述的样本标签的获取方法，其中，所述确定前置模型的模型质量评估结果包括：
    根据所述前置模型的查全率或查准率确定所述模型质量评估结果；
    或者，
    根据所述前置模型的查全率和查准率共同确定所述模型质量评估结果。
  7. 根据权利要求6所述的样本标签的获取方法，其中，所述根据所述前置模型的查全率和查准率共同确定所述模型质量评估结果包括：
    根据所述前置模型的查全率和查准率确定查全率-查准率曲线；
    根据所述查全率-查准率曲线的数值特征确定所述模型质量评估结果。
  8. 根据权利要求7所述的样本标签的获取方法，其中，所述查全率-查准率曲线的数值特征为所述查全率-查准率曲线与坐标轴之间的面积，记为评估面积。
  9. 根据权利要求1至8中任一项所述的样本标签的获取方法，其中，在所述确定前置模型的模型质量评估结果之前，所述方法还包括：
    在所述前置模型为目标检测模型的情况下，获取所述输入图像中检测目标的交并比；
    在所述交并比大于或者等于预设的交并比阈值的情况下，获取所述前置模型的模型质量评估结果。
  10. 一种镜头失效检测模型的训练方法，包括：
    获取待训练的镜头失效检测模型的输入图像以及所述输入图像的失效程度标签，其中，所述失效程度标签根据权利要求1至9中任一项所述的样本标签的获取方法得到；
    将所述输入图像作为所述待训练的镜头失效检测模型的输入数据，将所述失效程度标签作为所述镜头失效检测模型的目标数据，对所述镜头失效检测模型进行训练，其中，所述镜头失效检测模型至少用于确定待预测图像的镜头失效程度。
  11. 根据权利要求10所述的镜头失效检测模型的训练方法，其中，在对所述镜头失效检测模型进行训练之前，还包括：获取所述输入图像的镜头失效类型作为失效类型标签；
    所述对所述镜头失效检测模型进行训练包括：
    将输入图像作为待训练的镜头失效检测模型的输入数据，将所述失效程度标签和所述失效类型标签作为所述镜头失效检测模型的目标数据，对所述镜头失效检测模型进行训练。
  12. 一种镜头失效的检测方法，包括：
    获取待预测图像，通过训练好的镜头失效检测模型确定所述待预测图像的镜头失效程度，其中，所述训练好的镜头失效检测模型通过权利要求10至11中任一项所述的镜头失效检测模型的训练方法得到；
    将所述待预测图像的镜头失效程度作为图像采集设备的镜头失效程度，其中，所述待预测图像通过所述图像采集设备得到。
  13. 根据权利要求12所述的镜头失效的检测方法，其中，在所述获取待预测图像之后，所述方法还包括：
    通过所述训练好的镜头失效检测模型获取所述待预测图像的镜头失效类型；
    根据获取结果至少执行以下步骤之一：
    输出所述获取结果；
    根据所述获取结果输出镜头调整策略；
    其中，所述获取结果包括以下一项或多项：
    所述镜头失效程度和所述镜头失效类型。
  14. 根据权利要求13所述的镜头失效的检测方法，其中，所述通过所述训练好的镜头失效检测模型获取所述待预测图像的镜头失效类型包括：
    在所述待预测图像具有多个所述镜头失效类型的情况下，根据多个所述镜头失效类型对所述待预测图像的影响程度，确定与所述待预测图像对应的最终镜头失效类型。
  15. 一种车载控制系统，包括存储器和处理器，所述存储器存储有程序，所述程序在被所述处理器读取执行时，实现如权利要求1至14中任一项所述的方法。
  16. 一种车辆，包括：
    图像采集设备，设置为采集待预测图像；
    图像处理器，设置为实现如权利要求1至14中任一项所述的方法。
  17. 一种计算机可读存储介质，所述计算机可读存储介质存储有一个或者多个程序，所述一个或者多个程序可被一个或者多个处理器执行，以实现如权利要求1至14中任一项所述的方法。
PCT/CN2023/119612 2022-09-19 2023-09-19 样本标签的获取方法和镜头失效检测模型的训练方法 WO2024061194A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202211139892.4A CN115620079A (zh) 2022-09-19 2022-09-19 样本标签的获取方法和镜头失效检测模型的训练方法
CN202211139892.4 2022-09-19

Publications (1)

Publication Number Publication Date
WO2024061194A1 true WO2024061194A1 (zh) 2024-03-28

Family

ID=84857788

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2023/119612 WO2024061194A1 (zh) 2022-09-19 2023-09-19 样本标签的获取方法和镜头失效检测模型的训练方法

Country Status (2)

Country Link
CN (1) CN115620079A (zh)
WO (1) WO2024061194A1 (zh)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115620079A (zh) * 2022-09-19 2023-01-17 虹软科技股份有限公司 样本标签的获取方法和镜头失效检测模型的训练方法

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111753581A (zh) * 2019-03-27 2020-10-09 虹软科技股份有限公司 目标检测方法和装置
CN111950329A (zh) * 2019-05-16 2020-11-17 长沙智能驾驶研究院有限公司 目标检测及模型训练方法、装置、计算机设备和存储介质
CN112700408A (zh) * 2020-12-28 2021-04-23 中国银联股份有限公司 模型训练方法、图像质量评估方法及装置
CN113554597A (zh) * 2021-06-23 2021-10-26 清华大学 一种基于脑电特征的图像质量评价方法及装置
US20220245792A1 (en) * 2019-10-11 2022-08-04 Beijing Didi Infinity Technology And Development Co., Ltd. Systems and methods for image quality detection
CN115620079A (zh) * 2022-09-19 2023-01-17 虹软科技股份有限公司 样本标签的获取方法和镜头失效检测模型的训练方法

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111753581A (zh) * 2019-03-27 2020-10-09 虹软科技股份有限公司 目标检测方法和装置
CN111950329A (zh) * 2019-05-16 2020-11-17 长沙智能驾驶研究院有限公司 目标检测及模型训练方法、装置、计算机设备和存储介质
US20220245792A1 (en) * 2019-10-11 2022-08-04 Beijing Didi Infinity Technology And Development Co., Ltd. Systems and methods for image quality detection
CN112700408A (zh) * 2020-12-28 2021-04-23 中国银联股份有限公司 模型训练方法、图像质量评估方法及装置
CN113554597A (zh) * 2021-06-23 2021-10-26 清华大学 一种基于脑电特征的图像质量评价方法及装置
CN115620079A (zh) * 2022-09-19 2023-01-17 虹软科技股份有限公司 样本标签的获取方法和镜头失效检测模型的训练方法

Also Published As

Publication number Publication date
CN115620079A (zh) 2023-01-17

Similar Documents

Publication Publication Date Title
CN111553929B (zh) 基于融合网络的手机屏幕缺陷分割方法、装置及设备
CN112330601B (zh) 一种基于鱼眼相机的停车检测方法、装置、设备及介质
CN108229277B (zh) 手势识别、手势控制及多层神经网络训练方法、装置及电子设备
WO2021164234A1 (zh) 图像处理方法以及图像处理装置
WO2024061194A1 (zh) 样本标签的获取方法和镜头失效检测模型的训练方法
CN109799073B (zh) 一种光学畸变测量装置及方法、图像处理系统、电子设备和显示设备
CN107273870A (zh) 一种监控场景下融合上下文信息的行人位置检测方法
CN109711407B (zh) 一种车牌识别的方法及相关装置
CN103093458B (zh) 关键帧的检测方法及装置
CN111127358B (zh) 图像处理方法、装置及存储介质
WO2022116104A1 (zh) 图像处理方法、装置、设备及存储介质
CN113901961B (zh) 车位检测方法、装置、设备及存储介质
CN116385430A (zh) 一种机器视觉瑕疵检测方法、装置、介质及设备
CN110827375B (zh) 一种基于微光图像的红外图像真彩着色方法及系统
CN115965934A (zh) 一种停车位检测方法及装置
CN114037834B (zh) 一种基于振动信号和rgb图像融合的语义分割方法及装置
CN105229665A (zh) 对基于图像的蛇形带子磨损评估的增强分析
CN114219758A (zh) 缺陷检测方法、系统、电子设备和计算机可读存储介质
CN113627302A (zh) 一种登高施工合规性检测方法及系统
CN114596244A (zh) 基于视觉处理和多特征融合的红外图像识别方法及系统
CN111611835A (zh) 一种船只检测方法及装置
CN116664694A (zh) 图像亮度获取模型的训练方法、图像获取方法及移动终端
CN114821513B (zh) 一种基于多层网络的图像处理方法及装置、电子设备
CN116434202A (zh) 基于目标和关键点检测的低光照驾驶员分心检测方法
CN113807209B (zh) 车位检测方法、装置、电子设备及存储介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 23867481

Country of ref document: EP

Kind code of ref document: A1