CN113268870A - Monte Carlo-based image recognition reliability evaluation method under outdoor environment condition - Google Patents

Monte Carlo-based image recognition reliability evaluation method under outdoor environment condition Download PDF

Info

Publication number
CN113268870A
CN113268870A CN202110546526.XA CN202110546526A CN113268870A CN 113268870 A CN113268870 A CN 113268870A CN 202110546526 A CN202110546526 A CN 202110546526A CN 113268870 A CN113268870 A CN 113268870A
Authority
CN
China
Prior art keywords
image recognition
image
outdoor environment
flow
evaluation
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202110546526.XA
Other languages
Chinese (zh)
Other versions
CN113268870B (en
Inventor
任羿
易依
杨德真
冯强
孙博
钱诚
王自力
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beihang University
Original Assignee
Beihang University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beihang University filed Critical Beihang University
Priority to CN202110546526.XA priority Critical patent/CN113268870B/en
Publication of CN113268870A publication Critical patent/CN113268870A/en
Application granted granted Critical
Publication of CN113268870B publication Critical patent/CN113268870B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F30/00—Computer-aided design [CAD]
    • G06F30/20—Design optimisation, verification or simulation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/98—Detection or correction of errors, e.g. by rescanning the pattern or by human intervention; Evaluation of the quality of the acquired patterns
    • G06V10/993—Evaluation of the quality of the acquired pattern
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2119/00—Details relating to the type or aim of the analysis or the optimisation
    • G06F2119/02—Reliability analysis or reliability optimisation; Failure analysis, e.g. worst case scenario performance, failure mode and effects analysis [FMEA]

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Quality & Reliability (AREA)
  • General Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Computer Hardware Design (AREA)
  • Evolutionary Computation (AREA)
  • Geometry (AREA)
  • General Engineering & Computer Science (AREA)
  • Image Analysis (AREA)

Abstract

The invention relates to an image identification reliability evaluation method under an outdoor environment condition based on Monte Carlo. The method is based on the Monte Carlo simulation technology, and carries out simulation test on the image recognition process in the outdoor environment, so that reliability evaluation is realized. It comprises three steps: (1) establishing an image identification reliability influence factor set and an outdoor environment distribution model; (2) carrying out Monte Carlo sampling to respectively generate an outdoor environment state and a flow state; (3) according to the Monte Carlo sampling result, carrying out simulation test on the image identification; (4) and evaluating the image recognition reliability according to the simulation test result. Based on the method, guidance can be provided for image recognition reliability evaluation in an outdoor environment.

Description

Monte Carlo-based image recognition reliability evaluation method under outdoor environment condition
Technical Field
The invention provides an image recognition reliability evaluation method under outdoor environment conditions based on Monte Carlo, which is a method for evaluating an image recognition process under outdoor environment by utilizing Monte Carlo simulation so as to realize reliability evaluation. The invention belongs to the field of reliability engineering.
Background
Under the circumstances that computer vision and modern network technology are well developed, image recognition is taken as a typical biometric identification technology to facilitate people's lives, the technology is increasingly applied to a plurality of fields such as security systems, mobile payment and driving checking systems, and the reliability of image recognition becomes a remarkable problem.
The reliability of image recognition is affected by various factors such as the network structure of the algorithm, the continuous change of parameters, the software and hardware environment of the algorithm operation, the quality of training data and the like. Therefore, the conventional method suitable for hardware reliability evaluation is not suitable for image recognition reliability evaluation. At present, research aiming at an image recognition reliability evaluation method is poor. In view of this, the invention provides an image recognition reliability evaluation method under an outdoor environment condition, which provides a reference for image recognition reliability evaluation.
Disclosure of Invention
The invention aims to solve the problems in the background art and provide a feasible idea for evaluating the reliability of image recognition.
The reliability of image recognition under outdoor environmental conditions refers to the ability to correctly recognize an image in a predetermined time under a predetermined outdoor environment. The invention provides an image identification reliability evaluation method under outdoor environment conditions based on Monte Carlo, which mainly comprises the following steps:
the method comprises the following steps: and establishing an image identification reliability influence factor set and an outdoor environment distribution model. It comprises 2 sub-steps:
step 1: analyzing factors influencing the image recognition reliability under the outdoor environment, and constructing an influencing factor set { e1,e2，…，enIn which eiThe i-th influencing factor representing the reliability of image recognition. The influencing factors are not related to each otherAre independent of each other.
Step 2: obtaining the continuous time of different outdoor environment states in the reliability evaluation range according to the statistical data of the meteorological bureau, representing the probability of the possible occurrence of the states according to the proportion of the continuous time of each state in the evaluation time range, and determining the influence factor eiAt an evaluation time range tj(j ═ 1,2,3,4, and represents the distribution in spring, summer, autumn, and winter, respectively).
Step two: and carrying out Monte Carlo sampling to respectively generate an outdoor environment state and a flow state.
Determining the time range of image identification reliability evaluation, carrying out Monte Carlo sampling on the environment state according to the distribution of the environment state outside the chamber in the range, then fitting and generating a distribution model according to the statistical data of the flow under each outdoor environment, and sampling the flow according to the distribution model. The method comprises the following specific steps:
step 1: monte Carlo sampling is performed on the outdoor environmental state, and the outdoor environmental condition of each day is determined. The method comprises the following specific steps:
(1) recording the influencing factor eiThe distribution of the description parameter in the evaluation time range of (2) is F (e)i). In [0,1 ]]Random numbers v uniformly distributed in intervals are generatediAccording to F (e)i)＝viObtaining a obedient distribution function F (e)i) Of random variables ei＝F-1(vi)。
(2) Each influencing factor in the influencing factor set is sampled respectively, and the daily outdoor environment state is determined according to the sampling.
Step 2: monte Carlo simulation is performed on the flow through the image recognition system to generate the flow state for each day within the evaluation time. The method comprises the following specific steps:
(1) the factor having the largest influence on the flow in the influencing factor set is recorded as eoTo influence factor eoGrading and constructing influence factor eoClass set of { e }o1，eo2,…,eonB, wherein eolRepresenting influencing factors eoThe l-th level of (1).
(2) According to different outdoor ringsFlow data through the image recognition system in ambient conditions to influence factor eoThe mean flow at the same level is taken as the expected poisson distribution to characterize the flow distribution through the image recognition system at that level. When the flow rate through the image recognition system is represented by Y, the probability of the discrete random variable Y is
Figure BDA0003073872910000021
(m is the expected flow rate under the environmental conditions of the day, k is the numerical value of the flow rate, k is 0,1,2, …),
Figure BDA0003073872910000022
(3) in [0,1 ]]Generating random numbers q uniformly distributed in the interval if
Figure BDA0003073872910000023
Finding an integer n ═ m0+1,m0+2, …, satisfy
Figure BDA0003073872910000024
The sampling value of the discrete random variable Y is Y ═ n; if it is not
Figure BDA0003073872910000025
Finding an integer n ═ m0,m0-1, …, satisfying
Figure BDA0003073872910000026
The sampling value of the discrete random variable Y is Y ═ n, which indicates that the flow through the image recognition system is n on the same day. m is0The selection method comprises the following steps: order to
Figure BDA0003073872910000027
If it is not
Figure BDA0003073872910000028
Then get m0＝m0(ii) a If it is not
Figure BDA0003073872910000029
Figure BDA00030738729100000210
Then get m0＝m0+1。
Step three: and carrying out simulation test on the image identification according to the Monte Carlo sampling result.
Establishing an image identification embedded database and an image identification test set, inputting an outdoor environment state and a flow state obtained by Monte Carlo sampling into an image identification system, simulating normal use of image identification in an evaluation time period, and obtaining an image identification result after simulation. The method comprises the following specific steps:
step 1: and establishing an image recognition embedded database and an image recognition original test set.
(1) The method comprises the steps of giving IDs to photos containing clear images, selecting only one of the photos to be given with the ID, using the images with the IDs as an image identification embedded database, and recording a (a is a positive integer) images in the database.
(2) And taking a photos contained in the embedded database and b (b is a positive integer) other image photos without ID in the database as an image recognition original test set.
Step 2: based on the monte carlo sampling results, the simulated images identify normal use during the evaluation period. It comprises 3 sub-steps:
(1) the outdoor environmental conditions and flow rate are extracted for one day. Extracting outdoor environment state of one day, and extracting influence factor eoThe description parameter values correspond to the influence factor levels, sampling is carried out on the flow based on Poisson distribution according to the average value of the flow in the influence factor levels, and the simulation times of the outdoor environment state on the current day are determined.
(2) A one day image recognition simulation was started. It comprises 4 sub-steps:
randomly extracting a photo in an image recognition original test set;
processing the extracted pictures according to the extracted outdoor environment state to enable the description parameters of all the influencing factors in the pictures to be equal to the extracted values;
inputting the processed picture into an image recognition system for image recognition test, and recording the recognition result within the specified reaction time.
And fourthly, continuously circulating the step III until the extracted simulation times are reached.
(3) And (3) continuously circulating the steps (1) and (2) until the number of days specified by the evaluation time range is reached, and counting all test results to obtain test data.
Step four: and evaluating the image recognition reliability according to the simulation test result.
The working principle of the image recognition system is as follows: the feature vector similarity threshold of the image recognition system is recorded as T, when the detected sample passes through the image recognition system, the system traverses the images in the embedded database, outputs an ID corresponding to the image with the highest similarity in the images with the feature vector similarity higher than the threshold of the detected sample, and outputs 'unknown' if the embedded database does not have the image with the feature vector similarity higher than the threshold of the detected sample.
Accordingly, the image recognition result includes the following 6 cases:
firstly, the detected sample image exists in an embedded database, and the image recognition system outputs the ID corresponding to the detected sample image within the specified reaction time.
Secondly, the detected sample image exists in the embedded database, and the image recognition system outputs the ID corresponding to the non-detected sample image within the specified reaction time.
And thirdly, the detected sample image exists in the embedded database, and the image recognition system outputs 'unknown' within the specified reaction time.
Fourthly, the tested sample image does not exist in the embedded database, and the image recognition system outputs the ID existing in the embedded database within the specified response time.
The tested sample image does not exist in the embedded database, and the image recognition system outputs 'unknown' within the specified reaction time.
And sixthly, the image recognition system does not respond within the specified reaction time no matter whether the tested sample image exists in the embedded database or not.
The number of times of occurrence of 6 cases in the test results was counted, and the number of times of occurrence of the cases (i), (ii), (iii), (iv), (v), and (sixty) was A, B, C, D, E, F. The evaluation indexes of the image recognition reliability include the reliability, the unreliability and the false acceptance rate of the image recognition system in the specified reaction time in the specified outdoor environmental condition and the specified reaction time, and the signs, meanings and calculation formulas of the evaluation indexes are shown in table 1.
TABLE 1 evaluation index for image recognition reliability
Figure BDA0003073872910000041
In conclusion, the reliability evaluation indexes are selected according to the image recognition system test data obtained in the simulation within the evaluation time range, and the reliability evaluation is carried out on the image recognition under the outdoor environment condition.
Drawings
FIG. 1 is a flow chart of the method
FIG. 2 flow chart of face recognition test
Detailed Description
The process flow of the method of the invention is shown in figure 1. For a better understanding of the features and advantages of the present invention, reference is made to the following detailed description taken in conjunction with the accompanying drawings and examples. The selected case of the invention is a face recognition system at an entrance of a campus. The specific implementation steps are as follows:
the method comprises the following steps: and establishing an image identification reliability influence factor set and an outdoor environment distribution model. It comprises 2 sub-steps:
step 1: analyzing factors influencing the image recognition reliability under the outdoor environment, and constructing an influencing factor set { e1,e2，…，enIn which eiThe i-th influencing factor representing the reliability of image recognition. The influencing factors are not related to each other and are independent of each other.
Step 2: obtaining the duration time of different outdoor environment states in the reliability evaluation range according to the statistical data of the meteorological bureau, and representing the probability of the possible occurrence of the states according to the proportion of the duration time of each state in the evaluation time range, thereby determining the influence factorElement eiAt an evaluation time range tj(j ═ 1,2,3,4, and represents the distribution in spring, summer, autumn, and winter, respectively).
[ example ] the set of influence factors on the face recognition reliability at an entrance of a campus is { illumination }. At an illumination intensity e1As a descriptive parameter for measuring the illumination, the evaluation time range was selected as spring. Taking 10 parts per day: 00-16: the mean value of 00 light intensities was taken as the light intensity value on the day. The distribution of the illumination intensity in spring is obeyed N (6000,10000)2) The value range of the illumination intensity is (400, 10000), and the unit is lx.
Step two: and carrying out Monte Carlo sampling to respectively generate an outdoor environment state and a flow state.
Determining the time range of image identification reliability evaluation, carrying out Monte Carlo sampling on the environment state according to the distribution of the environment state outside the chamber in the range, then fitting and generating a distribution model according to the statistical data of the flow under each outdoor environment, and sampling the flow according to the distribution model. The method comprises the following specific steps:
step 1: monte Carlo sampling is performed on the outdoor environmental state, and the outdoor environmental condition of each day is determined. The method comprises the following specific steps:
(1) recording the influencing factor eiThe distribution of the description parameter in the evaluation time range of (2) is F (e)i). In [0,1 ]]Random numbers v uniformly distributed in intervals are generatediAccording to F (e)i)＝viObtaining a obedient distribution function F (e)i) Of random variables ei＝F-1(vi)。
(2) Each influencing factor in the influencing factor set is sampled respectively, and the daily outdoor environment state is determined according to the sampling.
[ example ] the distribution of the illumination intensity at an entrance of a campus in spring is subject to N (6000, 10000)2) The value range of the illumination intensity is (400, 10000), and the unit is lx. Monte Carlo sampling was performed for 120 days in spring to obtain daily light intensities as shown in Table 2.
TABLE 2 spring daily illumination intensity
Figure BDA0003073872910000051
Step 2: monte Carlo simulation is performed on the flow through the image recognition system to generate the flow state for each day within the evaluation time. The method comprises the following specific steps:
(1) the factor having the largest influence on the flow in the influencing factor set is recorded as eoTo influence factor eoGrading and constructing influence factor eoClass set of { e }o1,eo2,…,eonB, wherein eolRepresenting influencing factors eoThe l-th level of (1).
(2) According to the flow data passing through the image recognition system under different outdoor environment states, influencing factor eoThe mean flow at the same level is taken as the expected poisson distribution to characterize the flow distribution through the image recognition system at that level. When the flow rate through the image recognition system is represented by Y, the probability of the discrete random variable Y is
Figure BDA0003073872910000052
(m is the expected flow rate under the environmental conditions of the day, k is the numerical value of the flow rate, k is 0,1,2, …),
Figure BDA0003073872910000053
(3) in [0,1 ]]Generating random numbers q uniformly distributed in the interval if
Figure BDA0003073872910000054
Finding an integer n ═ m0+1,m0+2, …, satisfy
Figure BDA0003073872910000061
The sampling value of the discrete random variable Y is Y ═ n; if it is not
Figure BDA0003073872910000062
Finding an integer n ═ m0,m0-1,…Satisfy the following requirements
Figure BDA0003073872910000063
The sampling value of the discrete random variable Y is Y ═ n, which indicates that the flow through the image recognition system is n on the same day. m is0The selection method comprises the following steps: order to
Figure BDA0003073872910000064
If it is not
Figure BDA0003073872910000065
Then get m0＝m0(ii) a If it is not
Figure BDA0003073872910000066
Figure BDA0003073872910000067
Then get m0＝m0+1。
[ example ] the factor that has the greatest influence on the flow in the face recognition reliability influence factor set at the entrance of a campus is the illumination intensity. The illumination intensities are graded, and probability functions of the flow of people for face recognition under each illumination grade are determined, and are shown in table 3.
TABLE 3 grading of illumination intensity
Figure BDA0003073872910000068
The spring daily light intensity rating was obtained from the daily light intensity obtained by sampling and from table 3. And sampling according to the grades to obtain the flow of people who perform face recognition every day, which is shown in a table 4.
TABLE 4 spring daily pedestrian volume
Figure BDA0003073872910000069
Step three: and carrying out simulation test on the image identification according to the Monte Carlo sampling result.
Establishing an image identification embedded database and an image identification test set, inputting an outdoor environment state and a flow state obtained by Monte Carlo sampling into an image identification system, simulating normal use of image identification in an evaluation time period, and obtaining an image identification result after simulation. The method comprises the following specific steps:
step 1: and establishing an image recognition embedded database and an image recognition original test set.
(1) The method comprises the steps of giving IDs to photos containing clear images, selecting only one of the photos to be given with the ID, using the images with the IDs as an image identification embedded database, and recording a (a is a positive integer) images in the database.
(2) And taking a photos contained in the embedded database and b other image photos without ID in the database as an image recognition original test set.
8000 Master and student's clear 1 cun non-crown photographs were stored in a face recognition embedded database, and each photograph was labeled, i.e. the study or work number of the figure in the photograph. 8000 photos and 1000 stranger photos which are not the teacher are contained in the embedded database and are used as an original test set.
Step 2: based on the monte carlo sampling results, the simulated images identify normal use during the evaluation period. It comprises 3 sub-steps:
(1) the outdoor environmental conditions and flow rate are extracted for one day. Extracting outdoor environment state of one day, and extracting influence factor eoThe description parameter values correspond to the influence factor levels, sampling is carried out on the flow based on Poisson distribution according to the average value of the flow in the influence factor levels, and the simulation times of the outdoor environment state on the current day are determined.
(2) A one day image recognition simulation was started. It comprises 4 sub-steps:
randomly extracting a photo in an image recognition original test set;
processing the extracted pictures according to the extracted outdoor environment state to enable the description parameters of all the influencing factors in the pictures to be equal to the extracted values;
inputting the processed picture into an image recognition system for image recognition test, and recording the recognition result.
And fourthly, continuously circulating the step III until the extracted simulation times are reached.
(3) And (3) continuously circulating the steps (1) and (2) until the number of days specified by the evaluation time range is reached, and counting all test results to obtain test data.
The flow of the face recognition test at the campus entrance is shown in fig. 2, and the simulation test steps are as follows:
(1) and determining the daily outdoor environment state according to the extracted spring daily illumination intensity, and determining the face identification testing times in each outdoor environment state according to the extracted spring daily pedestrian volume.
(2) A one day face recognition simulation was started. It comprises 4 sub-steps:
randomly extracting a photo in an original face recognition test set;
and secondly, performing gray level transformation processing on the extracted pictures according to the extracted illumination intensity to ensure that the pictures are consistent with the pictures shot under the illumination intensity extracted from the real environment and serve as actual test samples. For example, the illumination intensity of 1 month and 1 day is 6376lx, and the photos extracted from the original test set are processed into photos with the same shooting effect as that of 6376lx illumination;
constructing a message for requesting the tested model service for the test sample, sending the message to the face recognition service at the entrance, comparing the result of the returned message of the face recognition service with the corresponding label, and recording the recognition result in 1 s.
And fourthly, continuously circulating the step III until the extracted simulation times are reached. For example, the circulation should be 2073 times in 1 month and 1 day outdoor environment.
(3) And (3) continuously circulating the steps (1) and (2) until 120-day spring simulation is completed, and counting all test results to obtain test data.
Step four: and evaluating the image recognition reliability according to the simulation test result.
The working principle of the image recognition system is as follows: the feature vector similarity threshold of the image recognition system is recorded as T, when the detected sample passes through the image recognition system, the system traverses the images in the embedded database, outputs the ID corresponding to the image with the highest similarity in the images with the feature vector similarity higher than the threshold of the detected sample, and outputs 'unknown' if the embedded database does not have the image with the feature vector similarity higher than the threshold T of the detected sample.
Accordingly, the image recognition result includes the following 6 cases:
firstly, the detected sample image exists in an embedded database, and the image recognition system outputs the ID corresponding to the detected sample image within the specified reaction time.
Secondly, the detected sample image exists in the embedded database, and the image recognition system outputs the ID corresponding to the non-detected sample image within the specified reaction time.
And thirdly, the detected sample image exists in the embedded database, and the image recognition system outputs 'unknown' within the specified reaction time.
Fourthly, the tested sample image does not exist in the embedded database, and the image recognition system outputs the ID existing in the embedded database within the specified response time.
The tested sample image does not exist in the embedded database, and the image recognition system outputs 'unknown' within the specified reaction time.
And sixthly, the image recognition system does not respond within the specified reaction time no matter whether the tested sample image exists in the embedded database or not.
The number of times of occurrence of 6 cases in the test results was counted, and the number of times of occurrence of the cases (i), (ii), (iii), (iv), (v), and (sixty) was A, B, C, D, E, F. The evaluation indexes of the image recognition reliability include the reliability, the unreliability and the false acceptance rate of the image recognition system in the specified reaction time in the specified outdoor environmental condition in the evaluation time period, and the signs, meanings and calculation formulas of these evaluation indexes are shown in table 5.
TABLE 5 image recognition reliability evaluation index
Figure BDA0003073872910000081
In conclusion, the reliability evaluation indexes are selected according to the image recognition system test data obtained in the simulation within the evaluation time range, and the reliability evaluation is carried out on the image recognition under the outdoor environment condition.
After 183927 tests of face recognition in 120 days of spring were completed, test data were obtained, see table 6.
TABLE 6 face recognition test data
Test conditions ① ② ③ ④ ⑤ ⑥
Number of occurrences A B C D E F
Value of 160037 95 393 154 22870 378
Calculating the evaluation index of the face recognition reliability:
Figure BDA0003073872910000091
Figure BDA0003073872910000092
Figure BDA0003073872910000093

Claims (1)

1. an image identification reliability evaluation method under outdoor environmental conditions based on Monte Carlo is characterized by comprising the following steps:
the method comprises the following steps: establishing an image recognition reliability influence factor set and an outdoor environment distribution model, which comprises 2 sub-steps:
step 1: analyzing factors influencing the image recognition reliability under the outdoor environment, and constructing an influencing factor set { e1,e2,…,enIn which eiThe ith influence factor represents the image identification reliability, and the influence factors are not related to each other and are independent of each other;
step 2: obtaining the continuous time of different outdoor environment states in the reliability evaluation range according to the statistical data of the meteorological bureau, representing the probability of the possible occurrence of the states according to the proportion of the continuous time of each state in the evaluation time range, and determining the influence factor eiAt an evaluation time range tj(j ═ 1,2,3,4, and represents the distribution in spring, summer, autumn, and winter, respectively);
step two: carrying out Monte Carlo sampling to respectively generate an outdoor environment state and a flow state;
determining the time range of image identification reliability evaluation, carrying out Monte Carlo sampling on the environment state according to the distribution of the environment state outside an indoor chamber in the range, then fitting and generating a distribution model according to the statistical data of the flow under each outdoor environment, and sampling the flow according to the statistical data, wherein the specific steps are as follows:
step 1: monte Carlo sampling is carried out on the outdoor environment state, the outdoor environment condition of each day is determined, and the method comprises the following specific steps:
(3) recording the influencing factor eiThe distribution of the description parameter in the evaluation time range of (2) is F (e)i) In [0,1 ]]Random numbers v uniformly distributed in intervals are generatediAccording to F (e)i)＝viObtaining a obedient distribution function F (e)i) Of random variables ei＝F-1(vi)；
(4) Sampling each influence factor in the influence factor set respectively, and determining the daily outdoor environment state according to the sampling;
step 2: carrying out Monte Carlo simulation on the flow passing through the image recognition system to generate a flow state of each day in the evaluation time, and specifically comprising the following steps:
(4) the factor having the largest influence on the flow in the influencing factor set is recorded as eoTo influence factor eoGrading and constructing influence factor eoClass set of { e }o1,eo2,…,eonIn which eolRepresenting influencing factors eoThe l-th grade of (1);
(5) according to the flow data passing through the image recognition system under different outdoor environment states, influencing factor eoTaking the average value of the flow at the same level as an expected value to generate a Poisson distribution so as to represent the flow distribution passing through the image recognition system at the level, and expressing the flow passing through the image recognition system by Y, the probability of the discrete random variable Y is
Figure FDA0003073872900000011
(m is the expectation of the flow under the environmental conditions of the day, k is the flowThe value of the quantity, k-0, 1,2, …),
Figure FDA0003073872900000012
(6) in [0,1 ]]Generating random numbers q uniformly distributed in the interval if
Figure FDA0003073872900000013
Finding an integer n ═ m0+1,m0+2, …, satisfy
Figure FDA0003073872900000014
The sampling value of the discrete random variable Y is Y ═ n; if it is not
Figure FDA0003073872900000021
Finding an integer n ═ m0,m0-1, …, satisfying
Figure FDA0003073872900000022
The sampling value of the discrete random variable Y is Y ═ n, which indicates that the flow through the image recognition system is n on the same day. m is0The selection method comprises the following steps: order to
Figure FDA0003073872900000023
If it is not
Figure FDA0003073872900000024
Then get m0＝m0(ii) a If it is not
Figure FDA0003073872900000025
Figure FDA0003073872900000026
Then get m0＝m0+1；
Step three: according to the Monte Carlo sampling result, carrying out simulation test on the image identification;
establishing an image identification embedded database and an image identification test set, inputting an outdoor environment state and a flow state obtained by Monte Carlo sampling into an image identification system, simulating the normal use of image identification in an evaluation time period, and obtaining an image identification result after simulation, wherein the image identification method specifically comprises the following steps:
step 1: establishing an image recognition embedded database and an image recognition original test set;
(3) endowing the photos containing clear images with IDs, only selecting one of the photos to be endowed with the ID, using the images endowed with the IDs as an image identification embedded database, and recording that the database contains a (a is a positive integer) images;
(4) taking a photo contained in the embedded database and b other image photos without ID in the database as an image recognition original test set;
step 2: from the monte carlo sampling results, the normal use of image recognition during the evaluation period is simulated, which comprises 3 sub-steps:
(4) extracting outdoor environment state of one day, and extracting influence factor eoThe description parameter values correspond to the influence factor levels, sampling flow based on Poisson distribution is carried out according to the average value of the flow in the influence factor levels, and the simulation times of the outdoor environment state on the same day are determined;
(5) starting a day's image recognition simulation, it contains 4 sub-steps:
randomly extracting a photo in an image recognition original test set;
processing the extracted pictures according to the extracted outdoor environment state to enable the description parameters of all the influencing factors in the pictures to be equal to the extracted values;
inputting the processed picture into an image recognition system for image recognition test, and recording the recognition result within the specified reaction time;
fourthly, continuously circulating the step III until the extracted simulation times are reached;
(6) continuously circulating the steps (1) and (2) until the number of days specified by the evaluation time range is reached, and counting all test results to obtain test data;
step four: evaluating the image recognition reliability according to the simulation test result;
the working principle of the image recognition system is as follows: recording a feature vector similarity threshold of the image identification system as T, when the detected sample passes through the image identification system, traversing the image in the embedded database by the system, outputting an ID corresponding to the image with the highest similarity in the image with the feature vector similarity higher than the threshold of the detected sample, and outputting 'unbnow' if the image with the feature vector similarity higher than the threshold of the detected sample does not exist in the embedded database;
accordingly, the image recognition result includes the following 6 cases:
firstly, a detected sample image exists in an embedded database, and an image identification system outputs an ID corresponding to the detected sample image within a specified reaction time;
secondly, the detected sample image exists in the embedded database, and the image recognition system outputs the ID corresponding to the non-detected sample image within the specified reaction time;
thirdly, the tested sample image exists in the embedded database, and the image recognition system outputs 'unbnow' within the specified reaction time;
fourthly, the detected sample image does not exist in the embedded database, and the image recognition system outputs the ID existing in the embedded database within the specified response time;
fifthly, the tested sample image does not exist in the embedded database, and the image recognition system outputs 'unbnow' within the specified reaction time;
whether the detected sample image exists in the embedded database or not, the image recognition system does not respond within the specified reaction time;
counting the occurrence frequency of 6 conditions in the test result, wherein the occurrence frequency of the conditions of (i) is recorded as A, B, C, D, E, F, the occurrence frequency of (iii) is recorded as (iv) is recorded as (A, B, C, D, E, F), the evaluation indexes of reliability, including the reliability, unreliability and the false acceptance rate in the specified reaction time within the specified outdoor environment condition of the image recognition system under the specified outdoor environment condition, and the signs, meanings and the calculation formulas of the evaluation indexes are recorded as (1):
TABLE 1 evaluation index for image recognition reliability
Figure FDA0003073872900000031
In conclusion, the reliability evaluation indexes are selected according to the image recognition system test data obtained in the simulation within the evaluation time range, and the reliability evaluation is carried out on the image recognition under the outdoor environment condition.
CN202110546526.XA 2021-05-19 2021-05-19 Monte Carlo-based image recognition reliability evaluation method under outdoor environment condition Active CN113268870B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110546526.XA CN113268870B (en) 2021-05-19 2021-05-19 Monte Carlo-based image recognition reliability evaluation method under outdoor environment condition

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110546526.XA CN113268870B (en) 2021-05-19 2021-05-19 Monte Carlo-based image recognition reliability evaluation method under outdoor environment condition

Publications (2)

Publication Number Publication Date
CN113268870A true CN113268870A (en) 2021-08-17
CN113268870B CN113268870B (en) 2022-06-28

Family

ID=77232022

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110546526.XA Active CN113268870B (en) 2021-05-19 2021-05-19 Monte Carlo-based image recognition reliability evaluation method under outdoor environment condition

Country Status (1)

Country Link
CN (1) CN113268870B (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114030488A (en) * 2022-01-11 2022-02-11 清华大学 Method and device for realizing automatic driving decision, computer storage medium and terminal
CN116306027A (en) * 2023-05-16 2023-06-23 启迪万众网络科技(北京)有限公司 Cloud data processing system based on environment simulation

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20100046840A1 (en) * 2008-08-20 2010-02-25 Canon Kabushiki Kaisha Image processing apparatus and image processing method for evaluating a plurality of image recognition processing units
CN109671274A (en) * 2019-01-24 2019-04-23 交通运输部公路科学研究所 A kind of highway risk automatic evaluation method based on latent structure and fusion
CN112506797A (en) * 2020-12-22 2021-03-16 南京航空航天大学 Performance test method for medical image recognition system

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20100046840A1 (en) * 2008-08-20 2010-02-25 Canon Kabushiki Kaisha Image processing apparatus and image processing method for evaluating a plurality of image recognition processing units
CN109671274A (en) * 2019-01-24 2019-04-23 交通运输部公路科学研究所 A kind of highway risk automatic evaluation method based on latent structure and fusion
CN112506797A (en) * 2020-12-22 2021-03-16 南京航空航天大学 Performance test method for medical image recognition system

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
方冠男 等: "视频人脸图像质量评估及其在课堂点名系统中的应用", 《计算机应用与软件》 *
杨德真 等: "基于公理设计的产品可靠性要求实现方法", 《北京航空航天大学学报》 *

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114030488A (en) * 2022-01-11 2022-02-11 清华大学 Method and device for realizing automatic driving decision, computer storage medium and terminal
US11673562B1 (en) 2022-01-11 2023-06-13 Tsinghua University Method, apparatus, computer storage medium and terminal for implementing autonomous driving decision-making
CN116306027A (en) * 2023-05-16 2023-06-23 启迪万众网络科技(北京)有限公司 Cloud data processing system based on environment simulation

Also Published As

Publication number Publication date
CN113268870B (en) 2022-06-28

Similar Documents

Publication Publication Date Title
CN109766872B (en) Image recognition method and device
CN113268870B (en) Monte Carlo-based image recognition reliability evaluation method under outdoor environment condition
CN110503000B (en) Teaching head-up rate measuring method based on face recognition technology
CN111382690B (en) Vehicle re-identification method based on multi-loss fusion model
CN113065516B (en) Sample separation-based unsupervised pedestrian re-identification system and method
CN114913923A (en) Cell type identification method aiming at open sequencing data of single cell chromatin
CN115617882B (en) GAN-based time sequence diagram data generation method and system with structural constraint
CN111401105B (en) Video expression recognition method, device and equipment
CN111738044A (en) Campus violence assessment method based on deep learning behavior recognition
CN113269903A (en) Face recognition class attendance system
CN109582974A (en) A kind of student enrollment's credit estimation method and device based on deep learning
CN115798022A (en) Artificial intelligence identification method based on feature extraction
CN111861038A (en) College entrance examination score prediction method and system based on machine learning algorithm
CN115393666A (en) Small sample expansion method and system based on prototype completion in image classification
CN114639152A (en) Multi-modal voice interaction method, device, equipment and medium based on face recognition
CN112541579B (en) Model training method, lean degree information identification method, device and storage medium
CN113988519A (en) Method for representing risk of cultural relic preservation environment in collection of cultural relics
CN116189063B (en) Key frame optimization method and device for intelligent video monitoring
CN113486202A (en) Method for classifying small sample images
CN112258481A (en) Portal photo reproduction detection method
Yu et al. A multi-scale feature selection method for steganalytic feature GFR
CN114898137A (en) Face recognition-oriented black box sample attack resisting method, device, equipment and medium
CN111160263B (en) Method and system for acquiring face recognition threshold
CN113688789A (en) Online learning investment recognition method and system based on deep learning
CN111444877A (en) Classroom people number identification method based on video photos

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant