JP7433782B2 - Information processing device, information processing method, and program - Google Patents

Description

The present invention relates to information processing technology in learning type recognition.

In a learning type recognition device, a so-called domain adaptation method is known in which the recognition accuracy for patterns specific to the installed environment is increased by additionally learning data obtained during operation.
In recent years, in order to increase the effectiveness of additional learning, methods have been proposed to screen out inappropriate teaching data to prevent it from being mixed in. For example, Patent Document 1 proposes a method of determining whether input data is to be adopted as training data based on whether it matches the morpheme structure of previously collected training data. Moreover, Patent Document 2 proposes a method of determining whether overfitting has occurred based on the transition of the evaluation value of additional learning data, and relearning if overfitting has occurred.

Japanese Patent Application Publication No. 2010-198189 Japanese Patent Application Publication No. 2010-257140

However, with the method described in the above-mentioned patent document, it is difficult to reliably prevent the mixing of inappropriate training data and the occurrence of overfitting, even if the training data is selected and reworked in case of overfitting. . Particularly in the case of long-term operation, it is not realistic to ensure that there is no reduction in recognition accuracy due to inappropriate additional learning. Further, the effects of such a decrease in recognition accuracy do not necessarily become apparent immediately. A decline in recognition accuracy becomes noticeable only when erroneous recognition or non-recognition becomes a problem. When the influence of a decrease in recognition accuracy appears, it is not easy to analyze what kind of additional learning was the cause of the decrease in recognition accuracy. Therefore, it is desirable to be able to appropriately eliminate the influence of inappropriate additional learning that reduces recognition accuracy.

Therefore, an object of the present invention is to make it possible to appropriately eliminate the influence of additional learning that reduces recognition accuracy, and to suppress the reduction in recognition accuracy.

The information processing device of the present invention includes a recognition unit that recognizes features from an image based on a model, a learning unit that performs additional learning of the model, and a management unit that manages information on the history of additional learning of the model. The evaluation means includes an evaluation means for evaluating the accuracy of the recognition by the model, and a selection means for selecting a model from among the models whose history information is managed based on the accuracy of the recognition, and the evaluation means maintains an evaluation set that records a set of information to be recognized, and the selection means performs additional learning based on the result of evaluating accuracy using a subset of information to be recognized from the evaluation set. A model is selected, and the recognition means uses the model selected by the selection means for the recognition.

According to the present invention, it is possible to appropriately eliminate the influence of additional learning that reduces recognition accuracy, and it is possible to suppress a reduction in recognition accuracy.

FIG. 1 is a diagram illustrating a configuration example of an information processing device in a first embodiment. It is a figure showing an example of a display screen of a display part. It is a flowchart of information processing in a 1st embodiment. FIG. 3 is a diagram showing an example of data managed by the management unit of the first embodiment. It is a flowchart of the evaluation set creation process by an evaluation part. FIG. 2 is a schematic diagram showing an example of an evaluation set. It is a flowchart of the search process by the selection part of 1st Embodiment. FIG. 6 is a diagram illustrating an example of the transition of performance values created by the selection unit. FIG. 3 is a diagram showing an example of a selection screen displayed by the display unit. It is a flowchart of information processing in a 2nd embodiment. It is a flowchart of the performance value calculation process in an evaluation part. FIG. 7 is a diagram illustrating an example of data managed by a management unit according to a third embodiment. It is a flowchart of the score calculation process by the selection part of 3rd Embodiment. It is a figure showing an example of data managed by a management part of a 4th embodiment. It is a flowchart of the score calculation process by the selection part of 4th Embodiment.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. Note that the configurations shown in the following embodiments are merely examples, and the present invention is not limited to the illustrated configurations.
<First embodiment>
FIG. 1 is a diagram showing a schematic configuration example of an information processing apparatus according to a first embodiment.
The information processing device of this embodiment includes a photographing section 101, a feature amount calculation section 102, a recognition section 103, a learning section 104, a storage section 105, a management section 106, an evaluation section 107, a selection section 108, a display section 109, and an operation section. It has 110.

The photographing unit 101 is a camera device that includes a lens, an image sensor, a lens drive motor, an MPU that controls the image sensor and the lens drive motor, and the like. The photographing unit 101 photographs a moving image, for example, and outputs data of the moving image.
The feature amount calculation unit 102 is configured by an MPU or the like. The feature amount calculation unit 102 calculates feature amounts from images included in the video captured by the imaging unit 101.

The recognition unit 103 is configured by an MPU or the like. The recognition unit 103 recognizes the target object that appears in the video image based on the feature amount calculated by the feature amount calculation unit 102. Although details will be described later, the recognition unit 103 calculates a recognition score for use in recognizing the target object in the image based on the statistical model selected by the selection unit 108 (described later) for the feature quantity calculated by the feature quantity calculation unit 102. calculate. Then, the recognition unit 103 recognizes a target object that has performed an abnormal motion among the target objects recognized in the captured video. In the case of this embodiment, the recognition unit 103 recognizes a person as a target object, and further recognizes a person who has behaved abnormally among these people.

The learning unit 104 is configured by an MPU or the like. The learning unit 104 adds the feature quantities calculated by the feature quantity calculation unit 102 from the teacher data to the model used by the recognition unit 103 and performs re-learning, thereby creating a model that has additionally learned the features of the teacher data. .

The storage unit 105 includes a recording medium such as a hard disk, an MPU, and the like. The storage unit 105 stores the data of the video captured by the imaging unit 101, the feature quantities computed by the feature quantity calculation unit 102, information on the results of recognition performed by the recognition unit 103, the model created by the learning unit 104, and the relationships among these. Stores the information to be represented and metadata such as creation time information. Note that the storage unit 105 may use storage on a network such as a NAS (Network Attached Storage), a SAN (Storage Area Network), or a cloud service instead of a recording medium.

The management unit 106 is composed of an MPU and the like. The management unit 106 records information such as the model created by the learning unit 104, the date and time of model creation, teacher data used for learning, and usage status in the recognition unit 103, and records the history of the learned model. Manage. Details of the information managed by the management unit 106 will be described later.
The evaluation unit 107 is configured by an MPU or the like. The evaluation unit 107 evaluates the performance of the specified model. Details of the performance evaluation will be described later.
The selection unit 108 is configured by an MPU or the like. The selection unit 108 selects a model to be used by the recognition unit 103 from among the models managed by the management unit 106 based on the evaluation result of the evaluation unit 107 and the like. Details of model selection will be described later.

The display unit 109 includes a liquid crystal screen and an MPU that controls the screen. The display unit 109 presents various information to the user. Furthermore, the display unit 109 creates and displays a user interface (UI) screen used when the user inputs instructions. Details of the UI screen will be described later.
The operation unit 110 includes a switch, a touch panel, and the like. The operation unit 110 senses a user's operation and inputs it to the information processing device. Note that the operation unit 110 may include another pointing device such as a mouse or a trackball instead of the touch panel.

Note that although the present embodiment has been described using as an example a device that recognizes abnormal behavior of a person as a target object from a captured video, the recognition target is not limited to this. The recognition target may be, for example, a specific person, a specific vehicle, or a specific car model, or may be an event, a time period, or the like. In addition to moving images, the recognition target may also be audio, documents, or a combination thereof.

FIGS. 2(A) to 2(D) are diagrams used to explain an example of the operation of the display unit 109 and the user's operation in this embodiment. The video 201 in FIG. 2A, the video 204 in FIG. 2B, the video 207 in FIG. 2C, and the video 211 in FIG. This shows an example of the image displayed on the screen.

An image 201 in FIG. 2A shows an example of a moving image during normal movement. The photographing unit 101 is installed as a surveillance camera at a location to be monitored. The display unit 109 displays the moving image being photographed by the photographing unit 101 as a live video. The user watches the live video displayed on the display unit 109 to monitor whether or not an abnormality has occurred in the monitoring target location. When a person appears at the monitoring target location, the recognition unit 103 recognizes the person and notifies the display unit 109 of the person. The display unit 109 superimposes a predetermined object (for example, a rectangular object that inscribes the person) on the image of the person, and displays the superimposed object on the live image. A video 201 in FIG. 2A shows an example in which rectangular objects 202 and 203 are displayed superimposed on a person recognized from a video of a monitoring target location.

The display unit 109 also displays a time bar 209 on the screen of the video 201. The time bar 209 can be operated by the user via the operation unit 110. The time bar 209 is associated with the video shooting time stored in the storage unit 105. For example, when a user wants to view a live video that was displayed in the past, the user can instruct the display of the past video by operating the time bar 209 via the operation unit 110. When the display unit 109 receives a notification from the operation unit 110 that the time bar 209 has been operated, the display unit 109 reads the video data of the shooting time corresponding to the operating position of the time bar 209 from the storage unit 105, and displays the video data of the shooting time corresponding to the operating position of the time bar 209. Display the image on the screen.

Furthermore, the recognition unit 103 performs a process of recognizing a person appearing at a monitoring target location as described above, as well as a process of recognizing abnormal behavior of the person. A video 204 in FIG. 2B shows an example of a video in which a person 205 falls, as an example of the person's abnormal behavior. When the person 205 falls, the recognition unit 103 recognizes that an abnormality different from normal walking has occurred and notifies the display unit 109. When receiving a notification from the recognition unit 103 that an abnormality has occurred in the person 205, the display unit 109 displays an alarm object 206 near the person 205, indicating that an abnormality has occurred in the person's behavior. This allows the user to recognize that an abnormality has occurred, confirm the state of the abnormality, and take necessary measures (for example, necessary measures such as rescuing a person who has fallen). Note that although the alarm object 206 is displayed on the screen in the example of FIG. It may be something like sending an email or short message to someone.

Although the recognition unit 103 is capable of recognizing abnormal behavior of a person as described above, for example, it may erroneously recognize that a person who is walking normally has acted abnormally. A video 207 in FIG. 2C shows an example in which a warning object 208 is displayed on the screen because the recognition unit 103 mistakenly recognizes that an abnormal behavior has occurred even though the person is walking normally. It shows. At this time, the user will judge whether the alarm is a false alarm (that is, a false alarm) by looking at the image on the screen.

Here, it is assumed that the user operates the time bar 209 to instruct to select a past shooting time, and then further instructs the user to select a display location of the alarm object 208. It is assumed that the user's selection instruction for the display location of the alarm object 208 is, for example, an operation such as tapping the position of the alarm object 208. When these selection instructions are given, the recognition unit 103 recognizes that the user has determined that the alarm issued by the alarm object 208 is a false alarm, and notifies the display unit 109. At this time, the display unit 109 displays a dialog box 210 for canceling the warning on the screen. The dialog box 210 includes, for example, a message "Do you want to cancel?" and buttons "OK" and "Cancel." The "OK" button is a button that is tapped when the user instructs to cancel the warning. The "cancel" button is a button that is tapped when the user does not instruct to cancel the warning. When the display unit 109 receives information from the operation unit 110 indicating that the user has selected the "OK" button, the display unit 109 determines that the user has instructed to cancel the alarm, and removes the alarm object 208 from the screen. Then, the management unit 106 causes the storage unit 105 to store information regarding the canceled false alarm.The information regarding the canceled false alarm is stored in the image area of the person whose behavior was recognized by the recognition unit 103 as abnormal. It includes information on feature amounts, shooting times, and past shooting times selected by the user by operating the time bar 209.As will be described in detail later, information regarding false alarms stored in the storage unit 105 is stored in the evaluation unit 107. is used when evaluating the performance of the model.

Further, while the above-described monitoring operation is being performed, the learning unit 104 of the information processing device performs additional learning on the recognition result by the recognition unit 103 as background processing. The learning unit 104 uses the recognition results performed by the recognition unit 103 for each person appearing in the video as described above, the photographing time of the person, the image area of the person, the feature amount acquired by the feature amount calculation unit 102, etc. The information is stored in the storage unit 105 as information regarding the person. Furthermore, the learning unit 104 performs additional learning, for example, every day, using the information accumulated in the storage unit 105 as teacher data. Then, the recognition unit 103 uses the learned model for person recognition processing and abnormal behavior recognition processing at the monitoring target location. Thereby, the recognition accuracy in the recognition unit 103 is improved. Further, as long as the recognition accuracy in the recognition unit 103 is maintained, the user does not need to be aware of this, and no additional learning information is particularly displayed on the display unit 109.

However, even though the above-described additional learning is performed, the recognition accuracy may not improve or may conversely deteriorate. The cause of this is, for example, when a person's behavior is erroneously recognized as abnormal even though it is normal, resulting in a false alarm, which is overlooked by the user, and then additional learning is performed using the recognition results related to the false alarm as training data. I can think of a case where this was done. In addition, for example, when an event that is different from normal is held at a monitored location, a large amount of training data related to the behavior of the person at the time of the event, that is, the behavior that is different from normal, is additionally learned. is possible. If a low-quality model based on such learning results is used for recognition processing, the recognition unit 103 may, for example, overlook many abnormal behaviors of a person without recognizing them as abnormal behaviors. There is. If the recognition accuracy decreases in this way, it becomes difficult to perform monitoring by the information processing device. Therefore, it is desirable to be able to appropriately remove inappropriate additional learning that causes a decrease in recognition accuracy, thereby suppressing the decrease in recognition accuracy. In this embodiment, by returning to the state before the recognition accuracy deteriorates, the recognition accuracy is returned to the state before inappropriate additional learning is performed.

Therefore, in the information processing apparatus of this embodiment, the evaluation unit 107 performs performance evaluation on the model based on additional learning. If the evaluation unit 107 detects that the performance of the recognition function deteriorates when the model is used as a result of the performance evaluation of the model, it notifies the display unit 109 to that effect. The display unit 109 that receives this notification displays information indicating that the performance of the recognition function has deteriorated. This allows the user to recognize that the performance of the recognition function has deteriorated.

An image 211 in FIG. 2(D) shows an example of an image displayed on the display unit 109 when a decrease in the performance of the recognition function is detected. The display unit 109 displays a dialog box 212 shown in FIG. 2(D) within the video 211. Dialog box 212 includes a message to be presented to the user, buttons for "OK," "cancel," and "confirm details." The message presented to the user consists of a message indicating that a decline in cognitive function has occurred and a message asking whether to return to the state of the model before the decline in cognitive function occurred. Note that it is preferable that the model before the cognitive function deteriorates is a model stored in the storage unit 105 at a date and time closest to the present time and before the time when the cognitive function deteriorates, for example.

In this embodiment, the model created by learning by the learning unit 104 and stored in the storage unit 105 is managed by the management unit 106. Further, in this embodiment, the model used by the recognition unit 103 in recognition processing is selected by the selection unit 108 from among the models managed by the management unit 106. When the evaluation unit 107 detects a decrease in the performance of the recognition function, the selection unit 108 selects a model from before the decrease in the recognition function occurs from among the models managed by the management unit 106, and a model before the decrease in the recognition function occurs. Select as a model to be returned when returning to the previous model state. Then, the selection unit 108 notifies the display unit 109 of the model to be sent back. The display unit 109 prompts the user to return the model to the state before the recognition function deteriorated, as shown in FIG. Generate and display a message to ask the question. By displaying such a dialog box 212, the user can know that the recognition accuracy has decreased due to deterioration of the model and the model at the time of the remand candidate before the recognition accuracy decreased. .

For example, when receiving a notification from the operation unit 110 that the user has tapped an “OK” button in the dialog box 212, the selection unit 108 selects the return candidate model as the model to be used by the recognition unit 103. As a result, the recognition unit 103 performs recognition processing using the returned candidate model, that is, the model before the recognition accuracy deteriorates. For example, when receiving a notification from the operation unit 110 that the user has tapped a “cancel” button, the selection unit 108 selects the current model as the model used by the recognition unit 103. As a result, the recognition unit 103 performs recognition processing using the current model. Further, for example, when the user taps a "details confirmation" button, a notification thereof is sent from the operation unit 110 to the display unit 109. At this time, the display unit 109 displays detailed information regarding the returned candidate model from the storage unit 105. The detailed information is information necessary for the user to determine whether to select the returned candidate model. The detailed information includes, for example, the date and time information when the returned candidate model was trained, the recognition result when the recognition unit 103 performed recognition processing using this model, and the image of the person used when learning this model. and the feature amount of the image area.

FIG. 3 is a flowchart showing the flow of information processing in the information processing apparatus of this embodiment as described above. The information processing apparatus of this embodiment executes the process shown in the flowchart of FIG. 3 every time one frame of a moving image is input from the imaging unit 101. Note that the frames to be processed according to the flowchart of FIG. 3 are not limited to frames of the moving image from the imaging unit 101. For example, the frames to be processed may be frames of individual video files that have been previously acquired, or videos that have been extracted with a specified date and time range from video files stored in a database or video management system (VMS), etc. It may be a frame of Alternatively, for example, frames of video data sequentially recorded in the storage unit 105 by specifying a video stream of a camera may be used.

First, in step S301, the feature amount calculation unit 102 extracts the feature amount of an image from the frame of the video that the imaging unit 101 is currently shooting. The feature calculation unit 102 detects a human region from the frame image using a known human body detection method, and calculates a HOG (Histogram Of Gradient) for each human body image region to obtain a real value vector. Obtain the features by Note that the method for extracting feature quantities is not limited to this.

Next, in step S302, the recognition unit 103 performs recognition processing using the model selected by the selection unit 108 for each of the feature amounts acquired for each person in step S301, and acquires a recognition score. In the case of this embodiment, a Gaussian mixture distribution model is used as the model. The recognition unit 103 obtains, as a recognition score, a value obtained by substituting the value of the feature amount into the distribution function in the Gaussian mixture distribution model. For example, the recognition unit 103 calculates the recognition score S(x;M) of the feature x using the model M using the following equation (1).

In equation (1), K is the number of mixtures, N(x; μ, σ) is the value of x in a multivariate Gaussian distribution (normal distribution) with mean μ and variance-covariance matrix σ, and ω _i is i of model M. The weight of the ith distribution, μ _i is the mean of the ith distribution of model M, and σ _i is the variance-covariance matrix of the ith distribution of model M. The weights take positive real values, and the sum of ω1 to ωK is 1. The range of the recognition score is a real value in the range [0, 1], where the closer it is to 1, the more normal it is, and the closer it is to 0, the more abnormal it is. Note that the model is not limited to this example, and the model may be, for example, a neural network model or a nearest neighbor model. The nearest neighbor model will be explained in the third embodiment described later.

Next, in step S303, the recognition unit 103 performs processing based on the recognition score calculated in step S302. The recognition unit 103 determines that the person's behavior is abnormal behavior when the recognition score is lower than a predetermined threshold. The display unit 109 then displays an alarm object 206 as shown in the video 204 of FIG. 2(B).
Next, in step S304, the storage unit 105 stores information on the feature amount extracted in step S301.

Next, in step S305, the learning unit 104 determines whether the conditions for performing additional learning are satisfied. If the learning unit 104 determines that the condition is satisfied, the process proceeds to step S306. On the other hand, if it is determined in step S305 that the condition is not satisfied, the information processing apparatus ends the process of the flowchart in FIG. In this embodiment, the condition for executing additional learning is whether the process of the flowchart in FIG. 3 is executed for the first time after 0:00 on that day, for example. That is, additional learning is executed only when the condition that the process of the flowchart of FIG. 3 is executed for the first time after 0:00 on the day is satisfied. As a result, additional learning is performed once every day using the feature amounts obtained on that day.

The conditions for performing additional learning are not limited to the conditions described above. For example, the period as a condition for performing additional learning may be not only every day, but also every half day, or every week. Further, the execution condition for additional learning may be the timing at which a predetermined number or more of the feature amounts saved in step S304 are accumulated. If the predetermined number is, for example, 1000, the learning unit 104 performs additional learning at each timing when 1000 or more feature quantities saved in step S304 are accumulated. Further, the execution condition for additional learning may be, for example, the time point when the feature amount is obtained. In this case, the learning unit 104 always performs additional learning every time a feature amount is obtained. Further, the execution condition for additional learning may be the timing instructed by the user. Furthermore, the execution condition for additional learning may be a combination of two or more of the above-mentioned conditions.

Next, in step S306, the learning unit 104 performs additional learning using the EM algorithm for the model selected by the selection unit 108, using the feature quantities stored in the storage unit 105.
Next, in step S307, the evaluation unit 107 creates and holds a set of recognition targets recognized by the recognition unit 103 as an evaluation set used for performance evaluation. The evaluation set includes a set of feature quantities, a correct label for each feature quantity, and information on weight, which is the degree to which the feature quantity is emphasized. The method for creating the evaluation set will be described later.
Next, in step S308, the evaluation unit 107 determines whether the model selected by the selection unit 108 is degraded based on the evaluation set created in step S307.
At this time, the evaluation unit 107 calculates the recognition scores when the recognition unit 103 recognizes the feature amounts included in the evaluation set created in step S307 using the models before and after the additional learning, and takes a weighted average. . The performance value S(M) of the model M is calculated using the following equation (2).

T in equation (2) is a feature set of the evaluation set, and #T is the number of elements of T. w _x is the weight of the feature x. p _x is +1 if the correct label of x is normal, and -1 if it is abnormal. That is, the performance value S(M) of the model increases as the score is higher when the person's behavior is normal and lower when the behavior of the person is abnormal. For example, if the performance value of the model after additional learning is less than 0.98 times the performance value of the model before additional learning, the evaluation unit 107 determines that the model has deteriorated; otherwise, the model has deteriorated. It is determined that the In the case of this embodiment, even if the performance value decreases by about 2% in order to absorb the blurring in the additional learning, it is not considered as deterioration, but this may be another value such as 1.00 times. In addition, the performance value of the model may be changed depending on the progress of additional learning.If the history of additional learning is small, the performance can be expected to improve further, so it is set to 1.02 times, and after the history has accumulated enough, For example, it may be multiplied by 0.98. Furthermore, the determination of model deterioration is not limited to these examples, and may be performed based on the difference in performance values of the models. Further, the comparison target may be a model showing the maximum performance value in the model history, instead of the performance value of the model before additional learning.

Next, in step S320, the evaluation unit 107 determines whether the determination result in step S308 is degraded or not. If it is determined that the determination result is degraded, the process proceeds to step S309; otherwise, the process proceeds to step S313. move on.
If it is determined in step S320 that there has been no deterioration, in step S313, the management unit 106 adds the model additionally learned in step S306 to the model history. Then, in the next step S314, the selection unit 108 selects the model added in step S313 as the current model. After this step 314, the information processing device ends the process of the flowchart in FIG.

FIG. 4 is a diagram schematically showing model history data managed by the management unit 106.
As shown in FIG. 4, the model history data consists of a string of model information records. The model information record includes a model ID, model data, original model ID, teacher data, and additional learning date and time.
The model ID is a serial number that identifies the model information record.
Although not shown in FIG. 4, the model data is the main body of the model used by the recognition unit 103, and is stored in a format such as binary or XML.
The original model ID is the model ID of the model that is the source when the learning unit 104 performs additional learning. The model ID is "none" if there is no original model, such as a model stored at the time of shipment or a model created during initial settings.
The teacher data is information on teacher data used when the learning unit 104 performs additional learning. The teacher data is a feature quantity stored in the storage unit 105, and the model information record has a set of reference information such as a file name and an inode number. If there is no original model, the training data will be "none".
The additional learning date and time is the date and time when the learning unit 104 performed additional learning.

The selection unit 108 selects one of these model information records as the model used by the recognition unit 103.
Note that as long as it is determined in step S308 that the additionally learned model has not deteriorated, the management unit 106 adds the additionally learned model to the model history, and the selection unit 108 does not select the added model. I will continue. That is, the model on which the latest additional learning has been performed continues to be used in the recognition unit 103.

On the other hand, in step S320, when the evaluation unit 107 determines that the additionally learned model is degraded and proceeds to step S309, the selection unit 108 selects a remand candidate model from among the models managed by the management unit 106. Explore. A method of searching for a return candidate model will be described later.

Next, in step S310, the display unit 109 prompts the user to confirm by displaying information about the remand candidate model searched in step S308. At this time, the user determines whether or not to return to the displayed remand candidate model, and inputs the result of the determination via the operation unit 110.

Next, in step S311, the selection unit 108 determines whether the user has input an instruction to return to the remand candidate model presented in step S310. If an instruction to return to the remand candidate model is input, the selection unit 108 selects the remand candidate model as the model to be used by the recognition unit 103 in step S312.

On the other hand, if the user inputs an instruction not to return to the remand candidate model in step S311, the information processing apparatus of this embodiment ends the process of the flowchart of FIG. 3. In this case, the model selected before performing additional learning in step S306 is used as is by the recognition unit 103. By using the returned candidate model as is, it is guaranteed that the state returns to the state at the time the returned candidate model was obtained. Note that, instead of using the remand candidate model as is, the additional learning performed after the time when the remand instruction was input may be redone for the remand candidate model. That is, when returning to the returned candidate model and using it as is, the model history is followed as is and the model becomes the same as the model created earlier in step S306. Therefore, the learning unit 104 re-sorts the teacher data using the method described in Patent Document 1, for example, and then performs additional learning again. This makes it possible to utilize the teacher data obtained after the point in time when the remand instruction was input. Whether to perform additional learning again or use the returned candidate model as is may be determined by the user's operation.

FIG. 5 is a flowchart showing the flow of processing in which the evaluation unit 107 creates an evaluation set for use in performance evaluation in step S307.
The evaluation unit 107 holds an initial evaluation set made from general human images at the time of shipment. The evaluation unit 107 creates an evaluation set by updating the initial evaluation set as needed using information obtained during operation, in order to adapt the initial evaluation set to the monitoring target location.

FIG. 6 is a diagram showing an example of an evaluation set. As shown in FIG. 6, the evaluation set consists of a column of evaluation data records. The evaluation data record includes an evaluation data ID, a feature amount, a weight, a correct label, and a score history. The evaluation ID is a serial number that identifies the evaluation data record. The weight indicates the importance of the feature amount as evaluation data, and is used to calculate the performance value S(M). The correct label is information on the correct answer for the feature indicated by the feature amount, and is information indicating "normal" or "abnormal". The score history is made up of a string of pairs of model IDs and score values, and includes records of scores when the recognition unit 103 recognizes feature amounts using the model corresponding to the model ID. The contents of the score history are created in step S702 in FIG. 7, which will be described later.

In step S501 of the flowchart of FIG. 5, the evaluation unit 107 first determines and acquires priority evaluation data. The priority evaluation data is a set of feature quantities stored in the storage unit 105 for which the user has canceled an alert using the operation unit 110. These feature quantities are the ones that the user directly judged and instructed as being normal after a false alarm of abnormality occurred, so they have a high level of normality for the user, and the circumstances behind the occurrence of the false alarm are clear. Because it is a feature that is easy to mistake, it is estimated that it is an important feature.

Next, in step S502, the evaluation unit 107 searches the feature quantities included in the storage unit 105 for a feature quantity similar to any of the priority evaluation data. Although the method for calculating the similarity of feature amounts is not limited, for example, an inner product can be used. Since the amount of priority evaluation data is generally small, in order to maintain the accuracy of evaluation, data with similar characteristics to the priority evaluation data is added to prevent the evaluation set from being biased. Note that the evaluation unit 107 may determine the similarity from the images instead of directly determining the degree of similarity of the feature amounts. For example, the evaluation unit 107 may estimate the posture of a human body and determine whether the postures are similar based on the arrangement of human body parts. Furthermore, the evaluation unit 107 may perform general object recognition and determine that objects are similar if they belong to the same category, for example.

Next, in step S503, the evaluation unit 107 adds the similar data obtained in steps S501 and S502 to the evaluation set. Furthermore, the weight of priority evaluation data is set to a large value, such as 10, and the weight of similar data is set to a lower value, such as 2, than the priority evaluation data. Note that the weight of the data included in the initial evaluation set may be set to an even lower value, such as 1, for example.

Next, in step S504, the evaluation unit 107 reduces data with similar score history from the data included in the evaluation set. In other words, since the amount of data increases if data is continuously added to the evaluation set, the evaluation unit 107 uses the similarity of the score history to reduce highly redundant data.

Here, two pieces of data that have similar score history transitions obtained in the process of model history evaluation show similar score change behavior even in different models. Therefore, it is presumed that these data will continue to produce similar scores even with subsequent models created through additional learning. In other words, these data are considered to have strong redundancy for the purpose of model evaluation. The degree of similarity between two score histories is determined, for example, as the root mean square value of the absolute value of the difference between the scores of the same model ID. If the degree of similarity is, for example, 0.01 or less, the evaluation unit 107 determines that the two are similar and deletes one of the two. For example, when the weights are different, the evaluation unit 107 deletes the one with the smaller weight, and when the weights are the same, the evaluation unit 107 deletes the randomly determined one. That is, the evaluation unit 107 uses at least one that is similar. Note that the evaluation unit 107 performs the deletion process in step S504 when the evaluation set is sufficiently small, such as 500 or less, or when the history has not been accumulated enough to make the similarity reliable, such as when the number of model histories is 10 or less. You may choose not to do so.

The evaluation unit 107 creates an evaluation set as described above. Deterioration determination in step S308 in FIG. 3 is performed based on this evaluation set. Further, the search process for a return candidate model in step S309 is performed based on this evaluation set.

FIG. 7 is a flowchart showing the flow of processing in which the selection unit 108 searches for a model to be sent back from the model history in step S309 of FIG.
First, in step S701, the selection unit 108 calculates a performance value S(M) for the evaluation set created in step S307 based on each model included in the model history.
Next, in step S702, the evaluation unit 107 adds the recognition score for each evaluation data calculated in the process of calculating the performance value S(M) in step S701 to the score history of the evaluation data record.

Next, in step S703, the selection unit 108 searches for a model candidate with a maximum performance value from the performance value calculated in step S701. For a model with a maximum performance value, the model M _max with the highest performance value in the model history is first searched for. Next, if there is a model in which the difference between the model M _max and the performance value S (M _max ) is smaller than a predetermined threshold value, the selection unit 108 also selects these models as maximum candidate models. However, if the model M _max is the first model, that is, an initial model on which no additional learning has been performed, the selection unit 108 selects a model for which there is no maximum model candidate, since additional learning has no effect at all. As such, we encourage the re-creation of the initial model.

FIGS. 8(A) to 8(C) are diagrams showing examples of changes in performance values.
For example, as shown in FIG. 8A, when the performance value S (M _max ) of the model M _max with the highest performance value is high and there is no model with a performance value close to it in the model history, the selection unit 108 selects the model M Select _max as is.
For example, as shown in FIG. 8(B), if there are multiple models with performance values close to the performance value S (M _max ) of the model M _max as shown in the frame indicated by the dotted line 804, the model M _max is It is not obvious that simply selecting is effective. Therefore, the display unit 109 presents the change in the performance value transition to the user.
For example, as shown in FIG. 8(C), if the model M _max is an initial model, no effect of additional learning has appeared in the first place. In this case, there may be some abnormality in the initial model.

Returning to the explanation of the flowchart in FIG. 7, in step S704, the selection unit 108 determines the number of candidate models that have the maximum performance value. If the number of candidate models with the maximum performance value is 1, the selection unit 108 directly selects the maximum model as the selection candidate model in step S705, and ends the process. On the other hand, if the number of candidate models with the maximum performance value is 0, that is, the model M _max is the initial model, the selection unit 108 advances the process to step S706 executed by the display unit 109. If there are multiple candidate models with maximum performance values, the selection unit 108 advances the process to step 709, which is executed by the display unit 109.

Proceeding to step S706, the display unit 109 displays a dialog for prompting the user to recreate the model. In the next step S707, the selection unit 108 determines whether the user has instructed, via the operation unit 110, to recreate the model or to return to the initial model. The selection unit 108 proceeds to step 708 when the model has been regenerated based on the user's instructions, and proceeds to step 708 when the regeneration has not been performed, that is, when an instruction to return to the initial model has been given. The process proceeds to step S705.

When the selection unit 108 determines in step S707 that the model has been re-created and proceeds to step S708, the selection unit 108 sets the re-created model as the selection candidate model and ends the process. On the other hand, if the selection unit 108 does not perform re-creation and returns to the initial model and proceeds to step S705, the selection unit 108 ends the process with the maximum model M _max , that is, the initial model, as the selection candidate model.

Further, in step S704, if there are multiple candidate models with maximum performance values and the process proceeds to step S709, the display unit 109 displays a selection screen from the maximum candidate models.
Next, in step S710, the selection unit 108 ends the process with the maximum candidate model selected by the user via the selection screen displayed in step S709 as the selected candidate model.

FIG. 9 is a diagram showing an example of a selection screen from maximum candidates displayed on the display unit 109 in step S709.
In the selection screen of FIG. 9, a combo box 901 has the creation date and time of the maximum candidate model as an item. The user can select one of the maximum candidate models by selecting the creation date and time item in the combo box 901. Thumbnail areas 902 and 903 are areas in which thumbnail images of evaluation data different from the recognition result of the model currently selected for use by the recognition unit 103 in the evaluation set are displayed. The thumbnail area 902 is an area where thumbnail images of data that is abnormal in the current model but normal in the maximum candidate model are displayed. On the other hand, the thumbnail area 903 is an area where thumbnail images of data that are normal in the current model but abnormal in the maximum candidate model are displayed. The thumbnail images displayed in the thumbnail areas 902 and 903 change as the user switches the selection in the combo box 901 using the operation unit 110. The user can look at these and select the maximum candidate model that is considered better while checking how the results change. The button 904 is a button that is tapped when the user further determines the creation date/time maximum candidate model selected from the combo box 901 as the selected candidate model. The selection candidate model selected in this manner is confirmed by the user in step S310, and is subsequently used by the recognition unit 103 as a return candidate model.

As explained above, the information processing device of the first embodiment is capable of reverting to the state at the time when the accuracy was high based on the previous history when it is found that the recognition performance deteriorates due to additional learning. I can do it. That is, the information processing device of the first embodiment can appropriately remove additional learning that lowers recognition accuracy, and can suppress a decrease in recognition accuracy.

<Second embodiment>
In the first embodiment, an example has been described in which an evaluation set is created and a remand candidate model is searched for using the evaluation set. However, if the evaluation set is evaluated for each model in the model history, the amount of calculation may increase.
In the second embodiment, a method of evaluating a model by reusing recognition results during normal operation will be described. The configuration of the information processing apparatus of the second embodiment is the same as that in FIG. 1, so illustration and description thereof will be omitted. Hereinafter, in the second embodiment, only the parts added or changed with respect to the first embodiment will be explained, and the explanation of the parts common to the first embodiment will be omitted.

FIG. 10 is a flowchart showing the flow of processing in the information processing apparatus of the second embodiment. Note that the steps in which the same processing as in the flowchart of FIG. 3 described above is performed are given the same reference numerals as in FIG. 3, and the description thereof will be omitted. The process in the flowchart of FIG. 10 is executed every time one frame of image data of a moving image is input from the imaging unit 101, similarly to FIG. 3. In the case of the flowchart in FIG. 10, while the process in step S307 in FIG. 3 is omitted, the process in step S1001 is added between step S305 and step S306.

If the information processing apparatus of the second embodiment determines in step S305 that the conditions for performing additional learning are satisfied, the process proceeds to step S1001. Proceeding to step S1001, the evaluation unit 107 calculates the performance value of the model currently used by the recognition unit 103 by executing the process shown in the flowchart of FIG. 11, which will be described later. After the processing in step S1001, the processing of the information processing apparatus proceeds to step S306, and additional learning is performed in the same manner as described above in step S306. After step S306, the processing of the information processing device proceeds to step S308.

In the case of the second embodiment, the process of step S307 in FIG. 3 is omitted. Therefore, in the next step S308, the evaluation unit 107 always uses the initial evaluation set in the first embodiment to determine model deterioration. Note that, as in the first embodiment, step S307 may be provided to update the evaluation set, but in that case, the evaluation set will be used only in this step.

FIG. 11 is a flowchart showing the flow of processing in which the evaluation unit 107 calculates the performance value S(M) of the model M currently used by the recognition unit 103 in step S1001 of FIG.
First, in step S1101, the evaluation unit 107 acquires the feature quantities whose recognition scores were calculated in step S302 from the storage unit 105, and performs clustering. Here, clustering can be considered as a process of classifying feature amounts based on the attributes of the recognition target. For example, when the recognition target is a person, clustering is a process of detecting posture as an attribute of the person and classifying the features by posture category such as walking, running, or sitting. It will be done. Clustering is a process that classifies feature amounts according to categories of person attributes, such as the person's walking direction, walking speed, distance from the camera, height, age, gender, clothing, belongings, or person density. You can. Furthermore, clustering may be a process of classifying feature amounts based on a combination of two or more highly related attributes, such as gender and clothing, clothing and belongings, age, sex, and walking speed. Alternatively, clustering may be a process of classifying feature amounts using, for example, the k-means method without prior knowledge.

Next, in step S1102, the evaluation unit 107 determines important clusters from the clusters subjected to clustering in step S1101. For example, the evaluation unit 107 selects the cluster with the largest size and determines it as an important cluster. For example, if the cluster of running people is large, the monitoring target location is considered to be an environment where many running people appear. In this case, it is presumed that it is reasonable to determine important clusters using a running person as a standard for model evaluation. Furthermore, the method of selecting important clusters used for evaluation is not limited to this. For example, if the user determines that a sitting person is important based on prior knowledge, the evaluation unit 107 determines an important cluster from the clusters of sitting people. Further, as the important cluster, a cluster in which an abnormality is likely to occur may be selected. Furthermore, a plurality of these clusters may be combined.

Next, in step S1103, the evaluation unit 107 calculates a performance value S(M) using the important cluster determined in step S1102 and the recognition score calculated in step S302 for the feature amount included therein. The correct label when calculating the performance value is that features whose warnings have been canceled by the user using the operation unit 110 are treated as normal, and those that have not been canceled are treated as abnormal if an alarm has been generated. is considered normal.

Next, in step S1104, the evaluation unit 107 stores the current model performance value calculated in step S1103 in the storage unit 105 in association with the model history record.
In the second embodiment, the performance value of the model currently used by the recognition unit 103 is calculated as described above.

In the case of the information processing apparatus of this embodiment, the performance value of the model has already been calculated in step S1001. For this reason, the information processing apparatus of the second embodiment performs the processing in the flowchart that omits the processing in step S701 and step S702 in FIG. 7 described above. That is, in the case of the second embodiment, the processing in steps S701 and S702 in FIG. 7 is omitted, and the selection unit 108 starts the processing in the flowchart from step S703, and performs the processing from step S703 onwards.

In this manner, the information processing apparatus of the second embodiment does not use an evaluation set, and determines selection candidate models based on performance in important clusters when selecting models. Although comparing performance using important clusters does not compare performance using the same data as using evaluation sets, it is expected that data with somewhat similar properties will be gathered through clustering. Furthermore, the recognition score of the important cluster uses the recognition result calculated in step S302 during normal operation. Therefore, the information processing device of the second embodiment does not require the recognition unit 103 to perform new calculations when selecting a model, and is useful in an environment where the amount of calculation is strongly restricted.

Note that although the second embodiment describes a method in which the evaluation set used in the first embodiment is not used in model selection, a method that partially uses the evaluation set may also be used. For example, the information processing device may separately obtain performance values from a small evaluation set and combine the performance values from important clusters, or limit the contents of the evaluation set to those similar to the important clusters. . In this case, the amount of calculation for the evaluation set can be reduced.

<Third embodiment>
The first embodiment has been described using a Gaussian mixture distribution as an example of the model. In the third embodiment, an example will be described in which the storage capacity of model history and the search for remand candidate models can be reduced by using a nearest neighbor model. The configuration of the information processing device of the third embodiment is the same as that in FIG. 1, so illustration and description thereof will be omitted. Hereinafter, in the third embodiment, only the parts added or changed with respect to the first embodiment will be explained, and the explanation of the parts common to the first embodiment will be omitted. Note that the method described in the third embodiment is also applicable to the second embodiment described above.

In the case of the third embodiment, the model is a set of points in feature space. Furthermore, the recognition score in the third embodiment is the distance to the point closest to the feature amount (nearest neighbor). In the case of the third embodiment, the recognition score S(x;M) of the feature x when using the model M is calculated by the following equation (3).

Here, K in equation (3) is the number of points in the model, y _i is the vector value of the points included in the model, and d(y, x) is the Euclidean distance between y and x. The recognition score takes a real value in the range of (0, 1), and the larger the score, the more normal it is, and the smaller it is, it is abnormal. Note that the range of d is [0, ∞). Here, since it is handled as a score, the range of d is transformed to take a range from 0 to 1, but the method is not limited to the one shown here. Further, the range of d is not limited to Euclidean distance, but may be any range as long as it satisfies the axiom of pseudo-distance, such as Manhattan distance or Chebyshev distance. Furthermore, the method of this embodiment can be similarly used with a method of approximating using Hamming distance using a method such as LSH (Locally-Sensitive Hashing). Further, in the case of the third embodiment, the generation number of additional learning for each point of the model is also held as metadata. In the initial model, all points in the model have generation number 0. Further, in the third embodiment, the additional learning in step S306 shown in FIG. 3 is performed by adding points of the feature amount whose correct answer label is normal to the model. The generation number is incremented each time step S306 is executed, and is given as metadata to the points added by additional learning.

The management unit 106 in the third embodiment does not hold model data in the model information record, but instead holds a generation number. Furthermore, the management unit 106 has model data of a single model corresponding to the latest model, and reproduces other historical model data from the model data of the latest model.

FIG. 12 is a diagram illustrating a history management method of the management unit 106 in the third embodiment.
As shown in FIG. 12, the model history data managed by the management unit 106 consists of a column of model information records 1201. The model information record 1201 includes a generation number, original model ID, teacher data, and additional learning date and time. That is, the model history data in the third embodiment does not include the model data of the model history data shown in FIG. 4, but instead includes a generation number.
The generation number is a number that is incremented every time additional learning is performed. However, the generation number does not necessarily match the model ID if the history is deleted or branched.

The management unit 106 of the third embodiment separately manages model data 1202 of the latest model. The model data 1202 of the latest model includes vector values representing points of the model, and a generation number is assigned as metadata to each vector value representing the points of the model. Note that in FIG. 12, specific vector values are omitted. A subset of vector values with the same generation number forms a set of vector values added with the same generation number.

For example, when obtaining model data with a generation number of 99, the management unit 106 extracts only vector values with generation numbers of 99 or less from the model data of the latest model. As a result, a set of vector values added through additional learning from the initial model to generation number 99, ie, model data of generation number 99, is obtained. In the example of FIG. 12, vector values with IDs from 9501st to 10000th are values added by additional learning corresponding to generation number 100. Therefore, when obtaining model data with generation number 99, the management unit 106 extracts each vector value with IDs from 1 to 9500, excluding the vector value with generation number 100, to obtain model data of generation number 99. Obtain model number 99.

FIG. 13 is a flowchart showing the flow of processing when the information processing apparatus according to the third embodiment calculates the performance value S(M) of the model. In the case of the third embodiment, the information processing device performs the process of the flowchart shown in FIG. 13 instead of calculating the performance values of the models of all model histories as described in step S701 of FIG. 7 described above. This increases the efficiency of calculating model performance values. Hereinafter, with reference to the flowchart of FIG. 13, a method for the information processing apparatus of the third embodiment to calculate the performance value of the model will be described. In the following description, the generation number is assumed to be g, and the model of the generation number g is assumed to be M _g .

First, in step S1301, the selection unit 108 acquires the generation number of the latest model from the management unit 106, and sets it as generation number G.
Next, in step S1302, the selection unit 108 acquires the model M _G with generation number G from the management unit 106. That is, as described above, the selection unit 108 acquires a set of vector values whose generation numbers are G or less from the management unit 106 and sets it as a model M _G .
Next, in step S1303, the evaluation unit 107 calculates the recognition score S(x; M _G ) using the above-mentioned formula. Furthermore, the evaluation unit 107 stores the point y of the model M _G at which the nearest neighbor y determined in the calculation process, that is, d(y, x) is the minimum.

Next, in step S1304, the selection unit 108 obtains the generation number of the nearest neighbor y obtained in step S1303, and sets it as the generation number G'.
Next, in step S1305, the selection unit 108 stores the recognition score of x in the models from generation number G' to generation number G as the value of recognition score S(x; M _G ) obtained in step S1303. That is, the recognition score S(x; M _g ) of the generation number g is the value of the following equation (4).

S(x; M _g )=S(x; M _G )G'≦g≦G Formula (4)

The nearest neighbor y is the vector value added in the additional learning of the generation number G', and since it is still the nearest neighbor in the model of the generation number G, it is the vector value added in the additional learning from the generation number G'+1 to the generation number G. Vector values closer to x than the neighbor y are not added. Therefore, it can be seen that the nearest neighbors of the models from generation number G' to generation number G are all y and take the same value S(x; M _G ).

Next, in step S1306, the selection unit 108 determines whether the generation number G'=0. If the generation number G'=0, the selection unit 108 ends the process of the flowchart of FIG. 13 because recognition scores have already been calculated for all models up to the initial model. On the other hand, if the generation number G' is not 0, the selection unit 108 advances the process to step S1307. Proceeding to step S1307, the selection unit 108 sets the generation number G to G'-1, returns to the process of step S1302, and calculates recognition scores for models before the generation number G'-1.

The information processing device of the third embodiment can calculate the recognition score S(x; M _g ) for each model included in the model history as described above. The information processing apparatus of the third embodiment can omit the calculation of the models with generation numbers G' to G-1 in step S1305, so when sequential calculation is performed for each model of the model history as in the embodiment described above. The calculation efficiency is higher than that of For example, even in the worst case where the nearest neighbor with generation number G'=G is always selected in step S1305, the calculation efficiency is equivalent to the sequential calculation described above.
By performing the above-described processing, the information processing apparatus according to the third embodiment can improve the efficiency of calculation of model history data size and model performance when using the nearest neighbor model.

<Fourth embodiment>
In the case of the third embodiment described above, the additional learning in step S306 increases the number of normal data that can be nearest neighbors by adding training data whose correct answer label is normal to the model. In contrast, the information processing device of the fourth embodiment performs additional learning that lowers the score in the vicinity of the teaching data by using the teaching data whose correct answer label is abnormal and reducing the normal data of the model. possible. The model according to the fourth embodiment works toward making it easier to recognize unrecognized abnormalities. The addition of a method for performing additional learning to reduce normal data in the fourth embodiment will be described below. Note that since the configuration of the information processing apparatus according to the fourth embodiment is the same as that in FIG. 1, illustration and description thereof will be omitted. Hereinafter, in the fourth embodiment, only the parts added or changed from the above-described embodiment will be explained, and the explanation of the parts common to the above-described embodiment will be omitted.

In the case of the fourth embodiment, in step S306, the learning unit 104 first adds points of the feature amount whose correct answer label is normal to the model, as in the third embodiment. Then, if there are features whose correct labels are abnormal in the training data, the learning unit 104 calculates the distance d from each of the vector values of the points of the model for each feature, and determines whether any of the distances is a predetermined threshold value. Points in the model that are smaller than t are deleted. That is, the learning unit 104 deletes points of the model included in a hypersphere having a radius (threshold t) centered on the teacher data.

For example, the learning unit 104 deletes by further adding the generation number at the time of deletion as metadata of the model point. Further, the learning unit 104 increments the generation number added as the generation number at the time of deletion to make it the next generation after the generation for additional learning by adding another correct answer data. That is, all additional learning generations with the same generation number are either added or all deleted.

FIG. 14 is a diagram illustrating a history management method of the management unit 106 in the fourth embodiment.
The model history data managed by the management unit 106 consists of a column of model information records 1201. The model information record 1201 is similar to that described in the third embodiment. In the model data 1401, a generation number and a generation number at the time of deletion are assigned as metadata to each vector value representing a point of the model. Note that for vector values that have not been deleted, the generation number at the time of deletion is "none". In the case of the example shown in FIG. 14, the second and 4890th vector values of the ID are deleted by additional learning (by deletion) of the generation number 101. In other words, for example, the vector value with the 4890th ID is data that was added at generation number 55 and deleted at generation number 101. That is, for example, the vector value with the 4890th ID is a vector value that exists in models from generation number 55 to generation number 100.
In this embodiment, in order to obtain model data with generation number 99 from this model data, only vector values whose generation number is 99 or less and whose generation number at the time of deletion is "none" or whose generation number is greater than 99 are required. All you have to do is extract it.

FIG. 15 is a flowchart showing the flow of processing when the information processing apparatus according to the fourth embodiment calculates the performance value S(M) of the model. Hereinafter, with reference to the flowchart of FIG. 15, the information processing apparatus of the fourth embodiment calculates the recognition score S(x; _{M g} ) will be explained. In the following explanation, processing different from the flowchart shown in FIG. 13 will be explained.

After step S1304, the information processing apparatus according to the fourth embodiment proceeds to step S1501. In the case of the third embodiment described above, the nearest neighbor y searched in step S1303 is the nearest neighbor all the way from the generation number G', which is the time when the nearest neighbor y was added, to the generation number G. On the other hand, in the case of the fourth embodiment, the point deleted from generation number G' to generation number G may be closer than the nearest neighbor y, so the selection unit 108 searches for it. Then, in step S1501, the selection unit 108 sets the generation number H to G'. Thereafter, the generation number H is a variable that starts from the generation number G' and increases up to the generation number G in the series of steps S1501 to S1507.

Next, in step _S1502 , the evaluation unit 107 calculates d(z, , x) is searched for and stored. Then, if such a nearest neighbor z exists, the evaluation unit 107 calculates the recognition score S(x; M _H ) using the above-mentioned formula. The nearest neighbor z is closer to x than y, but it does not exist at the time of generation number G (because y, which is further away than z, is the nearest neighbor), so the nearest neighbor z was deleted between generation numbers H and G. It is a point. Conversely, candidates for the nearest neighbor z are limited to such points, so distances are calculated only for points deleted between generation numbers H and G, that is, points whose generation number at the time of deletion is greater than or equal to H and less than or equal to G. This makes it possible to efficiently search for the nearest neighbor z.

Next, in step S1503, the selection unit 108 determines whether the nearest neighbor z in step S1502 has been found. If the selection unit 108 does not find the nearest neighbor z, the selection unit 108 advances the process to step S1504.
Here, if the nearest neighbor z is not found, there is no deleted data between generation number H and generation number G that is closer than y, so the data between generation number H and generation number G is It can be seen that all the nearest neighbors are y. Therefore, in step S1504, the selection unit 108 stores the recognition score of x in the models from generation number H to generation number G as the value of recognition score S(x; M _G ) obtained in step S1303. The selection unit 108 then proceeds to processing in step S1306.

On the other hand, if the nearest neighbor z is found in step S1503, the selection unit 108 advances the process to step S1505. Proceeding to step S1505, the selection unit 108 obtains the generation number of the nearest neighbor z at the time of deletion, and sets it as the generation number H'. It can be seen that the nearest neighbor z is the nearest neighbor from generation number H to generation number H'-1, which is immediately before deletion.

Next, in step S1506, the selection unit 108 stores the recognition score of x in the models from generation number H to generation number H'-1 as the value of recognition score S(x; M _H ) obtained in step S1502. .
Next, in step S1507, the selection unit 108 sets the generation number H to H', and then returns to the process of step S1502.

In the fourth embodiment, all the recognition scores of the feature amount x are determined for the models from generation number G' to generation number G through the series of processes described above. This is similar to the effect of step S1305.
According to the fourth embodiment, as described above, the processing of additional learning by deletion is added to the processing of the third embodiment, and the processing is performed efficiently like the third embodiment described above. It can be performed.

<Other embodiments>
The present invention provides a system or device with a program that implements one or more functions of each of the above-described embodiments via a network or a storage medium, and one or more processors in the computer of the system or device reads the program. It can also be realized by executing processing. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.
The above-described embodiments are merely examples of implementation of the present invention, and the technical scope of the present invention should not be construed as limited by these embodiments. That is, the present invention can be implemented in various forms without departing from its technical idea or main features.

101: Photographing section, 102: Feature calculation section, 103: Recognition section, 104: Learning section, 105: Storage section, 106: Management section, 107: Evaluation section, 108: Selection section, 109: Display section, 110: Operation Department

Claims (18)

a recognition means for recognizing features from an image based on the model;
learning means for additionally learning the model;
a management means for managing information on the history of additional learning of the model;
evaluation means for evaluating the accuracy of the recognition by the model;
a selection means for selecting a model based on the recognition accuracy from among the models for which the history information is managed;
The evaluation means holds an evaluation set that records a collection of recognition target information recognized by the recognition means,
The selection means selects an additionally learned model based on a result of evaluating accuracy using a subset of information on the recognition target extracted from the evaluation set;
The information processing apparatus is characterized in that the recognition means uses the model selected by the selection means for the recognition. The evaluation means includes a feature amount of the recognition target recognized by the recognition means, an ID of the model, and a score regarding the degree of the feature of the recognition target obtained as a result of recognition of the recognition target using the model. A set of correspondences is maintained as an evaluation set,
The selection means selects models included in the evaluation set based on the results of evaluating the accuracy of recognition of the evaluation set by the recognition means using each model for which the history information is managed. The method is characterized in that a model that has been additionally learned is selected using an evaluation set generated by a subset of recognition targets excluding any of a plurality of recognition targets whose IDs and scores of the models are similar to each other. The information processing device according to claim 1. further comprising display control means for displaying the image on a display device,
3. The information processing apparatus according to claim 1, wherein the display control means displays information on the recognition target included in the evaluation set. a recognition means for recognizing features from an image based on the model;
learning means for additionally learning the model;
a management means for managing information on the history of additional learning of the model;
evaluation means for evaluating the accuracy of the recognition by the model;
a selection means for selecting a model based on the recognition accuracy from among the models for which the history information is managed;
The selection means selects a part of the recognition target recognized by the recognition means,
The evaluation means evaluates the accuracy of the recognition based on the recognition results for a part of the selected recognition target,
The information processing apparatus is characterized in that the recognition means uses the model selected by the selection means for the recognition. The selection means clusters the recognition target recognized by the recognition means into a plurality of clusters, and selects at least one or more clusters from the plurality of clusters,
5. The information processing apparatus according to claim 4, wherein the evaluation means evaluates the accuracy of the recognition based on a recognition result of a recognition target belonging to the selected cluster. The information processing apparatus according to any one of claims 1 to 5, wherein the selection means selects a model with maximum accuracy in the accuracy transition in the history. further comprising a selection input means for a user to select a model;
The selection means presents the user with a plurality of models determined based on changes in accuracy in the history, and the user selects a model based on the result input by the selection input means. The information processing device according to any one of claims 1 to 5. 6. The information processing apparatus according to claim 1, wherein the selection means determines whether it is necessary to recreate the model based on a change in accuracy in the history. 6. The information processing apparatus according to claim 5 , wherein the selection means clusters the recognition targets based on attributes of the recognition targets. The learning means performs additional learning by adding or deleting a recognition target to the model,
The management means manages the history of the model by assigning a generation number representing a generation in which the additional learning has been performed to the recognition target,
According to any one of claims 1 to 9 , the recognition means determines a subset of data included in the model based on the generation number, and sets the subset as a model included in the history. The information processing device described. 11. The information processing apparatus according to claim 1, wherein the model includes a nearest neighbor model. The model is a collection of data indicating the recognition target in a feature space,
The recognition accuracy is a score indicating the distance to the point closest to the data of the predetermined recognition target when using the model with the generation number,
The selection means selects data of the recognition target that minimizes the score based on the data of the recognition target included in each model from the generation number of the latest model to the initial model and the predetermined recognition target. Select the model containing
If the generation number of the selected model is different from the generation number of the initial model, the evaluation means may exclude the recognition target data included in the models after the generation number of the selected model. evaluating the score based on the data of the recognition target and the predetermined recognition target, and when the generation number of the selected model is the generation number of the initial model, terminating the evaluation; The information processing device according to claim 10 . further comprising display control means for displaying the image on a display device,
13. The display control means displays information indicating objects recognized as normal and objects recognized as abnormal by the recognition means, respectively. The information processing device described in . The display control means further displays information for instructing the user to cancel the information, out of the information indicating the object recognized as having an abnormality by the recognition means. 14. The information processing device according to 13 . When there are a plurality of models selected by the selection means, the display control means includes information indicating the time when the plurality of models were generated, the recognition result of the current model, and the recognition of each of the plurality of models. The information processing apparatus according to claim 13 or 14 , wherein an image of an object having a different result is displayed. An information processing method executed by an information processing device, the method comprising:
a recognition step of recognizing features from the image based on the model;
a learning step of performing additional learning of the model;
a management step of managing information on the history of additional learning of the model;
an evaluation step of evaluating the recognition accuracy of the model;
a selection step of selecting a model based on the recognition accuracy from among the models for which the history information is managed;
The evaluation step includes maintaining an evaluation set that records a collection of information to be recognized;
The selection step selects an additionally learned model based on a result of evaluating accuracy using a subset of information on the recognition target extracted from the evaluation set;
The information processing method is characterized in that the recognition step uses the model selected in the selection step for the recognition. An information processing method executed by an information processing device, the method comprising:
a recognition step of recognizing features from the image based on the model;
a learning step of performing additional learning of the model;
a management step of managing information on the history of additional learning of the model;
an evaluation step of evaluating the accuracy of the recognition by the model;
A model is selected based on the recognition accuracy from among the models for which the history information is managed.
a selection step of selecting;
The selection step selects a part of the recognition target to be recognized by the recognition step,
The evaluation step evaluates the accuracy of the recognition based on the recognition results for a part of the selected recognition target,
The information processing method is characterized in that the recognition step uses the model selected in the selection step for the recognition. A program for causing a computer to function as each means of the information processing apparatus according to claim 1 .

Images