WO2020049636A1 - Système d'identification, procédé de présentation de modèle et programme de présentation de modèle - Google Patents

Système d'identification, procédé de présentation de modèle et programme de présentation de modèle Download PDF

Info

Publication number
WO2020049636A1
WO2020049636A1 PCT/JP2018/032761 JP2018032761W WO2020049636A1 WO 2020049636 A1 WO2020049636 A1 WO 2020049636A1 JP 2018032761 W JP2018032761 W JP 2018032761W WO 2020049636 A1 WO2020049636 A1 WO 2020049636A1
Authority
WO
WIPO (PCT)
Prior art keywords
model
identification
unit
identification system
image
Prior art date
Application number
PCT/JP2018/032761
Other languages
English (en)
Japanese (ja)
Inventor
芙美代 鷹野
竹中　崇
誠也 柴田
浩明 井上
高橋　勝彦
哲夫 井下
Original Assignee
日本電気株式会社
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 日本電気株式会社 filed Critical 日本電気株式会社
Priority to JP2020540901A priority Critical patent/JP6981553B2/ja
Priority to PCT/JP2018/032761 priority patent/WO2020049636A1/fr
Publication of WO2020049636A1 publication Critical patent/WO2020049636A1/fr

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N99/00—Subject matter not provided for in other groups of this subclass
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis

Definitions

  • the present invention relates to an identification system for identifying an object represented by data by applying the data to the model, and a model providing method and a model providing program for the identification system to provide a model to another identification system.
  • a general identification system learns a model in advance by machine learning using, as teacher data, a set of an image obtained by photographing with a camera included in the identification system and a label representing an object appearing in the image. . Then, the general identification system identifies an object shown in the image by applying an image newly obtained by photographing by the camera to the model.
  • Such a general identification system is used for the purpose of detecting a suspicious vehicle or a suspicious person, preventing crime, etc., detecting a user using a white stick or a wheelchair, and using a white stick or a wheelchair. It is used for the purpose of support such as guiding people.
  • the identification system for identifying an object appearing in an image has been described as an example, but an identification system for identifying an object represented by voice data is also conceivable as a general identification system.
  • an identification system for identifying an object appearing in an image will be described as an example.
  • Patent Document 1 describes an image recognition method that avoids prolonged additional learning due to a difference in an imaging environment.
  • the image recognition method described in Patent Document 1 is an image recognition method in a camera system including a plurality of camera devices.
  • a first image and first imaging environment information are acquired from a first camera device.
  • a parameter table that manages imaging environment information indicating each imaging environment when each camera device has previously captured an image and each recognition control parameter indicating each detector function corresponding to each imaging environment.
  • a first recognition control parameter indicating a first detector function corresponding to the same or similar imaging environment as the first imaging environment indicated in the first imaging environment information.
  • the first image obtained from the first camera device is recognized using the first detector function indicated by the first recognition control parameter.
  • Patent Document 2 discloses an image monitoring device.
  • the image monitoring device described in Patent Literature 2 normalizes a face area using the detected facial feature points and performs matching with a person dictionary.
  • a bias occurs in how an object is captured in an image obtained by one camera by shooting.
  • one camera has many opportunities to photograph a car traveling from right to left as viewed from the camera, but has few opportunities to photograph a car traveling in the opposite direction.
  • many images of a car traveling in the right-to-left direction are obtained, but only a few images of a car traveling in the opposite direction are obtained.
  • the teacher data includes many images of vehicles traveling in the right-to-left direction, and includes only a few images of vehicles traveling in the opposite direction.
  • the present invention provides an identification system that can re-learn its own model so as to improve the identification accuracy of its own model, and can also contribute to improving the identification accuracy of models of other identification systems, It is an object to provide a model providing method and a model providing program.
  • An identification system includes: learning means for learning a model for identifying an object represented by data using teacher data; first model storage means for storing a model learned by the learning means; First identifying means for identifying an object represented by data using a model learned by the first and second model storages for storing individual models learned by a plurality of predetermined first identification systems, respectively.
  • Model updating means for updating to a model received from the first identification system; and, in a predetermined case, for each model stored in the second model storage means,
  • a second identification means for identifying an object represented by the data identified by the first identification means;
  • a learning means comprising: a label for data determined based on an identification result derived by the second identification means; Is re-learned using the teacher data including the following, the model stored in the first model storage unit is updated to the re-learned model, and the model learned by the learning unit is set to a predetermined one.
  • a model transmitting means for transmitting to one or more second identification systems.
  • a model for identifying an object represented by data is learned using teacher data, the model is stored in the first model storage unit, and the model is stored in the first model storage unit.
  • a model learned by the first identification system is received from the first identification system, the model learned by the first identification system stored in the second model storage means is stored in the second model storage unit.
  • the model is updated to the model received from the first identification system, and in a predetermined case, for each model stored in the second model storage means, the data to be identified in the first identification process is determined.
  • the model providing program is a learning process for causing a computer to learn a model for identifying an object represented by data using teacher data, and to store the model in a first model storage unit.
  • a first identification process for identifying an object represented by data using a model stored in the model storage means, and individual models learned by a plurality of predetermined first identification systems are respectively assigned to a second model. Processing to be stored in the storage means; when a model learned by the first identification system is received from the first identification system, the model is learned by the first identification system stored in the second model storage means.
  • Model updating processing for updating the model that has been updated to the model received from the first identification system, and in a predetermined case, the individual model stored in the second model storage means.
  • a second identification process for identifying an object represented by data to be identified in the first identification process, a label for data determined based on an identification result derived in the second identification process, and the data
  • a re-learning process for re-learning the model using the included teacher data and updating the model stored in the first model storage unit to the re-learned model, and stored in the first model storage unit
  • a model transmission process for transmitting the model to one or more predetermined second identification systems is performed.
  • the present invention it is possible to re-learn the own model so as to improve the identification accuracy of the own model, and to contribute to the improvement of the identification accuracy of the model of another identification system.
  • FIG. 1 is a block diagram illustrating a configuration example of an identification system 100 according to an embodiment of the present invention. It is a schematic diagram which shows the example of an internally generated model and an externally generated model. It is a schematic diagram which shows the example of the screen which a determination part displays on a display apparatus in the 1st determination method. It is a schematic diagram which shows the example of the screen which a determination part displays on a display apparatus in the 3rd determination method.
  • FIG. 4 is an explanatory diagram illustrating a specific example of a first calculation method.
  • FIG. 9 is an explanatory diagram illustrating a specific example of a second calculation method.
  • 11 is a flowchart illustrating an example of processing progress from the time when the camera captures an image to the time when a second identification unit performs an identification process on an image.
  • FIG. 11 is a schematic block diagram illustrating a configuration example of a computer included in an identification system according to an embodiment of the present invention or a modified example thereof. It is a block diagram showing the outline of the identification system of the present invention.
  • FIG. 1 is a schematic diagram showing a situation where a plurality of identification systems of the present invention are provided.
  • FIG. 1 illustrates a case where six identification systems 100 are provided at various locations, but the number of identification systems 100 provided at various locations is not particularly limited. In the present embodiment, a description will be given assuming that the plurality of identification systems 100 have the same configuration.
  • Each identification system 100 can communicate, for example, via a communication network.
  • Each of the identification systems 100 includes a data collection unit (a data collection unit 101 shown in FIG. 3 described later).
  • the data collection units (not shown in FIG. 1; see FIG. 3 described later) of each identification system 100 are installed in various places where data is collected.
  • the data collection unit collects data at a location where the data collection unit is installed.
  • the data collection unit collects image and audio data at the installation location.
  • the data collection unit is realized by a camera or a microphone.
  • the data collection unit may collect images by photographing a monitoring place.
  • audio data may be collected by recording at an installation location.
  • Each individual identification system 100 includes a computer separate from the data collection unit, and the computer identifies an object represented by data (image, audio data, and the like).
  • Each identification system 100 learns a model using data collected by the data collection unit as teacher data.
  • This model is a model for identifying an object represented by data.
  • each identification system 100 provides a model, and re-learns its own model by using a model provided by another identification system 100.
  • Another identification system 100 that transmits a model to the identification system 100 of interest is predetermined.
  • Another identification system 100 that transmits a model to the identification system 100 of interest is referred to as a first identification system.
  • a description will be given assuming that a plurality of first identification systems are determined for the identification system 100 of interest. It can be said that the identification system 100 of interest receives the model from the first identification system.
  • another identification system 100 that is a transmission destination when the identification system 100 of interest transmits the model is predetermined.
  • Another identification system 100 that is a transmission destination when the identification system 100 of interest transmits the model is referred to as a second identification system.
  • a description will be given assuming that one or more second identification systems are determined for the identification system 100 of interest.
  • the identification system 100 of interest can be said to provide a model to the second identification system.
  • the first identification system and the second identification system are predetermined for each individual identification system 100.
  • an administrator who manages each identification system 100 may determine a first identification system and a second identification system for each individual identification system 100 in advance.
  • Each identification system 100 has the same configuration, but the first identification system and the second identification system are individually determined for each identification system 100.
  • FIG. 2 is an explanatory diagram showing an example of the first identification system and the second identification system.
  • the identification system 100a shown in FIG.
  • arrows indicate the direction in which the model is sent.
  • the identification systems 100b and 100c are defined as first identification systems
  • the identification systems 100b, 100c and 100d are defined as second identification systems.
  • a first identification system and a second identification system are individually defined.
  • FIG. 3 is a block diagram showing a configuration example of the identification system 100 according to the embodiment of the present invention.
  • the identification system 100 includes a data collection unit 101 and a computer 102.
  • the data collection unit 101 and the computer 102 are communicably connected by wire or wirelessly.
  • a case where the data collection unit 101 is a camera will be described as an example, and the data collection unit 101 will be referred to as a camera 101.
  • the camera 101 performs shooting from the installation location of the camera 101.
  • the installation location of the camera 101 and the installation location of the computer 102 may be different.
  • the computer 102 includes a learning unit 103, a first model storage unit 104, a data acquisition unit 105, a first identification unit 106, a determination unit 107, an area correction GUI (Graphical User Interface) display control unit 108, , Region extraction unit 109, second model storage unit 110, model update unit 121, second identification unit 111, display control unit 112, attribute data storage unit 113, integration unit 114, display device 115, a mouse 116, a result storage unit 117, and a model transmission unit 122.
  • GUI Graphic User Interface
  • the learning unit 103 learns a model by machine learning using the image obtained by the camera 101 as the teacher data.
  • the teacher data is, for example, a set of a set of an image obtained by photographing with the camera 101 and a label indicating an object appearing in the image.
  • the label may be determined by the operator of the identification system 100.
  • a model is learned (generated) using such a set of sets as teacher data.
  • the learning unit 103 adds a set of the image and the label to the teacher data, and re-learns the model by deep learning.
  • the predetermined image is an image determined by the determination unit 107 described later to cause the second identification unit 111 to execute the identification processing. Note that, in the present embodiment, an example will be described in which a region in which an object is extracted is extracted from an image determined as described above, and a pair of an image of the extracted region and a label is added to teacher data. I do.
  • a model generated by the learning unit 103 through learning may be hereinafter referred to as an internally generated model.
  • the second model storage unit 110 stores a model similarly generated by the first identification system predetermined for the identification system 100 shown in FIG.
  • a model generated by the first identification system may be referred to as an externally generated model to distinguish it from an internally generated model.
  • a model updating unit 121 described below receives the model from the first identification system, and stores the model in the second model storage unit 110 as an externally generated model.
  • the internally generated model and the externally generated model are models for identifying an object appearing in a given new image.
  • the objects appearing in the image are “car”, “motorcycle”, “bus”, and “background (that is, car, motorcycle and bus are not shown)”.
  • the model is a model for determining which one.
  • the operator determines one of “automobile”, “motorcycle”, “bus”, and “background” as a label to be paired with an image in the teacher data for each image.
  • the operator of the first identification system may use the “automobile”, “motorcycle”, “bus”, One of the “background” is determined for each image.
  • the first identification unit 106 determines whether the object shown in the image is “car”, “motorcycle”, “bus”, or “background” using a model is described.
  • the target determined using the model is not limited to “automobile”, “motorcycle”, “bus”, and “background”.
  • the operator may prepare teacher data according to the purpose of the identification processing, and let the learning unit 103 learn the model using the teacher data.
  • the objects to be determined using the model are common to each identification system 100.
  • the learning unit 103 causes the first model storage unit 104 to store the internally generated model generated by the deep learning.
  • the first model storage unit 104 is a storage device that stores an internally generated model.
  • FIG. 4 is a schematic diagram illustrating an example of an internally generated model and an externally generated model.
  • the image can be represented as a vector (X1, X2,..., Xn) T having each pixel value of n pixels as an element.
  • X1 represents the pixel value of the first pixel in the image.
  • T means transposition.
  • the model has a plurality of layers and includes a plurality of coefficients for each layer. In the example shown in FIG. 4, the first layer includes coefficients a1 to am, and the second layer includes coefficients b1 to bj.
  • the individual elements X1 to Xn of the vector representing the image are associated with the respective coefficients a1 to am of the first layer.
  • this association is represented by a line.
  • each coefficient of a certain layer is associated with each coefficient of the next layer.
  • this association is also represented by a line.
  • a weight is determined between the associated elements. For example, weights are determined for the associated a1 and b1, the associated a1 and b2, and the like.
  • the learning unit 103 determines the number of layers, the number of coefficients included in each layer, the value of each coefficient in each layer, and the value of the weight between associated elements by performing deep learning using the teacher data. . Determining these values corresponds to generating an internally generated model.
  • the internally generated model and the externally generated model can be represented as shown in the format illustrated in FIG. 4, but the number of layers, the number of coefficients included in each layer, the value of each coefficient in each layer, the , Etc., differ between the internally generated model and the externally generated model.
  • the second model storage unit 110 stores each of the externally generated models learned by the plurality of first identification systems. Each of the externally generated models is also generated based on different teacher data in different identification systems 100, and thus the number of layers and the like are different for each externally generated model.
  • the data acquisition unit 105 acquires a new image obtained by the camera 101 by shooting from the camera 101.
  • the data acquisition unit 105 is an interface for receiving an image from the camera 101.
  • the first identification unit 106 applies the image to an internally generated model stored in the first model storage unit 104, thereby Identify the object represented by the image.
  • the first identification unit 106 applies the image to the internally generated model to determine whether the object shown in the image is “car”, “motorcycle”, or “bus”. Alternatively, it is determined whether only the “background” is captured.
  • a vector (X1, X2,..., Xn) T representing the image is determined.
  • the first identification unit 106 calculates “automobile” by using the vector (X1, X2,..., Xn) T , each coefficient of each layer included in the internally generated model, and each weight included in the model. , “Motorcycle”, “bus”, and “background” are calculated. Then, the first identification unit 106 determines the label with the highest reliability among “car”, “motorcycle”, “bus”, and “background” as a label indicating an object appearing in the image. .
  • the reliability of “car”, “motorcycle”, “bus”, and “background” is “0.6”, “0”, .2 ",” 0.1 ", and” 0.1 "are obtained.
  • the first identification unit 106 identifies the object appearing in the image as an “automobile” having the highest reliability “0.6”.
  • the first identifying unit 106 represents a rectangular area surrounding the object (“car”, “motorcycle” or “bus”) shown in the image as the image.
  • the determination is made by an operation using the vector and the internally generated model.
  • the fact that the determined label is “background” means that it is determined that the object is not shown in the image. In this case, the first identification unit 106 determines that the object Does not determine the rectangular area surrounding.
  • the first identification unit 106 associates the image subjected to the identification processing, the label corresponding to the identification result, and the reliability corresponding to the label, and stores the image in the result storage unit 117.
  • the first identification unit 106 determines that the object shown in the image is “car” having the highest reliability “0.6”.
  • the first identification unit 106 causes the result storage unit 117 to store the image, the label “car”, and the reliability “0.6” in association with each other.
  • the result storage unit 117 is a storage device that stores identification results and the like. However, the result storage unit 117 additionally stores information indicating a rectangular area in the image, as described later.
  • the second model storage unit 110 is a storage device that stores a model different from the internally generated model (the model generated by the learning unit 103). More specifically, the second model storage unit 110 stores individual models (externally generated models) learned by a plurality of first identification systems predetermined for the identification system 100 shown in FIG. Remember. Each model stored in the second model storage unit 110 is represented in the same format as the model schematically shown in FIG.
  • the model updating unit 121 receives a model from each of a plurality of first identification systems predetermined for the identification system 100 shown in FIG. 3, and uses the model as an externally generated model in the second model storage unit 110. To memorize.
  • Each first identification system includes a model transmission unit 122 described later, similarly to the identification system 100 shown in FIG. Then, the model transmitting unit 122 of each first identification system transmits the model learned by the first identification system to the identification system 100 shown in FIG. 3 as appropriate.
  • the model updating unit 121 receives the model transmitted by the first identification system. It is assumed that the first identification system transmits a model to the identification system 100 shown in FIG. 3 for the first time. In that case, the second model storage unit 110 has not yet stored the model learned by the first identification system. At this time, the model updating unit 121 stores the received model as an externally generated model in the second model storage unit 110 in association with the information indicating the first identification system that is the transmission source. It is also assumed that the first identification system has previously transmitted a model to the identification system 100 shown in FIG. In that case, the second model storage unit 110 has already stored the model learned by the first identification system. At this time, the model updating unit 121 updates a model already stored in the second model storage unit 110 as a model learned by the first identification system to a newly received model.
  • the second model storage unit 110 stores all the individual models (externally generated models) learned by a plurality of first identification systems predetermined for the identification system 100 shown in FIG. It is assumed that it has been stored. In this state, when receiving the model from the first identification system, the model updating unit 121 updates the model stored in the second model storage unit 110 to the received model.
  • the second identification unit 111 applies a predetermined image among the images identified by the first identification unit 106 to the externally generated model stored in the second model storage unit 110, An object in the predetermined image is identified.
  • the second identification unit 111 performs this processing for each externally generated model.
  • the second identification unit 111 calculates the reliability of “automobile”, “motorcycle”, “bus”, and “background” by applying a predetermined image to the externally generated model. Then, the second identification unit 111 determines a label with the highest reliability among “automobile”, “motorcycle”, “bus”, and “background” as a label indicating an object appearing in the image. .
  • the predetermined image of the images identified by the first identification unit 106 is a predetermined image among the images identified by the first identification unit 106. This is an image determined to cause the image processing unit 111 to execute the identification processing.
  • the determination unit 107 determines an image that causes the second identification unit 111 to execute the identification process, from among the images that have been identified by the first identification unit 106.
  • three types of determination methods will be exemplified as a method in which the determination unit 107 determines an image to be subjected to the identification processing by the second identification unit 111 among the images identified by the first identification unit 106. Will be explained.
  • the determination unit 107 may employ only one of the following three types of determination methods. Alternatively, the determination unit 107 may employ a plurality of determination methods among the following three types of determination methods.
  • the determining unit 107 determines to cause the second identifying unit 111 to execute the identifying process on a certain image by any one of the plurality of determining methods, the determining unit 107 Thus, it is determined that the second identification unit 111 executes the identification processing.
  • the first determination method is that, when the label determined by the first identification unit 106 as a label representing an object appearing in an image is incorrect, the determination unit 107 determines the second This is a method of determining that the identification unit 111 performs the identification processing. That is, this is a method in which the determination unit 107 determines that the second identification unit 111 performs the identification process on the image that is incorrectly identified by the first identification unit 106. Whether or not the label determined by the first identification unit 106 is incorrect may be determined by, for example, an operator of the identification system 100. Hereinafter, this case will be described as an example.
  • the determination unit 107 When the first identification unit 106 determines a label for an image, the determination unit 107 provides a GUI for allowing the operator to input the image, the label determined for the image, and whether the label is correct. (In this example, two buttons are used.) Is displayed on the display device 115.
  • FIG. 5 is a schematic diagram illustrating an example of a screen displayed on the display device 115 by the determination unit 107 in the first determination method.
  • the determination unit 107 determines whether the image 301 that is the identification target of the first identification unit 106 and the first identification unit 106, as illustrated in FIG. A screen showing the determined label 302 (“motorcycle” in the example shown in FIG. 5) and the first button 304 and the second button 305 is displayed on the display device 115.
  • the first button 304 is a button for inputting that the label for the image is correct. Clicking on the first button 304 means that information indicating that the label for the image is correct has been input by the operator. I do.
  • the second button 305 is a button for inputting that the label for the image is incorrect.
  • the second button 305 When the second button 305 is clicked, information indicating that the label for the image is incorrect is displayed by the operator. Means input from In the example shown in FIG. 5, the image 301 shows the self-disembarkation, but “Motorcycle” is displayed as the label determined by the first identification unit 106. Therefore, the operator clicks the second button 305 using the mouse 116. In the example illustrated in FIG. 5, if “automobile” is displayed as the label determined by the first identification unit 106, the operator clicks the first button 304.
  • the determining unit 107 determines that the label determined by the first identifying unit 106 is incorrect, and the first identifying unit 106 identifies the label. It is determined that the second identification unit 111 performs the identification processing on the target image 301.
  • the determination unit 107 determines that the second identification unit 111 does not perform the identification processing on the image 301 that is the identification target of the first identification unit 106. I do.
  • the determination unit 107 identifies the image to the second identification unit 111. This is a method of determining to execute the processing.
  • the determination unit 107 causes the second identification unit 111 to execute the identification process on the image. Is determined. If the reliability corresponding to the label determined by the first identification unit 106 for the image exceeds the threshold, the determination unit 107 causes the second identification unit 111 to perform identification processing on the image. Is determined not to be executed.
  • the threshold value is, for example, “0.5”, but may be a value other than “0.5”.
  • the determination unit 107 determines whether or not to cause the second identification unit 111 to execute identification processing on an image by comparing the reliability derived by the first identification unit 106 with a threshold. decide. Therefore, it is not necessary to display the screen illustrated in FIG. 5 in the second determination method.
  • the determination unit 107 determines that the second identification unit 111 performs the identification process on the image. In other words, although the third determination method determines that the image does not include any of “car”, “motorcycle”, and “bus” in the image, , "Motorcycle” or "bus”, the determination unit 107 determines that the second identification unit 111 performs the identification process on the image.
  • the specified label is “background”, the operator of the identification system 100 determines whether or not “car” or the like is included in the image.
  • the determination unit 107 sets a screen representing the image, the label “background”, and the first button 304 and the second button 305 described above. Is displayed on the display device 115.
  • FIG. 6 is a schematic diagram illustrating an example of a screen displayed on the display device 115 by the determination unit 107 in the third determination method.
  • the determination unit 107 determines, as illustrated in FIG. 6, the image 301 that is the identification target of the first identification unit 106 and a label A screen representing 302 and first button 304 and second button 305 is displayed on display device 115. On the screen displayed by the third determination method, “background” is displayed as the label 302.
  • the first button 304 and the second button 305 are the same as the first button 304 and the second button 305 shown in FIG.
  • the image 301 includes The car is shown. Therefore, the operator clicks the second button 305 using the mouse 116. If none of the car, motorcycle, and bus is shown in the image 301, the operator clicks the first button 304.
  • the determination unit 107 specifies the label “background”, but the image includes any of “car”, “motorcycle”, and “bus”. It is determined that the image is captured, and it is determined that the second identification unit 111 performs the identification process on the image.
  • the determination unit 107 does not show any of “car”, “motorcycle”, and “bus” in the image, and the label “ It is determined that the “background” is correct, and it is determined that the second identifying unit 111 does not execute the identification process on the image.
  • the area correction GUI display control unit 108 will be described. As described above, when the label defined for the image is other than “background”, the first identification unit 106 surrounds the object (“car”, “motorcycle” or “bus”) shown in the image. Determine the rectangular area. The area correction GUI display control unit 108 displays the image determined by the determination unit 107 to cause the second identification unit 111 to execute the identification processing on the display device 115 together with the rectangular area, and further corrects the rectangular area. Is displayed on the display device 115. However, since the label “background” is defined for the image determined by the above-described third method, the rectangular area is not determined. In this case, the area correction GUI display control unit 108 does not display the rectangular area.
  • FIG. 7 is a schematic diagram showing an example of a screen displayed on the display device 115 by the area correction GUI display control unit 108.
  • a rectangular area 309 illustrated in FIG. 7 is a rectangular area defined by the first identification unit 106 as an area surrounding “car” in the image 301.
  • the area correction GUI display control unit 108 includes a determination button 307 and a correction button 308 in the screen.
  • the confirm button 307 is a button for the operator to instruct to confirm the displayed rectangular area.
  • the correction button 308 is a button for the operator to instruct to accept the correction of the rectangular area 309.
  • the rectangular area 309 is appropriate as a rectangular area surrounding “car” in the image 301.
  • the operator clicks the OK button 307.
  • the area extracting unit 109 determines the rectangular area 309 in the image 301 at that time.
  • FIG. 8 is a schematic diagram showing another example of a screen displayed on the display device 115 by the area correction GUI display control unit 108.
  • the rectangular area 309 is not appropriate as a rectangular area surrounding “car” in the image 301.
  • the area correction GUI display control unit 108 accepts an appropriate rectangular area as a rectangular area surrounding “automobile” in accordance with the operation of the operator.
  • the area correction GUI display control unit 108 receives the correction of the positions of the vertices and sides of the rectangular area 309 in accordance with the operation performed by the operator using the mouse 116.
  • the operator can correct the position of the vertices and sides to correct the rectangular area 309 to an appropriate position and size as illustrated in FIG.
  • the area correction GUI display control unit 108 receives such correction.
  • the region extracting unit 109 decides the rectangular region 309 in the image 301 at that time.
  • the area extracting unit 109 determines the rectangular area 309 after the correction.
  • the area correction GUI display control unit 108 does not display the rectangular area 309 on the screen illustrated in FIG.
  • the area correction GUI display control unit 108 displays a rectangular area 309 at an arbitrary size in an arbitrary place of the image 301, and responds to an operation performed by the operator using the mouse 116. Then, the correction of the positions of the vertices and sides of the rectangular area 309 is accepted. The operator corrects the displayed rectangular area 309 to an appropriate position and size surrounding the object shown in the image 301, and then clicks the OK button 307.
  • the area extracting unit 109 determines the rectangular area 309 in the image 301 at that time.
  • the area extracting unit 109 determines the rectangular area 309 in the image 301 at that time. Then, the area extracting unit 109 extracts the determined rectangular area from the image. This rectangular area is an area surrounding the object shown in the image.
  • the region extraction unit 109 represents the determined rectangular region in association with the image stored in the result storage unit 117, the label as the identification result by the first identification unit 106, and the reliability corresponding to the label. Information is also stored in the result storage unit 117.
  • the information representing the rectangular area is, for example, the coordinates of each vertex of the rectangular area.
  • the second identification unit 111 identifies an object appearing in the image of the rectangular area with respect to the image of the rectangular area extracted by the area extracting unit 109.
  • the second identification unit 111 executes this process for each of the externally generated models stored in the second model storage unit 110.
  • the second identification unit 111 calculates the reliability of “automobile”, “motorcycle”, “bus”, and “background” by applying the extracted image of the rectangular area to the externally generated model. Then, the second identification unit 111 determines a label with the highest reliability among “automobile”, “motorcycle”, “bus”, and “background” as a label indicating an object appearing in the image. . In addition, the second identification unit 111 has already stored in the result storage unit 117 the reliability calculated for each label, the label indicating the object appearing in the image, and the reliability corresponding to the label. The result is stored in the result storage unit 117 in association with the image. The second identification unit 111 executes this process for each externally generated model.
  • the result storage unit 117 stores the image, the label determined by the first identification unit 106 performing the identification processing on the image, the reliability corresponding to the label, and the Information indicating a rectangular area is stored. Further, in association with the information, the reliability of each label obtained by the second identification unit 111 applying the image of the rectangular area to the externally generated model A, and the label having the highest reliability and the label The corresponding reliability, the reliability for each label obtained by applying the image of the rectangular area to the externally generated model B by the second identification unit 111, and the label having the highest reliability and the reliability corresponding to the label. The degree is also stored in the result storage unit 117.
  • the result storage unit 117 stores the set of information as described above.
  • the image and a label determined by the first identifying unit 106 performing the identifying process on the image are stored in the result storage unit 117, and information indicating a rectangular area in the image is not stored.
  • the display control unit 112 reads one set of information from the information stored in the result storage unit 117, and displays the image, the label derived by the first identification unit 106, the reliability corresponding to the label,
  • the display unit 115 displays on the display device 115 a screen including the label derived for each externally generated model by the second identification unit 111 and the reliability corresponding to the label.
  • FIG. 9 is a schematic diagram showing an example of a screen displayed by the display control unit 112.
  • the display control unit 112 corresponds to the label derived by the first identification unit 106 and the reliability 501 corresponding to the label, and corresponds to the label derived by the second identification unit 111 using the externally generated model A and the label.
  • a screen in which the reliability 502 and the label derived by the second identification unit 111 using the externally generated model B and the reliability 503 corresponding to the label are superimposed on the image 301 is displayed on the display device 115.
  • the display control unit 112 also displays the determined rectangular area 309 so as to be superimposed on the image 301.
  • the case where the number of the externally generated models stored in the second model storage unit 110 is two is illustrated, but the number of the externally generated models may be three or more.
  • the display control unit 112 further displays a check box 504, a re-learning button 505, and screen switching buttons 506 and 507 on this screen.
  • the check box 504 is a GUI for designating whether or not to include the image 301 displayed on the screen (more specifically, the image of the rectangular area 309 extracted from the image 301) in the teacher data.
  • the check box 504 is checked, it means that the image of the rectangular area 309 extracted from the image 301 is included in the teacher data. If the check box 504 is not checked, it means that the image 301 is not included in the teacher data.
  • the display control unit 112 may display the check box 504 in a state where the check box is checked in advance, according to the reliability derived using the externally generated model.
  • the display control unit 112 may display a checked state.
  • the operator can check or uncheck the check box 504 by clicking the check box 504 with the mouse 116.
  • the operator may determine whether to include the image of the rectangular area 309 extracted from the image 301 in the teacher data by referring to the image 301 and the label and reliability derived for each externally generated model. Then, based on the determination, the operator may determine whether to check the check box 504.
  • the screen switching buttons 506 and 507 are buttons for switching to a screen displaying a different image. For example, when the screen switching button 506 is clicked, the display control unit 112 switches to a screen similar to the screen illustrated in FIG. 9 including an image preceding the image 301 in chronological order. Further, for example, when the screen switching button 507 is clicked, the display control unit 112 switches to a screen similar to the image illustrated in FIG. 9 that includes an image later than the image 301 in chronological order. The operator may determine whether or not to check the check box 504 on each of the switched screens.
  • the re-learning button 505 is a button for the operator to instruct the identification system 100 to re-learn the internally generated model.
  • the integration unit 114 specifies a label for each screen image in which the check box 504 is checked.
  • the integration unit 114 specifies the label of the image 301 illustrated in FIG.
  • the attribute data storage unit 113 stores data (attribute data) indicating the attributes of the camera 101 connected to the computer 102 (computer 102 shown in FIG. 3) including the attribute data storage unit 113, and stores the data in the second model storage unit 110.
  • This is a storage device for storing the attribute data of the camera 101 of each identification system 100 (that is, each first identification system) that has generated each of the stored externally generated models.
  • the attribute data of the camera 101 of the identification system 100 (first identification system) that has generated an externally generated model is referred to as attribute data corresponding to the externally generated model.
  • the attributes of the camera 101 include an attribute of the camera 101 itself, an attribute depending on an environment in which the camera 101 is installed, and the like.
  • the value of each attribute is represented by a numerical value.
  • the value of each attribute may be determined in advance by the administrator of each identification system 100 according to the setting of the camera 101 and the installation environment.
  • the attribute data is represented by a vector having such attribute values (numerical values) as elements.
  • the attribute data of the camera 101 includes at least “angle of view of the camera 101”, “whether the camera 101 is installed indoors or outdoors”, “photographing target of the camera 101”, and “ It includes the value of at least some of the attributes “moving direction”. Further, which attribute value is represented by a vector as attribute data is common to all the identification systems 100. Regarding which attribute value is a vector element, Common to the identification system 100. The numerical value of each element of the vector may be different for each identification system 100.
  • the administrator may determine the numerical value representing the angle of view as a vector element.
  • the value of this attribute is set to “0” and the camera 101 Is installed outdoors, the value of this attribute may be set to “1”.
  • the attribute of “the object to be captured by the camera 101” for example, when the camera 101 is installed to capture an image of a vehicle (for example, when the camera 101 is installed toward a road), this attribute The value is set to “0”.
  • the value of this attribute is set to “1”.
  • the value of this attribute is set. Is set to “0.5”.
  • a reference axis based on the main axis direction of the camera 101 and the like is determined, and the angle between the reference axis and the main moving direction of the photographing target is defined as the value of this attribute It may be determined as
  • attribute values other than the above may be included in the attribute data.
  • values such as “the height of the installation location of the camera 101”, “depression angle of the camera 101”, and “resolution of the camera 101” may be included in the attribute data. Since “the height of the installation location of the camera 101”, “depression angle of the camera 101”, and “resolution of the camera 101” are all represented by numerical values, these numerical values may be determined as vector elements.
  • the attribute data storage unit 113 stores the attribute data (vector) of the camera 101 connected to the computer 102 (computer 102 shown in FIG. 3) including the attribute data storage unit 113. This attribute data is referred to as reference attribute data. Further, the attribute data storage unit 113 stores the attribute data of the camera 101 of each first identification system that has generated each externally generated model stored in the second model storage unit 110. In the present embodiment, the second model storage unit 110 stores the externally generated model A and the externally generated model B. Therefore, the attribute data storage unit 113 stores, in addition to the reference attribute data, attribute data corresponding to the externally generated model A (described as attribute data A) and attribute data corresponding to the externally generated model B (attribute data B and Note) is also stored.
  • the attribute data A is attribute data of the camera 101 of the first identification system that has generated the externally generated model A.
  • the attribute data B is attribute data of the camera 101 of the first identification system that has generated the externally generated model B.
  • the administrator who manages each identification system 100 may store the attribute data of the camera 101 in FIG. 3 in the attribute data storage unit 113 as reference attribute data. Further, the administrator sets the attribute data of the camera 101 of each of the two first identification systems defined for the identification system 100 shown in FIG. 3 in the attribute data storage unit 113 as attribute data A and attribute data B. What is necessary is just to memorize.
  • the integration unit 114 determines the reliability of each label (in the present embodiment, “car”, “motorcycle”, “bus”, “bus”) derived by the second identification unit 111 for each externally generated model for the image.
  • the reliability of each "background” is integrated for each label, and the label of the image is specified based on the integration result.
  • the integration unit 114 includes the reference attribute data (that is, the attribute data of the camera 101 of the identification system 100 including the integration unit 114) and the plurality of first identification systems that have generated the externally generated model A and the externally generated model B. Is calculated for each of the first identification systems.
  • the integration unit 114 calculates the similarity between the reference attribute data and the attribute data A and the similarity between the reference attribute data and the attribute data B, respectively.
  • the similarity between the reference attribute data and the attribute data A is referred to as a similarity corresponding to the externally generated model A.
  • the similarity between the reference attribute data and the attribute data B is referred to as a similarity corresponding to the externally generated model B.
  • Attribute data is represented by a vector.
  • the integration unit 114 may calculate the reciprocal of the distance between the two vectors as the similarity.
  • the integration unit 114 weights the reliability of each label derived for each externally generated model when the label is integrated for each label, with the similarity corresponding to the externally generated model, and integrates them.
  • the integrating unit 114 may specify the label having the highest reliability integration result as the image label.
  • the integrating unit 114 may calculate the product of Li and Wi for each externally generated model, and use the average value of the product as the integrated result of the reliability of the label of interest. The integration unit 114 performs the same operation for other labels. Then, the integration unit 114 specifies the label with the highest integration result as the label of the image.
  • FIG. 10 is an explanatory diagram showing a specific example of the first calculation method. It is assumed that there are two externally generated models A and B. The reliability of “car”, “motorcycle”, “bus”, and “background” derived using the externally generated model A is “0.1”, “0.7”, “0.1”, “ 0.1 ". It is also assumed that the similarity calculated for the externally generated model A is “0.9”.
  • the integrating unit 114 calculates a result obtained by multiplying the degree of similarity by “0.9” for each reliability. As a result, the multiplication results (product) of “0.09”, “0.63”, “0.09”, “0.09” for “car”, “motorcycle”, “bus”, and “background” respectively. Is obtained.
  • the reliability of “car”, “motorcycle”, “bus” and “background” derived using the externally generated model B are “0.1”, “0.6”, and “0.2”, respectively. , “0.1”. It is also assumed that the similarity calculated for the externally generated model B is “0.8”.
  • the integrating unit 114 calculates a result of multiplying the degree of similarity “0.8” for each of the above degrees of reliability. As a result, the multiplication results (product) of “0.08”, “0.48”, “0.16”, “0.08” for “car”, “motorcycle”, “bus”, and “background” respectively. Is obtained.
  • the integration unit 114 calculates an average value of the multiplication results (products) obtained for each of the “car”, “motorcycle”, “bus”, and “background”.
  • the average values calculated for each of "car”, “motorcycle”, “bus” and “background” are "0.085”, “0.555”, “0.125”, and "0.085”. Therefore, the integrating unit 114 specifies “motorcycle” having the highest average value (integrated result) as the image label.
  • the reliability of the label of interest obtained using the i-th externally generated model is Li.
  • the similarity calculated for the i-th externally generated model is Wi.
  • the sum of the individual similarities calculated for the individual externally generated models is Wt.
  • the number of externally generated models stored in the second model storage unit 110 is N.
  • the integrating unit 114 may calculate Wt by the calculation of the following equation (2).
  • the integrating unit 114 may integrate the reliability of the label of interest by calculating the following expression (3).
  • the integration unit 114 calculates, for each externally generated model, the ratio of the similarity corresponding to the externally generated model to the sum of the similarities, and uses the calculation result of the ratio as a weight to determine the reliability of the label of interest.
  • the weighted sum may be calculated, and the calculation result may be used as the integrated result of the reliability of the label of interest.
  • the integration unit 114 performs the same operation for other labels. Then, the integration unit 114 specifies the label with the highest integration result as the label of the image.
  • the ratio of the similarity “0.9” corresponding to the externally generated model A to the total similarity “1.7” is “0.9 / 1.7”.
  • the ratio of the similarity “0.8” corresponding to the externally generated model B to the total similarity “1.7” is “0.8 / 1.7”.
  • the integrating unit 114 calculates a weighted sum of reliability for each label, using “0.9 / 1.7” and “0.8 / 1.7” as weights, and uses the calculation result as the reliability of the label. And the integration result. Then, the integration results of “car”, “motorcycle”, “bus”, and “background” are “0.0999”, “0.6528”, “0.1470”, and “0.0999”. Therefore, the integration unit 114 specifies the “motorcycle” having the highest integration result as the image label.
  • Each of the first and second calculation methods is a calculation in which the reliability of the label derived for each externally generated model is weighted by the similarity corresponding to the externally generated model and integrated. .
  • the learning unit 103 extracts a fixed rectangular area in the image, and integrates the rectangular area image with the image of the rectangular area.
  • the pair with the label specified by the unit 114 is included in the existing teacher data.
  • the learning unit 103 uses the teacher data to re-learn the internally generated model by deep learning. Further, the learning unit 103 updates the existing internally generated model stored in the first model storage unit 104 to a new internally generated model generated by re-learning.
  • the model transmitting unit 122 transmits the model learned by the learning unit 103 to a predetermined second identification system.
  • the number of the second identification system to which the model is transmitted may be one or plural.
  • the second identification system stores the received model as an externally generated model.
  • model transmission unit 122 transmits a model to the second identification system
  • four model transmission modes will be described as examples.
  • the model transmission unit 122 transmits the model to the second identification system.
  • the model transmission unit 122 transmits a model newly obtained by the re-learning to the second identification system. Therefore, in the first model transmission mode, the model transmission unit 122 can transmit the latest model to the second identification system.
  • the model transmission unit 122 periodically transmits the model stored in the first model storage unit 104 to the second identification system. That is, the model transmission unit 122 stores the model stored in the first model storage unit 104 in the first model storage unit 104 again after a certain period of time has elapsed after transmitting the model to the second identification system. And transmitting the model to the second identification system. Even if the model stored in the first model storage unit 104 has been updated a plurality of times during this fixed period, the model transmission unit 122 will continue to operate the first model storage at the time when the fixed period has elapsed since the previous model transmission. The model stored in the storage unit 104 is transmitted to the second identification system. If the model stored in the first model storage unit 104 has not been updated during this fixed period, the model transmission unit 122 retransmits the same model as the previously transmitted model. Become.
  • the model transmission unit 122 determines whether to transmit the model to the second identification system.
  • the image to be specified by the integrating unit 114 as a label is determined by the determining unit 107 of the images identified by the first identifying unit 106 by the first determining method, the second determining method, or the second determining method. It is an image determined to cause the second identification unit 111 to execute the identification process by the determination method of No. 3. That is, the image that the integrating unit 114 specifies as a label is an image that has been erroneously identified by the first identification unit 106 or has not obtained a reliability higher than the threshold. Before the learning unit 103 re-learns the model, the integration unit 114 specifies the label of such an image by using the reliability of each label obtained for each externally generated model.
  • the learning unit 103 when the learning unit 103 re-learns a model (internally generated model), the first identification unit 106 applies the image to the re-learned model, thereby obtaining the image. Is derived again.
  • the model transmitting unit 122 uses the model re-learned by the first identifying unit 106 to again derive the identification result (label) of the image and the label specified by the integrating unit 114. Then, it is determined that the relearned model is to be transmitted to the second identification system, and the relearned model is transmitted to the second identification system. On the other hand, when the two labels do not match, the model transmission unit 122 determines that the relearned model is not transmitted to the second identification system, and does not transmit the model to the second identification system.
  • the model transmission unit 122 determines whether to transmit the model to the second identification system as described above. To the second identification system.
  • the integration unit 114 specifies the label of the image by using the reliability of each label obtained for each externally generated model. Therefore, it is considered that the accuracy of the label specified by the integration unit 114 is high even if the image has been erroneously identified by the first identification unit 106 or has not obtained a reliability higher than the threshold. Therefore, the fact that the identification result (label) of the image derived again by using the model re-learned by the first identification unit 106 and the label specified by the integration unit 114 coincides with each other means that It can be said that the identification accuracy of the model obtained by learning is higher than the identification accuracy of the model before relearning.
  • the model obtained by the relearning is transmitted to the second identification system. It can be said that the transmission mode is used. Further, in the third model transmission mode, if the model before re-learning cannot obtain a correct identification result, but the model obtained by re-learning can obtain a correct identification result, It can also be said that the model obtained by learning is transmitted to the second identification system.
  • the model transmission unit 122 determines whether the model transmission unit 122 transmits the model to the second identification system. Is determined.
  • the model transmission mode when the model is relearned, the model transmission is performed within a predetermined time based on the accuracy rate of the identification result (label) derived by the first identification unit 106 using the model.
  • the unit 122 determines whether to send the model to the second identification system.
  • the accuracy rate is equal to or greater than a predetermined threshold
  • the model transmitting unit 122 determines that the relearned model is to be transmitted to the second identification system, and transmits the model to the second identification system.
  • the accuracy rate is less than the threshold
  • the model transmitting unit 122 determines that the relearned model is not transmitted to the second identification system, and does not transmit the model to the second identification system.
  • the number of times the first button 304 is clicked can be said to be the number of times that the label derived by the first identification unit 106 is correct.
  • the model transmission unit 122 calculates the ratio of the number of times the first button 304 is clicked to the number of times the determination unit 107 displays the screen illustrated in FIG. 5 within a predetermined time, The result may be the correct answer rate. Then, the model transmission unit 122 may determine whether to transmit the re-learned model to the second identification system as described above by comparing the accuracy rate with the threshold.
  • the model updating unit 121 and the model transmitting unit 122 are realized by, for example, a CPU (Central Processing Unit) of the computer 102 operating according to the model providing program and a communication interface of the computer 102.
  • the CPU may read the model providing program from a program recording medium such as a program storage device of the computer 102 and operate as the model updating unit 121 and the model transmitting unit 122 using the communication interface according to the model providing program.
  • the learning unit 103, the first identification unit 106, the determination unit 107, the area correction GUI display control unit 108, the area extraction unit 109, the second identification unit 111, the display control unit 112, and the integration unit 114 This is realized by the CPU of the computer 102 that operates according to the provided program.
  • the CPU reads the model providing program from a program recording medium such as a program storage device of the computer 102, and according to the model providing program, the learning unit 103, the first identification unit 106, the determination unit 107, the area correction GUI display control unit 108 , The region extraction unit 109, the second identification unit 111, the display control unit 112, and the integration unit 114.
  • the first model storage unit 104, the second model storage unit 110, the attribute data storage unit 113, and the result storage unit 117 are realized by a storage device included in the computer 102.
  • FIG. 12 is a flowchart illustrating an example of processing progress from the time when the camera 101 captures an image to the time when the second identification unit 111 performs an identification process on an image. The detailed description of the operation already described is omitted.
  • the learning unit 103 has previously learned the internally generated model by deep learning, and has stored the internally generated model in the first model storage unit 104.
  • the model updating unit 121 receives a model from each of a plurality of first identification systems predetermined for the identification system 100 shown in FIG. 3, and stores the model as an externally generated model in a second model storage. It is assumed that the information is stored in the unit 110. That is, it is assumed that the second model storage unit 110 stores the models learned by the individual first identification systems as externally generated models. Note that, when a new model is received from the first identification system, the model updating unit 121 stores the model already stored in the second model storage unit 110 as a model learned by the first identification system. May be updated to the newly received model.
  • the camera 101 obtains an image by photographing at the installation location of the camera 101 (step S1).
  • the camera 101 transmits the image to the computer 102.
  • the first identification unit 106 of the computer 102 receives the image via the data acquisition unit 105. Then, the first identification unit 106 identifies the object shown in the image by applying the image to the internally generated model (Step S2). In step S2, the first identification unit 106 derives a label representing the object appearing in the image and the reliability of the label. The first identification unit 106 stores the image in the result storage unit 117 in association with the derived label and reliability. When the specified label is not “background”, the first identification unit 106 determines a rectangular area surrounding the object shown in the image.
  • the determination unit 107 determines whether or not to cause the second identification unit 111 to execute an identification process on the image identified by the first identification unit 106 in step S2 (step S3). When it is determined that the second identification unit 111 does not execute the identification processing (No in step S3), the processing after step S1 is repeated.
  • the area correction GUI display control unit 108 displays an image on the display device 115.
  • the area correction GUI display control unit 108 displays the screens illustrated in FIGS. 7 and 8 on the display device 115.
  • the region extracting unit 109 determines a rectangular region surrounding the object shown in the image in accordance with the operation of the operator on the screen, and extracts the rectangular region from the image (step S4).
  • the second identification unit 111 identifies an object appearing in the image of the rectangular area extracted in step S4 for each of the externally generated models stored in the second model storage unit 110 in advance. (Step S5).
  • the second identification unit 111 derives the reliability of each label (“car”, “motorcycle”, “bus”, and “background”) for each externally generated model.
  • the result storage unit 117 stores the reliability of each label derived for each externally generated model.
  • the second identification unit 111 also causes the result storage unit 117 to store, for each externally generated model, a pair of the label having the highest reliability and the reliability corresponding to the label. The label with the highest reliability indicates an object determined to be present in the image.
  • step S5 the processes after step S1 are repeated.
  • FIG. 13 is a flowchart showing an example of processing progress when a model (internally generated model) is re-learned and the model is transmitted to the second identification system. In the following description, a detailed description of the operation already described is omitted. In FIG. 13, the above-described “first model transmission mode” will be described as an example.
  • the display control unit 112 determines the label derived by the first identification unit 106 and the reliability corresponding to the label, and the label derived by the second identification unit 111 for each externally generated model and the reliability corresponding to each label.
  • a screen in which the degree is superimposed on the image is displayed on the display device 115 (step S11).
  • the display control unit 112 includes a check box 504, a re-learning button 505, and screen switching buttons 506 and 507 in this screen.
  • the display control unit 112 displays, for example, a screen illustrated in FIG.
  • the operator checks the screen illustrated in FIG. 9 and determines whether or not to include the displayed image 301 (more specifically, the image of the rectangular area 309 determined in the image 301) in the teacher data. .
  • the operator specifies that the displayed image 301 is to be included in the teacher data. That is, the image displayed on the screen with the check box 504 checked is the image specified as the image to be included in the teacher data.
  • the integration unit 114 calculates the similarity between the reference attribute data and each attribute data corresponding to each externally generated model (step S12).
  • the attribute data is represented by a vector.
  • the integration unit 114 may calculate the reciprocal of the distance between the two vectors as the similarity.
  • the integration unit 114 integrates the reliability of the labels derived for each of the externally generated models using the similarities calculated in step S12. The integrating unit 114 performs this process for each label, and identifies the label with the highest reliability integration result as the label for the image to be included in the teacher data (step S13).
  • the integrating unit 114 executes the process of step S13 for each of the images.
  • the learning unit 103 extracts a determined rectangular area in the image to be included in the teacher data, and includes a set of the image of the rectangular area and the label specified by the integration unit 114 in the existing data. Then, using the teacher data, the learning unit 103 re-learns the internally generated model by deep learning, and stores the internally generated model obtained by the re-learning in the first model storage unit 104 (step S14). The learning unit 103 updates the existing internally generated model stored in the first model storage unit 104 to a new internally generated model generated by re-learning.
  • the first identification unit 106 uses a new internally generated model generated by relearning.
  • step S14 the model transmitting unit 122 transmits the model (internally generated model) relearned in step S14 to the second identification system (step S15).
  • Each identification system 100 has a similar configuration.
  • the model updating unit 121 of the second identification system updates the model stored in the second model storage unit 110 in the second identification system to the received model. I do. Therefore, the model transmitted by the model transmitting unit 122 to the second identification system in step S15 is stored as an externally generated model in the second identification system.
  • the “first model transmission mode” has been described as an example.
  • the computer 102 may end the process in step S14.
  • the model transmission unit 122 may periodically transmit the model stored in the first model storage unit 104 to the second identification system, separately from the processing in steps S11 to S14.
  • FIG. 14 is a flowchart showing an example of the processing progress of the above-mentioned “third model transmission mode”.
  • the processing up to step S14 is the same as that in the flowchart shown in FIG. 13, and a description thereof will be omitted.
  • the first identification unit 106 uses the label of the image (see FIG. 9) displayed in the screen with the check box 504 checked, using the internally generated model relearned in step S14. (Step S21).
  • the image displayed on the screen with the check box 504 checked is an image specified by the operator to be included in the teacher data.
  • the model transmission unit 122 determines whether or not the label derived in step S21 matches the label identified by the integration unit 114 in step S13 (see FIG. 13) for the same image (step S21). S22).
  • step S22 If the two labels match (Yes in step S22), the model transmitting unit 122 determines that the model (internally generated model) retrained in step S14 (see FIG. 13) is to be transmitted to the second identification system, The model is transmitted to the second identification system (Step S23).
  • the model transmitting unit 122 determines that the model re-learned in step S14 is not to be transmitted to the second identification system, and ends the process.
  • FIG. 15 is a flowchart showing an example of the processing progress of the “fourth model transmission mode” described above.
  • the processing up to step S14 is the same as that in the flowchart shown in FIG. 13, and a description thereof will be omitted.
  • the model transmission unit 122 calculates the correct answer rate of the image identification result when the re-learned model (internally generated model) is used within a predetermined time (step S31). Since the calculation example of the correct answer rate has already been described, the description is omitted here.
  • step S32 the model transmitting unit 122 determines whether or not the correct answer rate calculated in step S31 is equal to or greater than a predetermined threshold.
  • the model transmitting unit 122 determines that the model (internally generated model) retrained in step S14 (see FIG. 13) is to be transmitted to the second identification system. , And transmits the model to the second identification system (step S33).
  • the model transmitting unit 122 determines that the model re-learned in step S14 is not transmitted to the second identification system, and ends the process.
  • the determination unit 107 determines that the first identification unit 106 performs the identification process by at least one of the first determination method, the second determination method, and the third determination method described above. It is determined whether or not to cause the second identification unit 111 to execute the identification processing on the target image. Therefore, the image for which the identification processing is performed by the second identification unit 111 is an image in which the label determined by the first identification unit 106 is incorrect, and the reliability corresponding to the label determined for the image is a threshold. The following image or an image in which an object (“automobile”, “motorcycle” or “bus”) is captured despite the label determined by the first identification unit 106 being “background” is there.
  • such an image is generated by using a model different from the internally generated model (more specifically, a model generated by a first identification system predetermined for the identification system 100 of interest. (Externally generated model)), a pair of a label specified based on the result of applying the image and the image is added to the existing teacher data, and the learning unit 103 re-learns the internally generated model. Therefore, the identification accuracy of the internally generated model can be improved.
  • a model different from the internally generated model more specifically, a model generated by a first identification system predetermined for the identification system 100 of interest. (Externally generated model)
  • a pair of a label specified based on the result of applying the image and the image is added to the existing teacher data, and the learning unit 103 re-learns the internally generated model. Therefore, the identification accuracy of the internally generated model can be improved.
  • the model transmitting unit 122 transmits the model learned by the learning unit 103 to, for example, the first model transmitting mode, the second model transmitting mode, the third model transmitting mode, or the fourth model transmitting mode.
  • the data is transmitted to a predetermined second identification system.
  • the model updating unit 121 stores the received model in the second model storage unit 110 as an externally generated model. Therefore, each identification system 100 can relearn its own model so as to improve the identification accuracy of its own model, and can contribute to the improvement of the identification accuracy of the models of other identification systems 100. it can.
  • the region extracting unit 109 determines a rectangular region surrounding the object shown in the image, and extracts the rectangular region from the image. I do. Then, the second identification unit 111 identifies an object appearing in the extracted image of the rectangular area for each externally generated model. The second identification unit 111 performs a process of identifying an object appearing in the image not on the extracted image of the rectangular area but on the entire one image that has been processed by the first identification unit 106. You may. In this case, the identification system 100 (see FIG. 3) does not have to include the area correction GUI display control unit 108 and the area extraction unit 109. And the identification system 100 does not need to perform step S4 (see FIG. 12). The second identification unit 111 only needs to identify an object in the image in step 5 with respect to one entire image processed by the first identification unit 106.
  • the learning unit 103 only needs to include the entire set of one image and the label specified by the integration unit 114 in existing teacher data, and re-learn the internally generated model using the teacher data.
  • the learning unit 103 may re-learn an internally generated model by deep learning using a set of an image and a label specified by the integration unit 114 and an existing internally generated model as teacher data.
  • the second model storage unit 110 may store one externally generated model.
  • the learning unit 103 includes the pair of the image and the label with the highest reliability derived by the second identification unit 111 in the existing teacher data, and uses the teacher data to generate the internally generated model. Should be re-learned.
  • FIG. 16 is a schematic block diagram illustrating a configuration example of a computer 102 included in an identification system 100 according to an embodiment of the present invention or a modified example thereof.
  • a computer is represented by a reference numeral “1000”.
  • the computer 1000 includes an interface between a CPU 1001, a main storage device 1002, an auxiliary storage device 1003, an interface 1004, a display device 1005, an input device 1006, a communication interface 1007, and a data collection unit 101 (for example, a camera). 1008.
  • the operation of the computer included in the identification system 100 is stored in the auxiliary storage device 1003 in the form of a model providing program.
  • the CPU 1001 reads the model providing program from the auxiliary storage device 1003 and expands the program on the main storage device 1002. Then, the CPU 1001 executes the processing of the computer 102 (see FIG. 3) in the above-described embodiment and its modifications according to the model providing program.
  • the auxiliary storage device 1003 is an example of a non-transitory tangible medium.
  • Other examples of non-transitory tangible media include a magnetic disk, a magneto-optical disk, a CD-ROM (Compact Disk Read Only Memory), a DVD-ROM (Digital Versatile Disk Read Only Memory) connected via the interface 1004, A semiconductor memory and the like are included.
  • the program When the program is distributed to the computer 1000 via a communication line, the computer 1000 that has received the program may load the program into the main storage device 1002 and execute the above processing.
  • the program may be for realizing a part of the processing of the computer 102 shown in the embodiment and its modified example. Furthermore, the program may be a difference program that implements the above-described processing in combination with another program already stored in the auxiliary storage device 1003.
  • Some or all of the components may be realized by a general-purpose or dedicated circuit (processor), a processor, or a combination thereof. These may be configured by a single chip, or may be configured by a plurality of chips connected via a bus. Some or all of the components may be realized by a combination of the above-described circuit and the like and a program.
  • processor general-purpose or dedicated circuit
  • processor processor
  • a combination thereof may be configured by a single chip, or may be configured by a plurality of chips connected via a bus.
  • Some or all of the components may be realized by a combination of the above-described circuit and the like and a program.
  • the plurality of information processing devices, circuits, and the like may be centrally arranged or may be distributed.
  • the information processing device, the circuit, and the like may be realized as a form in which each is connected via a communication network, such as a client and server system or a cloud computing system.
  • FIG. 17 is a block diagram showing an outline of the identification system of the present invention.
  • the identification system according to the present invention includes a learning unit 701, a first model storage unit 702, a first identification unit 703, a second model storage unit 704, a model update unit 705, and a second identification unit 706. And a model transmitting unit 707.
  • the learning unit 701 (for example, the learning unit 103) learns a model for identifying an object represented by data (for example, an image) using teacher data.
  • the first model storage unit 702 (first model storage unit 104) stores the model learned by the learning unit 701.
  • the first identification unit 703 (for example, the first identification unit 106) identifies an object represented by data using the model learned by the learning unit 701.
  • the second model storage unit 704 (for example, the second model storage unit 110) stores the individual models learned by the plurality of first identification systems determined in advance.
  • the model updating unit 705 (for example, the model updating unit 121) receives a model learned by the first identification system from the first identification system, the model updating unit 705 stores the model stored in the second model storage unit 704. The model learned by the first identification system is updated to the model received from the first identification system.
  • the second identification unit 706 determines whether the first identification unit 703 is an identification target for each model stored in the second model storage unit 704. The object represented by the set data is identified.
  • the learning means 701 re-learns the model using the label for the data determined based on the identification result derived by the second identification means 706 and the teacher data including the data, and stores the model in the first model storage means 702. Update the stored model with the retrained model.
  • the model transmitting unit 707 (for example, the model transmitting unit 122) transmits the model learned by the learning unit 701 to one or a plurality of predetermined second identification systems.
  • the identification system of the present invention can re-learn its own model so as to improve the identification accuracy of its own model, and also contributes to the improvement of the identification accuracy of the models of other identification systems. can do.
  • the model transmitting means 707 may be configured to transmit the model to the second identification system when the learning means 701 re-learns the model.
  • the model transmitting unit 707 may periodically transmit the model stored in the first model storing unit 702 to the second identification system.
  • the second identification unit 706 integrates the identification results derived for each model stored in the second model storage unit 704 to identify the label for the data to be identified by the first identification unit 703.
  • the learning unit 701 re-learns the model using the teacher data including the label specified by the integrating unit and the data, and the model transmitting unit 707 performs When the identification result of the data derived by the first identifying means 703 using the relearned model matches the label for the data identified by the integrating means, the relearned model is assigned to the second learning means. It may be configured to transmit to the identification system.
  • the model transmission unit 707 determines that the accuracy rate of the identification result derived by the first identification unit 703 using the model within a predetermined time period is equal to or greater than a predetermined threshold value. , The model may be transmitted to the second identification system.
  • the present invention is suitably applied to an identification system that identifies an object represented by data by applying the data to a model.
  • REFERENCE SIGNS LIST 100 identification system 101 data collection unit 102 computer 103 learning unit 104 first model storage unit 105 data acquisition unit 106 first identification unit 107 determination unit 108 region correction GUI display control unit 109 region extraction unit 110 second model storage unit 111 second identification unit 112 display control unit 113 attribute data storage unit 114 integration unit 115 display device 116 mouse 117 result storage unit 121 model update unit 122 model transmission unit

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Image Analysis (AREA)

Abstract

Lorsqu'un modèle appris par un premier système d'identification est reçu en provenance du premier système d'identification, un moyen de mise à jour de modèle (705) met à jour un modèle, qui a été appris par le premier système d'identification et stocké dans un second moyen de stockage de modèle (704), vers le modèle reçu en provenance du premier système d'identification. Un moyen d'apprentissage (701) réapprend le modèle à l'aide des données d'apprentissage, qui comprennent une étiquette pour les données déterminées sur la base d'un résultat d'identification déduit par un second moyen d'identification (706) et desdites données et met à jour le modèle stocké dans un premier moyen de stockage de modèle (702) vers le modèle réappris. Un moyen de transmission de modèle (707) transmet le modèle appris par le moyen d'apprentissage (701) à un ou plusieurs seconds systèmes d'identification déterminés à l'avance.
PCT/JP2018/032761 2018-09-04 2018-09-04 Système d'identification, procédé de présentation de modèle et programme de présentation de modèle WO2020049636A1 (fr)

Priority Applications (2)

Application Number Priority Date Filing Date Title
JP2020540901A JP6981553B2 (ja) 2018-09-04 2018-09-04 識別システム、モデル提供方法およびモデル提供プログラム
PCT/JP2018/032761 WO2020049636A1 (fr) 2018-09-04 2018-09-04 Système d'identification, procédé de présentation de modèle et programme de présentation de modèle

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/JP2018/032761 WO2020049636A1 (fr) 2018-09-04 2018-09-04 Système d'identification, procédé de présentation de modèle et programme de présentation de modèle

Publications (1)

Publication Number Publication Date
WO2020049636A1 true WO2020049636A1 (fr) 2020-03-12

Family

ID=69721469

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2018/032761 WO2020049636A1 (fr) 2018-09-04 2018-09-04 Système d'identification, procédé de présentation de modèle et programme de présentation de modèle

Country Status (2)

Country Link
JP (1) JP6981553B2 (fr)
WO (1) WO2020049636A1 (fr)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2022185596A1 (fr) * 2021-03-04 2022-09-09 パナソニックＩｐマネジメント株式会社 Système d'estimation, procédé d'estimation et programme

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20080103996A1 (en) * 2006-10-31 2008-05-01 George Forman Retraining a machine-learning classifier using re-labeled training samples
JP2015129988A (ja) * 2014-01-06 2015-07-16 日本電気株式会社 データ処理装置
JP2017519282A (ja) * 2014-05-12 2017-07-13 クゥアルコム・インコーポレイテッドＱｕａｌｃｏｍｍ Ｉｎｃｏｒｐｏｒａｔｅｄ 分散モデル学習

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20080103996A1 (en) * 2006-10-31 2008-05-01 George Forman Retraining a machine-learning classifier using re-labeled training samples
JP2015129988A (ja) * 2014-01-06 2015-07-16 日本電気株式会社 データ処理装置
JP2017519282A (ja) * 2014-05-12 2017-07-13 クゥアルコム・インコーポレイテッドＱｕａｌｃｏｍｍ Ｉｎｃｏｒｐｏｒａｔｅｄ 分散モデル学習

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
LEISTNER, C. ET AL.: "Visual on-line Learning in Distributed Camera Networks", 2008 SECOND ACM/ IEEE INTERNATIONAL CONFERENCE ON DISTRIBUTED SMART CAMERAS, 30 September 2008 (2008-09-30), pages 1 - 10, XP031329233, Retrieved from the Internet <URL:https://ieeexplore.ieee.org/abstract/document/4635700> [retrieved on 20181127] *

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2022185596A1 (fr) * 2021-03-04 2022-09-09 パナソニックＩｐマネジメント株式会社 Système d'estimation, procédé d'estimation et programme

Also Published As

Publication number Publication date
JP6981553B2 (ja) 2021-12-15
JPWO2020049636A1 (ja) 2021-05-20

Similar Documents

Publication Publication Date Title
US11222239B2 (en) Information processing apparatus, information processing method, and non-transitory computer-readable storage medium
EP1530157B1 (fr) Systeme de mise en correspondance d&#39;images a l&#39;aide d&#39;un modele d&#39;objet en trois dimensions, procede de mise en correspondance d&#39;images et programme de mise en correspondance d&#39;images
JP6397379B2 (ja) 変化領域検出装置、方法、及びプログラム
WO2014080613A1 (fr) Programme, procédé et dispositif de correction de couleur
JP6866095B2 (ja) 学習装置、画像識別装置、学習方法、画像識別方法及びプログラム
WO2019215779A1 (fr) Système, procédé et programme de fourniture de modèle
JP7147753B2 (ja) 情報処理装置、情報処理方法、及びプログラム
JP2019057815A (ja) 監視システム
JP7001150B2 (ja) 識別システム、モデル再学習方法およびプログラム
CN115699082A (zh) 缺陷检测方法及装置、存储介质及电子设备
JP2021039625A (ja) 物体数推定装置、物体数推定方法、および物体数推定プログラム
WO2019215778A1 (fr) Système de fourniture de données et système de collecte de données
KR20150065370A (ko) 행동 인식 장치 및 그 방법
WO2020049636A1 (fr) Système d&#39;identification, procédé de présentation de modèle et programme de présentation de modèle
JP6425405B2 (ja) 情報処理装置、方法、プログラム
US11544926B2 (en) Image processing apparatus, method of processing image, and storage medium
WO2020054058A1 (fr) Système d&#39;identification, procédé de mise à jour de la valeur de paramètre et programme
JPWO2020003764A1 (ja) 画像処理装置、移動装置、および方法、並びにプログラム
JP2019046278A (ja) 情報処理装置、制御方法、コンピュータプログラム、記憶媒体、及びモデル作成装置
WO2022009652A1 (fr) Système de collecte de données, dispositif de capteur, dispositif de collecte de données et procédé de collecte de données
JP7211495B2 (ja) 教師データ生成装置
JP7211496B2 (ja) 教師データ生成装置
US20220383631A1 (en) Object recognition system and object recognition method
CN116798109A (zh) 一种动作类型的识别方法及装置
CN112700494A (zh) 定位方法、装置、电子设备及计算机可读存储介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 18932952

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2020540901

Country of ref document: JP

Kind code of ref document: A

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 18932952

Country of ref document: EP

Kind code of ref document: A1