WO2019003355A1

WO2019003355A1 - System for providing image analysis result, method for providing image analysis result, and program

Info

Publication number: WO2019003355A1
Application number: PCT/JP2017/023807
Authority: WO
Inventors: 俊二菅谷
Original assignee: 株式会社オプティム
Priority date: 2017-06-28
Filing date: 2017-06-28
Publication date: 2019-01-03
Also published as: JP6474946B1; JPWO2019003355A1

Abstract

[Problem] To provide a system for providing an image analysis result, a method for providing an image analysis result, and a program with which it is possible to output a highly accurate image analysis result for an unknown image that needs to be newly analyzed, without taking learning time. [Solution] The present invention is provided with: a storage module 231 for storing a learned model having finished machine learning in which a known image is image-analyzed; an acquisition module 211 for acquiring an unknown image for which no learned model has yet been created; a selection module 212 for selecting, from among the stored learned models, the learned model of a known image that is similar in imaging conditions to the acquired unknown image; an image analysis module 213 for using the selected learned model to image-analyze the unknown image; and a provision module 241 for providing the result of the image analysis.

Description

Image analysis result providing system, image analysis result providing method, and program

The present invention selects a learned model of a known image whose imaging condition is similar to an unknown image to be newly subjected to image analysis from among a plurality of patterns of machine-learned learned models in which artificial intelligence has image-analyzed a known image. The present invention relates to an image analysis result providing system, an image analysis result providing method, and a program capable of outputting an accurate image analysis result without spending learning time.

There has been proposed a method of providing a mechanism for automatically classifying a person image by determining who is shown by performing an image analysis process on the person image (Patent Document 1).

JP 2015-69580

Supervised learning is a well-known method as a machine learning method for artificial intelligence to perform image analysis.

However, for this supervised learning, it is generally necessary to prepare tens of thousands of images or more and hundreds of millions of images, add correct teacher data to the images, and then learn. In order to raise the accuracy of image analysis in supervised learning, it takes time and effort to prepare images for learning, and it takes a long time to learn.

For example, when the entrance detection is performed by image analysis to determine whether a person has entered a level crossing, the position of the monitoring camera is usually fixed. Therefore, there is a supervised at each level crossing such as level crossing A, level crossing B, level crossing C, etc. Machine learning, it is possible to improve the accuracy of entry detection for each of crossing A, crossing B, crossing C, and for the image analysis processing after that, the learned model of crossing A has been learned of crossing B Model and level crossing C learned models are used respectively. However, when it is intended to newly detect entry at level crossing D, a large number of images with tags to be supervised data for supervised learning for level crossing D are prepared and machine learning is performed from one on the basis of that. Because it is necessary to have the following procedure, it takes time and effort until entrance detection is introduced.

In order to solve this problem, the inventor selects and uses a learned model of a level crossing similar to the level crossing D and the imaging condition from the level crossing A, the level crossing B, and the level crossing C already subjected to machine learning. So, also at the crossing D, we focused on the fact that the image analysis accuracy of the approach detection above a certain degree can be obtained from the beginning.

The present invention selects a learned model of a known image whose imaging condition is similar to an unknown image to be newly subjected to image analysis from among a plurality of patterns of machine-learned learned models in which artificial intelligence has image-analyzed a known image. It is an object of the present invention to provide an image analysis result providing system, an image analysis result providing method, and a program capable of outputting an accurate image analysis result without spending learning time.

The present invention provides the following solutions.

The invention according to the first feature is
Storage means for storing a machine-learned learned model obtained by image analysis of a known image;
Acquisition means for acquiring an unknown image for which a learned model has not yet been created;
Selecting means for selecting a learned model of the known image having similar imaging conditions to the acquired unknown image from the stored learned models;
Image analysis means for performing image analysis of the unknown image using the selected learned model;
Providing means for providing the result of the image analysis;
Providing an image analysis result providing system.

According to the first aspect of the present invention, there is provided storage means for storing a machine-learned learned model obtained by image analysis of a known image, acquisition means for acquiring an unknown image for which a learned model has not yet been created, and The unknown image is selected using selection means for selecting a learned model of the known image whose imaging condition is similar to the acquired unknown image from the stored learned models, and using the selected learned model. An image analysis means for analyzing an image, and a providing means for providing a result of the image analysis.

The invention according to the first aspect is a category of an image analysis result providing system, but the same operation and effect can be obtained even with an image analysis result providing method and a program.

An invention according to a second feature is an image analysis result providing system according to the first feature,
The image analysis result providing system is characterized in that the imaging condition is an imaging position and an imaging angle with respect to an imaging target.

According to the invention of the second aspect, in the image analysis result providing system according to the first aspect, the imaging condition is an imaging position and an imaging angle with respect to an imaging target.

An invention according to a third feature is an image analysis result providing system according to the first feature or the second feature, wherein
The image analysis result providing system is characterized in that the learned model is a mathematical expression and parameters used for the image analysis calculated by the machine learning and an image used for the machine learning.

According to the third aspect of the present invention, in the image analysis result providing system according to the first or second aspect, the learned model is used for the image analysis calculated by the machine learning. They are an equation and parameters and an image used for the machine learning.

An invention according to a fourth feature is an image analysis result providing system according to any of the first feature through the third feature, wherein
There is provided an image analysis result providing system including: a creation unit configured to create a new machine-learned learned model subjected to image analysis for the unknown image.

According to the invention as set forth in the fourth feature, in the image analysis result providing system which is the invention as set forth in any one of the first feature to the third feature, new machine learning has already been performed in which the unknown image is analyzed It comprises a creation means for creating a learned model.

An invention according to a fifth feature is an image analysis result providing system according to the fourth feature, wherein
The image analysis means characterized in that the image analysis of the unknown image is performed using the selected learned model in a period until the new machine-learned learned model is created. Provide a provision system.

According to the invention as set forth in the fifth feature, in the image analysis result providing system according to the invention as set forth in the fourth feature, the image analysis means is in a period until creating the new machine-learned learned model. The unknown image is subjected to image analysis using the selected learned model.

An invention according to a sixth feature is an image analysis result providing system according to any of the first feature through the fifth feature, wherein
Creating a new machine-learned learned model in which the unknown image is image-analyzed using the analysis result of the image analysis of the unknown image as teacher data using the selected learned model for the unknown image There is provided an image analysis result providing system characterized by including a creating unit.

According to the invention as set forth in the sixth aspect, in the image analysis result provision system according to any one of the first to fifth aspects, the selected learned model is used for the unknown image. And generating means for generating a new machine-learned learned model in which the unknown image is image-analyzed using the analysis result obtained by image-analyzing the unknown image as teacher data.

The invention according to the seventh feature is
Storing a machine-learned learned model obtained by image analysis of a known image;
Acquiring an unknown image for which a learned model has not yet been created;
Selecting a learned model of the known image whose imaging condition is similar to the acquired unknown image from the stored learned models;
Analyzing the unknown image using the selected learned model;
Providing the result of the image analysis;
Providing an image analysis result providing method.

The invention according to the eighth feature is
Image analysis result provision system,
Storing a machine-learned learned model obtained by image analysis of known images;
Acquiring an unknown image for which a learned model has not yet been created;
Selecting a learned model of the known image whose imaging condition is similar to the acquired unknown image from the stored learned models;
Analyzing the unknown image using the selected learned model;
Providing the result of the image analysis;
Provide a program to run the program.

According to the present invention, from among a plurality of machine-trained learned patterns in which artificial intelligence has analyzed a known image, a learned model of a known image whose imaging condition is similar to an unknown image to be newly analyzed It is possible to provide an image analysis result providing system, an image analysis result providing method, and a program capable of outputting an accurate image analysis result without spending learning time by selecting and using Become.

FIG. 1 is a schematic diagram of a preferred embodiment of the present invention. FIG. 2 is a diagram showing the relationship between the functional blocks of the camera 100 and the computer 200 and the respective functions. FIG. 3 is a flow chart in the case of acquiring an unknown image from the camera 100, performing image analysis processing by the computer 200, and providing an image analysis result. FIG. 4 is a view showing the relationship between the functional blocks of the camera 100 and the computer 200 and the respective functions in the case of performing a learned model creation process of an unknown image. FIG. 5 is a flowchart of the camera 100 and the computer 200 in the case of performing a learned model creation process of an unknown image. FIG. 6 is a diagram showing the relationship between the functional blocks of the camera 100 and the computer 200 and the respective functions when the image analysis process is switched depending on whether the learned model creation process of the unknown image is finished. FIG. 7 is a process corresponding to A of the flowchart of FIG. 6, and when the learning model creation process of the unknown image is not completed, the learned model of the unknown image performed by the computer 200 when the machine learning of the unknown image is possible. It is a flowchart figure of preparation processing. FIG. 8 is a process corresponding to B of the flowchart of FIG. 6, and when the learned model creation process of the unknown image is not finished, the learned model selection process performed by the computer 200 when the machine learning of the unknown image is impossible. FIG. FIG. 9 is a flowchart of the computer 200 in the case of performing machine learning of an unknown image by adding an image analysis result of the unknown image as teacher data when performing a learned model creation process of the unknown image. FIG. 10 shows, as learned models of level crossing A, level crossing B and level crossing C, examples of mathematical expressions and parameters calculated by machine learning, examples of images used for machine learning, and examples of images of level crossing D which are unknown images FIG. FIG. 11 is a diagram for schematically describing the relationship between the camera 100, the computer 200, and the subject 400. FIG. 12 is an example of a table showing a data structure of a learned model for each camera. FIG. 13 is an example of a table showing a learned model used for image analysis for each camera when there is no learned model of the unknown image captured by the camera D. FIG. 14 is an example of a table showing a learned model used for image analysis for each camera when a learned model of an unknown image captured by a camera D is created.

Hereinafter, the best mode for carrying out the present invention will be described with reference to the drawings. This is merely an example, and the technical scope of the present invention is not limited to this.

[Overview of image analysis result provision system]
FIG. 1 is a schematic diagram of a preferred embodiment of the present invention. The outline of the present invention will be described based on FIG. The image analysis result providing system includes a camera 100, a computer 200, and a communication network 300.

In FIG. 1, the number of cameras 100 is not limited to one, and may be plural. The computer 200 is not limited to an existing device, and may be a virtual device.

As illustrated in FIG. 2, the camera 100 includes an imaging unit 10, a control unit 110, a communication unit 120, and a storage unit 130. The computer 200 also includes a control unit 210, a communication unit 220, a storage unit 230, and an input / output unit 240, as also shown in FIG. The control unit 210 cooperates with the communication unit 220 and the storage unit 230 to realize the acquisition module 211. Further, the control unit 210 cooperates with the storage unit 230 to implement the selection module 212 and the image analysis module 213. The storage unit 230 implements the storage module 231 in cooperation with the control unit 210. The input / output unit 240 implements the provision module 241 in cooperation with the control unit 210 and the storage unit 230. The communication network 300 may be a public communication network such as the Internet or a dedicated communication network, and enables communication between the camera 100 and the computer 200.

The camera 100 is an imaging device that includes an imaging device such as an imaging element and a lens that can perform data communication with the computer 200 and can measure the distance to the subject 400. Here, a web camera is illustrated as an example, but an imaging device provided with necessary functions such as a digital camera, digital video, a camera mounted on an unmanned aerial vehicle, a camera of a wearable device, a security camera, an on-vehicle camera, and a 360 degree camera It may be. Further, the captured image may be stored in the storage unit 130.

The computer 200 is a computing device capable of data communication with the camera 100. Here, a desktop computer is illustrated as an example, but in addition to a mobile phone, a portable information terminal, a tablet terminal, a personal computer, electric appliances such as a netbook terminal, a slate terminal, an electronic book terminal, a portable music player, etc. And wearable terminals such as smart glasses and head mounted displays.

In the image analysis result providing system of FIG. 1, first, the storage module 231 of the computer 200 stores a plurality of learned models in the storage unit 230 (step S01). The learned model may be acquired from another computer or storage medium, or may be created by the computer 200. In addition, the storage unit 230 may be provided with a dedicated database.

FIG. 10 is a diagram showing examples of mathematical expressions and parameters calculated by machine learning and examples of images used for machine learning as learned models of level crossing A, level crossing B and level crossing C. In addition, it is a figure which shows an example of the unknown image of the level crossing D in which the learning completed model is not yet created.

FIG. 12 is an example of a table showing a data structure of a learned model for each camera. In the present invention, a learned model is a model in which an equation for analyzing an image of a subject is associated with a parameter for each camera. At the same time, the image file with supervised data used in machine learning for calculating a learned model may be associated. Further, as an imaging condition for each camera, an imaging angle and an imaging position may be associated and stored. The learned model created using the supervised data by the camera A which captured the crossing A in FIG. 10 is the learned model created using the supervised data by the camera B which captured the learned model A and the crossing B. The learned model C created by using the trained model B and the supervised data from the camera C that has captured the level crossing C is the trained model C. Further, for an image obtained by imaging the level crossing D with the camera D, a learned model is not created.

FIG. 11 is a diagram for schematically describing the relationship between the camera 100, the computer 200, and the subject 400. It is assumed that the camera 100 and the computer 200 can communicate with each other via the communication network 300. The camera 100 in the present invention is an imaging device capable of measuring the distance to a subject. As for the method of measuring the distance to the subject, in the case where the subject can be simultaneously imaged from a plurality of different directions in addition to acquisition from the sensor or the like of the camera 100, the deviation of the image captured by each of the plurality of cameras It is also possible to measure the distance by learning the length and the actual distance. Also, it is possible to calculate the imaging angle using the measured distance. Furthermore, when the location of the camera 100 is fixed, the distance to the imaging location may be explicitly specified. Further, with regard to the imaging angle, the number of times the camera 100 is inclined from the horizontal direction is taken as the imaging angle. In the example of FIG. 11, for the subject 400, the imaging angle of the camera 100 is 30 degrees, and the imaging position, that is, the imaging distance is 5-6 m. Here, as an imaging condition, an imaging angle and an imaging position were taken as an example, but in addition to this, in the case of entrance detection to a level crossing, the presence or absence of an alarm, the presence or absence of a circuit breaker, and whether a line is a single or double , Etc., may be included to help determine the degree of similarity between the unknown image and the learned model.

As a preliminary step of storing a plurality of learned models in the storage unit 230, for machine learning by supervised learning, it is necessary to prepare a large number of images with appropriate labels as teaching data and perform machine learning. In addition, it takes a long time for learning. Therefore, it is preferable to prepare a learned model in which a width is given to imaging conditions for each camera on the assumption that a new unknown image is to be analyzed later. In addition, with regard to the label for supervised learning, assuming that it is necessary to actually provide a detailed image analysis result, in the case of entry detection of level crossing, only “with entry / without entry”. "No entry, there is an entry (adult), there is an entry (adult), there is an entry (child), there is an entry (old man), there is an entry (vehicle), there is an entry (bicycle), there is an entry (animal), there is a possibility of entry" It is necessary to add in accordance with the purpose of the system, considering whether detailed division is necessary.

Returning to FIG. 1, the camera 100 transmits imaging data, which is an unknown image, to the computer 200 (step S02), and the acquisition module 211 of the computer 200 acquires the unknown image (step S03). The acquisition module 211 acquires imaging conditions such as an imaging angle and an imaging position from the camera 100 together with the unknown image. Here, a flow for transmitting imaging data which is an unknown image from the camera 100 has been described, but the acquisition module 211 instructs the camera 100 to transmit imaging data, and the camera 100 transmits imaging data upon receiving the instruction. You may In addition, the acquisition module 211 may acquire not only images acquired in real time by the camera 100 but also images acquired by the camera 100 in the past and stored in the storage unit 130. .

Next, the selection module 212 of the computer 200 selects, from among the learned models stored in step S01, a learned model whose imaging condition is similar to the unknown image acquired in step S03 (step S04). Here, for example, when the captured unknown image is the crossing D in FIG. 10, the imaging condition is from the learned model from any of the crossing A, the crossing B, and the crossing C where the learned model exists. Choose something similar. Assuming that the imaging angle of the camera D that captures the crossing D is 20 degrees, the imaging distance is 4-5 m, and further analysis of the composition of the image that captures the crossing D results in selecting the learned model B here I assume.

Next, the image analysis module 213 of the computer 200 performs image analysis of the unknown image captured by the camera D using the learned model B (step S05).

FIG. 13 is an example of a table showing a learned model used for image analysis for each camera when there is no learned model of the unknown image captured by the camera D. In step S04, since the learned model B is selected as the learned model whose imaging condition is similar to the level crossing D imaged by the camera D, the learned model B as the used model is displayed in the column of the camera D in FIG. The table is filled assuming that yyyyyy is used as a mathematical expression and BBB, b and β are used as parameters. Here, it is desirable to fill in the imaging conditions with the actual camera D at the imaging angle and the imaging position which are the imaging conditions. In addition, since supervised learning is not performed on an image captured by the camera D, the field of the teacher data may be left blank. After that, until the learned model of the camera D is created or the number of other learned models increases, the selection module 212 selects the learned model most suitable for the camera D again, as shown in FIG. Image analysis of the camera D can be performed using the table.

Finally, the provision module 241 of the computer 200 provides the input / output unit 240 of the computer 200 with the image analysis result (step S06). In the case of entry detection to the railroad crossing D, only "with entry, no entry" or "without entry, with entry (adult), with entry (child), with entry (old man), with entry (vehicle), with entry (Bicycle) · There is an entry (animal) · There is a possibility of entry etc. Or image display results such as displaying the result with a warning sound or light as well as displaying the image Output shall be made in accordance with the purpose of the provided system. In addition, here, the detection of entrance to a level crossing has been described as an example, but image analysis includes, for example, face recognition for personal determination, discrimination of pest damage status of agricultural products, inventory confirmation in a warehouse, affected area for medical diagnosis It is assumed that the present invention is applicable to an appropriate one according to the purpose of the system, such as image recognition.

[Description of each function]
FIG. 2 is a diagram showing the relationship between the functional blocks of the camera 100 and the computer 200 and the respective functions. The camera 100 includes an imaging unit 10, a control unit 110, a communication unit 120, and a storage unit 130. The computer 200 also includes a control unit 210, a communication unit 220, a storage unit 230, and an input / output unit 240. The control unit 210 cooperates with the communication unit 220 and the storage unit 230 to realize the acquisition module 211. Further, the control unit 210 cooperates with the storage unit 230 to implement the selection module 212 and the image analysis module 213. The storage unit 230 implements the storage module 231 in cooperation with the control unit 210. The input / output unit 240 implements the provision module 241 in cooperation with the control unit 210 and the storage unit 230. The communication network 300 may be a public communication network such as the Internet or a dedicated communication network, and enables communication between the camera 100 and the computer 200.

The camera 100 includes a lens, an imaging element, various buttons, an imaging device such as a flash as an imaging unit 10, and captures an image as a captured image such as a moving image or a still image. Further, an image obtained by imaging is a precise image having an amount of information necessary for image analysis. Further, the resolution at the time of imaging, the camera angle, the camera magnification, and the like may be designated.

The control unit 110 includes a central processing unit (CPU), a random access memory (RAM), a read only memory (ROM), and the like.

A device for enabling communication with other devices as the communication unit 120, for example, an IMT-2000 standard such as a WiFi (Wireless Fidelity) compliant device compliant with IEEE 802.11 or a third generation or fourth generation mobile communication system A compliant wireless device is provided. It may be a wired LAN connection.

The storage unit 130 includes a storage unit of data using a hard disk or a semiconductor memory, and stores captured images, necessary data such as imaging conditions, and the like.

The control unit 210 includes a CPU, a RAM, a ROM, and the like. The control unit 210 cooperates with the communication unit 220 and the storage unit 230 to realize the acquisition module 211. Further, the control unit 210 cooperates with the storage unit 230 to implement the selection module 212 and the image analysis module 213.

A device for enabling communication with other devices as the communication unit 220, for example, a wireless device compliant with IEEE 802.11 or a wireless device compliant with IMT-2000 such as a third generation or fourth generation mobile communication system Etc. It may be a wired LAN connection.

The storage unit 230 includes a storage unit of data using a hard disk or a semiconductor memory, and stores data necessary for processing of a captured image, teacher data, an image analysis result, and the like. The storage unit 230 implements the storage module 231 in cooperation with the control unit 210. In addition, the storage unit 230 may include a database of learned models.

The input / output unit 240 has a function necessary to use the image analysis result providing system. The input / output unit 240 implements the provision module 241 in cooperation with the control unit 210 and the storage unit 230. As an example for realizing the input, it is possible to provide a liquid crystal display for realizing a touch panel function, a keyboard, a mouse, a pen tablet, hardware buttons on the device, a microphone for performing voice recognition, and the like. Further, as an example for realizing the output, a form such as a liquid crystal display, a display of a PC, a display such as a projection on a projector, and an audio output can be considered. The present invention is not particularly limited in function by the input / output method.

[Image analysis result provision processing]
FIG. 3 is a flow chart in the case of acquiring an unknown image from the camera 100, performing image analysis processing by the computer 200, and providing an image analysis result. The processing executed by each module described above will be described along with this processing.

First, the storage module 231 of the computer 200 stores a plurality of learned models in the storage unit 230 (step S301). The learned model may be acquired from another computer or storage medium, or may be created by the computer 200. Further, the storage unit 230 may be provided with a dedicated database for storing the learned model. The process of step S301 may be skipped if there is no new learned model, if a plurality of learned models have already been stored.

FIG. 10 is a diagram showing an example of a mathematical expression and parameters calculated by machine learning and an image used for machine learning as a learned model of level crossing A, level crossing B and level crossing C. In addition, it is a figure which shows an example of the unknown image of the level crossing D in which the learning completed model is not yet created.

FIG. 11 is a diagram for schematically describing the relationship between the camera 100, the computer 200, and the subject 400. It is assumed that the camera 100 and the computer 200 can communicate with each other via the communication network 300. The camera 100 in the present invention is an imaging device capable of measuring the distance to a subject. As for the method of measuring the distance to the subject, in the case where the subject can be simultaneously imaged from a plurality of different directions in addition to acquisition from the sensor or the like of the camera 100, the deviation of the image captured by each of the plurality of cameras It is also possible to measure the distance by learning the length and the actual distance. Also, it is possible to calculate the imaging angle using the measured distance. Furthermore, when the location of the camera 100 is fixed, the distance to the imaging location may be explicitly specified. Further, with regard to the imaging angle, the number of times the camera 100 is inclined from the horizontal direction is taken as the imaging angle. In the example of FIG. 11, for the subject 400, the imaging angle of the camera 100 is 30 degrees, and the imaging position, that is, the imaging distance is 5-6 m. Here, as an imaging condition, an imaging angle and an imaging distance were taken as an example, but in addition to this, in the case of entrance detection to a level crossing, the presence or absence of an alarm, the presence or absence of a circuit breaker, and whether the line is a single line or a double line , Etc., may be included to help determine the degree of similarity between the unknown image and the learned model.

Returning to FIG. 3, the acquisition module 211 of the computer 200 requests the camera 100 to transmit an image (step 302). If there is no learned model for the image of the camera 100 at the time of the transmission request, the image acquired from the camera 100 is an unknown image.

In response to the image transmission request from the computer 200, the camera 100 performs imaging with the imaging unit 10 (step S303).

Then, the camera 100 transmits imaging data, which is an unknown image, to the computer 200 via the communication unit 120 (step S304).

The acquisition module 211 of the computer 200 acquires an unknown image (step S305). Here, imaging conditions such as an imaging angle and an imaging position are acquired from the camera 100 together with the unknown image. The acquisition module 211 may acquire an image captured by the camera 100 in the past and stored in the storage unit 130, in addition to acquiring an image captured by the camera 100 in real time.

Next, the selection module 212 of the computer 200 selects a learned model having similar imaging conditions to the unknown image acquired in step S305 from the learned models stored in step S301 (step S306). Here, for example, when the captured unknown image is the crossing D in FIG. 10, the imaging condition is from the learned model from any of the crossing A, the crossing B, and the crossing C where the learned model exists. Choose something similar. Assuming that the imaging angle of the camera D that captures the crossing D is 20 degrees, the imaging distance is 4-5 m, and further analysis of the composition of the image that captures the crossing D results in selecting the learned model B here I assume.

Next, the image analysis module 213 of the computer 200 performs image analysis of the unknown image captured by the camera D using the learned model B (step S307).

FIG. 13 is an example of a table showing a learned model used for image analysis for each camera when there is no learned model of the unknown image captured by the camera D. In step S306, since the learned model B is selected as a learned model whose imaging conditions are similar to the level crossing D imaged by the camera D, the learned model B as a use model is selected in the column of camera D in FIG. The table is filled assuming that yyyyyy is used as a mathematical expression and BBB, b and β are used as parameters. Here, it is desirable to fill in the imaging conditions with the actual camera D at the imaging angle and the imaging position which are the imaging conditions. In addition, since supervised learning is not performed on an image captured by the camera D, the field of the teacher data may be left blank. After that, until the learned model of the camera D is created or the number of other learned models increases, the selection module 212 selects the learned model most suitable for the camera D again, as shown in FIG. Image analysis of the camera D can be performed using the table.

Finally, the provision module 241 of the computer 200 provides the input / output unit 240 of the computer 200 with the image analysis result (step S308). In the case of entry detection to the railroad crossing D, only "with entry, no entry" or "without entry, with entry (adult), with entry (child), with entry (old man), with entry (vehicle), with entry (Bicycle) · There is an entry (animal) · There is a possibility of entry etc. Or image display results such as displaying the result with a warning sound or light as well as displaying the image Output shall be made in accordance with the purpose of the provided system. In addition, here, the detection of entrance to a level crossing has been described as an example, but image analysis includes, for example, face recognition for personal determination, discrimination of pest damage status of agricultural products, inventory confirmation in a warehouse, affected area for medical diagnosis It is assumed that the present invention is applicable to an appropriate one according to the purpose of the system, such as image recognition. Further, the provision of the image analysis result does not have to be limited to the output to the input / output unit 240 of the computer 200, and the output according to the system is performed such as outputting to other devices via the communication unit 220. I assume.

As described above, according to the present invention, among a plurality of patterns of machine-learned learned models in which artificial intelligence has analyzed a known image, a known image whose imaging condition is similar to an unknown image to be newly analyzed An image analysis result providing system, an image analysis result providing method, and a program capable of outputting an accurate image analysis result without spending a learning time by selecting and using a learned model of It becomes possible.

[Learned model creation processing]
FIG. 4 is a view showing the relationship between the functional blocks of the camera 100 and the computer 200 and the respective functions in the case of performing a learned model creation process of an unknown image. In addition to the configuration of FIG. 2, the control unit 210 of the computer 200 cooperates with the storage unit 230 to implement the creation module 214. FIG. 5 is a flowchart of the camera 100 and the computer 200 in the case of performing a learned model creation process of an unknown image. The processing executed by each module described above will be described along with this processing. The processes in steps S501 to S503 in FIG. 5 correspond to the processes in steps S301 to S303 in FIG. As in step S301, the process of step S501 may be skipped if a plurality of learned models are already stored and if a new learned model does not exist.

In response to the image transmission request from the computer 200, the camera 100 transmits the unknown image captured by the imaging unit 10 to the computer 200 via the communication unit 120 (step S504). Here, in order to perform learning model creation processing of an unknown image, it is desirable to acquire as many unknown images as possible captured by the camera 100. Therefore, not only an image captured by the camera 100 in real time, but also an image captured by the camera 100 in the past and stored in the storage unit 130 may be transmitted.

The acquisition module 211 of the computer 200 acquires a plurality of unknown images (step S505). Here, imaging conditions such as an imaging angle and an imaging position are acquired from the camera 100 together with the respective unknown images.

Next, the creation module 214 of the computer 200 assigns teacher data to the unknown image acquired in step S505 (step S506). Here, an operation of adding a label that is a correct answer of the image analysis result to a plurality of acquired unknown images is assigned as teacher data. As mentioned above, with regard to labels for supervised learning, assuming that it is necessary to provide detailed image analysis results in actuality, if there is entry detection at a level crossing, “with entry / without entry” There is a possibility of no entry, entry (adult), entry (adult), entry (child), entry (old man), entry (vehicle), entry (bicycle), entry (animal), entry It is necessary to add in accordance with the purpose of the system, considering whether detailed division is necessary.

The creation module 214 of the computer 200 performs machine learning by supervised learning using the unknown image to which the teacher data is added (step S507).

Next, the creation module 214 creates a learned model of the unknown image based on the result of the machine learning in step S507 (step S508).

Finally, the storage module 231 stores the learned model of the unknown image in the storage unit 230 (step S509).

FIG. 14 is an example of a table showing a learned model used for image analysis for each camera when a learned model of an unknown image captured by a camera D is created. The learned model D of the camera D created in step S508 is described in the column of the camera D in FIG. The table is filled, assuming that a learned model D as a use model of the camera D, vvvvvvvv as a mathematical expression, and DDD, d, dD as parameters are used. Also, since supervised learning was performed, the teacher data column also describes teacher data. From this point onward, it is possible to perform image analysis of the camera D using the table of FIG. 14 until the training data is increased again by the camera D to create a learned model.

Since it is necessary to prepare a large number of images with appropriate labels and to take time to learn, in order to raise the accuracy of image analysis for a new unknown image, this trained model creation processing needs time-consuming learning. It becomes a heavy burden when accepting a new unknown image after starting the operation of the image analysis result provision system. Therefore, when new learned model creation processing can not be performed, it is possible to provide an image analysis result with a certain degree of accuracy without spending a learning time by the method described in FIGS. 2 and 3. is suggesting. However, in order to obtain better accuracy, it is desirable to perform machine learning in accordance with an unknown image and create a learned model of mathematical expressions and parameters more suitable. Therefore, according to the timing at which a large number of unknown images required for machine learning can be stored and the timing at which an appropriate correct label can be added, the system should not be stressed so as not to affect other image analysis tasks. It is necessary to properly execute the system according to the operation of the image analysis result provision system, such as execution.

As described above, according to the present invention, among a plurality of patterns of machine-learned learned models in which artificial intelligence has analyzed a known image, a known image whose imaging condition is similar to an unknown image to be newly analyzed By selecting and using the already-learned model, while outputting accurate image analysis results without spending learning time, machine learning in accordance with the new unknown image is performed by learning-made model creation processing. By creating a learned model, it is possible to provide an image analysis result providing system, an image analysis result providing method, and a program that can further improve the image analysis result.

[Image analysis switching process]
FIG. 6 is a diagram showing the relationship between the functional blocks of the camera 100 and the computer 200 and the respective functions when the image analysis process is switched depending on whether the learned model creation process of the unknown image is finished. FIG. 7 is a process corresponding to A of the flowchart of FIG. 6, and when the learning model creation process of the unknown image is not completed, the learned model of the unknown image performed by the computer 200 when the machine learning of the unknown image is possible. It is a flowchart figure of preparation processing. FIG. 8 is a process corresponding to B of the flowchart of FIG. 6, and when the learned model creation process of the unknown image is not finished, the learned model selection process performed by the computer 200 when the machine learning of the unknown image is impossible. FIG. The configurations of the camera 100 and the computer 200 are the same as in FIG. Further, since the processes of step S601 to step S605 of FIG. 6 correspond to the processes of step S301 to step S305 of FIG. 3, step S606 and subsequent steps will be described.

The creation module 214 of the computer 200 confirms whether the learned model of the unknown image has been created for the imaging data acquired in step S605 (step S606). Here, if the imaging data acquired in step S605 is acquisition data from the camera 100 for the first time, it is an unknown image, so it is considered that a learned model of the unknown image is not created.

If a learned model of the unknown image is not created, the creation module 214 determines whether machine learning is possible using the unknown image acquired in step S605 or the previously stored unknown image ( Step S607). Here, whether there is a large amount of images prepared as appropriately labeled teacher data necessary for machine learning, whether the system is stressed and whether it takes time for machine learning or not, It can be judged on the basis of Further, the determination may be made according to the operation status of the image analysis result providing system, and the determination according to the system is performed.

Here, if it is determined that machine learning of the unknown image is possible, the process proceeds to the flowchart of process A of FIG. 7 (step S608).

In process A of FIG. 7, the creation module 214 of the computer 200 adds teacher data to the unknown image acquired in step S605 or the unknown image stored before that (step S701). Here, an operation of adding a label that is a correct answer of the image analysis result to a plurality of acquired unknown images is assigned as teacher data. As mentioned above, with regard to labels for supervised learning, assuming that it is necessary to provide detailed image analysis results in actuality, if there is entry detection at a level crossing, “with entry / without entry” There is a possibility of no entry, entry (adult), entry (adult), entry (child), entry (old man), entry (vehicle), entry (bicycle), entry (animal), entry It is necessary to add in accordance with the purpose of the system, considering whether detailed division is necessary.

The creating module 214 of the computer 200 performs machine learning by supervised learning using the unknown image to which the teacher data is added (step S702).

Next, the creation module 214 creates a learned model of the unknown image based on the result of the machine learning in step S702 (step S703).

Next, the storage module 231 stores the learned model of the unknown image in the storage unit 230 (step S704).

FIG. 14 is an example of a table showing a learned model used for image analysis for each camera when a learned model of an unknown image captured by a camera D is created. The learned model D of the camera D created in step S703 is described in the column of the camera D in FIG. The table is filled, assuming that a learned model D as a use model of the camera D, vvvvvvvv as a mathematical expression, and DDD, d, dD as parameters are used. Also, since supervised learning was performed, the teacher data column also describes teacher data. From this point onward, it is possible to perform image analysis of the camera D using the table of FIG. 14 until the training data is increased again by the camera D to create a learned model.

Next, the image analysis module 213 of the computer 200 performs image analysis of the unknown image captured by the camera D using the created learned model D (step S705).

Next, the provision module 241 of the computer 200 provides the input / output unit 240 of the computer 200 with the image analysis result (step S706). Thereafter, the process returns to the flowchart of FIG. 6 and proceeds to step S614.

If it is determined in step S607 that machine learning of the unknown image is not possible, the creation module 214 stores the unknown image acquired in step S605 in the storage unit 230 (step S609). This is for later use as teacher data when performing machine learning for learned model creation processing on an unknown image.

After the storage process of step S609, the process proceeds to the flowchart of process B of FIG. 8 (step S610).

The selection module 212 of the computer 200 selects a learned model whose imaging condition is similar to the unknown image acquired in step S605 from the learned models stored in step S601 (step S801). Here, for example, when the captured unknown image is the crossing D in FIG. 10, the imaging condition is from the learned model from any of the crossing A, the crossing B, and the crossing C where the learned model exists. Choose something similar. Assuming that the imaging angle of the camera D that captures the crossing D is 20 degrees, the imaging distance is 4-5 m, and further analysis of the composition of the image that captures the crossing D results in selecting the learned model B here I assume.

Next, the image analysis module 213 performs image analysis of the unknown image captured by the camera D using the learned model B (step S802).

FIG. 13 is an example of a table showing a learned model used for image analysis for each camera when there is no learned model of the unknown image captured by the camera D. In step S801, the learned model B is selected as a learned model having similar imaging conditions to the level crossing D captured by the camera D. Therefore, in the field of the camera D in FIG. The table is filled assuming that yyyyyy is used as a mathematical expression and BBB, b and β are used as parameters. Here, it is desirable to fill in the imaging conditions with the actual camera D at the imaging angle and the imaging position which are the imaging conditions. In addition, since supervised learning is not performed on an image captured by the camera D, the field of the teacher data may be left blank. After that, until the learned model of the camera D is created or the number of other learned models increases, the selection module 212 selects the learned model most suitable for the camera D again, as shown in FIG. Image analysis of the camera D can be performed using the table.

Next, the provision module 241 provides the input / output unit 240 of the computer 200 with the image analysis result (step S803). Thereafter, the process returns to the flowchart of FIG. 6 and proceeds to step S614.

When the learned model of the unknown image is created in step S606 in FIG. 6, it is considered that the learned model of the unknown image has been created through the flow of process A. In this case, the selection module 212 selects and applies the learned model D created in step S703 of process A (step S611).

Next, the image analysis module 213 of the computer 200 performs image analysis of the unknown image captured by the camera D using the learned model D (step S612).

Next, the provision module 241 of the computer 200 provides the input / output unit 240 of the computer 200 with the image analysis result (step S613).

In step S706, step S803, and step S613, the provision module 241 provides the image analysis result to the input / output unit 240 of the computer 200. "No entry, no entry (adult), entry (adult), entry (child), entry (old man), entry (vehicle), entry (bicycle), entry (animal), entry possibility" It is assumed that the output according to the purpose of the image analysis result providing system is performed, such as displaying in detail, or displaying the result with a warning sound or light as well as displaying the image. In addition, here, the detection of entrance to a level crossing has been described as an example, but image analysis includes, for example, face recognition for personal determination, discrimination of pest damage status of agricultural products, inventory confirmation in a warehouse, affected area for medical diagnosis It is assumed that the present invention is applicable to an appropriate one according to the purpose of the system, such as image recognition. Further, the provision of the image analysis result does not have to be limited to the output to the input / output unit 240 of the computer 200, and the output according to the system is performed such as outputting to other devices via the communication unit 220. I assume.

Finally, it is confirmed whether the image analysis result provision process may be ended (step S614). If not completed, the process returns to step S602 to continue the process, and when completed, the image analysis result provision process is ended. Do.

As described above, according to the present invention, among a plurality of patterns of machine-learned learned models in which artificial intelligence has analyzed a known image, a known image whose imaging condition is similar to an unknown image to be newly analyzed By selecting and using the already-learned model, while outputting accurate image analysis results without spending learning time, machine learning in accordance with the new unknown image is performed by learning-made model creation processing. , It is possible to create a trained model. Also, until a new learned model is created, a trained model of known images is selected and used, and after a new learned model is created, a more accurate dedicated learned model By using this, it is possible to provide an image analysis result providing system, an image analysis result providing method, and a program having high accuracy and capable of further improving the accuracy from the start of citation of the system.

[Trained model creation processing using image analysis result of unknown image]
FIG. 9 is a flowchart of the computer 200 in the case of performing machine learning of an unknown image by adding an image analysis result of the unknown image as teacher data when performing a learned model creation process of the unknown image. The configurations of the camera 100 and the computer 200 are the same as in FIG. In FIG. 9, although described from step S901, before this, processing equivalent to step S301 to step S308 in FIG. 3 is performed, and image analysis of an unknown image by the selected learned model is performed, It is assumed that the image analysis result has been obtained.

The creation module 214 of the computer 200 assigns the image analysis result in step S307 as teacher data to the unknown image acquired in step S305 (step S901). Here, by adding the image analysis result of the unknown image according to the selected learned model as teaching data as it is, the cost of manually adding teacher data to be correct data to a large amount of images necessary for machine learning is It will be possible to reduce significantly.

The creating module 214 performs machine learning by supervised learning using the unknown image to which the teacher data is added (step S902).

Next, the creation module 214 creates a learned model of the unknown image based on the result of the machine learning in step S902 (step S903).

Finally, the storage module 231 stores the learned model of the unknown image in the storage unit 230 (step S904).

As described above, according to the present invention, among a plurality of patterns of machine-learned learned models in which artificial intelligence has analyzed a known image, a known image whose imaging condition is similar to an unknown image to be newly analyzed By selecting and using the already-learned model, while outputting the accurate image analysis result without taking a long learning time, furthermore, the image analysis result of the unknown image by the selected learned model is directly used as teaching data. As a result, it becomes possible to significantly reduce the cost of manually adding teacher data as correct data to a large amount of images required for machine learning, and finally improving the image analysis result. It is possible to provide an image analysis result providing system, an image analysis result providing method, and a program that can

The above-described means and functions are realized by a computer (including a CPU, an information processing device, and various terminals) reading and executing a predetermined program. The program may be provided, for example, from a computer via a network (SaaS: software as a service), a flexible disk, a CD (CD-ROM, etc.), a DVD (DVD-ROM, DVD) Provided in the form of being recorded in a computer readable recording medium such as a RAM, a compact memory, etc. In this case, the computer reads the program from the recording medium, transfers the program to an internal storage device or an external storage device, stores it, and executes it. Alternatively, the program may be recorded in advance in a storage device (recording medium) such as, for example, a magnetic disk, an optical disk, or a magneto-optical disk, and may be provided from the storage device to the computer via a communication line.

As mentioned above, although embodiment of this invention was described, this invention is not limited to these embodiment mentioned above. Further, the effects described in the embodiments of the present invention only list the most preferable effects resulting from the present invention, and the effects according to the present invention are limited to those described in the embodiments of the present invention is not.

100 cameras, 200 computers, 300 networks, 400 subjects

Claims

Storage means for storing a machine-learned learned model obtained by image analysis of a known image;
Acquisition means for acquiring an unknown image for which a learned model has not yet been created;
Selecting means for selecting a learned model of the known image having similar imaging conditions to the acquired unknown image from the stored learned models;
Image analysis means for performing image analysis of the unknown image using the selected learned model;
Providing means for providing the result of the image analysis;
An image analysis result providing system comprising:
The image analysis result providing system according to claim 1, wherein the imaging condition is an imaging position and an imaging angle with respect to an imaging target.
The image analysis result according to claim 1 or 2, wherein the learned model is a formula and parameters used for the image analysis calculated by the machine learning and an image used for the machine learning. system.
The image analysis result providing system according to any one of claims 1 to 3, further comprising: creation means for creating a new machine-learned learned model obtained by image analysis for the unknown image. .
The image analysis means is characterized in that the unknown image is subjected to image analysis using the selected learned model during a period until the new machine-learned learned model is created. Image analysis result providing system described in.
Creating a new machine-learned learned model in which the unknown image is image-analyzed using the analysis result of the image analysis of the unknown image as teacher data using the selected learned model for the unknown image The image analysis result providing system according to any one of claims 1 to 4, further comprising creation means.
Storing a machine-learned learned model obtained by image analysis of a known image;
Acquiring an unknown image for which a learned model has not yet been created;
Selecting a learned model of the known image whose imaging condition is similar to the acquired unknown image from the stored learned models;
Analyzing the unknown image using the selected learned model;
Providing the result of the image analysis;
An image analysis result providing method comprising:
Image analysis result provision system,
Storing a machine-learned learned model obtained by image analysis of known images;
Acquiring an unknown image for which a learned model has not yet been created;
Selecting a learned model of the known image whose imaging condition is similar to the acquired unknown image from the stored learned models;
Analyzing the unknown image using the selected learned model;
Providing the result of the image analysis;
A program to run a program.