WO2020158954A1 - Service building device, service building method, and service building program - Google Patents

Service building device, service building method, and service building program Download PDF

Info

Publication number
WO2020158954A1
WO2020158954A1 PCT/JP2020/003915 JP2020003915W WO2020158954A1 WO 2020158954 A1 WO2020158954 A1 WO 2020158954A1 JP 2020003915 W JP2020003915 W JP 2020003915W WO 2020158954 A1 WO2020158954 A1 WO 2020158954A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
learning
unit
model
function
Prior art date
Application number
PCT/JP2020/003915
Other languages
French (fr)
Japanese (ja)
Inventor
常人 萱沼
古賀　直樹
Original Assignee
株式会社コンピュータマインド
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 株式会社コンピュータマインド filed Critical 株式会社コンピュータマインド
Priority to JP2020539874A priority Critical patent/JPWO2020158954A1/en
Publication of WO2020158954A1 publication Critical patent/WO2020158954A1/en
Priority to JP2021198561A priority patent/JP2022033153A/en

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F8/00—Arrangements for software engineering
    • G06F8/60—Software deployment
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N20/00—Machine learning

Definitions

  • the present invention relates to an information processing device.
  • Patent Document 1 discloses Devices that have a learning function using technologies such as deep learning.
  • the conventional apparatus having a learning function is not versatile because it is manufactured as a dedicated machine for performing only specific learning.
  • the present invention has been made in view of such a situation, and an object of the present invention is to improve the efficiency of sales of devices having a learning function.
  • an information processing device of one embodiment of the present invention is An annotation means that adds predetermined information that can be an annotation of the image to the image, A learning unit that performs learning by using the image provided with the predetermined information as a teacher image and generates a model, Deploying means for making the generated model usable under a predetermined environment, Equipped with.
  • FIG. 3 is a functional block diagram showing an example of a functional configuration required for various processes executed by the information processing device of FIG. 2. It is a figure which shows the flow of a process in a modeling function. It is a figure which shows the outline
  • FIG. 1 is a flowchart showing an outline of functions of an embodiment of an information processing apparatus of the present invention. Note that, hereinafter, the processing of the information processing apparatus 1 targets image data, but hereinafter, unless otherwise specified, “data” is omitted and simply referred to as “image”. The "image” is a broad concept including a still image and a moving image.
  • the functions of the information processing device 1 include a “modeling function” and an “inference library function”.
  • the “modeling function” means learning using deep running or the like with respect to the image BF as the material of the teacher data, using the data with annotations such as the correct answer as the teacher data.
  • This is a function of generating a file MF (hereinafter, referred to as “model file MF”) in a predetermined format by performing the image identification/object detection/segmentation model.
  • the model file MF is a file generated according to a file format used as a model in an inference process described later.
  • the information processing apparatus 1 exerts a modeling function to perform image acquisition processing (step S1), annotation processing (step S2), learning processing (step S3), and deployment processing (step S4).
  • the model files MF are generated by sequentially executing.
  • the "image acquisition process” refers to a process of acquiring an image BF which is a material of teacher data.
  • the “annotation process” refers to a process of adding information (annotations such as correct answers) used as teacher data as metadata to the acquired image BF.
  • the image BF to which the metadata is added is stored and managed in the teacher DB 401 as an image TF as teacher data (hereinafter referred to as “teacher image TF”).
  • teacher image TF teacher data
  • the “learning process” refers to a process of generating or updating a model of image identification/object detection/segmentation by performing learning using a technique such as deep learning using the teacher image TF.
  • the “deployment process” is a process of making the model file MF of the generated model into a model file MF so that it can be used in a predetermined environment.
  • the model file MF generated by exhibiting the modeling function in the information processing device 1 is stored and managed in the model DB 402 described later.
  • the model file MF stored in the model DB 402 is managed for each version. For example, as shown in FIG. Each of (version) 1 to n (n is an integer value of 1 or more) is managed.
  • the “inference library function” is a function of reading a model file MF generated by the modeling function to make a component of a program capable of executing inference processing into a library.
  • the library created by the inference library function is referred to as "deep learning package solution”.
  • the modeling function and the inference library function of the information processing device 1 can provide a package solution of the learning function that enables the promotion and sales of the person who develops the device having the learning function to be facilitated. it can.
  • the present embodiment it is possible to solve the business problem of the device having the deep learning function.
  • the following problems have existed as problems in sales of a device having a conventional deep learning function. That is, there is no fixed package solution for the device having the deep learning function. For this reason, when doing business, it was forced to rely on the manpower of the staff of the sales department. There is also a problem that it is difficult to explain the price basis to the customer.
  • the feature of this embodiment is that "anyone can easily challenge the development of AI (deep learning)".
  • AI deep learning
  • sufficient knowledge of Linux (registered trademark) and software is required in order to build an environment in which the deep learning function is exerted by itself. For this reason, even if a person who does not have such knowledge tries to build an environment for exerting the deep learning function by himself, there are many hurdles.
  • the modeling function includes annotation processing (step S2), learning processing (step S3), and deployment processing (step S4).
  • the inference library function is provided with a program component that reads the model file generated by the modeling function to perform the deployment process and actually infers the model file.
  • the following services can be realized. That is, it can be provided as educational content for universities as a browser version of "DeepEye Machine Vision". In this case, it can be provided at a low price as an Acoustic version. It can also be provided as OEM (Original Equipment Manufacturer) for manufacturer products. In this case, a GUI (Graphical User Interface) can be customized for the customer and provided as a service or option for the product.
  • GUI Graphic User Interface
  • FIG. 2 is a block diagram showing the hardware configuration of the information processing device 1 of FIG.
  • the information processing device 1 includes a GPU (Graphics Processing Unit) 10, a CPU (Central Processing Unit) 11, a ROM (Read Only Memory) 12, a RAM (Random Access Memory) 13, a bus 14, and an input/output interface 15. 1, an output unit 16, an input unit 17, a storage unit 18, a communication unit 19, and a drive 20.
  • GPU Graphics Processing Unit
  • CPU Central Processing Unit
  • ROM Read Only Memory
  • RAM Random Access Memory
  • the GPU 10 executes routine arithmetic processing according to a program recorded in the ROM 12 or a program loaded from the storage unit 18 into the RAM 13. Specifically, the GPU 10 speeds up deep learning operations by repeatedly executing parallel processing of enormous operations required for learning processing and inference processing. In addition, the GPU 10 performs arithmetic processing required when performing image depiction.
  • the RAM 13 also appropriately stores data and the like necessary for the GPU 10 to execute arithmetic processing.
  • the CPU 11 executes various processes according to a program recorded in the ROM 12 or a program loaded from the storage unit 18 into the RAM 13.
  • the RAM 13 also appropriately stores data and the like necessary for the CPU 11 to execute various processes.
  • the GPU 10, the CPU 11, the ROM 12, and the RAM 13 are connected to each other via a bus 14.
  • An input/output interface 15 is also connected to the bus 14.
  • An output unit 16, an input unit 17, a storage unit 18, a communication unit 19, and a drive 20 are connected to the input/output interface 15.
  • the output unit 16 includes various liquid crystal displays and outputs various information.
  • the input unit 17 is composed of various types of hardware such as lead, and inputs various types of information.
  • the storage unit 18 is configured by a DRAM (Dynamic Random Access Memory) or the like, and stores various data.
  • the communication unit 19 controls communication with other devices via the network N including the Internet.
  • the drive 20 is provided as needed.
  • a removable medium 30, which is composed of a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like, is appropriately mounted on the drive 20.
  • the program read from the removable medium 30 by the drive 20 is installed in the storage unit 18 as needed.
  • the removable medium 30 can also store various data stored in the storage unit 18 in the same manner as the storage unit 18.
  • the OS is “Ubuntu 16.0.4 LTS”
  • the CPU eg CPU 11 in FIG. 2
  • the memory eg RAM 13 in FIG. 2
  • SSD SSD (Sold State Drive) is 500 G.
  • SATA SSD SATA SSD
  • HDD Hard Disk Drive
  • 3TB TeraByte
  • ODD Optical Disk Drive
  • DVD Super Multi power supply 1000W
  • GPU eg GPU 10 in FIG. 2 “GeforceTi RTX 20”. It is shown.
  • FIG. 3 is a functional block diagram showing an example of a functional configuration required for various processes executed by the information processing device 1 of FIG.
  • the modeling unit 101 functions when the modeling process is executed. Further, when the inference library processing is executed, the inference library unit 102 functions.
  • a teacher DB 401, a model DB 402, and a library DB 403 are provided in one area of the storage unit 18 of the information processing device 1.
  • the “modeling process” means a series of processes executed by the information processing apparatus 1 in which the modeling function of FIG. 1 described above is performed, that is, an image acquisition process (for example, step S1 of FIG. 1) and an annotation process ( For example, it refers to a process in which the learning process (for example, step S3 in FIG. 1) and the deployment process (for example, step S4 in FIG. 1) are sequentially executed.
  • the "inference library process” refers to a series of processes executed by the information processing device 1 in which the inference library function of FIG.
  • the modeling unit 101 has an image acquisition unit 111, an annotation unit 112, a learning unit 113, and a deployment unit 114.
  • the image acquisition unit 111 executes an image acquisition process (for example, step S1 in FIG. 1). Specifically, the image acquisition unit 111 acquires the image BF that is the material of the teacher data.
  • the annotation unit 112 executes annotation processing (for example, step S2 in FIG. 1). Specifically, the annotation unit 112 attaches, to the acquired image BF, information used as teacher data (annotations such as the contents of the correct answer) as metadata.
  • the learning unit 113 executes a learning process (eg, step S3 in FIG. 1). Specifically, the learning unit 113 generates or updates a model of image identification/object detection/segmentation by performing learning using a technique such as deep learning using the teacher image TF.
  • the deploy unit 114 executes a deploy process (eg, step S4 in FIG. 1). Specifically, the deploying unit 114 sets the generated model as the model file MF so that the model file MF can be used in a predetermined environment.
  • a deploy process eg, step S4 in FIG. 1.
  • the inference library unit 102 executes inference library processing.
  • the library forming unit 121 reads the model file MF generated by the above-described modeling function to make a library of parts that constitute a program that enables inference processing.
  • a deep learning package solution is generated by the information processing device 1 having the above functional configuration executing the modeling process and the inference library process described above.
  • a user's own system having a deep learning function can be easily constructed by simply installing the deep learning package solution in the application program developed by the customer.
  • FIG. 4 is a diagram showing the flow of processing in the modeling function.
  • the modeling function acquisition of an image BF, annotation (for example, the above-mentioned annotation processing), learning based on teacher data (for example, the above-mentioned teacher image TF) generated by the annotation (for example, the above-mentioned learning processing) ,
  • teacher data for example, the above-mentioned teacher image TF
  • the model generated by learning is deployed (for example, the deploy process of generating the model file MF described above).
  • the learning result can be output as a report.
  • the information processing apparatus 1 performs a modeling function as an image acquisition process (step S1) to display an image BF of a label (hereinafter, referred to as “wine label”) attached to a wine bottle, for example. get.
  • the annotation process step S2
  • the information processing device 1 adds metadata (for example, information on wine specified by the wine label, specifically, brand, origin, year of manufacture, etc.) to the image BF.
  • the teacher image TF is generated.
  • the information processing device 1 performs learning using the teacher image TF to infer the wine brand or the like indicated by the wine label from the wine label (the image including the subject as a subject). Generate a model.
  • the information processing apparatus 1 generates a wine label “.DEEP file” as a model file MF for the model as a deploy process (step S4).
  • FIG. 5 is a diagram showing an outline of the inference library function.
  • the user can install the “DeepEye Predictor” (the deep learning package solution described above) in the application program developed by the user.
  • This allows the user to build a system using deep learning. It can also be compatible with OSs (Operating Systems) such as Windows (registered trademark) and Ubuntu.
  • DeepEye Predictor is provided as a library that supports various languages such as C++, C#, and Python. For example, it is provided in DLL (Dynamic Link Library) or the like.
  • DLL Dynamic Link Library
  • FIG. 6 is a diagram showing a method of issuing “DeepEye Predictor”.
  • “DeepEye Predictor” is license-managed by the information processing device 1 functioning as a license management server. Therefore, when the library is created from the created model (.DEEP file), “DeepEye Predictor” is issued from the license management information processing apparatus (information processing apparatus 1) via the Internet or the like.
  • FIG. 7 is a diagram showing “DeepEye Predictor” after issuance.
  • wine brand application an application program that enables extraction of information such as a brand of wine from an image obtained by capturing a wine label.
  • a system using deep learning can be provided by mounting "DeepEye Predictor” (deep learning package solution) on the wine brand application. Then, by reading the ".DEEP file” (model file MF) of the wine label and making it into a library, the customer can develop a wine brand application that can be used standalone.
  • the AI environment can be realized without the user being aware of the deep learning library (environment construction). That is, in the development of conventional deep learning, it is essential for an engineer to build an environment, which is a trouble.
  • the deep learning according to the present embodiment since a deep learning tool with integrated hardware is used, the trouble of constructing the environment is unnecessary. In other words, if you get a deep learning tool, you can immediately develop deep learning according to your purpose. As a result, the convenience of the user who develops deep learning can be improved.
  • the processes from annotation, learning, test, and deployment can be performed in a series of flows.
  • annotation, learning, and testing can be performed in a series of flows in the development of deep learning can be expected to have a significant time saving effect compared to the process of deep learning development using conventional methods.
  • the created inference model can be executed from another application.
  • the deep learning function can be easily incorporated via the above-mentioned “DeepEye Predictor” to which the present embodiment is applied. This makes it possible to add a deep learning function to an existing application program at low cost. Also, the fact that the user can embed annotations into existing application programs "on their own” means that deep learning can be used without leaking confidential data owned by a company to the outside.
  • a person who does not have knowledge of AI development can try deep learning on a GUI basis. Specifically, it becomes possible to set hyperparameters, select a network, and display a visualization function (Gradcam). Also, image classification annotations can be easily implemented by drag and drop operations. That is, various networks can be evaluated on the basis of GUI only by adjusting the parameters. As described above, according to the present embodiment, even a person who does not have knowledge of AI development need only perform an operation of selecting from given options, which is convenient for the user and enlarges the user. You can expect a big merit in.
  • the wine label has been described as the subject of the image, but this is only an example, and any object can be the subject.
  • the hardware configuration shown in FIG. 2 is merely an example for achieving the object of the present invention, and is not particularly limited.
  • the functional block diagram shown in FIG. 3 is merely an example and is not particularly limited. That is, it is sufficient if the information processing system has a function capable of executing the above-described series of processes as a whole, and what kind of functional block is used to realize this function is not particularly limited to the example of FIG. ..
  • the location of the functional block is not limited to that shown in FIG. 3 and may be arbitrary. Further, one functional block may be configured by hardware alone, software alone, or a combination thereof.
  • the program forming the software is installed in a computer or the like from a network or a recording medium.
  • the computer may be a computer embedded in dedicated hardware.
  • the computer may be a computer capable of executing various functions by installing various programs, for example, a general-purpose smartphone or a personal computer other than an information processing device.
  • the recording medium containing such a program is not only constituted by a removable medium which is distributed separately from the apparatus main body in order to provide the program to each user, but is also pre-installed in the apparatus main body to each user. It is composed of a recording medium provided.
  • the steps of writing the program recorded on the recording medium include, not only the processing performed in time series according to the order, but also the processing performed in parallel or individually not necessarily in time series. It also includes the processing to be executed.
  • system means the entire device including a plurality of devices and a plurality of means.
  • the information processing apparatus to which the present invention is applied may have various configurations as long as it has the following configuration. That is, the information processing apparatus (for example, the information processing apparatus 1) to which the present invention is applied is An annotation means (for example, the annotation unit 112 in FIG. 3) that adds predetermined information (for example, metadata) that can be an annotation of the image to the image (for example, the image BF in FIG. 1), A learning unit (for example, the learning unit 113 in FIG. 3) that performs learning by using the image to which the predetermined information is given as a teacher image (for example, a teacher image TF) and generates a model, Deploying means (for example, the deploying unit 114 in FIG. 3) that puts the generated model into a usable state (for example, a model file MF) under a predetermined environment (for example, a system constructed by a user), Equipped with.
  • a usable state for example, a model file MF
  • a predetermined environment for example, a system constructed by
  • a library means for example, a library forming unit 121 which forms a library of components of a program that executes inference processing by reading the model.
  • the user can install the deep learning package solution in the application program developed by the user, so that the system using the deep learning can be easily constructed.
  • non-browser applications will be able to perform image classification, object detection, and segmentation tasks using deep learning. Not only learning but also annotation function for each task. In other words, even the annotation function is included.
  • the Data Augmentation can be set. There is a data replication function. It is possible to manage the verification results of deep learning in project units like Visual studio. Manage data, trained data, architects, and models consistently. Annotation is possible with simple operations such as drag and drop (from Explorer to app). It can be deployed not only on PC but also on various devices such as Jetson, FPGA, iPhone (registered trademark), MOVIDIUS, ARM. The status of the current work is easily possible. You can visualize the status of annotations>learning>inference> deployment.
  • the deep learning model visualization function (Gradcam) is packaged. By visualizing the value of the weight in each layer and the intermediate calculation result, it is possible to feed back to the learning data and to devise the learning data. The result of classification can be visually displayed in a confusion matrix.

Landscapes

  • Engineering & Computer Science (AREA)
  • Software Systems (AREA)
  • Theoretical Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Evolutionary Computation (AREA)
  • Medical Informatics (AREA)
  • Data Mining & Analysis (AREA)
  • Computing Systems (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Mathematical Physics (AREA)
  • Artificial Intelligence (AREA)
  • Stored Programmes (AREA)
  • Image Analysis (AREA)

Abstract

The present invention addresses the problem of improving the efficiency of business relating to devices and the like having a learning function.　An image acquiring unit 111 executes an image acquisition process. An annotation unit 112 assigns, to an image BF, prescribed information that may serve as an annotation for the image BF. A learning unit 113 generates a model by performing learning using the image BF as a teacher image TF. A deployment unit 114 enables the generated model to be used in a prescribed environment. An inference library unit 102 executes an inference library process. A model file MF is generated as a usable state. The abovementioned problem is thus resolved.

Description

[規則37.2に基づきISAが決定した発明の名称]　サービス構築装置、サービス構築方法及びサービス構築プログラム[Invention name determined by ISA based on Rule 37.2] Service construction device, service construction method, and service construction program
　本発明は、情報処理装置に関する。 The present invention relates to an information processing device.
　ディープラーニング等の技術を用いた学習機能を有する装置は従来から存在する。（特許文献１参照）。 Devices that have a learning function using technologies such as deep learning have existed in the past. (See Patent Document 1).
特開２０１８－２０６２６２号公報JP, 2008-206262, A
　しかしながら、従来の学習機能を有する装置は、特定の学習等のみを行う専用機として製造されるため汎用性がなかった。
　つまり、ディープラーニング等の技術を用いた学習機能には、パッケージソリューションが存在しない。このため、各種各様な学習機能を有する装置等の開発提供を請け負う者が顧客に見積金額を提示する際、その算定根拠の説明に困難を伴うことが多かった。 However, the conventional apparatus having a learning function is not versatile because it is manufactured as a dedicated machine for performing only specific learning.
In other words, there are no package solutions for learning functions that use technologies such as deep learning. For this reason, when a person who undertakes the development and provision of a device having various learning functions presents the estimated amount to the customer, it is often difficult to explain the calculation basis.
　本発明は、このような状況に鑑みてなされたものであり、学習機能を有する装置等についての営業の効率化を図ることを目的とする。 The present invention has been made in view of such a situation, and an object of the present invention is to improve the efficiency of sales of devices having a learning function.
　上記目的を達成するため、本発明の一態様の情報処理装置は、
　画像に対し、当該画像の注釈となり得る所定情報を付与するアノテーション手段と、
　前記所定情報が付与された前記画像を教師画像とする学習を行い、モデルを生成する学習手段と、
　生成された前記モデルを所定の環境下で使用可能な状態にするデプロイ手段と、
　を備える。 In order to achieve the above object, an information processing device of one embodiment of the present invention is
An annotation means that adds predetermined information that can be an annotation of the image to the image,
A learning unit that performs learning by using the image provided with the predetermined information as a teacher image and generates a model,
Deploying means for making the generated model usable under a predetermined environment,
Equipped with.
　本発明によれば、学習機能を有する装置等についての営業の効率化を図ることができる。 According to the present invention, it is possible to improve the efficiency of sales of a device having a learning function.
本発明の情報処理装置の一実施形態に係る情報処理装置の機能の概要を示すフロー図である。It is a flow figure showing an outline of a function of an information processor concerning one embodiment of an information processor of the present invention. 図１の情報処理装置のハードウェア構成を示すブロック図である。It is a block diagram which shows the hardware constitutions of the information processing apparatus of FIG. 図２の情報処理装置が実行する各種処理に必要となる機能的構成の一例を示す機能ブロック図である。FIG. 3 is a functional block diagram showing an example of a functional configuration required for various processes executed by the information processing device of FIG. 2. モデリング機能における処理の流れを示す図である。It is a figure which shows the flow of a process in a modeling function. 推論ライブラリ機能の概要を示す図である。It is a figure which shows the outline|summary of an inference library function. 「ＤｅｅｐＥｙｅ　Ｐｒｅｄｉｃｔｏｒ」の発行方法を示す図である。It is a figure which shows the issuing method of "DeepEye Predictor." 発行後の「ＤｅｅｐＥｙｅ　Ｐｒｅｄｉｃｔｏｒ」を示す図である。It is a figure which shows "DeepEye Predictor" after issuance.
　以下、本発明の実施形態について図面を用いて説明する。 Embodiments of the present invention will be described below with reference to the drawings.
　図１は、本発明の情報処理装置の一実施形態の機能の概要を示すフロー図である。
　なお、以下、情報処理装置１の処理は、画像のデータが対象となるが、以下、特に断りのない限り「データ」を省略して、単に「画像」と略記する。
　また、「画像」は、静止画像と動画像とを含む広義な概念である。 FIG. 1 is a flowchart showing an outline of functions of an embodiment of an information processing apparatus of the present invention.
Note that, hereinafter, the processing of the information processing apparatus 1 targets image data, but hereinafter, unless otherwise specified, “data” is omitted and simply referred to as “image”.
The "image" is a broad concept including a still image and a moving image.
　図１に示すように、情報処理装置１の機能には、「モデリング機能」と、「推論ライブラリ機能」とが含まれる。 As shown in FIG. 1, the functions of the information processing device 1 include a “modeling function” and an “inference library function”.
　情報処理装置１の機能のうち「モデリング機能」とは、教師用データの素材としての画像ＢＦに対して、正解等の注釈がついたデータを教師用データとして、ディープランニング等を用いた学習を行うことで、画像識別・物体検出・セグメンテーションのモデルを所定フォーマットのファイルＭＦ（以下、「モデルファイルＭＦ」と呼ぶ）を生成する機能のことをいう。
　ここで、モデルファイルＭＦとは、後述する推論処理においてモデルとして使用されるファイルフォーマットに従って生成されるファイルをいう。
　具体的には例えば、情報処理装置１は、モデリング機能を発揮することで、画像取得処理（ステップＳ１）、アノテーション処理（ステップＳ２）、学習処理（ステップＳ３）、及びデプロイ処理（ステップＳ４）を順次実行して、モデルファイルＭＦを生成する。 Among the functions of the information processing apparatus 1, the “modeling function” means learning using deep running or the like with respect to the image BF as the material of the teacher data, using the data with annotations such as the correct answer as the teacher data. This is a function of generating a file MF (hereinafter, referred to as “model file MF”) in a predetermined format by performing the image identification/object detection/segmentation model.
Here, the model file MF is a file generated according to a file format used as a model in an inference process described later.
Specifically, for example, the information processing apparatus 1 exerts a modeling function to perform image acquisition processing (step S1), annotation processing (step S2), learning processing (step S3), and deployment processing (step S4). The model files MF are generated by sequentially executing.
　ここで、「画像取得処理」とは、教師用データの素材となる画像ＢＦを取得する処理をいう。
　「アノテーション処理」とは、取得された画像ＢＦに対し、教師用データとして用いる情報（正解の内容等の注釈）をメタデータとして付与する処理をいう。メタデータが付与された画像ＢＦは、教師用データとしての画像ＴＦ（以下、「教師画像ＴＦ」と呼ぶ）として教師ＤＢ４０１に記憶されて管理される。
　「学習処理」とは、教師画像ＴＦを用いて、ディープラーニング等の技術を用いた学習を行うことで、画像識別・物体検出・セグメンテーションのモデルを生成又は更新する処理をいう。
　「デプロイ処理」とは、生成されたモデルについて、モデルファイルＭＦについて、所定の環境下で使用することができるように、モデルファイルＭＦとする処理をいう。
　情報処理装置１においてモデリング機能が発揮されるにより生成されたモデルファイルＭＦは、後述するモデルＤＢ４０２に記憶されて管理される。なお、モデルＤＢ４０２に記憶されているモデルファイルＭＦは、バージョン毎に管理されており、例えば図１に示すように、Ｖｅｒ．（バージョン）１乃至ｎ（ｎは１以上の整数値）の夫々が管理されている。 Here, the "image acquisition process" refers to a process of acquiring an image BF which is a material of teacher data.
The “annotation process” refers to a process of adding information (annotations such as correct answers) used as teacher data as metadata to the acquired image BF. The image BF to which the metadata is added is stored and managed in the teacher DB 401 as an image TF as teacher data (hereinafter referred to as “teacher image TF”).
The “learning process” refers to a process of generating or updating a model of image identification/object detection/segmentation by performing learning using a technique such as deep learning using the teacher image TF.
The “deployment process” is a process of making the model file MF of the generated model into a model file MF so that it can be used in a predetermined environment.
The model file MF generated by exhibiting the modeling function in the information processing device 1 is stored and managed in the model DB 402 described later. The model file MF stored in the model DB 402 is managed for each version. For example, as shown in FIG. Each of (version) 1 to n (n is an integer value of 1 or more) is managed.
　情報処理装置１の機能のうち「推論ライブラリ機能」とは、モデリング機能により生成されたモデルファイルＭＦを読み込むことで、推論処理を実行可能とするプログラムを構成する部品をライブラリ化させる機能のことをいう。以下、推論ライブラリ機能によりライブラリ化されたものを「ディープラーニングパッケージソリューション」と呼ぶ。
　このようにすることで、ユーザ（図示せず）により開発されたアプリケーションプログラムに対して、ディープラーニングパッケージソリューションを搭載させるだけで、ディープラーニングの機能を有する独自のシステムを容易に構築することができる。 Among the functions of the information processing apparatus 1, the “inference library function” is a function of reading a model file MF generated by the modeling function to make a component of a program capable of executing inference processing into a library. Say. Hereinafter, the library created by the inference library function is referred to as "deep learning package solution".
By doing so, it is possible to easily build a unique system having a deep learning function simply by mounting the deep learning package solution on the application program developed by the user (not shown). ..
　以上のように、情報処理装置１のモデリング機能及び推論ライブラリ機能により、学習機能を有する装置の開発を行う者における宣伝・営業の容易化を可能とする、学習機能のパッケージソリューションを提供することができる。 As described above, the modeling function and the inference library function of the information processing device 1 can provide a package solution of the learning function that enables the promotion and sales of the person who develops the device having the learning function to be facilitated. it can.
　即ち、ディープラーニングを用いた画像識別・物体検出・セグメンテーションのモデルの作成を行う統合ソフトウェア（例えば上述のディープラーニングパッケージソリューション）を開発して販売することができる。
　また、ソフトウェア単体での販売を行わずに、ハードウェアとのセットで販売することができる。また、ソフトウェア環境が既に構築された状態でディープラーニングパッケージソリューションを出荷することができる。
　これにより、ユーザは、ハードウェアの導入とともに即時使用可能とすることができる。また例えば、ディープラーニングの営業の効率化を図ることができる。また例えば、受託開発への展開を図ることができる。また例えば、ライセンスビジネスへの展開を図ることができる。 That is, it is possible to develop and sell integrated software (for example, the deep learning package solution described above) that creates a model for image identification, object detection, and segmentation using deep learning.
In addition, it is possible to sell as a set with hardware without selling the software as a single unit. We can also ship deep learning package solutions with the software environment already built.
This allows the user to immediately use the hardware as soon as it is installed. Further, for example, it is possible to improve the efficiency of deep learning sales. In addition, for example, it is possible to expand to contract development. Further, for example, it is possible to develop into a license business.
　また、本実施形態によれば、ディープラーニング機能を有する装置の営業上の問題を解消することができる。例えば、従来からあるディープラーニング機能を有する装置の営業上の問題点として、以下のような問題が存在していた。即ち、ディープラーニング機能を有する装置には、定型のパッケージソリューションが存在しない。このため、営業を行う場合、営業部門のスタッフのマンパワーに依存せざるを得なかった。また、顧客への価格根拠の説明が難しいという問題があった。
　このような問題に対して、本実施形態によれば、（１）パッケージ販売の宣伝・営業が行い易くなる、（２）パッケージ販売で完結すれば、「それでよし」とすることができる、（３）顧客からの要求があれば、コンサルタント契約の締結等に繋げることができる、といったことが実現可能となる。 Further, according to the present embodiment, it is possible to solve the business problem of the device having the deep learning function. For example, the following problems have existed as problems in sales of a device having a conventional deep learning function. That is, there is no fixed package solution for the device having the deep learning function. For this reason, when doing business, it was forced to rely on the manpower of the staff of the sales department. There is also a problem that it is difficult to explain the price basis to the customer.
With respect to such a problem, according to the present embodiment, (1) it becomes easy to advertise and sell the package sale, and (2) if the package sale is completed, it is possible to say “it is good” ( 3) It becomes possible to connect to the conclusion of a consultant contract, etc., if requested by the customer.
　本実施形態の特徴は、換言すると、「誰でも簡単にＡＩ（ディープラーニング）の開発にチャレンジできる」というものである。
　ここで、従来からある技術のみを用いて、ディープラーニング機能を発揮させる環境を構築することもできる。例えば、パーソナルコンピュータを自前で用意し、これにオープンソースソフトウェアをいくつかインストールすれば、ディープラーニング機能を発揮させる環境を自前で構築することができる。
　しかしながら、ディープラーニング機能を発揮させる環境を自前で構築するためには、Ｌｉｎｕｘ（登録商標）やソフトウェアに関する十分な知識が必要となる。このため、このような知識を持ち合わせていない者が、ディープラーニング機能を発揮させる環境を自前で構築しようとしても、そこには多くのハードルが存在する。また、ディープラーニング機能を発揮させる環境を構築することができたとしても、データ管理、アノテーション、学習、及び推論を実行するためのソフトウェアは、夫々異なるのが一般的である。また、データ管理、及び学習済みのモデルの管理を自前で行わなければならない。さらに、多くのオープンソースソフトウェアは、システムエンジニアにより使用されることを前提として作られているため、専門用語を理解する必要があるだけではなく、多くの複雑なパラメータの設定を行う必要がある。本実施形態が適用される製品である「ＤｅｅｐＥｙｅ」は、これらの多くのハードルを取り除くことができる製品である。 In other words, the feature of this embodiment is that "anyone can easily challenge the development of AI (deep learning)".
Here, it is also possible to build an environment for exerting the deep learning function by using only conventional technology. For example, if you prepare your own personal computer and install some open source software on it, you can build your own environment to exercise the deep learning function.
However, sufficient knowledge of Linux (registered trademark) and software is required in order to build an environment in which the deep learning function is exerted by itself. For this reason, even if a person who does not have such knowledge tries to build an environment for exerting the deep learning function by himself, there are many hurdles. Further, even if an environment for exerting a deep learning function can be constructed, software for executing data management, annotation, learning, and inference are generally different from each other. In addition, data management and management of learned models must be done by themselves. Further, since many open source softwares are designed to be used by system engineers, it is necessary not only to understand technical terms but also to set many complicated parameters. The product to which this embodiment is applied, "DeepEye," is a product that can remove many of these hurdles.
　上述したように、モデリング機能には、アノテーション処理（ステップＳ２）、学習処理（ステップＳ３）、デプロイ処理（ステップＳ４）が含まれる。また、推論ライブラリ機能には、モデリング機能によりデプロイ処理がなされたこと生成された、モデルファイルを読み込んで実際に推論を行うプログラム部品が提供される。 As described above, the modeling function includes annotation processing (step S2), learning processing (step S3), and deployment processing (step S4). In addition, the inference library function is provided with a program component that reads the model file generated by the modeling function to perform the deployment process and actually infers the model file.
　また、本実施形態によれば、例えば以下のようなサービスを実現させることができる。即ち、ブラウザ版の「ＤｅｅｐＥｙｅ　Ｍａｃｈｉｎｅ　Ｖｉｓｉｏｎ」として、大学向けの教育コンテンツとして提供することができる。この場合、Ａｃａｍｅｍｉｃ版として安価で提供できることもできる。また、メーカ製品向けのＯＥＭ（Ｏｒｉｇｉｎａｌ　Ｅｑｕｉｐｍｅｎｔ　Ｍａｎｕｆａｃｔｕｒｅｒ）として提供することもできる。この場合、ＧＵＩ（Ｇｒａｐｈｉｃａｌ Ｕｓｅｒ Ｉｎｔｅｒｆａｃｅ）を顧客向けにカスタマイズして、製品のサービス又はオプションとして提供してもらうことができる。 Further, according to this embodiment, for example, the following services can be realized. That is, it can be provided as educational content for universities as a browser version of "DeepEye Machine Vision". In this case, it can be provided at a low price as an Acoustic version. It can also be provided as OEM (Original Equipment Manufacturer) for manufacturer products. In this case, a GUI (Graphical User Interface) can be customized for the customer and provided as a service or option for the product.
　次に、モデリング機能及び推論ライブラリ機能を発揮させるための各種処理を実行する情報処理装置１のハードウェア構成について説明する。
　図２は、図１の情報処理装置１のハードウェア構成を示すブロック図である。 Next, the hardware configuration of the information processing apparatus 1 that executes various processes for exhibiting the modeling function and the inference library function will be described.
FIG. 2 is a block diagram showing the hardware configuration of the information processing device 1 of FIG.
　情報処理装置１は、ＧＰＵ（Ｇｒａｐｈｉｃｓ　Ｐｒｏｃｅｓｓｉｎｇ　Ｕｎｉｔ）１０と、ＣＰＵ（Ｃｅｎｔｒａｌ　Ｐｒｏｃｅｓｓｉｎｇ　Ｕｎｉｔ）１１と、ＲＯＭ（Ｒｅａｄ　Ｏｎｌｙ　Ｍｅｍｏｒｙ）１２と、ＲＡＭ（Ｒａｎｄｏｍ　Ａｃｃｅｓｓ　Ｍｅｍｏｒｙ）１３と、バス１４と、入出力インターフェース１５と、出力部１６と、入力部１７と、記憶部１８と、通信部１９と、ドライブ２０とを備えている。 The information processing device 1 includes a GPU (Graphics Processing Unit) 10, a CPU (Central Processing Unit) 11, a ROM (Read Only Memory) 12, a RAM (Random Access Memory) 13, a bus 14, and an input/output interface 15. 1, an output unit 16, an input unit 17, a storage unit 18, a communication unit 19, and a drive 20.
　ＧＰＵ１０は、ＲＯＭ１２に記録されているプログラム、又は、記憶部１８からＲＡＭ１３にロードされたプログラムに従って定型的な演算処理を実行する。具体的には、ＧＰＵ１０は、学習処理及び推論処理に必要となる膨大な演算の並列処理を繰り返し実行することで、ディープラーニングの演算を高速化させる。また、ＧＰＵ１０は、画像描写を行う際に必要となる演算処理を行う。
　ＲＡＭ１３には、ＧＰＵ１０が演算処理を実行する上において必要なデータ等も適宜記憶される。 The GPU 10 executes routine arithmetic processing according to a program recorded in the ROM 12 or a program loaded from the storage unit 18 into the RAM 13. Specifically, the GPU 10 speeds up deep learning operations by repeatedly executing parallel processing of enormous operations required for learning processing and inference processing. In addition, the GPU 10 performs arithmetic processing required when performing image depiction.
The RAM 13 also appropriately stores data and the like necessary for the GPU 10 to execute arithmetic processing.
　ＣＰＵ１１は、ＲＯＭ１２に記録されているプログラム、又は、記憶部１８からＲＡＭ１３にロードされたプログラムに従って各種の処理を実行する。
　ＲＡＭ１３には、ＣＰＵ１１が各種の処理を実行する上において必要なデータ等も適宜記憶される。 The CPU 11 executes various processes according to a program recorded in the ROM 12 or a program loaded from the storage unit 18 into the RAM 13.
The RAM 13 also appropriately stores data and the like necessary for the CPU 11 to execute various processes.
　ＧＰＵ１０、ＣＰＵ１１、ＲＯＭ１２及びＲＡＭ１３は、バス１４を介して相互に接続されている。このバス１４にはまた、入出力インターフェース１５も接続されている。入出力インターフェース１５には、出力部１６、入力部１７、記憶部１８、通信部１９及びドライブ２０が接続されている。 The GPU 10, the CPU 11, the ROM 12, and the RAM 13 are connected to each other via a bus 14. An input/output interface 15 is also connected to the bus 14. An output unit 16, an input unit 17, a storage unit 18, a communication unit 19, and a drive 20 are connected to the input/output interface 15.
　出力部１６は各種液晶ディスプレイ等で構成され、各種情報を出力する。
　入力部１７は、各種ハードウェア鉛等で構成され、各種情報を入力する。
　記憶部１８は、ＤＲＡＭ（Ｄｙｎａｍｉｃ　Ｒａｎｄｏｍ　Ａｃｃｅｓｓ　Ｍｅｍｏｒｙ）等で構成され、各種データを記憶する。
　通信部１９は、インターネットを含むネットワークＮを介して他の装置との間で行う通信を制御する。 The output unit 16 includes various liquid crystal displays and outputs various information.
The input unit 17 is composed of various types of hardware such as lead, and inputs various types of information.
The storage unit 18 is configured by a DRAM (Dynamic Random Access Memory) or the like, and stores various data.
The communication unit 19 controls communication with other devices via the network N including the Internet.
　ドライブ２０は、必要に応じて設けられる。ドライブ２０には磁気ディスク、光ディスク、光磁気ディスク、或いは半導体メモリ等よりなる、リムーバブルメディア３０が適宜装着される。ドライブ２０によってリムーバブルメディア３０から読み出されたプログラムは、必要に応じて記憶部１８にインストールされる。またリムーバブルメディア３０は、記憶部１８に記憶されている各種データも、記憶部１８と同様に記憶することができる。 The drive 20 is provided as needed. A removable medium 30, which is composed of a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like, is appropriately mounted on the drive 20. The program read from the removable medium 30 by the drive 20 is installed in the storage unit 18 as needed. The removable medium 30 can also store various data stored in the storage unit 18 in the same manner as the storage unit 18.
　具体的には例えば、次のようなハードウェア構成とすることができる。即ち、ＯＳが「Ｕｂｕｎｔｕ　１６．０．４ＬＴＳ」、ＣＰＵ（例えば図２のＣＰＵ１１）が「Ｃｏｒｅ　ｉ７－８７００Ｋ」、メモリ（例えば図２のＲＡＭ１３）の容量が３２ＧＢ、ＳＳＤ（Ｓｏｉｌｄ　Ｓｔａｔｅ　Ｄｒｉｖｅ）が５００Ｇ　ＳＡＴＡ　ＳＳＤ、ＨＤＤ（Ｈａｒｄ　Ｄｉｓｋ　Ｄｒｉｖｅ）が３ＴＢ（ＴｅｒａＢｙｔｅ）、ＯＤＤ（Ｏｐｔｉｃａｌ　Ｄｉｓｋ　Ｄｒｉｖｅ）が「ＤＶＤ　Ｓｕｐｅｒ　Ｍｕｌｔｉ」、電源が１０００Ｗ、ＧＰＵ（例えば図２のＧＰＵ１０）が「Ｇｅｆｏｒｃｅ　ＲＴＸ　２０８０Ｔｉ」であることが示されている。 Specifically, for example, the following hardware configuration can be used. That is, the OS is “Ubuntu 16.0.4 LTS”, the CPU (eg CPU 11 in FIG. 2) is “Core i7-8700K”, the memory (eg RAM 13 in FIG. 2) has a capacity of 32 GB, and SSD (Sold State Drive) is 500 G. SATA SSD, HDD (Hard Disk Drive) 3TB (TeraByte), ODD (Optical Disk Drive) “DVD Super Multi”, power supply 1000W, GPU (eg GPU 10 in FIG. 2) “GeforceTi RTX 20”. It is shown.
　次に、図２のハードウェア構成を有する情報処理装置１の機能的構成について説明する。
　図３は、図２の情報処理装置１が実行する各種処理に必要となる機能的構成の一例を示す機能ブロック図である。 Next, the functional configuration of the information processing device 1 having the hardware configuration of FIG. 2 will be described.
FIG. 3 is a functional block diagram showing an example of a functional configuration required for various processes executed by the information processing device 1 of FIG.
　図３に示すように、情報処理装置１のＧＰＵ１０（また、図示はしないがＣＰＵ１１）においては、モデリング処理が実行される場合には、モデリング部１０１が機能する。また、推論ライブラリ処理が実行される場合には、推論ライブラリ部１０２が機能する。
　なお、情報処理装置１の記憶部１８の一領域には、教師ＤＢ４０１と、モデルＤＢ４０２と、ライブラリＤＢ４０３とが設けられている。 As shown in FIG. 3, in the GPU 10 (also the CPU 11 (not shown)) of the information processing device 1, the modeling unit 101 functions when the modeling process is executed. Further, when the inference library processing is executed, the inference library unit 102 functions.
A teacher DB 401, a model DB 402, and a library DB 403 are provided in one area of the storage unit 18 of the information processing device 1.
　ここで、「モデリング処理」とは、上述の図１のモデリング機能が発揮された情報処理装置１により実行される一連の処理、即ち、画像取得処理（例えば図１のステップＳ１）、アノテーション処理（例えば図１のステップＳ２）、学習処理（例えば図１のステップＳ３）、及びデプロイ処理（例えば図１のステップＳ４）が順次実行される処理をいう。
　「推論ライブラリ処理」とは、上述の図１の推論ライブラリ機能が発揮された情報処理装置１により実行される一連の処理をいう。 Here, the “modeling process” means a series of processes executed by the information processing apparatus 1 in which the modeling function of FIG. 1 described above is performed, that is, an image acquisition process (for example, step S1 of FIG. 1) and an annotation process ( For example, it refers to a process in which the learning process (for example, step S3 in FIG. 1) and the deployment process (for example, step S4 in FIG. 1) are sequentially executed.
The "inference library process" refers to a series of processes executed by the information processing device 1 in which the inference library function of FIG.
　モデリング部１０１は、画像取得部１１１と、アノテーション部１１２と、学習部１１３と、デプロイ部１１４とを有する。 The modeling unit 101 has an image acquisition unit 111, an annotation unit 112, a learning unit 113, and a deployment unit 114.
　画像取得部１１１は、画像取得処理（例えば図１のステップＳ１）を実行する。具体的には、画像取得部１１１は、教師用データの素材となる画像ＢＦを取得する The image acquisition unit 111 executes an image acquisition process (for example, step S1 in FIG. 1). Specifically, the image acquisition unit 111 acquires the image BF that is the material of the teacher data.
　アノテーション部１１２は、アノテーション処理（例えば図１のステップＳ２）を実行する。具体的には、アノテーション部１１２は、取得された画像ＢＦに対し、教師用データとして用いる情報（正解の内容等の注釈）をメタデータとして付与する The annotation unit 112 executes annotation processing (for example, step S2 in FIG. 1). Specifically, the annotation unit 112 attaches, to the acquired image BF, information used as teacher data (annotations such as the contents of the correct answer) as metadata.
　学習部１１３は、学習処理（例えば図１のステップＳ３）を実行する。具体的には、学習部１１３は、教師画像ＴＦを用いて、ディープラーニング等の技術を用いた学習を行うことで、画像識別・物体検出・セグメンテーションのモデルを生成又は更新する。 The learning unit 113 executes a learning process (eg, step S3 in FIG. 1). Specifically, the learning unit 113 generates or updates a model of image identification/object detection/segmentation by performing learning using a technique such as deep learning using the teacher image TF.
　デプロイ部１１４は、デプロイ処理（例えば図１のステップＳ４）を実行する。具体的には、デプロイ部１１４は、生成されたモデルについて、モデルファイルＭＦについて、所定の環境下で使用することができるように、モデルファイルＭＦとする The deploy unit 114 executes a deploy process (eg, step S4 in FIG. 1). Specifically, the deploying unit 114 sets the generated model as the model file MF so that the model file MF can be used in a predetermined environment.
　推論ライブラリ部１０２は、推論ライブラリ処理を実行する。 The inference library unit 102 executes inference library processing.
　即ち、ライブラリ化部１２１は、上述のモデリング機能により生成されたモデルファイルＭＦを読み込むことで、推論処理を実行可能とするプログラムを構成する部品をライブラリ化する。 That is, the library forming unit 121 reads the model file MF generated by the above-described modeling function to make a library of parts that constitute a program that enables inference processing.
　以上の機能的構成を有する情報処理装置１が上述のモデリング処理及び推論ライブラリ処理を実行することによりディープラーニングパッケージソリューションが生成される。これにより、顧客が開発するアプリケーションプログラムに、ディープラーニングパッケージソリューションを搭載させるだけで、ディープラーニングの機能を有するユーザ独自のシステムを容易に構築することができる。 A deep learning package solution is generated by the information processing device 1 having the above functional configuration executing the modeling process and the inference library process described above. As a result, a user's own system having a deep learning function can be easily constructed by simply installing the deep learning package solution in the application program developed by the customer.
　図４は、モデリング機能における処理の流れを示す図である。 FIG. 4 is a diagram showing the flow of processing in the modeling function.
　図４に示すように、モデリング機能では、画像ＢＦの取得、アノテーション（例えば上述のアノテーション処理）、アノテーションにより生成された教師データ（例えば上述の教師画像ＴＦ）に基づく学習（例えば上述の学習処理）、学習により生成されたモデルのデプロイ（例えば上述のモデルファイルＭＦを生成するデプロイ処理）を行う。なお、学習の結果は、レポートとして出力することもできる。 As shown in FIG. 4, in the modeling function, acquisition of an image BF, annotation (for example, the above-mentioned annotation processing), learning based on teacher data (for example, the above-mentioned teacher image TF) generated by the annotation (for example, the above-mentioned learning processing) , The model generated by learning is deployed (for example, the deploy process of generating the model file MF described above). The learning result can be output as a report.
　具体的には例えば、情報処理装置１は、画像取得処理（ステップＳ１）として、モデリング機能を発揮させて、例えばワインボトルに貼付されたラベル（以下、「ワインラベル」と呼ぶ）の画像ＢＦを取得する。
　情報処理装置１は、アノテーション処理（ステップＳ２）として、画像ＢＦに対して、メタデータ（例えばそのワインラベルにより特定されるワインの情報、具体的には例えば銘柄、産地、製造年等）を付与することで、教師画像ＴＦを生成する。
　情報処理装置１は、学習処理（ステップＳ３）として、教師画像ＴＦを用いた学習を行うことで、ワインラベル（それを被写体として含む画像）から、そのワインラベルが示すワインの銘柄等を推論するモデルを生成する。
　情報処理装置１は、デプロイ処理（ステップＳ４）として、そのモデルについてのモデルファイルＭＦとして、ワインラベルの「．ＤＥＥＰファイル」を生成する。 Specifically, for example, the information processing apparatus 1 performs a modeling function as an image acquisition process (step S1) to display an image BF of a label (hereinafter, referred to as “wine label”) attached to a wine bottle, for example. get.
As the annotation process (step S2), the information processing device 1 adds metadata (for example, information on wine specified by the wine label, specifically, brand, origin, year of manufacture, etc.) to the image BF. By doing so, the teacher image TF is generated.
As a learning process (step S3), the information processing device 1 performs learning using the teacher image TF to infer the wine brand or the like indicated by the wine label from the wine label (the image including the subject as a subject). Generate a model.
The information processing apparatus 1 generates a wine label “.DEEP file” as a model file MF for the model as a deploy process (step S4).
　図５は、推論ライブラリ機能の概要を示す図である。 FIG. 5 is a diagram showing an outline of the inference library function.
　図５に示すように、推論ライブラリ機能によれば、ユーザは、自身が開発したアプリケーションプログラムに「ＤｅｅｐＥｙｅ　Ｐｒｅｄｉｃｔｏｒ」（上述のディープラーニングパッケージソリューション）を搭載することができる。これにより、ユーザは、ディープラーニングを用いたシステムを構築することができる。Ｗｉｎｄｏｗｓ（登録商標）及びＵｂｕｎｔｕ等のＯＳ（Ｏｐｅｒａｔｉｎｇ　Ｓｙｓｔｅｍ）に対応させることもできる。 As shown in FIG. 5, according to the inference library function, the user can install the “DeepEye Predictor” (the deep learning package solution described above) in the application program developed by the user. This allows the user to build a system using deep learning. It can also be compatible with OSs (Operating Systems) such as Windows (registered trademark) and Ubuntu.
　また、「ＤｅｅｐＥｙｅ　Ｐｒｅｄｉｃｔｏｒ」は、Ｃ＋＋、Ｃ♯、Ｐｙｔｈｏｎなどの各種言語に対応したライブラリで提供される。例えばＤＬＬ（Ｄｙｎａｍｉｃ　Ｌｉｎｋ　Ｌｉｂｒａｒｙ）等で提供される。 ”DeepEye Predictor” is provided as a library that supports various languages such as C++, C#, and Python. For example, it is provided in DLL (Dynamic Link Library) or the like.
　図６は、「ＤｅｅｐＥｙｅ　Ｐｒｅｄｉｃｔｏｒ」の発行方法を示す図である。 FIG. 6 is a diagram showing a method of issuing “DeepEye Predictor”.
　図６に示すように、「ＤｅｅｐＥｙｅ　Ｐｒｅｄｉｃｔｏｒ」は、ライセンス管理用サーバとして機能する情報処理装置１においてライセンス管理される。このため、作成済みのモデル（．ＤＥＥＰファイル）からライブラリが作成される場合には、インターネット等を介してライセンス管理用情報処理装置（情報処理装置１）から「ＤｅｅｐＥｙｅ　Ｐｒｅｄｉｃｔｏｒ」が発行される。 As shown in FIG. 6, “DeepEye Predictor” is license-managed by the information processing device 1 functioning as a license management server. Therefore, when the library is created from the created model (.DEEP file), “DeepEye Predictor” is issued from the license management information processing apparatus (information processing apparatus 1) via the Internet or the like.
　図７は、発行後の「ＤｅｅｐＥｙｅ　Ｐｒｅｄｉｃｔｏｒ」を示す図である。 FIG. 7 is a diagram showing “DeepEye Predictor” after issuance.
　図７に示すように、ライブラリ化された後は、スタンドアロンで使用することができるため、アプリケーションプログラムとして各種の装置内に組み込むことができる。 As shown in Fig. 7, after being made into a library, it can be used standalone, so it can be incorporated into various devices as an application program.
　具体的には例えば、ユーザが、ワインラベルを撮像した画像からワインの銘柄等の情報を抽出可能とするアプリケーションプログラム（以下、「ワイン銘柄アプリ」と呼ぶ）を開発した場合を想定する。
　この場合、ワイン銘柄アプリに、「ＤｅｅｐＥｙｅ　Ｐｒｅｄｉｃｔｏｒ」（ディープラーニングパッケージソリューション）を搭載させることで、ディープラーニングを用いたシステムとすることができる。
　そして、ワインラベルの「．ＤＥＥＰファイル」（モデルファイルＭＦ）を読み込んでライブラリ化させることにより、顧客は、スタンドアロンで使用することができるワイン銘柄アプリを開発することができる。 Specifically, for example, it is assumed that the user has developed an application program (hereinafter, referred to as “wine brand application”) that enables extraction of information such as a brand of wine from an image obtained by capturing a wine label.
In this case, a system using deep learning can be provided by mounting "DeepEye Predictor" (deep learning package solution) on the wine brand application.
Then, by reading the ".DEEP file" (model file MF) of the wine label and making it into a library, the customer can develop a wine brand application that can be used standalone.
　以上をまとめると、本実施形態によれば、以下のような効果が期待できる。
　即ち、本実施形態によれば、ユーザがディープラーニングのライブラリ（環境構築）を意識せずにＡＩ環境を実現できる。即ち、従来のディープラーニングの開発では、エンジニアが環境構築を行うことが必須となっており、これが手間となっていた。これに対して、本実施形態によるディープラーニングは、ハードウェアが一体になったディープラーニングツールが用いられるため、環境構築の手間が不要となる。つまり、ディープラーニングツールを入手すれば、直ちに目的に応じたディープラーニングの開発が可能となる。これにより、ディープラーニングの開発を行うユーザの利便性を向上させることができる。 Summarizing the above, according to the present embodiment, the following effects can be expected.
That is, according to the present embodiment, the AI environment can be realized without the user being aware of the deep learning library (environment construction). That is, in the development of conventional deep learning, it is essential for an engineer to build an environment, which is a trouble. On the other hand, in the deep learning according to the present embodiment, since a deep learning tool with integrated hardware is used, the trouble of constructing the environment is unnecessary. In other words, if you get a deep learning tool, you can immediately develop deep learning according to your purpose. As a result, the convenience of the user who develops deep learning can be improved.
　また例えば、本実施形態によれば、アノテーション、学習、テスト、デプロイまでの処理を一連の流れで行うことができる。ディープラーニングの開発においてアノテーション、学習、テストを一連の流れで行えるということは、従来の手法を用いたディープラーニングの開発のプロセスに比べて大幅な時間削減効果を期待することができる。また、複雑な手順やノウハウを覚える必要が無いという意味でも、効率化につなげることができる。 Further, for example, according to this embodiment, the processes from annotation, learning, test, and deployment can be performed in a series of flows. The fact that annotation, learning, and testing can be performed in a series of flows in the development of deep learning can be expected to have a significant time saving effect compared to the process of deep learning development using conventional methods. In addition, it is possible to improve efficiency in the sense that there is no need to learn complicated procedures and know-how.
　また例えば、本実施形態によれば、作成された推論モデルを、他のアプリケーションから実行することができる。具体的には、本実施形態が適用される上述の「ＤｅｅｐＥｙｅ　Ｐｒｅｄｉｃｔｏｒ」経由で、ディープラーニングの機能を容易に組み込むことができる。これにより、既存のアプリケーションプログラムにディープラーニング機能を追加することを低コストで実現させることができる。また、アノテーションから既存のアプリケーションプログラムへの組み込みをユーザが「自前で」行えるということは、企業が持つ機密データを外部に漏らすことなくディープラーニングを活用できることにつながる。 Further, for example, according to the present embodiment, the created inference model can be executed from another application. Specifically, the deep learning function can be easily incorporated via the above-mentioned “DeepEye Predictor” to which the present embodiment is applied. This makes it possible to add a deep learning function to an existing application program at low cost. Also, the fact that the user can embed annotations into existing application programs "on their own" means that deep learning can be used without leaking confidential data owned by a company to the outside.
　また例えば、本実施形態によれば、ＡＩ開発の知識が無い者であっても、ＧＵＩベースでディープラーニングを試すことができる。具体的には、ハイパーパラメータの設定、ネットワークの選択、及び可視化機能（Ｇｒａｄｃａｍ）の表示が可能となる。また、画像分類アノテーションがドラッグ　アンド　ドロップの操作で容易に実施することができる。即ち、ＧＵＩベースで、様々なネットワークを、パラメータ調整のみで評価できる。このように、本実施形態によれば、ＡＩ開発の知識が無い者であっても、与えられた選択肢から選ぶ操作を行うだけで良いので、ユーザの利便性の面や、ユーザの拡大という面で大きなメリットが期待できる。 Further, for example, according to this embodiment, even a person who does not have knowledge of AI development can try deep learning on a GUI basis. Specifically, it becomes possible to set hyperparameters, select a network, and display a visualization function (Gradcam). Also, image classification annotations can be easily implemented by drag and drop operations. That is, various networks can be evaluated on the basis of GUI only by adjusting the parameters. As described above, according to the present embodiment, even a person who does not have knowledge of AI development need only perform an operation of selecting from given options, which is convenient for the user and enlarges the user. You can expect a big merit in.
　さらに例えば、本実施形態によれば、複数のエッジデバイスへのデプロイが可能となる。また、ブラウザであっても上記同様のアプリケーションプログラムが使用可能となる。また、マルチプラットフォーム（Ｗｉｎｄｏｗｓ・Ｕｂｕｎｔｕ）への対応が可能となる。 Further, for example, according to this embodiment, it is possible to deploy to a plurality of edge devices. Further, even with a browser, the same application program as above can be used. In addition, it is possible to support multi-platform (Windows/Ubuntu).
　以上、本発明の一実施形態について説明したが、本発明は、上述の実施形態に限定されるものではなく、本発明の目的を達成できる範囲での変形、改良等は本発明に含まれるものである。 Although one embodiment of the present invention has been described above, the present invention is not limited to the above-described embodiment, and modifications, improvements, etc. within the scope of achieving the object of the present invention are included in the present invention. Is.
　例えば、上述の実施形態では、画像の被写体としてワインラベルについて説明したが、これは一例に過ぎず、あらゆる物体を被写体とすることができる。 For example, in the above-described embodiment, the wine label has been described as the subject of the image, but this is only an example, and any object can be the subject.
　また、図２に示すハードウェア構成は、本発明の目的を達成するための例示に過ぎず、特に限定されない。 The hardware configuration shown in FIG. 2 is merely an example for achieving the object of the present invention, and is not particularly limited.
　また、図３に示す機能ブロック図は、例示に過ぎず、特に限定されない。即ち、上述した一連の処理を全体として実行できる機能が情報処理システムに備えられていれば足り、この機能を実現するためにどのような機能ブロックを用いるのかは、特に図３の例に限定されない。 Also, the functional block diagram shown in FIG. 3 is merely an example and is not particularly limited. That is, it is sufficient if the information processing system has a function capable of executing the above-described series of processes as a whole, and what kind of functional block is used to realize this function is not particularly limited to the example of FIG. ..
　また、機能ブロックの存在場所も、図３に限定されず、任意でよい。
　また、１つの機能ブロックは、ハードウェア単体で構成してもよいし、ソフトウェア単体で構成してもよいし、それらの組み合わせで構成してもよい。 The location of the functional block is not limited to that shown in FIG. 3 and may be arbitrary.
Further, one functional block may be configured by hardware alone, software alone, or a combination thereof.
　各機能ブロックの処理をソフトウェアにより実行させる場合には、そのソフトウェアを構成するプログラムが、コンピュータ等にネットワークや記録媒体からインストールされる。
　コンピュータは、専用のハードウェアに組み込まれているコンピュータであってもよい。また、コンピュータは、各種のプログラムをインストールすることで、各種の機能を実行することが可能なコンピュータ、例えば情報処理装置の他汎用のスマートフォンやパーソナルコンピュータであってもよい。 When the processing of each functional block is executed by software, the program forming the software is installed in a computer or the like from a network or a recording medium.
The computer may be a computer embedded in dedicated hardware. In addition, the computer may be a computer capable of executing various functions by installing various programs, for example, a general-purpose smartphone or a personal computer other than an information processing device.
　このようなプログラムを含む記録媒体は、各ユーザにプログラムを提供するために装置本体とは別に配布される、リムーバブルメディアにより構成されるだけではなく、装置本体に予め組み込まれた状態で各ユーザに提供される記録媒体等で構成される。 The recording medium containing such a program is not only constituted by a removable medium which is distributed separately from the apparatus main body in order to provide the program to each user, but is also pre-installed in the apparatus main body to each user. It is composed of a recording medium provided.
　なお、本明細書において、記録媒体に記録されるプログラムを記述するステップは、その順序に添って時系列的に行われる処理はもちろん、必ずしも時系列的に処理されなくとも、並列的或いは個別に実行される処理をも含むものである。 In the present specification, the steps of writing the program recorded on the recording medium include, not only the processing performed in time series according to the order, but also the processing performed in parallel or individually not necessarily in time series. It also includes the processing to be executed.
　また、本明細書において、システムの用語は、複数の装置や複数の手段等より構成される全体的な装置を意味するものである。 Further, in the present specification, the term system means the entire device including a plurality of devices and a plurality of means.
　以上まとめると、本発明が適用される情報処理装置は、次のような構成を取れば足り、各種各様な実施形態を取ることができる。
　即ち、本発明が適用される情報処理装置（例えば情報処理装置１）は、
　画像（例えば図１の画像ＢＦ）に対し、当該画像の注釈となり得る所定情報（例えばメタデータ）を付与するアノテーション手段（例えば図３のアノテーション部１１２）と、
　前記所定情報が付与された前記画像を教師画像（例えば教師画像ＴＦ）とする学習を行い、モデルを生成する学習手段（例えば図３の学習部１１３）と、
　生成された前記モデルを所定の環境下（例えばユーザが構築するシステム）で使用可能な状態（例えばモデルファイルＭＦ）にするデプロイ手段（例えば図３のデプロイ部１１４）と、
　を備える。 In summary, the information processing apparatus to which the present invention is applied may have various configurations as long as it has the following configuration.
That is, the information processing apparatus (for example, the information processing apparatus 1) to which the present invention is applied is
An annotation means (for example, the annotation unit 112 in FIG. 3) that adds predetermined information (for example, metadata) that can be an annotation of the image to the image (for example, the image BF in FIG. 1),
A learning unit (for example, the learning unit 113 in FIG. 3) that performs learning by using the image to which the predetermined information is given as a teacher image (for example, a teacher image TF) and generates a model,
Deploying means (for example, the deploying unit 114 in FIG. 3) that puts the generated model into a usable state (for example, a model file MF) under a predetermined environment (for example, a system constructed by a user),
Equipped with.
　これにより、学習機能を有する装置の開発を行う者における宣伝・営業の容易化を可能とする、ディープラーニングパッケージソリューションを提供することができる。 With this, it is possible to provide a deep learning package solution that enables easy promotion and sales for those who develop devices with learning functions.
　また、前記モデルを読み込むことで推論処理を実行するプログラムの構成部品をライブラリ化するライブラリ手段（例えばライブラリ化部１２１）をさらに備えることができる。 Further, it is possible to further include a library means (for example, a library forming unit 121) which forms a library of components of a program that executes inference processing by reading the model.
　これにより、ユーザは、ユーザが開発したアプリケーションプログラムに、ディープラーニングパッケージソリューションを搭載することができるので、ディープラーニングを用いたシステムを容易に構築することができる。 With this, the user can install the deep learning package solution in the application program developed by the user, so that the system using the deep learning can be easily constructed.
　その他として、非ブラウザアプリケーションで、ディープラーニングを用いた画像分類、物体検知、セグメンテーションタスクが実行可能になる。
　学習だけでなく、それぞれのタスクでアノテーション機能がある。つまり、アノテーション機能までが内包されている。
　Ｄａｔａ　Ａｕｇｍｅｎｔａｔｉｏｎの設定可能である。データの複製機能がある。
　ディープラーニングの検証結果等をＶｉｓｕａｌ　ｓｔｕｄｉｏのようにプロジェクト単位で管理可能である。データ、学習済みデータ、アーキテクト、モデルが一貫して管理可能である。
　（エクスプローラからアプリに）ドラッグ　アンド　ドロップなど簡易な操作でアノテーション可能である。
　ＰＣだけでなくＪｅｔｓｏｎ、ＦＰＧＡ、ｉＰｈｏｎｅ（登録商標）、ＭＯＶＩＤＩＵＳ、ＡＲＭなど多彩なデバイスにデプロイ可能である。
　現状行っている作業のステータスが容易に可能である。アノテーション＞学習＞推論＞デプロイなどの状態を見える化させることができる。
　深層学習モデルの可視化機能（Ｇｒａｄｃａｍ）がパッケージング化されている。
　各層での重みの値や途中計算結果を見える化させることにより、学習データにフィードバックが出来、学習データの工夫が可能となる。
　分類の結果をコンフュージョンマトリクスで視覚的に表示できる。 In addition, non-browser applications will be able to perform image classification, object detection, and segmentation tasks using deep learning.
Not only learning but also annotation function for each task. In other words, even the annotation function is included.
The Data Augmentation can be set. There is a data replication function.
It is possible to manage the verification results of deep learning in project units like Visual studio. Manage data, trained data, architects, and models consistently.
Annotation is possible with simple operations such as drag and drop (from Explorer to app).
It can be deployed not only on PC but also on various devices such as Jetson, FPGA, iPhone (registered trademark), MOVIDIUS, ARM.
The status of the current work is easily possible. You can visualize the status of annotations>learning>inference> deployment.
The deep learning model visualization function (Gradcam) is packaged.
By visualizing the value of the weight in each layer and the intermediate calculation result, it is possible to feed back to the learning data and to devise the learning data.
The result of classification can be visually displayed in a confusion matrix.
　１：情報処理装置、１０：ＧＰＵ、１１：ＣＰＵ、１２：ＲＯＭ、１３：ＲＡＭ、１４：バス、１５：入出力インターフェース、１６：出力部、１７：入力部、１８：記憶部、１９：通信部、２０：ドライブ、３０：リムーバブルメディア、１０１：モデリング部、１０２：推論ライブラリ部、１１１：画像取得部、１１２：アノテーション部、１１３：学習部、１１４：デプロイ部、１２１：ライブラリ化部、４０１：教師ＤＢ、４０２：モデルＤＢ、ＢＦ：画像、ＴＦ：教師画像、ＭＦ：モデルファイル 1: Information processing device, 10: GPU, 11: CPU, 12: ROM, 13: RAM, 14: Bus, 15: Input/output interface, 16: Output unit, 17: Input unit, 18: Storage unit, 19: Communication Part, 20: drive, 30: removable medium, 101: modeling part, 102: inference library part, 111: image acquisition part, 112: annotation part, 113: learning part, 114: deploying part, 121: library forming part, 401 : Teacher DB, 402: model DB, BF: image, TF: teacher image, MF: model file

Claims (2)

  1. 　画像に対し、当該画像の注釈となり得る所定情報を付与するアノテーション手段と、
    　前記所定情報が付与された前記画像を教師画像とする学習を行い、モデルを生成する学習手段と、
    　生成された前記モデルを所定の環境下で使用可能な状態にするデプロイ手段と、
    　を備える情報処理装置。     An annotation means that adds predetermined information that can be an annotation of the image to the image,
    A learning unit that performs learning by using the image provided with the predetermined information as a teacher image and generates a model,
    Deploying means for making the generated model usable under a predetermined environment,
    An information processing apparatus including.
  2. 　前記モデルを読み込むことで推論処理を実行するプログラムの構成部品をライブラリ化するライブラリ手段をさらに備える、
    　請求項１に記載の情報処理装置。     Further comprising library means for converting the components of a program for executing inference processing into a library by reading the model.
    The information processing apparatus according to claim 1.
PCT/JP2020/003915 2019-02-01 2020-02-03 Service building device, service building method, and service building program WO2020158954A1 (en)

Priority Applications (2)

Application Number Priority Date Filing Date Title
JP2020539874A JPWO2020158954A1 (en) 2019-02-01 2020-02-03 Service construction device, service construction method and service construction program
JP2021198561A JP2022033153A (en) 2019-02-01 2021-12-07 Information processing device

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2019-017194 2019-02-01
JP2019017194 2019-02-01

Publications (1)

Publication Number Publication Date
WO2020158954A1 true WO2020158954A1 (en) 2020-08-06

Family

ID=71841049

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2020/003915 WO2020158954A1 (en) 2019-02-01 2020-02-03 Service building device, service building method, and service building program

Country Status (2)

Country Link
JP (2) JPWO2020158954A1 (en)
WO (1) WO2020158954A1 (en)

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2018097671A (en) * 2016-12-14 2018-06-21 株式会社グルーヴノーツ Service construction device, service construction method, and service construction program
WO2019003485A1 (en) * 2017-06-30 2019-01-03 株式会社Ａｂｅｊａ Computer system and method for machine learning or inference

Family Cites Families (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2008191889A (en) * 2007-02-05 2008-08-21 Hitachi Software Eng Co Ltd Program license check system
JP2011170465A (en) * 2010-02-16 2011-09-01 Ricoh Co Ltd System, method, and program for software distribution
JP5821720B2 (en) * 2012-03-14 2015-11-24 株式会社ソシオネクスト Startup control program and startup control method
JP2014016659A (en) * 2012-07-05 2014-01-30 Nec Soft Ltd Electronic authentication system, terminal, server, and electronic authentication program
WO2018142766A1 (en) * 2017-02-03 2018-08-09 パナソニックＩｐマネジメント株式会社 Learned model provision method and learned model provision device
WO2018173121A1 (en) * 2017-03-21 2018-09-27 株式会社Ｐｒｅｆｅｒｒｅｄ Ｎｅｔｗｏｒｋｓ Server device, trained model providing program, trained model providing method, and trained model providing system
WO2018216379A1 (en) * 2017-05-26 2018-11-29 株式会社日立国際電気 Machine learning model illicitness sensing system and illicitness sensing method

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2018097671A (en) * 2016-12-14 2018-06-21 株式会社グルーヴノーツ Service construction device, service construction method, and service construction program
WO2019003485A1 (en) * 2017-06-30 2019-01-03 株式会社Ａｂｅｊａ Computer system and method for machine learning or inference

Also Published As

Publication number Publication date
JP2022033153A (en) 2022-02-28
JPWO2020158954A1 (en) 2021-02-18

Similar Documents

Publication Publication Date Title
Davenport From analytics to artificial intelligence
US20240281364A1 (en) Test case reduction for code regression testing
US10255081B2 (en) Method and system for intelligent cloud planning and decommissioning
KR102443654B1 (en) Automatically create machine learning models for software tools that operate on source code
CN107220172B (en) Method and system for automated User Interface (UI) testing via model-driven techniques
WO2021109928A1 (en) Creation method, usage method and apparatus for machine learning scheme template
US10929490B2 (en) Network search query
Silva et al. Using vistrails and provenance for teaching scientific visualization
US20050228767A1 (en) Method, system and program product for developing a data model in a data mining system
Turetken et al. The effect of modularity representation and presentation medium on the understandability of business process models in BPMN
US11080655B2 (en) Machine learning technical support selection
CN106095767A (en) Automatically generate the method and system of user's form interface
US9922136B2 (en) Non-intrusive, semantics-driven impact analysis for business applications
US10572247B2 (en) Prototype management system
EP3819760A1 (en) Methods and apparatus for generating a platform-agnostic mobile application configuration data structure with a dynamic quiz
Ihrig A new research architecture for the simulation era
JP2020505710A (en) Methods and systems for verifying software programs
Ahmed-Nacer et al. Model-Driven Simulation of Elastic OCCI Cloud Resources
WO2020158954A1 (en) Service building device, service building method, and service building program
US10740070B2 (en) Locating features in a layered software application
US10521751B2 (en) Using customer profiling and analytics to understand, rank, score, and visualize best practices
US20070088562A1 (en) Method and program product for identifying educational content for a business initiative
Velozo et al. Evaluation of a Mobile Software Development Company
Bourgeois et al. Information systems development
US20140244677A1 (en) Dynamic generation of demonstrative aids for a meeting

Legal Events

Date Code Title Description
ENP Entry into the national phase

Ref document number: 2020539874

Country of ref document: JP

Kind code of ref document: A

121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20749008

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20749008

Country of ref document: EP

Kind code of ref document: A1