WO2024023948A1 - Analysis device, analysis method, and analysis program - Google Patents

Analysis device, analysis method, and analysis program Download PDF

Info

Publication number
WO2024023948A1
WO2024023948A1 PCT/JP2022/028845 JP2022028845W WO2024023948A1 WO 2024023948 A1 WO2024023948 A1 WO 2024023948A1 JP 2022028845 W JP2022028845 W JP 2022028845W WO 2024023948 A1 WO2024023948 A1 WO 2024023948A1
Authority
WO
WIPO (PCT)
Prior art keywords
screenshot
component
screen
image
unit
Prior art date
Application number
PCT/JP2022/028845
Other languages
French (fr)
Japanese (ja)
Inventor
結香子 飯村
忍 斎藤
Original Assignee
日本電信電話株式会社
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 日本電信電話株式会社 filed Critical 日本電信電話株式会社
Priority to PCT/JP2022/028845 priority Critical patent/WO2024023948A1/en
Publication of WO2024023948A1 publication Critical patent/WO2024023948A1/en

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F8/00—Arrangements for software engineering
    • G06F8/30—Creation or generation of source code
    • G06F8/38—Creation or generation of source code for implementing user interfaces
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F8/00—Arrangements for software engineering
    • G06F8/70—Software maintenance or management
    • G06F8/74—Reverse engineering; Extracting design information from source code

Definitions

  • the present invention relates to an analysis device, an analysis method, and an analysis program.
  • the present invention has been made in view of the above, and an object of the present invention is to extract screen specifications of an information system.
  • an analysis device includes a storage unit that stores a collection of screenshots of the system screen recorded during operation, and a storage unit that stores a set of screenshots of the system screen recorded during operation, and a storage unit that stores a set of screenshots of the system screen recorded during operation.
  • an acquisition unit that acquires an image of the component
  • a determination unit that determines whether each screenshot in the storage unit includes an acquired image of the component
  • a generation unit that generates a screen specification indicating the components included in the image forming apparatus.
  • FIG. 1 is a schematic diagram illustrating a schematic configuration of an analyzer.
  • FIG. 2 is a diagram illustrating the processing of the acquisition unit.
  • FIG. 3 is a diagram illustrating the data structure of screen specifications.
  • FIG. 4 is a diagram illustrating the processing of the determination unit.
  • FIG. 5 is a diagram showing an example of presentation of screen specifications.
  • FIG. 6 is a diagram showing an example of presentation of screen specifications.
  • FIG. 7 is a flowchart showing the analysis processing procedure.
  • FIG. 8 is a diagram showing an example of a computer that executes an analysis program.
  • FIG. 1 is a schematic diagram illustrating a schematic configuration of an analyzer.
  • the analyzer 10 of this embodiment is realized by a general-purpose computer such as a personal computer, and includes an input section 11, an output section 12, a communication control section 13, a storage section 14, and a control section 15.
  • the input unit 11 is realized using an input device such as a keyboard or a mouse, and inputs various instruction information such as starting processing to the control unit 15 in response to an input operation by an operator.
  • the output unit 12 is realized by a display device such as a liquid crystal display, a printing device such as a printer, and the like. For example, the output unit 12 displays the results of analysis processing, which will be described later.
  • the communication control unit 13 is realized by a NIC (Network Interface Card) or the like, and controls communication between an external device and the control unit 15 via a telecommunication line such as a LAN (Local Area Network) or the Internet.
  • a NIC Network Interface Card
  • the communication control unit 13 controls communication between the control unit 15 and other servers that manage various data.
  • the storage unit 14 is realized by a semiconductor memory element such as a RAM (Random Access Memory) or a flash memory, or a storage device such as a hard disk or an optical disk.
  • a processing program for operating the analyzer 10 data used during execution of the processing program, and the like are stored in advance, or are temporarily stored each time processing is performed.
  • the storage unit 14 may be configured to communicate with the control unit 15 via the communication control unit 13.
  • the storage unit 14 stores a screenshot set 14a and screen specifications 14b.
  • the screen shot set 14a is a set of screen shots of system screens recorded while operating the information system. For example, a user of the information system obtains a group of screenshots of the operation screen during work in advance and stores them in the storage unit 14. Further, the screen specification 14b is generated by a process described later.
  • the control unit 15 is realized using a CPU (Central Processing Unit) or the like, and executes a processing program stored in a memory. Thereby, the control unit 15 functions as an acquisition unit 15a, a determination unit 15b, a generation unit 15c, and a classification unit 15d, and executes the analysis process, as illustrated in FIG. Note that each or a part of these functional units may be implemented in different hardware.
  • the classification unit 15d may be implemented in a different device from other functional units. Further, the control unit 15 may include other functional units.
  • the acquisition unit 15a acquires an image of a designated component on the system screen. Specifically, the acquisition unit 15a acquires a partial image of the area designated on the screenshot of the system screen as an image of the component.
  • FIG. 2 is a diagram for explaining the processing of the acquisition unit.
  • the user operates the mouse on the screenshot of the system screen displayed on the output unit 12 to select a rectangular area as shown by the thick frame in FIG.
  • An image of a component to be processed is designated via the unit 11.
  • the acquisition unit 15a acquires an image of the designated component.
  • the acquisition unit 15a may acquire an image of a component specified in advance and store it in the storage unit 14, or may immediately transfer it to the determination unit 15b described below without storing it in the storage unit 14. Good too.
  • the determining unit 15b determines whether each screenshot of the screenshot set 14a in the storage unit 14 includes an acquired image of a component. For example, the determining unit 15b determines whether each screenshot includes an image of a component by comparing each screenshot with an image of a component using template matching.
  • the determination unit 15b repeats the process of comparing the image of a single component with all the screenshots in the screenshot set 14a. Further, the determination unit 15b repeats the process of changing the image of the component to be processed and comparing it with all the screenshots in the screenshot set 14a.
  • the generation unit 15c uses the determined results to generate screen specifications 14b indicating the components included in each system screen. For example, the generation unit 15c causes the storage unit 14 to store, as the screen specification 14b, information in which information identifying each screenshot is associated with information identifying an image of a component included in each screenshot.
  • FIG. 3 is a diagram illustrating the data structure of the screen specifications.
  • the screen specification 14b is information that associates the operation screen of the information system with each component that appears on the operation screen.
  • a screen is information that identifies each screenshot in the screenshot set 14a.
  • the screen component is information that identifies an image of each component.
  • An "x" shown in FIG. 3 indicates that the relevant component appears in the relevant screenshot, and a component without an "x" represents that it does not appear in the relevant screenshot.
  • the determining unit 15b when determining that the acquired image of the component is included in the screenshot of the screenshot collection 14a of the storage unit 14, the determining unit 15b further specifies the appearance position in the screenshot. For example, the determining unit 15b specifies the appearance position of the component such as the coordinates on the screenshot, and the generation unit 15c associates the specified appearance position with information identifying the image of the component, and generates a screen specification. Add to 14b.
  • the generation unit 15c presents the screen specifications to the user via the output unit 12 based on the generated screen specifications 14b.
  • the generation unit 15c may present the screen specifications 14b as is, or may present the appearance position of each component instead of whether each component appears.
  • the "x" shown in FIG. 3 may be information representing the appearance position of each component.
  • the appearance position of each component may be written together with the "x" shown in FIG. 3.
  • the determining unit 15b when determining that the screenshot of the screenshot collection 14a of the storage unit 14 includes the acquired image of the component, the determining unit 15b further determines that the area of the image of the component with respect to the entire area of the screenshot is You may also calculate the ratio of
  • FIG. 4 is a diagram illustrating the processing of the determination unit.
  • FIG. 5 is a diagram showing an example of presentation of screen specifications.
  • the determination unit 15b calculates the ratio of the area of the image of each component in the entire area of the screenshot, as illustrated by the bold frame in FIG.
  • the generation unit 15c adds the calculated area ratio of the image of each component to the screen specification 14b.
  • the generation unit 15c presents the user with the ratio (%) of the area of each component in the entire area of the screenshot, instead of the appearance position of each component, as illustrated in FIG. You may.
  • the proportion of non-extracted regions other than the component region is also presented.
  • the classification unit 15d classifies the set of screenshots in the storage unit 14 according to the degree of similarity between each screenshot. For example, the classification unit 15d calculates the degree of similarity between the screenshots in the screenshot set 14a, and classifies each screenshot so that the screenshots for which the calculated degree of similarity is greater than or equal to a predetermined threshold are in the same class. Divide. The generation unit 15c then adds the class of each screenshot to the screen specification 14b.
  • FIG. 6 is a diagram showing an example of presentation of screen specifications.
  • the user is presented with a screen specification 14b in which screen classes classified for each screen are added.
  • screen specifications 14b in which screen classes classified for each screen are added.
  • FIG. 7 is a flowchart showing the analysis processing procedure.
  • the flowchart in FIG. 7 is started, for example, at the timing when the user performs an operation input instructing to start.
  • the acquisition unit 15a acquires an image of a designated component on the system screen (step S1). For example, the acquisition unit 15a acquires a partial image of a designated area on a screenshot of the system screen as an image of a component.
  • the determining unit 15b determines whether the acquired image of the component appears in each screenshot of the screenshot set 14a (step S2). If the image of the component does not appear in the screenshot (step S2, No), the determination unit 15b advances the process to step S4.
  • step S3 the appearance position of the image of the component in the screenshot is specified.
  • the determination unit 15b repeats the processing of steps S2 to S3 for all screenshots (step S4, No ⁇ S2). Then, when the determination unit 15b finishes processing all the screenshots (step S4, Yes), the determination unit 15b returns the process to step S1 until the processing of all component images is finished (step S5, No ⁇ S1). .
  • step S5 when the processing of images of all component parts is completed (step S5, Yes), the generation unit 15c generates the screen specification 14b. Furthermore, the generation unit 15c outputs the screen specifications to the user based on the generated screen specifications 14b (step S6). This completes the series of analysis processes.
  • the storage unit 14 stores the screen shot set 14a of the system screen recorded during operation.
  • the acquisition unit 15a acquires an image of a designated component on the system screen.
  • the determining unit 15b determines whether each screenshot of the screenshot set 14a in the storage unit 14 includes an acquired image of a component.
  • the generation unit 15c uses the determined results to generate screen specifications 14b indicating the components included in each system screen.
  • the acquisition unit 15a acquires a partial image of the area specified on the screenshot of the system screen as an image of the component.
  • the determining unit 15b when determining that the acquired image of the component is included in the screenshot of the screenshot collection 14a of the storage unit 14, the determining unit 15b further specifies the appearance position in the screenshot. This makes it possible to clarify the screen specifications of the information system in detail.
  • the determining unit 15b when determining that the screenshot of the screenshot collection 14a of the storage unit 14 includes the acquired image of the component, the determining unit 15b further determines that the area of the image of the component with respect to the entire area of the screenshot is Calculate the percentage of This also makes it possible to clarify the screen specifications of the information system in detail.
  • the classification unit 15d classifies the set of screenshots in the storage unit 14 according to the degree of similarity between each screenshot. This also makes it possible to clarify the screen specifications of the information system in more detail.
  • the analysis device 10 can be implemented by installing an analysis program that executes the above analysis process on a desired computer as packaged software or online software. For example, by causing the information processing device to execute the above analysis program, the information processing device can be made to function as the analysis device 10.
  • the information processing device referred to here includes a desktop or notebook personal computer.
  • information processing devices include mobile communication terminals such as smartphones, mobile phones, and PHSs (Personal Handyphone Systems), as well as slate terminals such as PDAs (Personal Digital Assistants).
  • the functions of the analysis device 10 may be implemented in a cloud server.
  • FIG. 8 is a diagram showing an example of a computer that executes an analysis program.
  • Computer 1000 includes, for example, memory 1010, CPU 1020, hard disk drive interface 1030, disk drive interface 1040, serial port interface 1050, video adapter 1060, and network interface 1070. These parts are connected by a bus 1080.
  • the memory 1010 includes a ROM (Read Only Memory) 1011 and a RAM 1012.
  • the ROM 1011 stores, for example, a boot program such as BIOS (Basic Input Output System).
  • Hard disk drive interface 1030 is connected to hard disk drive 1031.
  • Disk drive interface 1040 is connected to disk drive 1041.
  • a removable storage medium such as a magnetic disk or an optical disk is inserted into the disk drive 1041, for example.
  • a mouse 1051 and a keyboard 1052 are connected to the serial port interface 1050.
  • a display 1061 is connected to the video adapter 1060.
  • the hard disk drive 1031 stores, for example, an OS 1091, an application program 1092, a program module 1093, and program data 1094. Each piece of information described in the above embodiments is stored in, for example, the hard disk drive 1031 or the memory 1010.
  • the analysis program is stored in the hard disk drive 1031, for example, as a program module 1093 in which commands to be executed by the computer 1000 are written.
  • a program module 1093 in which each process executed by the analysis device 10 described in the above embodiment is described is stored in the hard disk drive 1031.
  • data used for information processing by the analysis program is stored as program data 1094 in, for example, the hard disk drive 1031.
  • the CPU 1020 reads out the program module 1093 and program data 1094 stored in the hard disk drive 1031 to the RAM 1012 as necessary, and executes each of the above-described procedures.
  • program module 1093 and program data 1094 related to the analysis program are not limited to being stored in the hard disk drive 1031; for example, they may be stored in a removable storage medium and read by the CPU 1020 via the disk drive 1041 or the like. may be done.
  • the program module 1093 and program data 1094 related to the analysis program are stored in another computer connected via a network such as a LAN or WAN (Wide Area Network), and read out by the CPU 1020 via the network interface 1070. It's okay.

Landscapes

  • Engineering & Computer Science (AREA)
  • Software Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Human Computer Interaction (AREA)
  • Debugging And Monitoring (AREA)

Abstract

A storage unit (14) stores a screenshot set (14a) of a system screen recorded during an operation. An acquisition unit (15a) acquires an image of a designated component of the system screen. A determination unit (15b) determines whether the acquired image of the component is included in each screenshot of the screenshot set (14a) in the storage unit (14). A generation unit (15c) generates screen specifications (14b) indicating components included in each system screen, using a result of the determination.

Description

分析装置、分析方法および分析プログラムAnalytical equipment, analytical methods and analytical programs
　本発明は、分析装置、分析方法および分析プログラムに関する。 The present invention relates to an analysis device, an analysis method, and an analysis program.
　情報システムのシステム更改の際、現行システムについて、システム画面を構成する部品とそれらの部品で構成される画面のバリエーションを示す画面仕様を理解する必要がある。従来、システムの仕様を理解するため、ソースコードリバースを活用したシステム分析の技術が知られている（特許文献１参照）。また、操作時の操作画面のキャプチャ画像と、操作対象ウィンドウの操作箇所の操作画面での位置を特定する情報を含む操作ログとを利用して、操作ログを処理順に並べて画面フローを明らかにする技術が開示されている（特許文献２参照）。 When updating an information system, it is necessary to understand the screen specifications that indicate the parts that make up the system screen and the variations of the screens that are made up of those parts for the current system. Conventionally, in order to understand the specifications of a system, a system analysis technique using source code reversal is known (see Patent Document 1). In addition, by using the captured image of the operation screen during operation and the operation log that includes information that specifies the position of the operation part of the operation target window on the operation screen, the operation log is arranged in the order of processing to clarify the screen flow. A technique has been disclosed (see Patent Document 2).
特開平１１－９５９９０号公報Japanese Patent Application Publication No. 11-95990 国際公開第２０２０／２３５０８５号International Publication No. 2020/235085
　しかしながら、従来技術によれば、情報システムの画面仕様を明らかにすることが困難な場合がある。例えば、システム画面の見た目は、表示データやユーザの操作により変化するため、ソースコードリバースでは、画面仕様を十分に抽出することができない。また、システム画面の構成要素は複数の画面で横断的に使用されることがあり、また、その出現位置は固定ではないため、画面フローを明らかにするだけでは、画面仕様を十分に抽出することができない。 However, according to the conventional technology, it may be difficult to clarify the screen specifications of an information system. For example, the appearance of a system screen changes depending on display data and user operations, so source code reversal cannot sufficiently extract screen specifications. In addition, system screen components may be used across multiple screens, and their appearance positions are not fixed, so simply clarifying the screen flow is not enough to fully extract the screen specifications. I can't.
　本発明は、上記に鑑みてなされたものであって、情報システムの画面仕様を抽出することを目的とする。 The present invention has been made in view of the above, and an object of the present invention is to extract screen specifications of an information system.
　上述した課題を解決し、目的を達成するために、本発明に係る分析装置は、操作中に記録されたシステム画面のスクリーンショットの集合を記憶する記憶部と、システム画面の指定された構成部品の画像を取得する取得部と、前記記憶部の各スクリーンショットに、取得された前記構成部品の画像が含まれるか否かを判定する判定部と、判定された結果を用いて、各システム画面に含まれる構成部品を示す画面仕様を生成する生成部と、を有することを特徴とする。 In order to solve the above-mentioned problems and achieve the purpose, an analysis device according to the present invention includes a storage unit that stores a collection of screenshots of the system screen recorded during operation, and a storage unit that stores a set of screenshots of the system screen recorded during operation, and a storage unit that stores a set of screenshots of the system screen recorded during operation. an acquisition unit that acquires an image of the component; a determination unit that determines whether each screenshot in the storage unit includes an acquired image of the component; and a generation unit that generates a screen specification indicating the components included in the image forming apparatus.
　本発明によれば、情報システムの画面仕様を抽出することが可能となる。 According to the present invention, it is possible to extract screen specifications of an information system.
図１は、分析装置の概略構成を例示する模式図である。FIG. 1 is a schematic diagram illustrating a schematic configuration of an analyzer. 図２は、取得部の処理を例示する図である。FIG. 2 is a diagram illustrating the processing of the acquisition unit. 図３は、画面仕様のデータ構成を例示する図である。FIG. 3 is a diagram illustrating the data structure of screen specifications. 図４は、判定部の処理を例示する図である。FIG. 4 is a diagram illustrating the processing of the determination unit. 図５は、画面仕様の提示例を示す図である。FIG. 5 is a diagram showing an example of presentation of screen specifications. 図６は、画面仕様の提示例を示す図である。FIG. 6 is a diagram showing an example of presentation of screen specifications. 図７は、分析処理手順を示すフローチャートである。FIG. 7 is a flowchart showing the analysis processing procedure. 図８は、分析プログラムを実行するコンピュータの一例を示す図である。FIG. 8 is a diagram showing an example of a computer that executes an analysis program.
　以下、図面を参照して、本発明の一実施形態を詳細に説明する。なお、この実施形態により本発明が限定されるものではない。また、図面の記載において、同一部分には同一の符号を付して示している。 Hereinafter, one embodiment of the present invention will be described in detail with reference to the drawings. Note that the present invention is not limited to this embodiment. In addition, in the description of the drawings, the same parts are denoted by the same reference numerals.
［分析装置の構成］
　図１は、分析装置の概略構成を例示する模式図である。図１に例示するように、本実施形態の分析装置１０は、パソコン等の汎用コンピュータで実現され、入力部１１、出力部１２、通信制御部１３、記憶部１４、および制御部１５を備える。 [Analyzer configuration]
FIG. 1 is a schematic diagram illustrating a schematic configuration of an analyzer. As illustrated in FIG. 1, the analyzer 10 of this embodiment is realized by a general-purpose computer such as a personal computer, and includes an input section 11, an output section 12, a communication control section 13, a storage section 14, and a control section 15.
　入力部１１は、キーボードやマウス等の入力デバイスを用いて実現され、操作者による入力操作に対応して、制御部１５に対して処理開始などの各種指示情報を入力する。出力部１２は、液晶ディスプレイなどの表示装置、プリンター等の印刷装置等によって実現される。例えば、出力部１２には、後述する分析処理の結果が表示される。 The input unit 11 is realized using an input device such as a keyboard or a mouse, and inputs various instruction information such as starting processing to the control unit 15 in response to an input operation by an operator. The output unit 12 is realized by a display device such as a liquid crystal display, a printing device such as a printer, and the like. For example, the output unit 12 displays the results of analysis processing, which will be described later.
　通信制御部１３は、ＮＩＣ（Network　Interface　Card）等で実現され、ＬＡＮ（Local　Area　Network）やインターネットなどの電気通信回線を介した外部の装置と制御部１５との通信を制御する。例えば、通信制御部１３は、各種データを管理する他のサーバ等と制御部１５との通信を制御する。 The communication control unit 13 is realized by a NIC (Network Interface Card) or the like, and controls communication between an external device and the control unit 15 via a telecommunication line such as a LAN (Local Area Network) or the Internet. For example, the communication control unit 13 controls communication between the control unit 15 and other servers that manage various data.
　記憶部１４は、ＲＡＭ（Random　Access　Memory）、フラッシュメモリ（Flash　Memory）等の半導体メモリ素子、または、ハードディスク、光ディスク等の記憶装置によって実現される。記憶部１４には、分析装置１０を動作させる処理プログラムや、処理プログラムの実行中に使用されるデータなどが予め記憶され、あるいは処理の都度一時的に記憶される。なお、記憶部１４は、通信制御部１３を介して制御部１５と通信する構成でもよい。 The storage unit 14 is realized by a semiconductor memory element such as a RAM (Random Access Memory) or a flash memory, or a storage device such as a hard disk or an optical disk. In the storage unit 14, a processing program for operating the analyzer 10, data used during execution of the processing program, and the like are stored in advance, or are temporarily stored each time processing is performed. Note that the storage unit 14 may be configured to communicate with the control unit 15 via the communication control unit 13.
　本実施形態において、記憶部１４は、スクリーンショット集合１４ａ、画面仕様１４ｂを記憶する。スクリーンショット集合１４ａは、情報システムを操作中に記録されたシステム画面のスクリーンショットの集合である。例えば、予め、情報システムの利用者が業務中に操作画面のスクリーンショット群を取得し、記憶部１４に蓄積する。また、画面仕様１４ｂは、後述する処理で生成される。 In this embodiment, the storage unit 14 stores a screenshot set 14a and screen specifications 14b. The screen shot set 14a is a set of screen shots of system screens recorded while operating the information system. For example, a user of the information system obtains a group of screenshots of the operation screen during work in advance and stores them in the storage unit 14. Further, the screen specification 14b is generated by a process described later.
　制御部１５は、ＣＰＵ（Central　Processing　Unit）等を用いて実現され、メモリに記憶された処理プログラムを実行する。これにより、制御部１５は、図１に例示するように、取得部１５ａ、判定部１５ｂ、生成部１５ｃおよび分類部１５ｄとして機能して、分析処理を実行する。なお、これらの機能部は、それぞれ、あるいは一部が異なるハードウェアに実装されてもよい。例えば、分類部１５ｄは、他の機能部とは異なる装置に実装されてもよい。また、制御部１５は、その他の機能部を備えてもよい。 The control unit 15 is realized using a CPU (Central Processing Unit) or the like, and executes a processing program stored in a memory. Thereby, the control unit 15 functions as an acquisition unit 15a, a determination unit 15b, a generation unit 15c, and a classification unit 15d, and executes the analysis process, as illustrated in FIG. Note that each or a part of these functional units may be implemented in different hardware. For example, the classification unit 15d may be implemented in a different device from other functional units. Further, the control unit 15 may include other functional units.
　取得部１５ａは、システム画面の指定された構成部品の画像を取得する。具体的には、取得部１５ａは、システム画面のスクリーンショット上で指定された領域の部分画像を、構成部品の画像として取得する。 The acquisition unit 15a acquires an image of a designated component on the system screen. Specifically, the acquisition unit 15a acquires a partial image of the area designated on the screenshot of the system screen as an image of the component.
　ここで、図２は、取得部の処理を説明するための図である。例えば、図２に例示するように、出力部１２に表示されたシステム画面のスクリーンショット上でユーザがマウスを操作して、図２に太枠で示すように矩形領域を選択することにより、入力部１１を介して処理対象の構成部品の画像を指定する。これにより、取得部１５ａが、指定された構成部品の画像を取得する。 Here, FIG. 2 is a diagram for explaining the processing of the acquisition unit. For example, as illustrated in FIG. 2, the user operates the mouse on the screenshot of the system screen displayed on the output unit 12 to select a rectangular area as shown by the thick frame in FIG. An image of a component to be processed is designated via the unit 11. Thereby, the acquisition unit 15a acquires an image of the designated component.
　なお、取得部１５ａは、予め指定された構成部品の画像を取得して記憶部１４に記憶させてもよいし、記憶部１４に記憶させずに直ちに以下に説明する判定部１５ｂに転送してもよい。 Note that the acquisition unit 15a may acquire an image of a component specified in advance and store it in the storage unit 14, or may immediately transfer it to the determination unit 15b described below without storing it in the storage unit 14. Good too.
　図１の説明に戻る。判定部１５ｂは、記憶部１４のスクリーンショット集合１４ａの各スクリーンショットに、取得された構成部品の画像が含まれるか否かを判定する。例えば、判定部１５ｂは、各スクリーンショットと構成部品の画像とをテンプレートマッチングにより比較することにより、各スクリーンショットに構成部品の画像が含まれるか否かを判定する。 Returning to the explanation of FIG. 1. The determining unit 15b determines whether each screenshot of the screenshot set 14a in the storage unit 14 includes an acquired image of a component. For example, the determining unit 15b determines whether each screenshot includes an image of a component by comparing each screenshot with an image of a component using template matching.
　判定部１５ｂは、単一の構成部品の画像を、スクリーンショット集合１４ａの全スクリーンショットと比較する処理を繰り返す。また、判定部１５ｂは、処理対象の構成部品の画像を変えて、スクリーンショット集合１４ａの全スクリーンショットと比較する処理を繰り返す。 The determination unit 15b repeats the process of comparing the image of a single component with all the screenshots in the screenshot set 14a. Further, the determination unit 15b repeats the process of changing the image of the component to be processed and comparing it with all the screenshots in the screenshot set 14a.
　生成部１５ｃは、判定された結果を用いて、各システム画面に含まれる構成部品を示す画面仕様１４ｂを生成する。例えば、生成部１５ｃは、各スクリーンショットを識別する情報と各スクリーンショットに含まれる構成部品の画像を識別する情報とを対応付けた情報を、画面仕様１４ｂとして記憶部１４に記憶させる。 The generation unit 15c uses the determined results to generate screen specifications 14b indicating the components included in each system screen. For example, the generation unit 15c causes the storage unit 14 to store, as the screen specification 14b, information in which information identifying each screenshot is associated with information identifying an image of a component included in each screenshot.
　ここで、図３は、画面仕様のデータ構成を例示する図である。図３に例示するように、画面仕様１４ｂは、情報システムの操作画面と操作画面に出現する各構成部品とを対応付けた情報である。画面とは、スクリーンショット集合１４ａの各スクリーンショットを識別する情報である。また、画面構成部品とは、各構成部品の画像を識別する情報である。図３に示す「×」は、該当の構成部品が当該スクリーンショットに出現することを表し、「×」がない構成部品は当該スクリーンショットに出現しないことを表している。 Here, FIG. 3 is a diagram illustrating the data structure of the screen specifications. As illustrated in FIG. 3, the screen specification 14b is information that associates the operation screen of the information system with each component that appears on the operation screen. A screen is information that identifies each screenshot in the screenshot set 14a. Further, the screen component is information that identifies an image of each component. An "x" shown in FIG. 3 indicates that the relevant component appears in the relevant screenshot, and a component without an "x" represents that it does not appear in the relevant screenshot.
　また、判定部１５ｂは、記憶部１４のスクリーンショット集合１４ａのスクリーンショットに、取得された構成部品の画像が含まれると判定した場合に、さらに該スクリーンショットにおける出現位置を特定する。例えば、判定部１５ｂは、スクリーンショット上の座標等の構成部品の出現位置を特定し、生成部１５ｃが、特定されたた出現位置と構成部品の画像を識別する情報と対応付けて、画面仕様１４ｂに追加する。 Further, when determining that the acquired image of the component is included in the screenshot of the screenshot collection 14a of the storage unit 14, the determining unit 15b further specifies the appearance position in the screenshot. For example, the determining unit 15b specifies the appearance position of the component such as the coordinates on the screenshot, and the generation unit 15c associates the specified appearance position with information identifying the image of the component, and generates a screen specification. Add to 14b.
　生成部１５ｃは、生成した画面仕様１４ｂに基づいて、出力部１２を介して画面仕様をユーザに提示する。なお、生成部１５ｃは、画面仕様１４ｂをそのまま提示してもよいし、各構成部品が出現するか否かに変えて、各構成部品の出現位置を提示してもよい。例えば、図３に示した「×」が、各構成部品の出現位置を表す情報であってもよい。あるいは、図３に示した「×」に各構成部品の出現位置を併記されたものであってもよい。 The generation unit 15c presents the screen specifications to the user via the output unit 12 based on the generated screen specifications 14b. Note that the generation unit 15c may present the screen specifications 14b as is, or may present the appearance position of each component instead of whether each component appears. For example, the "x" shown in FIG. 3 may be information representing the appearance position of each component. Alternatively, the appearance position of each component may be written together with the "x" shown in FIG. 3.
　また、判定部１５ｂは、記憶部１４のスクリーンショット集合１４ａのスクリーンショットに、取得された構成部品の画像が含まれると判定した場合に、さらに該スクリーンショットの全領域に対する構成部品の画像の領域の割合を算出してもよい。 Further, when determining that the screenshot of the screenshot collection 14a of the storage unit 14 includes the acquired image of the component, the determining unit 15b further determines that the area of the image of the component with respect to the entire area of the screenshot is You may also calculate the ratio of
　ここで、図４は、判定部の処理を例示する図である。また、図５は、画面仕様の提示例を示す図である。この場合に、判定部１５ｂは、図４に太枠で例示するように、スクリーンショットの全領域における各構成部品の画像の領域の割合を算出する。そして、生成部１５ｃが、算出された各構成部品の画像の領域の割合を画面仕様１４ｂに追加する。 Here, FIG. 4 is a diagram illustrating the processing of the determination unit. Further, FIG. 5 is a diagram showing an example of presentation of screen specifications. In this case, the determination unit 15b calculates the ratio of the area of the image of each component in the entire area of the screenshot, as illustrated by the bold frame in FIG. Then, the generation unit 15c adds the calculated area ratio of the image of each component to the screen specification 14b.
　そして、生成部１５ｃがユーザに提示する際には、図５に例示するように、各構成部品の出現位置に変えて、スクリーンショットの全領域における各構成部品の領域の割合（％）を提示してもよい。図５に示す例では、構成部品の領域以外の非抽出領域の割合が併せて提示されている。 Then, when the generation unit 15c presents the user with the ratio (%) of the area of each component in the entire area of the screenshot, instead of the appearance position of each component, as illustrated in FIG. You may. In the example shown in FIG. 5, the proportion of non-extracted regions other than the component region is also presented.
　図１の説明に戻る。分類部１５ｄは、記憶部１４のスクリーンショットの集合を各スクリーンショット間の類似度に応じてクラス分けする。例えば、分類部１５ｄは、スクリーンショット集合１４ａのスクリーンショット間の類似度を算出し、算出された類似度が所定の閾値以上であるスクリーンショット同士が同一クラスになるように、各スクリーンショットをクラス分けする。そして、生成部１５ｃは、画面仕様１４ｂに各スクリーンショットのクラスを追加する。 Returning to the explanation of FIG. 1. The classification unit 15d classifies the set of screenshots in the storage unit 14 according to the degree of similarity between each screenshot. For example, the classification unit 15d calculates the degree of similarity between the screenshots in the screenshot set 14a, and classifies each screenshot so that the screenshots for which the calculated degree of similarity is greater than or equal to a predetermined threshold are in the same class. Divide. The generation unit 15c then adds the class of each screenshot to the screen specification 14b.
　ここで、図６は、画面仕様の提示例を示す図である。図６に示す例では、各画面に対して分類された画面クラスが追加された画面仕様１４ｂがユーザに提示されている。これにより、同一クラスの類似した操作画面においても異なる構成部品が出現するなど、より詳細な画面仕様が明らかになる。 Here, FIG. 6 is a diagram showing an example of presentation of screen specifications. In the example shown in FIG. 6, the user is presented with a screen specification 14b in which screen classes classified for each screen are added. As a result, more detailed screen specifications become clear, such as the appearance of different components even on similar operation screens of the same class.
［分析処理］
　次に、図７を参照して、本実施形態に係る分析装置１０による分析処理について説明する。図７は、分析処理手順を示すフローチャートである。図７のフローチャートは、例えば、ユーザが開始を指示する操作入力を行ったタイミングで開始される。 [Analysis processing]
Next, with reference to FIG. 7, analysis processing by the analysis device 10 according to this embodiment will be described. FIG. 7 is a flowchart showing the analysis processing procedure. The flowchart in FIG. 7 is started, for example, at the timing when the user performs an operation input instructing to start.
　まず、取得部１５ａが、システム画面の指定された構成部品の画像を取得する（ステップＳ１）。例えば、取得部１５ａは、システム画面のスクリーンショット上で指定された領域の部分画像を、構成部品の画像として取得する。 First, the acquisition unit 15a acquires an image of a designated component on the system screen (step S1). For example, the acquisition unit 15a acquires a partial image of a designated area on a screenshot of the system screen as an image of a component.
　次に、判定部１５ｂが、スクリーンショット集合１４ａの各スクリーンショットに、取得された構成部品の画像が出現するか否かを判定する（ステップＳ２）。当該スクリーンショットに構成部品の画像が出現しない場合には（ステップＳ２、Ｎｏ）、判定部１５ｂは、ステップＳ４に処理を進める。 Next, the determining unit 15b determines whether the acquired image of the component appears in each screenshot of the screenshot set 14a (step S2). If the image of the component does not appear in the screenshot (step S2, No), the determination unit 15b advances the process to step S4.
　一方、当該スクリーンショットに構成部品の画像が出現している場合には（ステップＳ２、Ｙｅｓ）、当該構成部品の画像のスクリーンショットにおける出現位置を特定する（ステップＳ３）。 On the other hand, if the image of the component appears in the screenshot (step S2, Yes), the appearance position of the image of the component in the screenshot is specified (step S3).
　判定部１５ｂは、全てのスクリーンショットについて、ステップＳ２～Ｓ３の処理を繰り返す（ステップＳ４、Ｎｏ→Ｓ２）。そして、判定部１５ｂは、全てのスクリーンショットの処理が終わったら（ステップＳ４、Ｙｅｓ）、全ての構成部品の画像についての処理が終わるまで、ステップＳ１に処理を戻す（ステップＳ５、Ｎｏ→Ｓ１）。 The determination unit 15b repeats the processing of steps S2 to S3 for all screenshots (step S4, No→S2). Then, when the determination unit 15b finishes processing all the screenshots (step S4, Yes), the determination unit 15b returns the process to step S1 until the processing of all component images is finished (step S5, No→S1). .
　そして、全ての構成部品の画像についての処理が終わったら（ステップＳ５、Ｙｅｓ）、生成部１５ｃが、画面仕様１４ｂを生成する。また、生成部１５ｃは、生成した画面仕様１４ｂに基づいて、画面仕様をユーザに対して出力する（ステップＳ６）。これにより、一連の分析処理が終了する。 Then, when the processing of images of all component parts is completed (step S5, Yes), the generation unit 15c generates the screen specification 14b. Furthermore, the generation unit 15c outputs the screen specifications to the user based on the generated screen specifications 14b (step S6). This completes the series of analysis processes.
［効果］
　以上、説明したように、本実施形態の分析装置１０において、記憶部１４が、操作中に記録されたシステム画面のスクリーンショット集合１４ａを記憶する。取得部１５ａが、システム画面の指定された構成部品の画像を取得する。判定部１５ｂが、記憶部１４のスクリーンショット集合１４ａの各スクリーンショットに、取得された構成部品の画像が含まれるか否かを判定する。生成部１５ｃが、判定された結果を用いて、各システム画面に含まれる構成部品を示す画面仕様１４ｂを生成する。 [effect]
As described above, in the analysis device 10 of this embodiment, the storage unit 14 stores the screen shot set 14a of the system screen recorded during operation. The acquisition unit 15a acquires an image of a designated component on the system screen. The determining unit 15b determines whether each screenshot of the screenshot set 14a in the storage unit 14 includes an acquired image of a component. The generation unit 15c uses the determined results to generate screen specifications 14b indicating the components included in each system screen.
　具体的には、取得部１５ａは、システム画面のスクリーンショット上で指定された領域の部分画像を、構成部品の画像として取得する。 Specifically, the acquisition unit 15a acquires a partial image of the area specified on the screenshot of the system screen as an image of the component.
　これにより、分析装置１０は、システムの各操作画面に出現する構成部品を、その位置に関わらずに抽出して一覧化することが可能となる。このように、分析装置１０によれば、情報システムの画面仕様を明らかにすることが容易に可能となる。したがって、例えば既存の情報システムの更改に際して、似ている画面で構成部品が異なる等、問題点の分析が可能となる。 This makes it possible for the analysis device 10 to extract and list the components that appear on each operation screen of the system, regardless of their positions. In this way, according to the analysis device 10, it becomes possible to easily clarify the screen specifications of the information system. Therefore, for example, when updating an existing information system, it is possible to analyze problems such as similar screens with different component parts.
　また、判定部１５ｂは、記憶部１４のスクリーンショット集合１４ａのスクリーンショットに、取得された構成部品の画像が含まれると判定した場合に、さらに該スクリーンショットにおける出現位置を特定する。これにより、情報システムの画面仕様を詳細に明らかにすることが可能となる。 Further, when determining that the acquired image of the component is included in the screenshot of the screenshot collection 14a of the storage unit 14, the determining unit 15b further specifies the appearance position in the screenshot. This makes it possible to clarify the screen specifications of the information system in detail.
　また、判定部１５ｂは、記憶部１４のスクリーンショット集合１４ａのスクリーンショットに、取得された構成部品の画像が含まれると判定した場合に、さらに該スクリーンショットの全領域に対する構成部品の画像の領域の割合を算出する。これによっても、情報システムの画面仕様を詳細に明らかにすることが可能となる。 Further, when determining that the screenshot of the screenshot collection 14a of the storage unit 14 includes the acquired image of the component, the determining unit 15b further determines that the area of the image of the component with respect to the entire area of the screenshot is Calculate the percentage of This also makes it possible to clarify the screen specifications of the information system in detail.
　また、分類部１５ｄが、記憶部１４のスクリーンショットの集合を各スクリーンショット間の類似度に応じてクラス分けする。これによっても、より詳細に情報システムの画面仕様を明らかにすることが可能となる。 Furthermore, the classification unit 15d classifies the set of screenshots in the storage unit 14 according to the degree of similarity between each screenshot. This also makes it possible to clarify the screen specifications of the information system in more detail.
［プログラム］
　上記実施形態に係る分析装置１０が実行する処理をコンピュータが実行可能な言語で記述したプログラムを作成することもできる。一実施形態として、分析装置１０は、パッケージソフトウェアやオンラインソフトウェアとして上記の分析処理を実行する分析プログラムを所望のコンピュータにインストールさせることによって実装できる。例えば、上記の分析プログラムを情報処理装置に実行させることにより、情報処理装置を分析装置１０として機能させることができる。ここで言う情報処理装置には、デスクトップ型またはノート型のパーソナルコンピュータが含まれる。また、その他にも、情報処理装置にはスマートフォン、携帯電話機やＰＨＳ（Personal　Handyphone　System）などの移動体通信端末、さらには、ＰＤＡ（Personal　Digital　Assistant）などのスレート端末などがその範疇に含まれる。また、分析装置１０の機能を、クラウドサーバに実装してもよい。 [program]
It is also possible to create a program in which the processing executed by the analysis device 10 according to the embodiment described above is written in a computer-executable language. As one embodiment, the analysis device 10 can be implemented by installing an analysis program that executes the above analysis process on a desired computer as packaged software or online software. For example, by causing the information processing device to execute the above analysis program, the information processing device can be made to function as the analysis device 10. The information processing device referred to here includes a desktop or notebook personal computer. In addition, information processing devices include mobile communication terminals such as smartphones, mobile phones, and PHSs (Personal Handyphone Systems), as well as slate terminals such as PDAs (Personal Digital Assistants). Further, the functions of the analysis device 10 may be implemented in a cloud server.
　図８は、分析プログラムを実行するコンピュータの一例を示す図である。コンピュータ１０００は、例えば、メモリ１０１０と、ＣＰＵ１０２０と、ハードディスクドライブインタフェース１０３０と、ディスクドライブインタフェース１０４０と、シリアルポートインタフェース１０５０と、ビデオアダプタ１０６０と、ネットワークインタフェース１０７０とを有する。これらの各部は、バス１０８０によって接続される。 FIG. 8 is a diagram showing an example of a computer that executes an analysis program. Computer 1000 includes, for example, memory 1010, CPU 1020, hard disk drive interface 1030, disk drive interface 1040, serial port interface 1050, video adapter 1060, and network interface 1070. These parts are connected by a bus 1080.
　メモリ１０１０は、ＲＯＭ（Read　Only　Memory）１０１１およびＲＡＭ１０１２を含む。ＲＯＭ１０１１は、例えば、ＢＩＯＳ（Basic　Input　Output　System）等のブートプログラムを記憶する。ハードディスクドライブインタフェース１０３０は、ハードディスクドライブ１０３１に接続される。ディスクドライブインタフェース１０４０は、ディスクドライブ１０４１に接続される。ディスクドライブ１０４１には、例えば、磁気ディスクや光ディスク等の着脱可能な記憶媒体が挿入される。シリアルポートインタフェース１０５０には、例えば、マウス１０５１およびキーボード１０５２が接続される。ビデオアダプタ１０６０には、例えば、ディスプレイ１０６１が接続される。 The memory 1010 includes a ROM (Read Only Memory) 1011 and a RAM 1012. The ROM 1011 stores, for example, a boot program such as BIOS (Basic Input Output System). Hard disk drive interface 1030 is connected to hard disk drive 1031. Disk drive interface 1040 is connected to disk drive 1041. A removable storage medium such as a magnetic disk or an optical disk is inserted into the disk drive 1041, for example. For example, a mouse 1051 and a keyboard 1052 are connected to the serial port interface 1050. For example, a display 1061 is connected to the video adapter 1060.
　ここで、ハードディスクドライブ１０３１は、例えば、ＯＳ１０９１、アプリケーションプログラム１０９２、プログラムモジュール１０９３およびプログラムデータ１０９４を記憶する。上記実施形態で説明した各情報は、例えばハードディスクドライブ１０３１やメモリ１０１０に記憶される。 Here, the hard disk drive 1031 stores, for example, an OS 1091, an application program 1092, a program module 1093, and program data 1094. Each piece of information described in the above embodiments is stored in, for example, the hard disk drive 1031 or the memory 1010.
　また、分析プログラムは、例えば、コンピュータ１０００によって実行される指令が記述されたプログラムモジュール１０９３として、ハードディスクドライブ１０３１に記憶される。具体的には、上記実施形態で説明した分析装置１０が実行する各処理が記述されたプログラムモジュール１０９３が、ハードディスクドライブ１０３１に記憶される。 Further, the analysis program is stored in the hard disk drive 1031, for example, as a program module 1093 in which commands to be executed by the computer 1000 are written. Specifically, a program module 1093 in which each process executed by the analysis device 10 described in the above embodiment is described is stored in the hard disk drive 1031.
　また、分析プログラムによる情報処理に用いられるデータは、プログラムデータ１０９４として、例えば、ハードディスクドライブ１０３１に記憶される。そして、ＣＰＵ１０２０が、ハードディスクドライブ１０３１に記憶されたプログラムモジュール１０９３やプログラムデータ１０９４を必要に応じてＲＡＭ１０１２に読み出して、上述した各手順を実行する。 Furthermore, data used for information processing by the analysis program is stored as program data 1094 in, for example, the hard disk drive 1031. Then, the CPU 1020 reads out the program module 1093 and program data 1094 stored in the hard disk drive 1031 to the RAM 1012 as necessary, and executes each of the above-described procedures.
　なお、分析プログラムに係るプログラムモジュール１０９３やプログラムデータ１０９４は、ハードディスクドライブ１０３１に記憶される場合に限られず、例えば、着脱可能な記憶媒体に記憶されて、ディスクドライブ１０４１等を介してＣＰＵ１０２０によって読み出されてもよい。あるいは、分析プログラムに係るプログラムモジュール１０９３やプログラムデータ１０９４は、ＬＡＮやＷＡＮ（Wide　Area　Network）等のネットワークを介して接続された他のコンピュータに記憶され、ネットワークインタフェース１０７０を介してＣＰＵ１０２０によって読み出されてもよい。 Note that the program module 1093 and program data 1094 related to the analysis program are not limited to being stored in the hard disk drive 1031; for example, they may be stored in a removable storage medium and read by the CPU 1020 via the disk drive 1041 or the like. may be done. Alternatively, the program module 1093 and program data 1094 related to the analysis program are stored in another computer connected via a network such as a LAN or WAN (Wide Area Network), and read out by the CPU 1020 via the network interface 1070. It's okay.
　以上、本発明者によってなされた発明を適用した実施形態について説明したが、本実施形態による本発明の開示の一部をなす記述および図面により本発明は限定されることはない。すなわち、本実施形態に基づいて当業者等によりなされる他の実施形態、実施例および運用技術等は全て本発明の範疇に含まれる。 Although embodiments to which the invention made by the present inventor is applied have been described above, the present invention is not limited by the description and drawings that form part of the disclosure of the present invention by this embodiment. That is, all other embodiments, examples, operational techniques, etc. made by those skilled in the art based on this embodiment are included in the scope of the present invention.
　１０　分析装置
　１１　入力部
　１２　出力部
　１３　通信制御部
　１４　記憶部
　１４ａ　スクリーンショット集合
　１４ｂ　画面仕様
　１５　制御部
　１５ａ　取得部
　１５ｂ　判定部
　１５ｃ　生成部
　１５ｄ　分類部 10 Analyzer 11 Input section 12 Output section 13 Communication control section 14 Storage section 14a Screen shot collection 14b Screen specifications 15 Control section 15a Acquisition section 15b Judgment section 15c Generation section 15d Classification section

Claims (7)

  1. 　操作中に記録されたシステム画面のスクリーンショットの集合を記憶する記憶部と、
    　システム画面の指定された構成部品の画像を取得する取得部と、
    　前記記憶部の各スクリーンショットに、取得された前記構成部品の画像が含まれるか否かを判定する判定部と、
    　判定された結果を用いて、各システム画面に含まれる構成部品を示す画面仕様を生成する生成部と、
    　を有することを特徴とする分析装置。     a storage unit that stores a collection of system screen screenshots recorded during operation;
    an acquisition unit that acquires an image of a specified component of the system screen;
    a determination unit that determines whether each screenshot in the storage unit includes an acquired image of the component;
    a generation unit that uses the determined results to generate screen specifications indicating the components included in each system screen;
    An analysis device characterized by having:
  2. 　前記取得部は、システム画面のスクリーンショット上で指定された領域の部分画像を、前記構成部品の画像として取得することを特徴とする請求項１に記載の分析装置。 The analysis device according to claim 1, wherein the acquisition unit acquires a partial image of a region specified on a screenshot of a system screen as an image of the component.
  3. 　前記判定部は、前記記憶部のスクリーンショットに、取得された前記構成部品の画像が含まれると判定した場合に、さらに該スクリーンショットにおける出現位置を特定することを特徴とする請求項１に記載の分析装置。 2. The determining unit further specifies an appearance position in the screenshot when determining that the acquired image of the component is included in the screenshot in the storage unit. analysis equipment.
  4. 　前記判定部は、前記記憶部のスクリーンショットに、取得された前記構成部品の画像が含まれると判定した場合に、さらに該スクリーンショットの全領域に対する前記構成部品の画像の領域の割合を算出することを特徴とする請求項１に記載の分析装置。 When the determining unit determines that the acquired image of the component is included in the screenshot in the storage unit, the determination unit further calculates a ratio of the area of the image of the component to the total area of the screenshot. The analysis device according to claim 1, characterized in that:
  5. 　前記記憶部のスクリーンショットの集合を各スクリーンショット間の類似度に応じてクラス分けする分類部をさらに有することを特徴とする請求項１に記載の分析装置。 The analysis device according to claim 1, further comprising a classification unit that classifies the set of screenshots in the storage unit according to the degree of similarity between each screenshot.
  6. 　分析装置が実行する分析方法であって、
    　前記分析装置が、操作中に記録されたシステム画面のスクリーンショットの集合を記憶する記憶部を有し、
    　システム画面の指定された構成部品の画像を取得する取得工程と、
    　前記記憶部の各スクリーンショットに、取得された前記構成部品の画像が含まれるか否かを判定する判定工程と、
    　判定された結果を用いて、各システム画面に含まれる構成部品を示す画面仕様を生成する生成工程と、
    　を含んだことを特徴とする分析方法。     An analysis method performed by an analyzer,
    The analysis device has a storage unit that stores a collection of screenshots of system screens recorded during operation,
    an acquisition step of acquiring an image of a designated component of the system screen;
    a determination step of determining whether each screenshot in the storage unit includes an acquired image of the component;
    a generation step of generating screen specifications indicating the components included in each system screen using the determined results;
    An analysis method characterized by including.
  7. 　操作中に記録されたシステム画面のスクリーンショットの集合を記憶する記憶部を参照し、
    　システム画面の指定された構成部品の画像を取得する取得ステップと、
    　前記記憶部の各スクリーンショットに、取得された前記構成部品の画像が含まれるか否かを判定する判定ステップと、
    　判定された結果を用いて、各システム画面に含まれる構成部品を示す画面仕様を生成する生成ステップと、
    　をコンピュータに実行させるための分析プログラム。     Refers to the storage unit that stores a collection of system screen screenshots recorded during operation,
    an acquisition step of acquiring an image of a specified component of the system screen;
    a determination step of determining whether each screenshot in the storage unit includes an acquired image of the component;
    a generation step of generating screen specifications indicating the components included in each system screen using the determined results;
    An analysis program that allows a computer to execute
PCT/JP2022/028845 2022-07-26 2022-07-26 Analysis device, analysis method, and analysis program WO2024023948A1 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
PCT/JP2022/028845 WO2024023948A1 (en) 2022-07-26 2022-07-26 Analysis device, analysis method, and analysis program

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/JP2022/028845 WO2024023948A1 (en) 2022-07-26 2022-07-26 Analysis device, analysis method, and analysis program

Publications (1)

Publication Number Publication Date
WO2024023948A1 true WO2024023948A1 (en) 2024-02-01

Family

ID=89705653

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2022/028845 WO2024023948A1 (en) 2022-07-26 2022-07-26 Analysis device, analysis method, and analysis program

Country Status (1)

Country Link
WO (1) WO2024023948A1 (en)

Non-Patent Citations (3)

* Cited by examiner, † Cited by third party
Title
DAISUKE TASHIRO, RYOTA MIBE: "Data Output UI Specification Extraction using Differences of Texts in HTML Documents. ", IEICE TECHNICAL REPORT, ISEC, IEICE, JP, vol. 116, no. 289 (ISEC2016-67), 31 October 2016 (2016-10-31), JP, pages 89 - 94, XP009552264 *
TOM YEH ; TSUNG-HSIANG CHANG ; ROBERT C. MILLER: "Sikuli", PROCEEDINGS OF THE 22ND ANNUAL ACM SYMPOSIUM ON USER INTERFACE SOFTWARE AND TECHNOLOGY: VICTORIA, BRITISH COLUMBIA, OCTOBER 4 - 7, 2009, ACM, NEW YORK, NY, 4 October 2009 (2009-10-04) - 7 October 2009 (2009-10-07), New York, NY , pages 183 - 192, XP058124772, ISBN: 978-1-60558-745-5, DOI: 10.1145/1622176.1622213 *
WU JASON JSONWU@CMU.EDU; ZHANG XIAOYI XIAOYIZ@APPLE.COM; NICHOLS JEFF JWNICHOLS@APPLE.COM; BIGHAM JEFFREY P JBIGHAM@APPLE.COM: "Screen Parsing: Towards Reverse Engineering of UI Models from Screenshots", PROCEEDINGS OF THE 30TH ACM INTERNATIONAL CONFERENCE ON MULTIMEDIA, ACMPUB27, NEW YORK, NY, USA, 10 October 2021 (2021-10-10) - 14 August 2022 (2022-08-14), New York, NY, USA, pages 470 - 483, XP058896560, ISBN: 978-1-4503-9036-1, DOI: 10.1145/3472749.3474763 *

Similar Documents

Publication Publication Date Title
EP3215900B1 (en) Robotic process automation
WO2020235085A1 (en) Operation log visualization device, operation log visualization method, and operation log visualization program
WO2017001560A1 (en) Robotic process automation
CN111143188A (en) Method and equipment for automatically testing application
CN109871205B (en) Interface code adjustment method, device, computer device and storage medium
CN114282686A (en) Method and system for constructing machine learning modeling process
JP7380714B2 (en) Operation log acquisition device and operation log acquisition method
WO2024023948A1 (en) Analysis device, analysis method, and analysis program
JP7173308B2 (en) DETECTION DEVICE, DETECTION METHOD AND DETECTION PROGRAM
CN111401465A (en) Training sample optimization method, device, equipment and storage medium
CN111258875A (en) Interface test method and system, electronic device and storage medium
JP7501674B2 (en) Operation support device, operation support method, and operation support program
JP7338791B2 (en) User operation recording device and user operation recording method
US9015658B2 (en) Automatic program generation device and automatic program generation method
CN110968519A (en) Game testing method, device, server and storage medium
CN111414213B (en) Method for identifying boot stage of basic input and output system
EP4361909A1 (en) Method and system for task recording using robotic process automation techchnology
JP7485078B2 (en) DISPLAY CONTROL DEVICE, DISPLAY CONTROL METHOD, AND DISPLAY CONTROL PROGRAM
WO2024134843A1 (en) Abnormality detection device, abnormality detection method and abnormality detection program
US11899552B2 (en) Estimation device, estimation method and estimation program
CN116185411B (en) RPA interface determination method, RPA interface determination device, computer equipment and storage medium
EP3872640B1 (en) Information creation device, information creation method, and information creation program
CN116708945B (en) Media editing method, device, equipment and storage medium
CN114860611B (en) BIOS upgrading test method, device and medium
US20240004508A1 (en) Operation monitoring device, operation monitoring method, and operation monitoring program

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22953043

Country of ref document: EP

Kind code of ref document: A1