WO2015107671A1

WO2015107671A1 - Software execution operation assistance device and software execution operation assistance method

Info

Publication number: WO2015107671A1
Application number: PCT/JP2014/050764
Authority: WO
Inventors: 木原　隆幸
Original assignee: 株式会社島津製作所
Priority date: 2014-01-17
Filing date: 2014-01-17
Publication date: 2015-07-23
Also published as: JP6094691B2; JPWO2015107671A1

Abstract

A software execution operation assistance device (1) for assisting a user to perform an operation using the subject software, wherein the user is assisted to perform the operation intuitively using the subject software, through the provision of: a database (36) in which a registered image and a prescribed process are stored in a linked manner; an operation position identifying means (32) which identifies the position of an operation performed by the user on a screen of a display unit (14); a registered image detecting means (33) which determines whether or not the registered image is contained in a prescribed neighboring area of the operation position identified by the operation position identifying means (32); and an execution means (35) which executes the prescribed process linked to the registered image that the registered image detecting means (33) has determined is contained in the neighboring area.

Description

Software execution operation assistance device and software execution operation assistance method

The present invention relates to a software execution operation assistance device and a software execution operation assistance method for assisting a user's operation on target software.

When an analyzer such as a gas chromatograph (GC) or liquid chromatograph (LC) is used in combination with a plurality of detectors, a plurality of chromatograms are obtained simultaneously. When a photodiode array detector or a mass spectrometer is used as a detector, a three-dimensional chromatogram having a wavelength and mass-to-charge ratio as a third axis in addition to the time axis and the intensity axis is obtained. A plurality of two-dimensional chromatograms (for example, a mass chromatogram when the detector is a mass spectrometer) can be cut out by focusing on the mass number (for example, Patent Document 1).

In recent years, analysis processing of analysis data obtained as a result of control of an analysis apparatus or analysis is performed using analysis software that operates on a general computer. When a user analyzes analysis data using such analysis software, a specific operation for executing a desired process is often repeated. Therefore, for frequently executed processes, a lot of analysis software is provided that improves the operability by shortening the time required for operation by providing a GUI button that can be commanded to execute the process with a single click. Has been. In addition to these buttons, a menu that is expanded by right-clicking or clicking on a selected item on a toolbar is also common. These GUIs activate predetermined functions by accepting an operation such as a click by the user.

Japanese Patent Laid-Open No. 10-73582

When performing predetermined processing on analysis data on general analysis software, the user needs to perform a series of operations in order to instruct execution of the processing.

As a specific example, consider a case where the user wants to manually change the peak start point of the chromatogram automatically detected by the analysis software for the LC analysis data (FIG. 15). FIG. 15 is an example of a display screen of general LC analysis data analysis software, and shows a state in which the peak start point (up arrow) and end point (down arrow) are detected by the analysis software. First, the user clicks the peak detection point move button on the GUI button bar. Next, on the chromatogram displayed on the screen, the peak start point mark attached to the peak start point s1 detected by the analysis software is clicked with the cursor, and then the change destination point s2 is clicked. By such a series of operations, the position of the peak start point (and the up arrow) is changed.

Thus, normally, in order to change the peak start point, it is necessary to click the peak detection point moving button in advance. Therefore, even when the cursor is near the peak start point mark that is the operation target, the user once moves the cursor from the spot, clicks the peak detection point move button, and again reaches the peak start point mark. An operation such as moving the cursor and clicking is required. Such an operation is not intuitive because it requires a large number of clicks and the user's viewpoint and the range of movement of the cursor are large, and if it is repeated many times, it requires a lot of time and is inefficient. Similar problems occur when a touch panel is used instead of a mouse.

If the analysis software does not include a GUI button to which a process desired by the user is assigned, the operation becomes even more complicated.
Specifically, for example, a predetermined selection item displayed on the analysis software toolbar is clicked to expand a pull-down menu, and one of a plurality of selection items listed in the menu is clicked. Therefore, an operation for executing a desired process is required. In this series of operations, the viewpoint and the cursor move more than in the above-described button example, and the throughput is also reduced.

In order to avoid the above-mentioned annoyance, analysis software that does not require button or menu pre-operation for specific processing is commercially available in recent years, but such functions are externally applied to other existing analysis software. It is difficult to add if the source code of the analysis software is not known.

The present invention has been made in view of the above circumstances, and an object thereof is to provide a software execution operation assistance device and a software execution operation assistance method for assisting a user's intuitive operation on target software. There is.

A first invention made to solve the above problems is a software execution operation assisting device for assisting a user's operation on the target software,
a) a database in which registered images and predetermined processes are stored in association with each other;
b) operation position specifying means for specifying, as an operation position, a position at which a user performs a predetermined operation on the display screen;
c) registered image detection means for determining whether or not the registered image is included in a predetermined vicinity range of the operation position specified by the operation position specifying means;
d) execution means for executing the predetermined process associated with the registered image determined to be included in the vicinity range by the registered image detection means;
It is characterized by providing.

As the predetermined operation, a click through a mouse or a touch operation with an indicator such as a finger can be exemplified. The predetermined pattern of the temporal change of the cursor position is also included in the predetermined operation of the present invention.

According to the software execution operation assisting device of the first invention, when the user performs a predetermined operation on the screen of the display unit, the registered image included in the vicinity of the operation position on the screen is detected and detected. The process associated with the registered image is executed. Therefore, for example, if a predetermined image including at least a part of an object to be operated on the target software or attached in the vicinity of the object is set as a registered image, the user can perform the operation by operating in the vicinity of the object. It is possible to instruct execution of a process indirectly associated with an object. This eliminates the need for the user to move the viewpoint, the cursor, and the like in accordance with an operation conventionally required to execute the processing, and allows the user to seamlessly shift to the object operation.
Therefore, it is possible to assist the user's intuitive operation on the target software.

As the “object” in the present invention, a graph such as a chromatogram or spectrum, its waveform line, a mark attached to the graph, a display area of text data accompanying the graph or data, and the like can be exemplified. In addition, the “image” in the present invention includes any expression (for example, a character or a graph) composed of a combination of pixels.

The predetermined process is a process assigned to a predetermined GUI included in the target software,
The execution unit may execute a process assigned to the predetermined GUI associated with the registered image detected by the registered image detection unit with respect to the target software.
According to this configuration, if the user clicks on the vicinity of the registered image, processing corresponding to the situation in which an operation is performed on the GUI associated with the registered image is executed. The convenience can be further enhanced by using.

The software execution operation assisting device further includes:
e) It is preferable to provide a linking means for linking one or a plurality of the registered images and the predetermined process and storing them in the database in accordance with a user instruction.
According to this configuration, the user can arbitrarily associate the registered image with the predetermined process. For example, the user associates the process frequently used with the registered image with the registered image. Thus, the operation associated with the processing can be made more intuitive.

The registered image detection unit may determine that the registered image is not included in the vicinity range when a ratio of pixels having a predetermined color in the vicinity range is less than a predetermined value. Good.
According to this configuration, the process associated with the registered image is executed only when a predetermined color occupies a certain ratio or more in the vicinity range. Therefore, if this predetermined color is set as the background color of the graph, for example, the background color of the graph generally occupies most of the graph, so only when an operation is performed near the registered image in the graph. The process associated with the registered image is executed. That is, when an operation is performed on the vicinity of an image that is the same as a registered image or can be determined to be a registered image in an area outside the graph, the process associated with the registered image is prevented from being erroneously executed. Can do.

The software execution operation assisting device may further be configured to change the color of the registered image and / or an image in a predetermined area including the registered image, or to prompt the user to change the color.
According to this configuration, for example, the color of the registered image and / or the background area of the registered image is set to a color that is unlikely to overlap with other images on the screen of the display unit. It is possible to prevent another image from being erroneously detected as a registered image.

A second invention made to solve the above problems is a software execution operation assisting method for assisting a user's operation on the target software,
a) a linking means for linking a registered image and a predetermined process;
b) An operation position specifying step in which the operation position specifying means specifies, as the operation position, a position where a predetermined operation is performed by the user on the screen of the display unit;
c) a registered image detecting step for determining whether or not the registered image is included in a predetermined vicinity area of the operation position specified in the operation position specifying step;
d) an executing unit that executes the predetermined process associated with the registered image that is determined to be included in the vicinity area in the registered image detection step;
It is characterized by including.

According to the present invention, a process indirectly associated with an object is executed by an operation in the vicinity of the object by the user, so that the user's intuitive operation on the target software can be assisted. it can.

1 is a block diagram showing a schematic configuration of a sample analysis system including a software execution operation assisting device according to a first embodiment of the present invention. The flowchart which shows the flow of the software execution operation assistance process by the software execution operation assistance apparatus which concerns on the same embodiment. The example of a screen display of a display part when a click is performed on the peak starting point which the target software detected in the software execution operation assistance apparatus which concerns on the embodiment. The conceptual diagram which shows the vicinity area | region centering on the click position shown in FIG. The conceptual diagram which shows the structural example of the registration image-target GUI correspondence table stored in the database with which the software execution operation assistance apparatus which concerns on the same embodiment is provided. The flowchart which shows an example of the flow of the correlation setting process by the software execution operation assistance apparatus which concerns on the same embodiment. The flowchart which shows the flow of the registration image detection process by the software execution operation assistance apparatus which concerns on the 2nd Embodiment of this invention. (A) The example of a screen display of a display part when a click is performed by the point on a chromatogram waveform line on object software in the software execution operation assistance apparatus which concerns on the same embodiment. (B) The conceptual diagram which shows the vicinity area | region centering on the click position shown by (a). The flowchart which shows the flow of the software execution operation assistance process by the software execution operation assistance apparatus which concerns on the 3rd Embodiment of this invention. The example of a screen display of a display part when a right click operation is performed on the chromatogram expanded by object software in the software execution operation assistance apparatus which concerns on the embodiment. The block diagram which shows schematic structure of the sample analysis system containing the software execution operation assistance apparatus which concerns on the 4th Embodiment of this invention. The flowchart which shows the flow of the software execution operation assistance process by the software execution operation assistance apparatus which concerns on the same embodiment. (A) The example of a screen display of a display part when a click is performed by the point on the waveform line of a chromatogram on object software in the software execution operation assistance apparatus which concerns on the same embodiment. (B) Screen display example showing a state in which the extracted waveform line is displayed superimposed on another chromatogram. The flowchart which shows the flow of the superimposition assistance process by the software execution operation assistance apparatus which concerns on the same embodiment. An example of a display screen of analysis software for general LC analysis data.

Hereinafter, embodiments for carrying out the present invention will be described in detail with reference to the drawings. In the following description, members having the same functions as those in the above-described drawings are denoted by the same reference numerals, and description thereof is omitted.

[First Embodiment]
FIG. 1 shows an analysis data processing apparatus (software execution operation assisting apparatus) 1 according to a first embodiment of the present invention. The actual state of the analysis data processing apparatus 1 is a computer. A CPU (Central Processing Unit) 10 which is a central processing unit 10 has a memory 12, a monitor (display unit) 14 including an LCD (Liquid Crystal Display), a keyboard, a mouse, and the like. An input unit 16 and a storage unit 20 including a mass storage device such as a hard disk or an SSD (Solid State Drive) are connected to each other. The storage unit 20 is provided with an execution operation program 21, an analysis data analysis program 22, and an analysis data storage unit 23. The storage unit 20 also stores an OS (Operating System) 24. In addition to this, the storage unit 20 may be provided with a program for controlling the analysis by the analyzer A1, but the illustration is omitted because it is out of the scope of the present invention.

The analysis data processing apparatus 1 has an interface (I / F) (data output means) for direct connection with an external apparatus and connection with an external apparatus or the like via a network such as a LAN (Local Area Network). 18 is connected to the analyzer A1 via the network cable NW (or wireless LAN) from the I / F 18.
Note that there may be a plurality of analyzers connected to the analysis data processing apparatus 1. Further, the analysis data processing device 1 and the analysis device A1 may be configured as an integrated device.

The analyzer A1 is a liquid chromatograph mass spectrometer (LC-MS) in this embodiment. However, the actual condition of the analyzer A1 is not limited to this, and may be GC or LC, or a gas chromatograph mass spectrometer (GC-MS). The analysis result by the analysis device A1 is transmitted to the analysis data processing device 1 and accumulated in the analysis data storage unit 23 as analysis data.

The analysis data analysis program (target software) 22 is application software for analyzing analysis data by LC-MS, and creates graphs such as chromatograms and spectra based on the analysis data accumulated in the analysis data storage unit 23. This is displayed on the screen of the monitor 14.
The analysis data analysis program 22 also executes various analysis processes based on the analysis data automatically or based on a user operation via the input unit 16, and generates the text data indicating each analysis result and a predetermined image as the created graph. It has a function to be added on or near the top. Typical examples of the predetermined image include an arrow mark indicating the peak start point or end point, a baseline connecting the start point and end point, and the like as exemplified in FIG. . As shown in the figure, the analysis data analysis program 22 further displays a GUI for instructing execution of various waveform processes on the screen, and accepts a user input operation for each button. Note that the term “button” in the present embodiment refers to a GUI button displayed on the screen of the display unit.

The execution operation assistance program 21 is a function expansion program for the analysis data analysis program 22. The execution operation assistance program 21 may be installed later on a computer in which the analysis data analysis program 22 is already installed, or may be installed simultaneously with the analysis data analysis program 22.
In FIG. 1, as related to the execution operation assistance program 21, an association setting unit (association unit) 31, an operation position identification unit (operation position identification unit) 32, a registered image detection unit (registered image detection unit) 33, A target GUI detection unit 34, an execution command unit (execution means) 35, and a registered image linking database (DB) 36 are shown. These are basically functional means realized by software when the CPU 10 executes the execution operation auxiliary program 21.

[Software execution operation support process flow]
Hereinafter, the flow of the software execution operation assisting process will be specifically described with reference to FIG. 2 which is a flowchart and FIGS. 3 to 5 as necessary. As an example, it is assumed that the user manually changes the peak start point of the chromatogram automatically detected by the analysis data analysis program 22.

First, the operation position specifying unit 32 detects a click by the user (step S1), and specifies the click position (step S2: operation position specifying step). Specifically, when the operation position specifying unit 32 obtains an operation signal corresponding to the click through the input unit 16, the coordinate information included in the operation signal is read, and X and X on the screen of the monitor 14 are read. Get the coordinate value of Y.

FIG. 3 is a screen display example by the monitor 14 at this time. It is assumed that the user has placed the cursor C on the peak start point mark 301 attached to the point 302 of the chromatogram 300 automatically detected by the analysis data analysis program 22 and clicked.

Next, the operation position specifying unit 32 determines whether or not the click position is within the application window of the analysis data analysis program 22 (step S3). Specifically, the operation position specifying unit 32 determines whether the coordinate value of the click position is within the display range of the application window of the analysis data analysis program 22 based on the coordinate information read in step S2. If the click position is outside the application window (No in step S3), the software execution operation assistance process returns to the position before step S1 and is in a standby state until the next click operation is performed. Thereby, it is possible to prevent an image unrelated to the analysis data analysis program 22 from being detected as a registered image.

In the example shown in FIG. 3, since the click position is a point in the application window of the analysis data analysis program 22 (Yes in step S3), subsequently, the registered image detection unit 33 acquires an image of the neighborhood region R (step S4). Specifically, the registered image detection unit 33 instructs the OS 24 to capture the display content of the entire screen, and the coordinates of the click position specified in step S2 are centered from the full screen captured image that the OS 24 has copied to the clipboard. The image of the rectangular area to be extracted is extracted (trimmed).
FIG. 4 shows a conceptual diagram of the neighboring region (neighboring range) R. The neighboring region R is a range of several pixels in the vertical and horizontal directions centered on the click position 400 specified in step S2. The size of the neighboring region R may be changed as appropriate, and the shape is not limited to a rectangle but may be a circle.

Next, the registered image detection unit 33 determines whether or not a registered image is included in the vicinity region R (step S5: registered image detection step). Specifically, the registered image detection unit 33 first refers to the registered image-target GUI correspondence table 500 stored in the registered image association database 36. FIG. 5 is a conceptual diagram showing an example of the structure of the registered image-target GUI correspondence table 500. In the registered image-target GUI correspondence table 500, a registered image and a target GUI are associated with each record. For example, in the first record, a peak start point mark is registered as a registered image, and a peak detection point moving button is registered as a corresponding target GUI. The registered image detection unit 33 reads an image in the vicinity region R, and determines whether an image that matches any of the images registered in the “registered image” column of the registered image-target GUI correspondence table 500 is included. judge. If the registered image is not included in the neighboring region R (No in step S5), the software execution operation assisting process returns to the state before step S1 and enters a standby state.

In the example shown in FIG. 4, the peak start point mark 301 that is a registered image is included in the vicinity region R (Yes in step S5), and then the target GUI detection unit 34 is associated with the registered image. A GUI is specified (step S6). Specifically, the registered image detection unit 33 notifies the target GUI detection unit 34 of the record number of the registered image-target GUI correspondence table 500 including the image determined to match in step S5. Then, the target GUI detection unit 34 refers to the data value in the “target GUI” column of the record of the notified number. In this example, since the registered image is the peak start point mark 301, the first record of the registered image-target GUI correspondence table 500 shown in FIG. 5 is referred to, and the peak detection is performed as the target GUI assigned to the record. A point move button 311 is specified.

Next, the target GUI detection unit 34 determines whether or not the target GUI is displayed on the screen of the monitor 14 (step S7). Specifically, the target GUI detection unit 34 searches the display screen of the monitor 14 and determines whether or not the image of the peak detection point moving button 311 that is the target GUI specified in step S6 exists on the screen. To do. If the target GUI is not detected on the screen (No in step S7), the software execution operation assist process returns to the state before step S1 and enters a standby state.

In the example shown in FIG. 3, since the peak detection point moving button 311 is displayed, the target GUI detection unit 34 detects this target GUI (Yes in step S7), and the execution command unit 35 clicks the target GUI ( Step S8: execution step). Specifically, the execution command unit 35 operates a macro that clicks the peak detection point moving button 311 detected by the target GUI detection unit 34 in step S7.

Subsequently, the execution command unit 35 re-executes the click operation by the user (step S9: execution step). The click operation re-executed in this step is the click operation detected in step S1. The execution command unit 35 refers to the operation position specified by the operation position specifying unit in step S2 and operates a macro that clicks the position. Alternatively, a message notifying that the position has been clicked may be transmitted to the analysis data analysis program 22.

According to the processing described above, when a click on the peak start point mark 301 (or a point in the vicinity of the peak start point mark 301) is detected, the analysis data analysis program 22 displays the peak after the peak detection point moving button 311 is clicked. It is determined that the start point mark 301 has been clicked. Therefore, the user does not need to click the peak detection point moving button 311 in advance when moving the peak start point. Thus, according to the analysis data processing apparatus 1 according to the present embodiment, an intuitive operation of the user on the analysis data analysis program 22 is realized. Further, according to the above configuration, the source data information of the analysis data analysis program 22 is not necessary, so that user operations on any software can be assisted.

In the above description, the configuration in which the target GUI displayed on the screen of the monitor 14 is clicked by a macro operation has been described. In this configuration, the target GUI needs to be displayed on the screen, but is highly versatile in that it can be implemented even by a person who does not know the message corresponding to the button click. In addition, since the target GUI that is not displayed on the screen is not clicked, the GUI that is not used among the registered GUIs as the target GUI in the registered image-target GUI correspondence table 500 is hidden so that the user can hide it. Undesired operations can also be prevented.

As another example, if a message for notifying the analysis data analysis program 22 that the target GUI has been clicked is known, sending such a message can be used in place of clicking a button by a macro operation. According to this configuration, it is not necessary to display the target GUI on the screen of the monitor 14.

[Flow of linking setting process]
In the registered image-target GUI correspondence table 500 stored in the registered image linking database 36, a pair of a predetermined registered image and target GUI may be registered in advance, but any pair can be registered by the user. It is preferable that Hereinafter, the flow of the association setting process between the registered image and the target GUI by the analysis data processing apparatus 1 according to the present embodiment will be described with reference to FIG. 6 which is a flowchart. The processing shown in the figure starts when the user performs an operation for instructing the execution operation assistance program 21 to register a new registered image-target GUI pair.

First, the execution command unit 35 changes the color setting of the analysis data analysis program 22 (step S600). Specifically, the execution command unit 35 transmits a message for changing the settings of the chromatogram line color, background color, and character color to a predetermined color to the analysis data analysis program 22. The predetermined color is preferably a color that is difficult to be used for other images on the screen of the monitor 14. As a result, another image similar to the registered image can be prevented from being erroneously detected as the registered image.
If the transmission message to the analysis data analysis program 22 is not known, the execution command unit 35 may execute a macro according to the procedure for the user to change the color setting on the analysis data analysis program 22. Alternatively, an image or moving image tutorial may be displayed on the screen to prompt the user to execute the macro.

Next, the association setting unit 31 determines a registered image (step S601). Specifically, the association setting unit 31 sets a predetermined image designated by the user input as a data value in the “registered image” column in the new record of the registered image-target GUI correspondence table 500. The image may be determined by designating an image file stored in the external storage device, or an image in an arbitrary range designated by a user operation is extracted from the captured image on the display screen, It is good also as a structure determined to a registration image.
For images that do not have a fixed shape, such as waveform lines or peak baselines, a predetermined reference for recognizing these images may be acquired by user input and stored.

Next, the linking setting unit 31 sets the target GUI (step S602). Specifically, the association setting unit 31 sets a predetermined image (typically a GUI button image) designated by the user as a data value in the “target GUI” column in the record created in step S601. The image designation by the user may be performed by the method described above, similarly to the registered image. If the message for notifying the analysis data analysis program 22 that the target GUI has been clicked is known, it may be obtained by user input or the like instead of the image and stored.

Next, the linking setting unit 31 updates the registered image-target GUI correspondence table 500 (step S603: linking step). Specifically, the association setting unit 31 updates the registered image-target GUI correspondence table 500 with a new record including the registered image and the target GUI determined in steps S601 and S602 as data values.

Through the processing described above, the user can add a pair in which an arbitrary registered image and target GUI are linked to the registered image-target GUI correspondence table 500. Thus, the user can make the operation more intuitive by associating the registered image and the target GUI related to the operation frequently performed by the user in the registered image-target GUI correspondence table 500. . Further, it is preferable that each record in the registered image-target GUI correspondence table 500 can be arbitrarily changed or deleted by the user.

In the present embodiment, the configuration in which the operation position specifying unit 32 included in the execution operation assisting program 21 detects a user's click and specifies the click position has been described. However, the operation position specifying unit 32 has functions of other programs. It may be realized by using. The elements included in the execution operation assistance program 21 in FIG. 1 may be present in the analysis data processing apparatus 1, and the form is not particularly limited.
In addition, although the vicinity region R has been described as a region having a spread centered on the click point 400, the region below the click point 400 is considered to have low visibility for the user due to superimposition with the cursor C, and therefore the click region 400 A region that extends upward from the point 400 and is considered to have high visibility for the user may be used as the neighborhood region R.

[Second Embodiment]
Since the schematic configuration of the analysis data processing apparatus 1 according to the present embodiment is the same as that of the first embodiment, the illustration is omitted.
As an example, it is assumed that the user manually inserts a peak of a chromatogram 800 (see FIG. 8A) that has not been detected by automatic detection by the analysis data analysis program 22. Normally, when performing this process, the user first needs to click the peak insertion button 312 and then click the point 802 to be set as the start point of the peak to be inserted and the point 803 to be set as the end point in order.
Therefore, in the present embodiment, the waveform line 801 of the chromatogram 800 is used as a registered image, and the ratio of the background color in the neighboring region R is included in the determination condition when detecting the registered image.

Now, as shown in FIG. 8A, it is assumed that the user has placed the cursor C on a point 802 on the waveform line 801 of the chromatogram 800 and clicked. FIG. 8B is a conceptual diagram of the neighboring region R at this time. The size and shape of the neighboring region R in the present embodiment are as described in the first embodiment. A waveform region 801 of the chromatogram 800 is included in a neighboring region R centering on a point 802 shown in FIG.

The flow of the software execution operation assistance process in the present embodiment is different from the process in step S5 described in the first embodiment. FIG. 7 shows the flow of registered image detection processing according to step S5 in the present embodiment.
First, the registered image detection unit 33 detects a line of a predetermined color in the vicinity region R (step S701). The predetermined color is a line color of the waveform line 801 stored in advance, and the registered image detection unit 33 extracts pixels having the same color as the line color, and the extracted pixels are within a certain range ( For example, when a line having a thickness of 1 to 2 pixels is formed, this is detected as a line.

Next, the registered image detection unit 33 determines whether or not the proportion of the background color in the neighboring region R is 50% or more (step S702). Specifically, the registered image detection unit 33 refers to the values of all the pixels in the neighborhood region R, and counts pixels that exhibit the same color as the background color of the chromatogram stored in advance. Then, the ratio n / N of the count number n to the total number of pixels N in the neighboring region R is obtained, and it is determined whether 0.5 ≦ n / N is satisfied. In general, since the background color occupies most of the graph in the chromatogram, when the above inequality holds, the clicked point 802 is considered to be a point on the chromatogram 800, and the line detected in step S701 is a waveform line. 801 is likely to be a part of 801. If the above inequality is satisfied (Yes in step S702), the registered image detection unit 33 determines that the registered image is included in the neighboring region R (step S703), and the software execution operation assist process proceeds to steps S6 to S9. Transition. In this example, the peak insertion button 312 which is the target GUI associated with the waveform line 801 of the registered image is clicked, and then the click on the point 802 is executed again.

On the other hand, when the above inequality is not satisfied, that is, when n / N <0.5 (No in step S702), the registered image detection unit 33 determines that the registered image is not included in the neighboring region R ( Step S704). Therefore, the software execution operation assist process returns to the state before step S1 and enters a standby state.

According to the processing described above, when the user detects a click at a point 802 on the waveform line 801 of the chromatogram 800, the analysis data analysis program 22 clicks the point 802 after the peak insertion button 312 is clicked. It is determined that Therefore, when the user manually inserts a peak, it is only necessary to directly click the point 802, and the peak insertion button 312 need not be clicked in advance.
Furthermore, when detecting the registered image, if the ratio of the background color in the neighboring region R is less than 50%, the peak insertion button 312 is not clicked, so that a line outside the chromatogram 800 is erroneously detected as the waveform line 801. This can be prevented. The threshold value of 50% is merely an example, and the determination threshold value in step S702 may be set as appropriate.

[Third Embodiment]
Since the schematic configuration of the analysis data processing apparatus 1 according to the present embodiment is the same as that of the first embodiment, the illustration is omitted. However, in this embodiment, the target GUI detection unit 34 is not an essential element. The registered image association database 36 stores a registered image-macro correspondence table (not shown) in which a registered image is associated with a predetermined macro. Hereinafter, the flow of the software execution operation assisting process by the analysis data processing apparatus 1 according to the present embodiment will be specifically described with reference to FIG. 9 which is a flowchart and FIG. 10 as necessary.
As an example, it is assumed that the user returns the original display size (cancels the enlargement) after the chromatogram 1000 is enlarged and displayed on the analysis data analysis program 22 (see FIG. 10). Normally, when this processing is performed, the user first right-clicks on the chromatogram 1000 (for example, the point 1005) to display the menu 1010, and then selects a selection item 1011 in the menu 1010 ("undo enlargement"). The operation of clicking is required.

In this embodiment, a user operation that triggers a software execution operation assisting process related to cancellation of enlargement of the chromatogram 1000 is a double-click on the chromatogram 1000. This is a design example for preventing confusion with other operations and operations not desired by the user, and the type of operation may be set as appropriate. The processes according to steps S901 to S904 are the same as steps S1 to S4 described in the first embodiment except that the detected user operation is a double click. Here, it is assumed that a double click on the point 1005 in the chromatogram 1000 is detected. At this time, since the right click operation is not performed, the menu 1010 is not displayed on the screen of the monitor 14.

Subsequently, the registered image detection unit 33 determines whether or not the ratio of the background color in the neighboring region R is 50% or more (step S905: registered image detection step). The processing in this step is the same as that in step S702 described in the second embodiment. Generally, since the background color occupies most of the graph in the chromatogram, the click position is considered to be on the chromatogram 1000 when the determination result in this step is Yes. If it is determined that the ratio of the background color occupying the neighboring region R is less than 50% (No in step S905), the software execution operation assisting process returns to the state before step S901 and enters a standby state.

If it is determined that the proportion of the background color in the neighboring region R is 50% or more (Yes in step S905), then the execution command unit 35 executes a macro associated with the registered image (step S906: Execution step). Here, in the present embodiment, among the images of the neighboring region R acquired in step S904, registration in the registered image linking database 36 is performed so that an image having a background color ratio of 50% or more is recognized as a registered image. Determination criteria are registered in the image-macro correspondence table. The macro associated with the registered image is “right click + click on selection item 1011 in menu 1010 (“ return enlargement ””). If the message corresponding to the click of the selection item 1011 is known, the message may be transmitted to the analysis data analysis program 22 instead of executing the macro.

According to the processing described above, when the user double-clicks on the chromatogram 1000, the right click and the selection item 1011 are automatically clicked. In other words, the user can restore the enlargement with a single double click. According to the present embodiment, the operation time is shortened to 1/5 to 1/10 as compared with the case where the right click and the selection item 1011 are manually clicked (double click: 200 ms, right click + selection item). Click 1011: 1-2 seconds). Therefore, in waveform processing operations that require frequent enlargement / reduction, the present embodiment greatly contributes to the improvement of throughput.
In addition, when the ratio of the background color in the neighboring region R centering on the click point 1005 is less than 50%, the macro is not executed, and thus an operation unintended by the user in the region outside the chromatogram 1000 is prevented. Can do. Similar to the second embodiment, the background color ratio threshold of 50% is merely an example.
Note that the registered image-macro correspondence table in the present embodiment is similar to the registered image-target GUI correspondence table 500 described in the first embodiment, and the user assigns a registered image and a predetermined macro associated with the registered image. A configuration that can be arbitrarily registered is preferable.

[Fourth Embodiment]
When a user compares and examines a plurality of chromatograms created by the analysis data analysis program 22, it may be desired to superimpose a waveform line of a certain chromatogram on another chromatogram on the screen. Conventionally, in order to perform such superposition, images of a plurality of chromatograms to be compared are each saved by screen capture etc., and these are opened on image editing software, and a background color is selected and made transparent. The operation of superimposing on the top was necessary. It is not practical to perform such a complicated operation every time of comparison.
Therefore, the analysis data processing apparatus 100 according to the present embodiment uses the waveform line of the chromatogram as a registered image, and when the waveform line is included in the vicinity range, the entire waveform line is extracted and displayed on the foreground on the screen. It was set as the structure displayed.

FIG. 11 shows an analysis data processing apparatus (software execution operation assisting apparatus) 100 according to the fourth embodiment of the present invention. The analysis data processing device 100 is configured to include an execution operation assistance program 21a instead of the execution operation assistance program 21 provided in the analysis data processing device 1. FIG. 11 shows an operation position specifying unit 32, a registered image detecting unit 33, an execution command unit 35, and a registered image-program database (DB) 361 as related to the execution operation assisting program 21a. In addition to this, the execution operation assistance program 21a may include each element included in the execution operation assistance program 21 according to the first embodiment. These are basically functional means realized by software when the CPU 10 executes the execution operation assisting program 21a.

The registered image-program database 361 stores a registered image and a predetermined program in association with each other, and includes a superimposition support program 40 as an example described in the present embodiment. The superimposition support program includes an image extraction unit 41, a transparency processing unit 42, a list control unit 43, a display control unit 44, and an image storage unit 45. These are basically functional means realized by software when the CPU 10 executes the superimposition support program 40.

Hereinafter, the flow of the software execution operation assistance process will be specifically described with reference to FIG. 12 which is a flowchart, and FIGS. 13 and 14 as necessary. Here, as shown in FIG. 13A, a point on the waveform line 1301 of the chromatogram 1300 displayed on the screen of the monitor 14 by the analysis data analysis program 22 (a point in the vicinity of the waveform line 1305). Is also clicked. A combination with a modifier key (for example, the Ctrl key) may be used as a trigger in order to distinguish it from an operation that triggers another software execution operation assist process.

The processing related to steps S1201 to S1204 is the same as steps S1 to S4 described in the first embodiment.
Next, the registered image detection unit 33 determines whether or not there is a waveform line in the vicinity region R (step S1205). Specifically, for example, a pixel having the same color as the waveform color of the waveform line 801 stored in advance is extracted with reference to the image in the vicinity region R centered on the click position specified in step S1202. Then, it is determined whether or not the extracted pixels constitute a line having a thickness within a certain range (for example, 1 to 2 pixels). If it is determined that the waveform line is not included in the neighboring region R (No in step S1205), the software execution operation assisting process returns to the state before step S1201 and enters a standby state.

If it is determined that the waveform line is included in the vicinity region R (Yes in step S1205), the execution command unit 35 executes the program associated with the waveform line (step S1206). Specifically, the execution command unit 35 refers to a correspondence table (not shown) stored in the registered image-program database 361 and performs a superimposition support program 40 that is a program linked to the registered image “waveform line”. Execute.

FIG. 14 shows a flow of superimposition support processing by the superimposition support program 40 in this step.
First, the image extraction unit 41 extracts a rectangular area image including the entire waveform line 1301 (step S1401). Specifically, the OS 24 is instructed to capture the display content of the entire screen, and an image of a rectangular area including the entire display range of the waveform line 1301 is extracted (trimmed) from the full screen captured image copied to the clipboard by the OS 24. The extracted image is stored in the image storage unit 45 as an extracted image. The display range of the waveform line 1301 may be recognized as a range in which lines detected in the neighborhood region R are continuous, for example.
In addition to the waveform line 1301, a frame line of the chromatogram 1300 may be extracted. For example, the frame line can be detected as a line segment in contact with the waveform line 1301 and parallel to the X axis and the Y axis of the screen. In this case, the range extracted is the display range of the frame line. The user may be able to change the setting of whether to extract including the frame line.

Next, the transparency processing unit 42 makes the background of the extracted image transparent (step S1402). Specifically, the transparency processing unit 42 transparentizes a color stored in advance as a background color in the extracted image and stores it in the image storage unit 45. Or, since the background color generally occupies most of the graph in the chromatogram, the color having the largest number of pixels in the extracted image may be made transparent.

Next, the list control unit 43 displays the extracted image list 1320 on the screen of the monitor 14 (step S1403). FIG. 13B shows a screen display example by the monitor 14 at this time. The extracted image list 1320 is a GUI for allowing the user to visually recognize an extracted image that can be pasted on the screen and accepting a selection operation thereof. The extracted image thumbnail 1321 that has been subjected to transparency processing in step S1402 is displayed. Include in the list. In the figure, a waveform line whose background has been made transparent in step S1402 is shown. When a plurality of extracted images are stored in the image storage unit 45, the list control unit 43 may display thumbnails of the plurality of extracted images in the extracted image list 1320 as stock.
If a click operation on the display button 1322 is detected, the display control unit 44 displays the extracted waveform line 1301a on the foreground on the screen. If the non-display button 1323 is clicked, the display of the waveform line 1301a is canceled. The waveform line 1301a is displayed as a window with a transparent background, and can be arbitrarily moved and enlarged / reduced by a user operation. Thus, if the waveform line 1301a is superimposed on another chromatogram 1310 on the screen as shown in FIG. 13 (b), the user can easily compare the two

chromatograms

1300 and 1310, which is necessary. It can be switched between display and non-display according to.

According to the processing described above, when the user clicks in the vicinity of the waveform line 1301 on the chromatogram 1300, only the waveform line 1301 is automatically extracted, and the user places the extracted waveform line 1301a on the foreground on the screen. It can be displayed arbitrarily. Thereby, the effort required when superimposing a plurality of chromatograms can be suppressed, and the user can shorten the time taken for the comparative study.

[Modification]
In each embodiment mentioned above, it was set as the structure which software execution operation assistance processing starts by detection of a click etc. by a user as a trigger. This configuration assumes that the link relationship between the registered image and the predetermined process is known to the user, but a situation in which the user does not grasp the link relationship is also assumed. Therefore, the execution

operation assisting programs

21 and 21a refer to the image of the neighboring region R centered on the stationary position by using the cursor C as a trigger at the same position on the screen of the monitor 14 for a certain period of time. When a registered image is included in the region R, an explanatory comment associated with the registered image may be displayed near the cursor C as a tool tip. The explanatory comment may notify the user of processing executed by clicking at the current position, such as “movement of peak start point” in the first embodiment. As a result, even a user who does not sufficiently grasp the association relationship between the registered image and the predetermined process can smoothly perform operations on the analysis data analysis program 22 using the execution

operation assistance programs

21 and 21a. it can. The comment for explanation may be registered by the user together with the registration image and the target GUI in the association setting process (see FIG. 6) described in the first embodiment. Moreover, it is good also as a structure which replaces with the comment for description, displays the GUI button linked | related with the registration image in the vicinity of the cursor C, and receives operation with respect to this button.

In the first embodiment, the configuration in which the target GUI is determined in step S602 shown in FIG. 6 has been described, but a predetermined macro may be registered in accordance with user input instead of the target GUI.

Note that the above-described embodiments and modifications are examples of the present invention, and it should be understood that changes, modifications, additions, and combinations as appropriate within the scope of the present invention are included in the scope of the claims of the present application. It is.

DESCRIPTION OF SYMBOLS 1,100 ... Analysis data processing apparatus 10 ... CPU
12 ... Memory 14 ... Monitor 16 ... Input unit 18 ... I / F
DESCRIPTION OF SYMBOLS 20 ...

Storage part

21, 21a ... Execution operation assistance program 22 ... Analysis data analysis program 23 ... Analysis data storage part 24 ... OS
31 ... Association setting unit 32 ... Operation position specifying unit 33 ... Registered image detection unit 34 ... Target GUI detection unit 35 ... Execution command unit 36 ... Registered image association database 361 ... Registered image-program database R ... Neighborhood

Claims

A software execution operation assisting device for assisting a user's operation on the target software,
a) a database in which registered images and predetermined processes are stored in association with each other;
b) operation position specifying means for specifying, as an operation position, a position at which a user performs a predetermined operation on the display screen;
c) registered image detection means for determining whether or not the registered image is included in a predetermined vicinity range of the operation position specified by the operation position specifying means;
d) execution means for executing the predetermined process associated with the registered image determined to be included in the vicinity range by the registered image detection means;
A software execution operation assisting device.
The predetermined process is a process assigned to a predetermined GUI included in the target software,
The said execution means performs the process allocated to the said predetermined GUI linked | related with the said registration image detected by the said registration image detection means with respect to the said object software, The said software is characterized by the above-mentioned. Software execution operation assistance device.
3. The linking device according to claim 1, further comprising a linking unit that links one or a plurality of the registered images and the predetermined process and stores them in the database in accordance with a user instruction. Software execution operation support device
The registered image detecting means determines that the registered image is not included in the vicinity range when a ratio of pixels exhibiting a predetermined color in the vicinity range is less than a predetermined value. The software execution operation assisting device according to any one of claims 1 to 3.
5. The software execution operation according to claim 1, wherein a color of the registered image and / or an image in a predetermined area including the registered image is changed or the user is prompted to change the color. Auxiliary device.
A software execution operation assisting method for assisting a user's operation on the target software,
a) a linking means for linking a registered image and a predetermined process;
b) An operation position specifying step in which the operation position specifying means specifies, as the operation position, a position where a predetermined operation is performed by the user on the screen of the display unit;
c) a registered image detecting step for determining whether or not the registered image is included in a predetermined vicinity area of the operation position specified in the operation position specifying step;
d) an executing unit that executes the predetermined process associated with the registered image that is determined to be included in the vicinity area in the registered image detection step;
A software execution operation assisting method, comprising: