JP2009042835A - Input support apparatus and computer program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a technology to improve usability when substituting an arithmetic element as an operation object designated from a graph into an arithmetic expression to carry out an operation. <P>SOLUTION: In an input support apparatus 100, an input reception section 110 receives an instruction from a user, a (spectral) graph generation section 112 generates a graph from an image and makes a display device 160 display the graph, an arithmetic expression generation (display) section 113 makes the display device 160 display an arithmetic expression including a variable, an association processing section 114 designates an arithmetical element as an operation object from the graph on a screen, and when the input reception section 110 has received an association instruction for operation for associating the arithmetical element with a variable included in the arithmetic expression on the screen, obtains the value of the arithmetical element, and associates the obtained value of the arithmetical element with the variable of the arithmetic expression, and an arithmetic processing section 115 obtains an operation result thereof. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to an input support apparatus and a computer program.

  As a conventional spectroscopic device, for example, there is one described in Patent Document 1. The spectroscopic device described in Patent Document 1 includes one diffraction grating, a detector that receives light in each wavelength region (for example, 32 channels) dispersed by the diffraction grating, and data that processes data from the detector. And a processing device. In this spectroscopic device, the detector receives the light in each wavelength region dispersed by the diffraction grating, and transmits data indicating the light intensity in each wavelength region to the data processing device.

  Here, the data processing apparatus can perform various processes on the received data. For example, the data processing device generates a two-dimensional spectrum image from the received data. For example, the data processing apparatus generates and displays a spectrum graph of a region designated in the two-dimensional spectrum image.

  Furthermore, the data processing apparatus performs various calculations using the generated and displayed spectrum graph. For example, it is often performed that various calculations are performed by substituting an integral value (calculation element to be calculated) in a wavelength range designated in the displayed spectrum graph into an arbitrary calculation expression.

JP 2004-345672 A

  However, in order to substitute a calculation element (for example, the above integral value) obtained from the spectrum graph into the calculation formula, the user must manually input the numerical value, and in particular, the numerical value must be input many times. If this is the case, the operation is complicated for the user.

  An object of this invention is to provide the technique which improves the usability at the time of calculating by substituting the calculation element which is the object of the calculation designated from the graph into the calculation formula.

  An input support apparatus according to a first aspect of the present invention for solving the above problem is an input support apparatus connected to a display device and an observation apparatus that provides an image related to a sample, and accepts an instruction from a user. An input receiving unit; a graph generating unit that generates a corresponding graph from the image; and outputs first screen data for displaying the graph to the display device; and second screen data for displaying an arithmetic expression including a variable. On the screen of the display device, a calculation element to be calculated is designated from the graph based on the first screen data, and the calculation element is designated as the second calculation element. When an instruction corresponding to a variable included in the arithmetic expression based on the screen data is received from the input receiving unit, the value of the arithmetic element is obtained, and the calculated value of the arithmetic element is used as a variable of the arithmetic expression. Mapping An association processing unit; an arithmetic processing unit that obtains an operation result by substituting the value of the associated operation element into a variable of the operation expression; and an output unit that outputs the operation result to the display device. Prepare.

  According to the input support device of the present invention, it is possible to improve usability when performing computations by substituting computation elements that are computation targets designated from a graph into a computation expression.

  The best mode for carrying out the present invention will be described below with reference to the drawings.

  An example of the functional configuration of the input support apparatus 100 applied to the embodiment of the present invention is shown in the block diagram of FIG. As illustrated, the input support apparatus 100 includes a control unit 101, a storage unit 102, and an interface unit 103.

  The input support apparatus 100 is used in, for example, a microscope spectroscopy system 50 as shown in FIG. As illustrated, the microscope spectroscopic system 50 includes an input support apparatus 100, a microscope apparatus 200, a spectroscopic detector 300, and a controller 400.

  The microscope apparatus 200 includes a laser light source 201, an optical fiber 202, a collimator lens 203, a dichroic mirror 204, a galvano mirror 205, an objective lens 206, a table 207, a condenser lens 208, a pinhole 209, Have

  Here, the illumination optical system of the microscope apparatus 200 includes a laser light source 201, an optical fiber 202, a collimator lens 203, a dichroic mirror 204, a galvano mirror 205, and an objective lens 206. The observation optical system of the microscope apparatus 200 includes an objective lens 206, a galvanometer mirror 205, a dichroic mirror 204, and a condenser lens 208.

  In the microscope apparatus 200, the laser light emitted from the laser light source 201 is guided to the collimating lens 203 through the optical fiber 202. The collimating lens 203 collimates the guided laser beam. The dichroic mirror 204 receives the laser light collimated by the collimating lens 203 and reflects it in the direction of the sample S. The galvanometer mirror 205 scans the sample S two-dimensionally with the laser beam reflected by the dichroic mirror 204. The objective lens 206 condenses the light from the galvanometer mirror 205 on the observation surface of the sample S. The condensing lens 208 condenses the laser light reflected by the observation surface of the sample S placed on the table 207 at the position of the pinhole 209. Here, the light incident on the pinhole 209 is sent to the spectroscopic detector 300 via an optical fiber.

  The microscope apparatus 200 can obtain a two-dimensional image of the sample S by having the above configuration.

  The spectroscopic detector 300 includes a collimating lens 301, a diffraction grating 302, a light detection array 303, and a signal processing circuit 304.

  The collimating lens 301 collimates the light transmitted from the microscope apparatus 200. The diffraction grating 302 splits the light collimated by the collimating lens 301 into light having different wavelength ranges. The photodetection array 303 converts the intensity of light in each wavelength region that has been dispersed by the diffraction grating 302 into an electrical detection signal. The signal processing circuit 304 samples the detection signal converted by the light detection array 303 and supplies the sampling data to the controller 400.

  The spectroscopic detector 300 can obtain the light intensity for each wavelength range of the light transmitted from the microscope apparatus 200 by providing the above configuration.

  The controller 400 includes a control circuit 401, a laser driving circuit 402, a scanner driving circuit 403, and an interface circuit 404.

  The control circuit 401 temporarily stores a CPU that executes various programs, a ROM that stores various data and programs, a RAM that temporarily stores various data and programs, data from the spectral detector 300, and the like. Frame memory. The control circuit 401 configured as described above outputs a laser light output instruction signal and a laser light shutter opening instruction signal to the laser driving circuit 402. Further, the control circuit 401 outputs an operation control signal for the galvano mirror 205 to the scanner drive circuit 403.

  Further, the control circuit 401 generates a two-dimensional spectrum image related to the sample S. Specifically, the control circuit 401 sequentially receives sampling data supplied from the spectroscopic detector 300 while outputting a control signal instructing two-dimensional scanning to the scanner driving circuit 403. A two-dimensional spectral image of each wavelength region can be generated. That is, when the use channel of the light detection array 304 of the spectroscopic detector 300 is 32 channels, the control circuit 401 generates 32 two-dimensional spectrum images.

  Then, the control circuit 401 transmits the generated two-dimensional spectrum image to the input support apparatus 100 via the interface circuit 404.

  The laser driving circuit 402 outputs a control signal for outputting laser light to the laser light source 201 or a control signal for opening the laser light shutter of the laser light source 201 in accordance with the signal supplied from the control circuit 401, 201 is controlled.

  The scan drive circuit 403 outputs a signal corresponding to the signal supplied from the control circuit 401 to the galvanometer mirror 205, and scans the laser beam reflected by the dichroic mirror 204 two-dimensionally on the sample S.

  The interface circuit 404 is an interface for communicating with the input support apparatus 100. The interface circuit 404 includes a mechanism for performing USB communication with the input support apparatus 100, for example. In this case, communication data corresponding to the USB communication protocol is used.

  The controller 400 can provide the input support apparatus 100 with a two-dimensional spectrum image related to the sample S observed with the microscope apparatus 200 by having the above configuration.

  The input support apparatus 100 stores the two-dimensional spectrum image provided from the controller 400. The input support apparatus 100 can display the two-dimensional spectrum image on the connected display device 160.

  The input support apparatus 100 has a function for the user to perform various processes on the two-dimensional spectrum image. In the present embodiment, the input support apparatus 100 has a function for observing a Fret (Fluorescence resonance energy transfer) phenomenon in the sample S using a two-dimensional spectrum image. Here, the Fret phenomenon refers to a donor fluorescent molecule that emits fluorescence having a wavelength included in a specific wavelength region and an acceptor molecule that emits fluorescence having a wavelength different from that of the donor fluorescent molecule when the sample S is irradiated with laser light. This is a phenomenon in which the acceptor molecule absorbs the light energy emitted by the donor fluorescent molecule when the is close to each other. In order to observe this Fret phenomenon, for example, a spectrum graph showing the light intensity of a part of a region in a two-dimensional spectrum image obtained using the microscope apparatus 200 at a certain time, and a time graph obtained after that are obtained. What is necessary is just to compare with the spectrum graph which shows the light intensity of the same area | region. Therefore, the input support apparatus 100 according to the present embodiment has a function of generating a spectrum graph from a two-dimensional spectrum image in order to observe the Fret phenomenon in the sample S, and a calculation element (for example, a designated element) obtained from the spectrum graph. It is assumed that the calculation function is such that calculation is performed by substituting the integrated value in the wavelength region into a predetermined calculation formula. When the input device 150 is connected to the input support device 100 and the Fret phenomenon is observed, an instruction for specifying a calculation element from the spectrum graph, an instruction for substituting the calculation element into a predetermined arithmetic expression, etc. Can be received from the user.

  Returning to the description of FIG. 1, the control unit 101 of the input support apparatus 100 used in the microscope spectroscopic system 50 as described above includes an input reception unit 110, a spectrum image acquisition unit 111, a spectrum graph generation unit 112, and an operation. An expression generation unit 113, an association processing unit 114, an arithmetic processing unit 115, and a teaching processing unit 116 are included.

  The spectrum image acquisition unit 111 issues a command for generating the above-described two-dimensional spectrum image to the controller 400, and performs a process of storing the two-dimensional spectrum image provided from the controller 400 in the spectrum image DB 120. A command for generating a two-dimensional spectrum image is issued, for example, when causing the microscope apparatus 200 to perform time-lapse imaging, or when causing the microscope apparatus 200 to perform imaging for each instruction from the user.

  Further, the spectrum image acquisition unit 111 generates a spectrum image display screen 500 as shown in FIG. 3A from the two-dimensional spectrum image acquired from the controller 400 and displays the spectrum image display screen 500 on the display device 160. Here, the spectral image display screen 500 is generated by superimposing all the two-dimensional spectral images (two-dimensional spectral images in each wavelength region) acquired from the controller 400.

  The spectrum graph generation unit 112 performs processing for generating a spectrum graph from a two-dimensional spectrum image in each wavelength range included in the spectrum image display screen 500 displayed on the display device 160. When a user designates a partial region 501 in the spectrum image display screen 500, the spectrum graph generation unit 112 graphs the spectrum data in the region 501. Then, the spectrum graph generation unit 112 causes the display device 160 to display a spectrum graph display screen 600 as shown in FIG. A spectrum graph display screen 600 shown here is an example of a screen in which spectrum graphs generated from spectrum data included in regions located at the same coordinates of a plurality of two-dimensional spectrum images acquired at different times are displayed in an overlapping manner. (Solid line: spectrum graph at time T = 1, alternate long and short dash line: spectrum graph at time T = 2).

  When the spectrum graph generation unit 112 designates a plurality of regions (for example, an A region (solid line) 501 and a B region (broken line) 502) in the same spectral image display screen 500, FIG. As shown in FIG. 4, spectrum graphs (solid line graphs and broken line graphs) corresponding to the respective regions (501, 502) may be displayed on the same spectrum graph display screen 600 in an overlapping manner. When a plurality of areas are designated by the user, spectrum graphs corresponding to the respective areas (501, 502) may be displayed on separate spectrum graph display screens 600 (not shown).

  Further, the spectrum graph generation unit 112 displays the wavelength lower limit line 602 and the wavelength upper limit line 603 on the spectrum graph display screen 600. The wavelength lower limit line 602 and the wavelength upper limit line 603 can be moved in a direction parallel to the x axis (wavelength axis) on the spectrum graph display screen 600 in accordance with a user instruction. The spectrum graph display screen 600 is displayed when a mouse pointer is placed on a region 601 surrounded by a spectrum graph, a wavelength lower limit line 602, a wavelength upper limit line 603, and an x axis (wavelength axis). The color 601 may be reversed.

  The arithmetic expression generation unit 113 performs processing for generating an arithmetic expression used for observing the Fret phenomenon from the spectrum graph generated by the spectrum graph generation unit 112. For example, the arithmetic expression generation unit 113 causes the display device 160 to display an arithmetic expression display screen 700 for generating an arithmetic expression, and generates a predetermined arithmetic expression 701 based on an instruction from the user. At this time, blanks (702, 703) representing variables in the formula can be provided in the generated calculation formula 701. After the calculation formula 701 is generated, numerical values (calculation elements described above) are displayed in the blanks (702, 703). When X is substituted, the solution X of the arithmetic expression 701 can be obtained. Then, the arithmetic expression generation unit 113 stores the generated arithmetic expression 701 in the correspondence table 121 described later. The generation method of the arithmetic expression 701 used here is not limited to this. For example, the arithmetic expression 701 may be generated by selecting an arithmetic expression stored in the storage unit 102 in advance.

  The association processing unit 114 performs processing for associating the operation element designated by the user in the spectrum graph generated by the spectrum graph generation unit 112 with the operation expression 701 generated by the operation expression generation unit 113. That is, the association processing unit 114 performs processing for determining which blank (701, 702) in the arithmetic expression 701 is to be substituted with the arithmetic element designated by the user in the spectrum graph.

  Specifically, first, the association processing unit 114 displays the calculation element specified by the mouse pointer on the spectrum graph display screen 600 in which the spectrum graph is displayed on the display device 160 as shown in FIG. Find the value. For example, the calculation elements include a spectrum graph, a wavelength lower limit line 602, a wavelength upper limit line 603, an integral value of a region 601 surrounded by the x axis (wavelength axis), and the like. When the region 601 surrounded by the spectrum graph, the wavelength lower limit line 602, the wavelength upper limit line 603, and the x-axis (wavelength axis) is dragged based on the user's instruction, the association processing unit 114 displays the region 601. The process of obtaining the integral value of

  Then, the association processing unit 114 calculates the calculated element when the dragged area 601 based on the user's instruction is dropped in the blank field 702 (703) included in the calculation formula 701 on the calculation formula display screen 700. Is stored in the correspondence table 121 described later. Here, the data stored in the correspondence table 121 is, for example, in the format of “variable name (name specifying the blank 702) = value of the calculation element”, and is stored in association with the dropped calculation expression 701.

  Furthermore, the association processing unit 114 ends the association processing when the processing for associating such an arithmetic element with the arithmetic expression 701 is performed on all blanks (702, 703) in the arithmetic expression 701. Then, this is notified to the arithmetic processing unit 115.

  Here, as shown in FIG. 6A, when the region 601 dragged on the spectrum graph display screen 600 is dropped into a blank field 702 (703) on the arithmetic expression display screen 700, the association is performed. Although processing is performed, the present invention is not limited to this. A specific example of performing the association process in other cases will be described later.

  When the association processing unit 114 associates the operation element with the operation expression 701, the operation processing unit 115 performs processing for obtaining the operation result (solution X) of the operation expression. Specifically, the arithmetic processing unit 115 refers to the arithmetic elements and arithmetic expressions stored in association with the correspondence table 121 to obtain the arithmetic result (solution X). Further, the arithmetic processing unit 115 stores the calculated value of the calculation result (solution X) in the correspondence table 121.

  The teaching processing unit 116, when causing the microscope apparatus 200 to continuously observe the sample S (for example, time-lapse observation), calculates the calculation element on the spectrum graph, the calculation expression that substitutes the calculation element, and the calculation element as the calculation expression. The time to be substituted is set in advance (Teaching), and after the observation, the calculation result (solution X) of the calculation formula is obtained. Specifically, the teaching processing unit 116 first displays a teaching function setting screen 800 as shown in FIG. 6B on the display device 160 based on a user instruction before observing the sample S. In the Teachin function setting screen 800, a region for displaying a spectrum graph similar to the spectrum graph display screen 600, a region for creating the arithmetic expression 701 similar to the arithmetic expression display screen 700, and a time for assigning the arithmetic element to the arithmetic expression 701 are displayed. An input area (for example, blank) 801, an area for displaying a time-dependent graph of the calculation result (solution X) of the calculation formula, and an observation start button 802 for receiving an instruction to start observation from the user are provided. The Teaching processing unit 116 receives various settings from the user after displaying the Teaching function setting screen 800, and transmits a continuous observation start instruction signal to the controller 400 when the observation start button 802 is pressed. Here, for various settings received from the user, the specification of the wavelength lower limit value line 601 and the wavelength upper limit value line 602 for determining the calculation element, creation of the calculation formula 701, specification of the time at which the calculation element is substituted into the calculation formula, Is included. Then, the teaching processing unit 116 stores various setting contents in the teaching table 122 described later. Even when the Teaching function is set, after the continuous observation is started, the association processing unit 116 associates the operation element (the operation element obtained at the specified time) with the operation expression 701. I do.

  In addition, after starting continuous observation, the teaching processing unit 116 performs processing to update the spectrum graph on the teaching function setting screen 800 and the time-dependent graph of the calculation result (solution X) at predetermined time intervals. Also good.

  The input receiving unit 110 performs a process of receiving an input signal from the input device 150 described later. Specifically, the input receiving unit 110 specifies a supply destination (for example, each unit 111 to 117) of the received input signal, and supplies the input signal to the specified supply destination. For example, when the mouse 152 of the input device 150 is clicked when the mouse pointer is on the spectrum graph display screen 600 generated by the spectrum graph generation unit 112, the position of the mouse pointer on the spectrum graph display screen 600 is indicated. Information and information for specifying an operation are supplied to the association processing unit 112.

  The storage unit 102 includes a spectrum image database (DB) 120, a correspondence table 121, and a teaching table 122.

  The two-dimensional spectrum image provided from the controller 400 is stored in the spectrum image DB 120.

  The correspondence table 121 is a table for associating the calculation element designated by the user in the spectrum graph generated by the spectrum graph generation unit 112 with the calculation formula 701 generated by the calculation formula generation unit 113. As shown in FIG. 7, the correspondence table 121 includes a record 1217 for each arithmetic expression. In each record 1217, an ID 1211, an arithmetic expression 1212, variables A to C (1213 to 1215), and an operation result 1216 are stored in association with each other. In this embodiment, the correspondence table 121 has three columns for storing the variables A to C (1213 to 1215). However, the number of variables in the arithmetic expression 1212 is not limited to this. Depending on, the number of columns for storing variables may be increased or decreased.

  The ID 1211 is a code for identifying the arithmetic expression 701 generated by the arithmetic expression generation unit 113, and is a numeric string such as “1” or “2”, for example.

  An arithmetic expression 1212 is the arithmetic expression 701 generated by the arithmetic expression generation unit 113. On the calculation formula display screen 700, blanks (702, 703) included in the calculation formula 701 created by the user have different variable names (eg, “A”, “B”, etc.) in the blanks 702 (703). ) And stored in the correspondence table 121.

  Variables A to C (1213 to 1215) are values of the calculation elements obtained by the association processing unit 114, and are character strings in a format of “variable name = value of calculation elements”. The variable names included in the variables A to C correspond to the variable names included in the arithmetic expression 1212, and the calculation element value is dropped in the blank 702 (703) corresponding to the variables A to C. This is the value of the calculation element obtained when

  The calculation result 1216 is a numeric string indicating the value of the calculation result (solution X) obtained by the calculation processing unit 115.

  As shown in FIG. 8, the teaching table 122 includes a record 1228 for each arithmetic expression. Each record 1228 stores an ID 1221, an arithmetic expression 1222, a (wavelength) lower limit value 1223, a (wavelength) upper limit value 1224, and input times 1225 to 1227 of variables A to C in association with each other. In this embodiment, the Teaching table 122 is provided with three columns for storing the input times 1225 to 1227 of the variables A to C. However, the present invention is not limited to this, and the variables in the arithmetic expression 1221 are not limited to this. Depending on the number, the number of fields for storing variable input times may be increased or decreased.

  The ID 1221 corresponds to the ID 1211 stored in the correspondence table 121.

  The arithmetic expression 1222 corresponds to the arithmetic expression 1212 stored in the correspondence table 121.

  The (wavelength) lower limit value 1223 and the (wavelength) upper limit value 1224 specify the wavelength lower limit value and the wavelength upper limit value for determining the wavelength range of the calculation element when the microscope apparatus 200 performs continuous observation such as time-lapse observation. It is data to be. Specifically, the (wavelength) lower limit value 1223 is a number string indicating the wavelength lower limit value designated by the user on the Teaching function setting screen 800. For example, the (wavelength) lower limit 1223 may be a numeric string such as “580”. Similarly, the (wavelength) upper limit value 1224 is a numeric string indicating the wavelength upper limit value designated by the user on the Teaching function setting screen.

  The input times 1225 to 1227 of the variables A to C are the times when the teaching processing unit 117 calculates the value of the calculation element when the microscope apparatus 200 performs continuous observation such as time lapse observation, and substitutes the calculated value into the calculation formula. This data identifies Specifically, the input times 1225 to 1227 of the variables A to C are character strings indicating the times entered in the blank fields of the area 801 on the teaching function setting screen 800. For example, the input times 1225 to 1227 of the variables A to C may be character strings such as “T = 1” or “T = 10 (H): 24 (M): 34 (S)”.

  The interface unit 103 is an interface for transmitting and receiving communication data with the controller 400.

  The input device 150 includes a keyboard 151 and a mouse 152. By operating the keyboard 151 and the mouse 152 by the user, a user instruction is input to the input support apparatus 100. The display device 160 is a display for notifying the user of data and processing content generated by the input support device 100.

  The input support device 100 configured as described above includes, for example, a CPU 901, a main storage unit 902, an external storage device 903 such as an HDD, and a CD-ROM or DVD-ROM as shown in FIG. In order to perform communication such as USB communication with the reading device 905 that reads information from the portable storage medium 907 having portability, the input / output device 906 including the display device 160, the keyboard 151, the mouse 152, and the like, This is realized by a general computer including a communication device 904 for connecting to a network such as the Internet.

  For example, the storage unit 102 illustrated in FIG. 1 can be realized by the external storage device 903, and the control unit 101 loads a predetermined program stored in the external storage device 903 to the main storage device 902 and executes it by the CPU 901. This is possible.

  The predetermined program is downloaded from the portable storage medium 907 via the reading device 905 or from the network via the communication device 904 to the external storage device 903, and then loaded onto the main storage device 902 to be CPU 901. May also be executed. Alternatively, the program may be directly loaded onto the main storage device 902 from the portable storage medium 907 via the reading device 905 or from the network via the communication device 904 and executed by the CPU 901.

  With the configuration as described above, the input support apparatus 100 can improve usability when performing calculation by substituting calculation elements obtained from the spectrum graph into calculation expressions.

  FIG. 10 is a flowchart showing a first input support process performed by the input support apparatus 100.

  When the input receiving unit 110 receives an instruction to observe the Fret phenomenon from the user, the control unit 101 starts the first input support process. First, the input reception unit 110 notifies the spectrum image acquisition unit 111 that an instruction to observe the Fret phenomenon has been received.

  The spectrum image acquisition unit 111 displays the spectrum image display screen 500 on the display device 160 (step S101). Specifically, the spectral image acquisition unit 111 displays a spectral image as shown in FIG. 3A from the two-dimensional spectral image selected by the user among the two-dimensional spectral images stored in the spectral image DB 120. A screen 500 is generated. At this time, another spectrum image display screen 500 generated from two-dimensional spectrum images acquired at different times may be further generated. Then, the spectral image acquisition unit 111 transmits the generated spectral image display screen 500 to the display device 160 for display.

  Next, the spectrum graph generation unit 112 receives an instruction for designating a region to be graphed on the spectrum image display screen 500 displayed on the display device 160 (step S102). Specifically, when a circle as shown in FIG. 3A is drawn on the spectrum image display screen 500 by the operation of the mouse 152 by the user, the input reception unit 110 displays the area specified by the circle. The coordinates of the pixels included in 501 (502) are supplied to the spectrum graph generation unit 112.

  The spectrum graph generation unit 112 generates a spectrum graph obtained by graphing the spectrum data of the region 501 specified in step S102, and displays it on the display device 160 (step S103). Specifically, the spectrum graph generation unit 112 receives the coordinates of the region 501 designated by the user from the input reception unit 110, for all the two-dimensional spectrum images constituting the spectrum image display screen 500. Reference is made to the spectral data of the pixel located at the same coordinate. And the spectrum graph production | generation part 112 produces | generates the spectrum graph for every wavelength range corresponding to each two-dimensional spectrum image, synthesize | combines these, and produces | generates a final spectrum graph. Further, the spectrum graph generation unit 112 causes the display device 160 to display a spectrum graph display screen 600 including the generated spectrum graph. In step S102, when regions 501 having the same coordinates are designated on the plurality of spectrum image display screens 500, the spectrum graph generation unit 112 adds the spectrum graphs generated from the regions 501 respectively. A display screen 600 is generated and displayed on the display device 160.

  Next, the arithmetic expression generation unit 113 performs processing for generating an arithmetic expression (step S104). Specifically, the arithmetic expression generation unit 113 displays an arithmetic expression display screen 700 as shown in FIG. 5 on the display device 160, and on the arithmetic expression display screen 700 in accordance with a user operation, a mathematical expression or a blank field 702 ( 703) is generated. The arithmetic expression generation unit 113 associates the arithmetic expression 1212 in which the blank 702 (703) included in the generated arithmetic expression 701 is converted into a variable name for identifying the blank 702 (703) with the ID 1211 in correspondence. Store in the table 121. Here, numbers in ascending order are assigned to ID1211. When a plurality of arithmetic expressions 701 are generated, the arithmetic expression generation unit 113 generates, in the correspondence table 121, a plurality of records 1217 in which different IDs 1212 are assigned to the arithmetic expressions 1212 obtained by converting the blank 702 (703) into variable names. To do.

  In a state where the spectrum graph display screen 600 generated by the spectrum graph generation unit 112 in step S103 and the arithmetic expression display screen 700 in step S104 are displayed on the display device 160, the input receiving unit 110 receives input from the user. Accept (step S105). Here, for the input from the user, for example, an input for designating a region 601 to be a calculation element on the spectrum graph display screen 600, an input by an operation of dragging the designated region 601, and a dragged region 601 is displayed on the arithmetic expression display screen. 700 includes an input by an operation of dropping in a blank field 702 included in the arithmetic expression 701 in FIG. The input receiving unit 110 supplies inputs from these users to the association processing unit 114.

  The association processing unit 114 performs an association process based on the input supplied from the input receiving unit 110 (step S106). Specifically, the association processing unit 114 causes the display device 160 to display a spectrum graph display screen 600 as illustrated in FIG. When an input for dragging a region 601 surrounded by a spectrum graph, a wavelength lower limit line 602, a wavelength upper limit line 603, and an x axis (wavelength axis) on the spectrum graph display screen 600 is supplied in step S105, The association processing unit 114 obtains an integral value of the region 601. Then, the association processing unit 114 calculates the calculated element when the dragged area 601 based on the user's instruction is dropped in the blank field 702 (703) included in the calculation formula 701 on the calculation formula display screen 700. Are stored in the correspondence table 121. Thereby, the association processing unit 114 can associate the arithmetic expression 701 with the arithmetic element.

  The association processing unit 114 may associate the arithmetic expression 701 with the arithmetic element by a method other than drag and drop. For example, as shown in FIG. 11A, when an input for clicking on a region 601 on the spectrum graph display screen 600 is supplied in step S105, the association processing unit 114 calculates the integral value (calculation) of the region 601. Element value). Then, after the area 601 is clicked, the association processing unit 114 continuously calculates the calculation element obtained when the blank field 702 (703) included in the calculation expression 701 on the calculation expression display screen 700 is clicked. May be stored in the correspondence table 121.

  Further, as shown in FIG. 11B, the spectrum graph display screen 600 may be provided with a region for displaying a spectrum graph and a blank field 604 for substituting a calculation element obtained from the spectrum graph. In this case, when the input to drag the region 601 on the spectrum graph display screen 600 and drop it onto the blank field 604 is supplied in step S105, the association processing unit 114 calculates the integral value (calculation) of the region 601. Element value). Then, the association processing unit 114 stores the calculated value of the calculation element in the correspondence table 121.

  The association processing unit 114 repeats the processing of step S105 and step S106 until the association processing as described above is performed for all blanks (702, 703) in the arithmetic expression 701 (step S107; No). On the other hand, when the association process is completed for all blanks (702, 703) in the arithmetic expression 701 (step S107; Yes), the association processing unit 114 indicates that the association process has been completed. The arithmetic processing unit 115 is notified, and the process proceeds to step S108.

  When the input processing apparatus 100 according to the present embodiment performs the association process as described above, the user can easily substitute the arithmetic element obtained from the spectrum graph into the arithmetic expression 701.

  Receiving the notification that the association processing is completed in step S107, the arithmetic processing unit 115 performs processing for obtaining a calculation result (solution X) of the arithmetic expression (step S108). Specifically, the arithmetic processing unit 115 accesses the correspondence table 121 and performs an operation by substituting the values of the variables A to C (1213 to 1215) into the arithmetic expression 1212 stored in the same record 1217. . For example, in the example of FIG. 7, the arithmetic processing unit 115 adds the variable A1213 and the variable B1214 of “A = 52” and “B = 23” to the arithmetic expression 1212 of “X = 1− (A / B)”. Substitute the value of to perform the operation. Thereby, the arithmetic processing unit 115 can obtain the solution X “−1.26”. Then, the arithmetic processing unit 115 stores the obtained calculation result (solution X) 1216 in the same record 1217.

  Then, the arithmetic processing unit 115 displays the calculation result (solution X) obtained in step S108 on the display device 160 (step S109). Thereby, the input support apparatus 100 according to the present embodiment notifies the user of data serving as a determination material such as whether or not the Fret phenomenon has occurred in the sample S or how the Fret phenomenon has occurred. Can do.

  After displaying the calculation result (solution X), the control unit 101 ends the first input support process.

  FIG. 12 is a flowchart illustrating a second input support process when the Teaching function is set in the input support apparatus 100.

  The control unit 101 stands by until an instruction for setting the Teaching function is input from the user (step S201; No). When the instruction for setting the Teaching function is input from the user (step S201; Yes), the control unit 101 starts the second input support process. Specifically, when receiving an instruction to start setting of the Teaching function, the input receiving unit 110 of the control unit 101 notifies the Teaching processing unit 116 of starting the second input support process.

  Upon receiving this notification, the teaching processing unit 116 causes the display device 160 to display a teaching function setting screen 800 as shown in FIG. 6B (step S202).

  Here, the teaching processing unit 116 accepts various settings related to the teaching function from the user, and stores the setting contents (step S203). Specifically, the teaching processing unit 116 is created when an arithmetic expression 701 including a mathematical expression and a blank field 702 (703) is created on the teaching function setting screen 800 as shown in FIG. 6B. The calculated expression 701 is stored in the Teaching table 122. However, in this case, the teaching processing unit 116 converts the blank 702 (703) included in the calculation formula 701 into a variable name for specifying the blank 702 (703), and then calculates the calculation formula 1222 as follows. The information is stored in the Teaching table 122 in association with the ID 1221. Further, the teaching processing unit 116 also stores the ID 1211 and the arithmetic expression 1221 that are the same as the ID 1221 and the arithmetic expression 1222 in the correspondence table 121. Further, when the wavelength lower limit line 602 and the wavelength upper limit line 603 for determining the calculation element are designated, the teaching processing unit 116 sets the corresponding (wavelength) lower limit value 1223 and (wavelength) upper limit value 1224. And stored in the Teaching table 122. Further, when the time for assigning the calculation element to the calculation formula is input to the area 801 on the teaching function setting screen 800, the teaching processing unit 116 sets the input time corresponding to the input blank. Are stored in the Teaching table 122 as input times (1225 to 1227) of the variable names (variables A to C).

  After finishing the setting of the Teaching function, the Teaching processing unit 116 shifts the processing to Step S204 when the observation start button 802 on the Teaching function setting screen 800 is pressed.

  In step S204, the teaching processing unit 204 causes the microscope apparatus 200 to start continuous observation of the sample S such as time-lapse observation (step S204). Specifically, the teaching processing unit 204 supplies an instruction signal for starting continuous observation of the sample S such as time-lapse observation to the controller 400. At the same time, the teaching processing unit 204 starts a time measurement process using an internal timer of the CPU 901 or the like.

  During continuous observation of the sample S, the controller 400 provides a two-dimensional spectrum image at predetermined intervals. Here, the spectrum image acquisition unit 111 sequentially stores the two-dimensional spectrum image provided from the controller 400 in the spectrum image DB 120.

  Subsequently, the teaching processing unit 116 performs processing for associating the computing element set on the teaching function setting screen 800 with the created computing equation 701 (step S206). Specifically, during the period in which the microscope apparatus 200 is continuously observing the sample S, the teaching processing unit 204 refers to the input times (1225 to 1227) of the variables A to C in the teaching table 122, and the CPU 901 The system waits until the time counted by the internal timer matches the input time (1225-1227) (step S205; No). Here, when the time counted by the internal timer of the CPU 901 coincides with any of the input times (1225 to 1227) (step S205; Yes), the teaching processing unit 116 performs the calculation previously determined in step S203. Find the value of an element. When obtaining the calculation element, the teaching processing unit 116 reads the two-dimensional spectrum image for the used channel provided from the controller 400 at the coincident time from the spectrum image DB 120 and obtains the value of the calculation element. Then, the teaching processing unit 116 stores the calculated value of the calculation element in the corresponding variable A 1213 to variable C 1215 of the correspondence table 121. However, at this time, the value of the calculation element is stored in the record 1217 having the same calculation expression 1212 as the calculation expression 701 created on the Teaching function setting screen 800.

  The teaching processing unit 116 repeats the processing of the above steps S204 to S206 until all the blanks (702, 703) in the arithmetic expression 701 are associated with the values of the arithmetic elements (step S207; No). . On the other hand, when the matching process is completed for all the blanks (702, 703) in the calculation formula 701 (step S207; Yes), the teaching processing unit 116 calculates that the matching process has been completed. The processing unit 115 is notified, and the process proceeds to step S208.

  In the subsequent processing, similarly to the first input support processing, in step S207, the arithmetic processing unit 115 that has received the notification that the association processing has been completed performs processing for obtaining the arithmetic result (solution X) of the arithmetic expression. (Step S208). Then, the arithmetic processing unit 115 causes the display device 160 to display the calculation result (solution X) obtained in step S208 (step S209).

  After displaying the calculation result (solution X), the control unit 101 ends the second input support process.

  In addition, this invention is not limited to the said embodiment, A various deformation | transformation and application are possible.

  For example, in the above-described embodiment, the spectrum graph generation unit 112 generates a spectrum graph from the two-dimensional spectrum image of the sample S observed with the microscope apparatus 200. However, the present invention is not limited to this. For example, the microscope apparatus 200 may be an observation apparatus such as a camera, the two-dimensional spectrum image may be an image such as an image taken by a camera, and the spectrum graph is a function. Any graph can be used.

It is a functional block diagram of the input assistance apparatus which concerns on embodiment of this invention. It is a block diagram which shows an example of a structure of the microscope spectroscopy system which concerns on embodiment of this invention. (A) It is a figure which shows the example of a layout of a spectrum image display screen. (B) It is a figure which shows the example of a layout of a spectrum graph display screen. It is a figure which shows another layout example of a spectrum image display screen. It is a figure which shows the example of a layout of an arithmetic expression display screen. (A) It is a figure which shows the example of a layout at the time of displaying a spectrum image display screen and a calculation type | formula display screen on a display apparatus. (B) It is a figure which shows the example of a layout of Teaching function setting screen. It is a figure which shows notionally the data structure of a correspondence table. It is a figure which shows notionally the data structure of a Teaching table. It is a figure which shows an example of the hardware constitutions of the input assistance apparatus which concerns on embodiment of this invention. It is a figure which shows the flowchart of the 1st input assistance process performed with an input assistance apparatus. (A) (B) It is a figure which shows another layout example at the time of displaying a spectrum image display screen and a calculation type | formula display screen on a display apparatus. It is a figure which shows the flowchart of the 2nd input assistance process performed with an input assistance apparatus.

Explanation of symbols

50 Microscope Spectroscopic System 100 Input Support Device 101 Control Unit 102 Storage Unit 103 Interface Unit 110 Input Accepting Unit 111 Spectrum Image Acquisition Unit 112 Spectrum Graph Generation Unit 113 Arithmetic Expression Generation Unit 114 Association Processing Unit 115 Operation Processing Unit 116 Teaching Processing Unit 120 Spectral image database 121 Corresponding table 122 Teaching table 150 Input device 160 Display device 200 Microscope device 300 Spectroscopic detector 400 Controller 500 Spectral image display screen 600 Spectrum graph display screen 700 Arithmetic display screen 800 Teaching function setting screen 901 CPU

Claims (8)

An input support device connected to a display device and an observation device for providing an image related to a sample,
An input receiving unit for receiving instructions from the user;
A graph generation unit for generating a corresponding graph from the image and outputting first screen data for displaying the graph to the display device;
An arithmetic expression display unit that outputs to the display device second screen data for displaying an arithmetic expression including a variable;
On the screen of the display device, a calculation element to be calculated is specified from a graph based on the first screen data, and the calculation element is set as a variable included in the calculation formula based on the second screen data. When an instruction to be associated is received from the input receiving unit, a value of the calculation element is obtained, and an association processing unit that associates the calculated value of the calculation element with a variable of the calculation formula;
An arithmetic processing unit that obtains an arithmetic result by substituting the value of the arithmetic element associated with the variable of the arithmetic expression;
An output unit for outputting the calculation result to the display device;
An input support apparatus comprising: An input support apparatus according to claim 1,
The observation device is a microscope device;
The image is a two-dimensional spectral image;
The graph is a spectrum graph.
An input support apparatus characterized by that. An input support apparatus according to claim 1 or 2,
The association instruction is
Dragging an arithmetic element based on the first screen data and dropping the arithmetic element on a variable included in an arithmetic expression on the screen displayed based on the second screen data;
An input support apparatus characterized by that. An input support apparatus according to claim 1 or 2,
The association instruction is
Clicking on a calculation element based on the first screen data, and then clicking on a variable included in the calculation expression on the screen displaying the calculation element based on the second screen data.
An input support apparatus characterized by that. An input support apparatus according to claim 1 or 2,
The screen to be displayed based on the first screen data is provided with blanks corresponding to the variables included in the arithmetic expression,
The association instruction is
Dragging a calculation element designated on the screen displayed based on the first screen data, and dropping the calculation element on the blank on the screen displayed based on the first screen data. ,
An input support apparatus characterized by that. An input support apparatus according to claim 1 or 2,
The arithmetic expression displayed by the arithmetic expression display unit includes a plurality of variables,
The association processing unit
The input receiving unit designates a plurality of calculation elements to be calculated from a graph based on the first screen data, and displays the plurality of calculation elements based on the second screen data. When an instruction for associating operations corresponding to a plurality of variables included in the arithmetic expression is received, the values of the plurality of arithmetic elements are respectively obtained, and the values of the plurality of arithmetic elements obtained are determined in the arithmetic expression. Associate each with multiple variables,
An input support apparatus characterized by that. An input support device connected to a display device and an observation device for providing an image related to a sample,
An input receiving unit for receiving instructions from the user;
A storage unit for storing setting data;
A graph generation unit for generating a corresponding graph from the image and outputting first screen data for displaying the graph to the display device;
An arithmetic expression display unit that outputs to the display device second screen data for displaying an arithmetic expression including a variable;
A timekeeping section for timing,
When an instruction to specify a calculation element to be calculated from the graph based on the first screen data is received from the input receiving unit, specific data for specifying the calculation element is stored in the storage unit in advance. In addition, when the instruction for specifying the timing for associating the calculation element with the variable included in the calculation formula based on the second screen data is received from the input receiving unit, time information for specifying the timing is stored in advance in the storage unit And when the time measured by the timing unit coincides with the time information, the value of the calculation element specified by the specific data is obtained, and the calculated value of the calculation element is calculated as the calculation value. A Teaching processing unit for associating with an expression variable;
An arithmetic processing unit that obtains an arithmetic result by substituting the value of the arithmetic element associated with the variable of the arithmetic expression;
An output unit for outputting the calculation result to the display device;
An input support apparatus comprising: A program for causing a computer to function as an input support device to which an observation device and a display device for providing an image relating to a sample are connected,
The computer
Input receiving means for receiving instructions from the user,
In the computer,
A graph generation process for generating a corresponding graph from the image and outputting first screen data for displaying the graph to the display device;
An arithmetic expression display process for outputting second screen data for displaying an arithmetic expression including a variable;
A screen in which the input receiving means designates a calculation element to be calculated from a graph on the screen displayed based on the first screen data, and displays the calculation element based on the second screen data The association process of obtaining the value of the operation element when the instruction for associating the operation associated with the variable included in the arithmetic expression is received, and associating the calculated value of the operation element with the variable of the arithmetic expression When,
A calculation process for obtaining a calculation result by substituting the value of the calculation element associated with the variable of the calculation formula;
An output process for outputting the calculation result to the display device;
A program characterized by that.

Images