JP2012194517A - Image display device, observation apparatus, display program, image display method, and observation control method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To efficiently perform a display with a simple operation when performing the search and select of a specified target from an image obtained by picking up a wide range area.SOLUTION: An image display device includes a record section for recording a picked-up image, a monitor section for displaying the picked-up image, and display control means for reading out a partial image from the picked-up image according to a predetermined scanning condition and generating a first display image including indicator information showing the image position of the partial image in the picked-up image and a second display image corresponding to the partial image to display the first display image and second display image on the monitor section.

Description

  The present invention relates to an image display device, an observation device, a display program, an image display method, and an observation control method.

  2. Description of the Related Art Conventionally, a technique for switching and displaying an entire image and a partial enlarged image is widely known. Such a technique is required when confirming the details of each part while confirming the entire image.

  For example, in the field of biology and the like, such a technique is used when searching and selecting a specific object such as a colony from an image obtained by imaging a wide area. As an example, the invention of Patent Document 1 discloses a technique for associating positions of an image with a first magnification and an image with a second magnification of a higher magnification with respect to an image acquired using a microscope.

JP-T-2001-519944

  By the way, in the field of biology and the like described above, the process of searching for and selecting a specific object such as a colony is called screening. In such screening, a user such as a researcher needs to manually search and select a specific object while switching and displaying an image obtained by imaging a wide area and a partial enlarged image thereof. . Therefore, the operation is complicated, and there are many cases where screening omissions, duplicate screenings, and the like occur.

  The present invention has been made in view of the above problems, and an object thereof is to perform efficient display with a simple operation when searching for and selecting a specific object from an image obtained by imaging a wide area. .

  The image display device according to one aspect includes a recording unit that records a captured image, a monitor unit that displays the captured image, a partial image is read from the captured image under a predetermined scanning condition, and the image in the captured image is Display control means for generating a first display image including indicator information indicating an image position of the partial image and a second display image corresponding to the partial image and displaying the second display image on the monitor unit;

  The display control unit further includes a stop switch unit, the display control unit interrupts reading of the partial image based on a stop signal of the stop switch unit, and includes a scanned region designated as a partial image read in the past. Visual information that can be distinguished from other unscanned areas may be added to the first display image.

  The display control unit further includes a restart switch unit, and the display control unit is configured to select the part from a predetermined position in the unscanned area on the first display image based on an input signal from the restart switch unit. The image update designation may be resumed.

  In addition, the display control unit further includes an area designation input unit, and the display control unit is configured to specify the part based on an input value from the area designation input unit at the time of the update designation at the first time or at the time of restart by the display control unit. An image may be specified.

  The recording unit may record a plurality of partial images as the captured image, and the display control unit may generate the first display image by combining the plurality of partial images.

  An observation apparatus according to one aspect includes any one of the above-described image display apparatuses, an observation optical system for observing a specimen, and imaging the specimen via the observation optical system, and image data of the specimen is represented by the image display apparatus. In the observation device having an observation unit that outputs to, a stop switch unit that stops reading of the partial image according to the scan condition, and the second display image when reading of the partial image is stopped by the stop switch unit An image analysis unit for analysis.

  The image analysis unit includes a designation operation unit that designates a desired cell in the second display image, a measurement unit that measures morphological information or brightness information of the cell designated by the designation operation unit, An evaluation unit that evaluates the state of the cell from the shape information or brightness information of the cell measured by the measurement unit may be included.

  Further, the image analysis unit analyzes the first analysis unit for analyzing whether the cell as the specimen is a desired cell based on the second display image, and among the cells of the second display image, the first analysis unit A first processing unit that performs a marking process on a cell that has been determined as the desired cell by one analysis unit, and the display control unit includes the desired cell that has been marked by the first processing unit. You may perform selective reading of an image.

  Further, the image analysis unit analyzes the first analysis unit for analyzing whether the cell as the specimen is a desired cell based on the second display image, and among the cells of the second display image, the first analysis unit A first processing unit that performs a marking process on a cell that has been determined to be the desired cell by one analysis unit; and the desired cell that has been subjected to the marking process or a cell that has not been subjected to the marking process. An evaluation correction instruction unit for instructing correction of the evaluation of the analysis result of the first analysis unit whether or not there is, the first processing unit using the second display image displayed on the monitor unit Then, the correction instruction of the evaluation correction instruction unit may be executed.

  An observation apparatus according to another aspect includes any one of the above-described image display apparatuses, an observation optical system for observing a specimen, and imaging the specimen via the observation optical system, and image data of the specimen as the image display apparatus In the observation apparatus having an observation unit that outputs to the imaging unit, the specimen is imaged at a predetermined time interval via a stop switch unit that stops reading of the partial image according to the scan condition and the observation optical system, An imaging point on the sample corresponding to the second display image when reading of the partial image is stopped by the observation unit configured to output image data to the image display device and the stop switch unit. Is registered as an observation point for the time lapse observation.

  In addition, you may display the map which shows the observation point of the said time lapse observation registered in the said registration part in the 1st display image of the said image display apparatus.

A display program according to one aspect is a display program for executing processing of an image display apparatus including a recording unit that records a captured image and a monitor unit that displays the captured image, by a computer from the captured image. A step of reading a partial image under the scanning conditions of
Generating a first display image in which indicator information indicating an image position of the partial image is added to the captured image; and a second display image corresponding to the partial image; and Displaying the second display image.

  An image display method according to an aspect is an image display method of an image display device including a recording unit that records a captured image and a monitor unit that displays the captured image, and a partial image from the captured image under a predetermined scanning condition. A first display image obtained by adding indicator information indicating an image position of the partial image to the captured image, a second display image corresponding to the partial image, and the monitor unit Displaying a first display image and the second display image.

  An observation control method according to one aspect includes any one of the above-described image display devices, an observation optical system for observing a specimen, and imaging the specimen via the observation optical system, and displaying image data of the specimen as the image display In an observation control method for an observation apparatus having an observation unit that outputs to the apparatus, reading of the partial image according to the scan condition is stopped, and the second display image when the reading of the partial image is stopped is analyzed.

  The specimen is time-lapse observed at a predetermined time interval through the observation optical system, and when the reading of the partial image is stopped, the imaging point on the specimen corresponding to the second display image is observed in the time-lapse observation. You may register as an observation point.

  ADVANTAGE OF THE INVENTION According to this invention, when searching and selecting a specific target object from the image which imaged the wide area | region, it can display efficiently by simple operation.

Schematic diagram showing a configuration example of an observation apparatus according to an embodiment Diagram explaining tiling image and partial image Example of monitor display Monitor display example (enlarged view) Indicator display example The flowchart which shows the operation example of the control part 16 in one Embodiment. Flowchart showing an operation example of the control unit 16 in one embodiment (continued) Example of monitor display Example of monitor display Diagram explaining scan direction Example of monitor display Indicator display example Flowchart showing an operation example of the control unit 16 in one embodiment (continued) Example of monitor display Example of monitor display Indicator display example Indicator display example

  Hereinafter, embodiments will be described with reference to the drawings. In the following description, an observation apparatus is provided as an example with a display device.

  FIG. 1 is a schematic diagram illustrating a configuration example of an observation apparatus according to an embodiment. The observation device is, for example, a device for culturing cells to be observed and time-lapse observation of the cells to be observed.

  The observation apparatus includes a temperature-controlled room 11, an environment control unit 12, a stage 13, a container transport unit 14, an imaging unit 15, a control unit 16, a storage unit 17, an input unit 18, a monitor 19, and a communication unit 20. Here, the environment control unit 12, the stage 13, the container transport unit 14, the imaging unit 15, the storage unit 17, the input unit 18, the monitor 19, and the communication unit 20 are connected to the control unit 16.

  The temperature-controlled room 11 is a room for culturing cells. For example, in the temperature-controlled room 11, a plurality of culture containers (petri dishes, well plates, etc.) that hold the cells to be observed together with the culture medium are held. A container transport unit 14 and a stage 13 are disposed in the temperature-controlled room 11.

The environment control unit 12 controls parameters (temperature, humidity, CO2 concentration, etc.) of environmental conditions in the temperature-controlled room 11. By the operation of the environment control unit 12, the inside of the temperature-controlled room 11 is maintained at predetermined environmental conditions (for example, temperature 37 ° C., humidity 90%, CO 2 concentration 5%) suitable for cell culture.

  A culture vessel is placed on the stage 13 during observation. The stage 13 has a drive mechanism that is driven in accordance with an instruction from the control unit 16, and finely adjusts the position of the culture vessel during observation.

  The container transport unit 14 transports the culture container in the temperature-controlled room 11 according to an instruction from the control unit 16. For example, the container transport unit 14 transports the culture container at the storage position in the temperature-controlled room 11 onto the stage 13 during observation. And the container conveyance part 14 returns the culture container on the stage 13 to said accommodation position after completion | finish of observation. The container transport unit 14 carries the culture container into and out of the temperature-controlled room 11.

  The imaging unit 15 is an electronic camera module that captures an observation image of a cell to be observed under the environment of the temperature-controlled room 11. The optical system of the imaging unit 15 is an optical system of a transmission microscope (for example, a phase contrast microscope). Then, the imaging unit 15 captures an image of cells in the culture vessel placed on the stage 13 in accordance with an instruction from the control unit 16. The image output of the imaging unit 15 is connected to the storage unit 17.

  The imaging unit 15 in one embodiment has a still image and moving image shooting function. Note that the imaging unit 15 may be configured to capture still images and moving images with the same imaging device. Alternatively, the imaging unit 15 may be configured to branch the optical path with a half mirror or the like and to capture the still image and the moving image with different imaging elements.

  The control unit 16 is a processor that comprehensively controls the observation apparatus. The control unit 16 performs each process by executing a program.

  The control unit 16 controls the stage 13, the container transport unit 14, and the imaging unit 15 according to a preset observation schedule, and has a time-lapse observation function of cells. In addition, the control unit 16 controls the stage 13, the container transport unit 14, and the imaging unit 15 to repeatedly perform imaging while changing the imaging position, and by combining a plurality of obtained images, a wider field of view can be obtained. It has a function of generating a tiling image having detailed information.

  The storage unit 17 is configured by a storage medium such as a hard disk or a nonvolatile semiconductor memory, for example. The storage unit 17 stores a program executed by the control unit 16, information on the observation schedule of the culture vessel, information on the observation point of the culture vessel, teacher information for specifying a target cultured cell, and video shooting. More detailed observation condition information and the like are stored. The storage unit 17 can also store information on various images input from the imaging unit 15.

  The input unit 18 includes, for example, an input device such as a button or a touch panel. The input unit 18 supplies a signal corresponding to a user operation to the control unit 16. The monitor 19 executes various displays under the control of the control unit 16.

  The communication unit 20 performs data transmission / reception with a terminal device 22 outside the observation apparatus via a wireless or wired communication line 21. In addition, said terminal device 22 is comprised, for example with a personal computer.

  Hereinafter, an example of operation during screening will be described with reference to the drawings. Screening is a process of searching for and selecting a specific object such as a colony based on the above-described tiling image. Further, when performing screening, a plurality of partial images obtained by repeatedly imaging at high magnification while changing the imaging position (imaging range) by the imaging unit 15 for the entire field of view, and the plurality of partial images It is assumed that a captured image including a tiling image generated by combining (joining) is generated in advance and stored in the storage unit 17. When performing screening, the control unit 16 reads out partial images and tiling images from the storage unit 17 and continuously changes the display range displayed on the monitor 19 among the read images. Process. The user selects an image (image group) to be screened from the storage unit 17 and executes screening.

  In the following, 100 images (for example, about 1000 pixels per side) of partial images Img (1,1) to Img (10,10) shown in FIG. 2 and a tiling image Img ( T) (For example, a case where screening is performed for an object of about 500 pixels per side) will be described as an example.

  First, display of the tiling image Img (T) and the partial images Img (1,1) to Img (10,10) will be described. FIG. 3 is a display example when various images are displayed on the monitor 19. As shown in FIG. 3, an image to be displayed is displayed in the area A1. In the area A2, icons and the like related to reproduction and display of images (moving images) by time-lapse observation are displayed. In the area A3, icons such as a button for selecting presence / absence of center display of the area A1, a button for displaying a list of images at a plurality of points, and a button for ending display in the area A1 are displayed. In the area A4, information related to the image displayed in the area A1 is displayed. In the area A5, an indicator to be described later and an icon of a designation button related to screening are displayed. In the area A6, a button icon for changing the display magnification, the display position, and the like of the image displayed in the area A1 is displayed. In the area A7, icons for performing various settings for point designation (including screening) are displayed.

  Here, display of an image in the area A1 in FIG. 3 will be described. A tiling image Img (T) or partial images Img (1,1) to Img (10,10) are displayed in the area A1 in FIG. Further, based on a user operation via the input unit 18 for each icon described above, a tiling image Img (T) or an enlarged image or a reduced image of the partial images Img (1,1) to Img (10,10). Is also displayed.

  FIG. 4 is an enlarged view of the regions A5 and A6 of FIG. 3 described above. An indicator to be described later is displayed in the area a1. In the area a2, an icon related to execution of Frame scan described later is displayed. In the area a3, an icon related to execution of Auto browse to be described later is displayed. In the area a4, an icon for enlarging the image displayed in the area A1 in FIG. 3 is displayed, and in the area a5, an icon for reducing the image displayed in the area A1 in FIG. 3 is displayed. . In the area a6, an icon for returning the partial image (enlarged image) to the tiling image (entire image) is displayed. In the area a7, an icon for changing the position of the image displayed in the area A1 in FIG. 3 is displayed.

  The user can enlarge and reduce the image displayed in the area A1 in FIG. 3 and change the display position by operating each icon in the area A6 in FIG. 3 via the input unit 18. In addition, in the area a1 (see FIG. 4) of the area A5 in FIG. 3, which position on the tiling image (entire image) the image currently displayed in the area A1 in FIG. Is displayed (see FIG. 5). The user can easily grasp the position of the image currently displayed in the area A1 in FIG. 3 on the tiling image (overall image) by viewing the indicator.

  Next, the operation of the control unit 16 at the time of screening will be described with reference to the flowchart shown in FIG. The process described in FIG. 6 is always performed in a state where the tiling image Img (T) or the partial images Img (1,1) to Img (10,10) is displayed in the above-described area A1 in FIG. . That is, as described above, in the state where the tiling image Img (T) or the partial images Img (1,1) to Img (10,10) are displayed in the area A1 in FIG. Screening can be performed by manipulation via 18.

  In step S <b> 1, the control unit 16 determines whether a Frame scan magnification is selected via the input unit 18. The control unit 16 determines whether any of the icons in the area a2 in FIG. 4 described above is selected. If the control unit 16 determines that the icon is selected, the control unit 16 proceeds to step S2 to execute Frame scan. Frame scan is a screening method that scans the entire tiling image Img (T) at a magnification selected by the user, a predetermined start position, a predetermined scanning direction, and a predetermined scanning speed (details will be described later). The user can select the frame scan magnification by selecting any of the icons 9, 4, and 1 in the area a2 in FIG. Note that 9 means that the range of 9 partial images (3 × 3) is enlarged, 4 means that the range of 4 partial images (2 × 2) is enlarged, and 1 means that the range of one partial image is enlarged and displayed. That is, the enlargement magnification increases in the order of 9, 4, 1. Hereinafter, a case where 4 is selected will be described as an example.

  In step S2, the control unit 16 controls each unit to execute Frame scan. And the control part 16 will return to step S1, if Frame scan is complete | finished.

  In step S <b> 3, the control unit 16 determines whether or not start of auto browse is selected via the input unit 18. The control unit 16 determines whether or not the Start icon in the above-described region a3 in FIG. 4 is selected. If the control unit 16 determines that the Start icon is selected, the control unit 16 proceeds to Step S4 in order to execute Auto browse. Auto browse determines the enlargement magnification and start position based on the partial image displayed in the area A1 in FIG. 3, and also determines the size of the tiling image Img (T) at a predetermined scan direction and a predetermined scan speed. This is a screening method for scanning a part (details will be described later).

  In step S4, the control unit 16 controls each unit to execute Auto browse. And the control part 16 will return to step S1, if Auto browse is complete | finished.

  Next, the details of the Frame scan described in step S2 of FIG. 6 will be described with reference to the flowchart shown in FIG. As described above, Frame scan comprehensively scans a range corresponding to the entire tiling image Img (T) with an enlargement magnification based on user selection, a predetermined start position, a predetermined scan direction, and a predetermined scan speed. Screening method. As described with reference to FIG. 6, the frame scan is started by selecting any of the icons 9, 4, and 1 in the area a <b> 2 in FIG. 4. Note that the Frame scan displays the partial images Img (1, 1) to Img (10, 10) even if the tiling image Img (T) is displayed in the area A1 in FIG. Even if it is, it can be executed. FIG. 8 shows a display example of the monitor 19 when the tiling image Img (T) is displayed in the area A1 of FIG.

  In step S <b> 11, the control unit 16 displays a confirmation message on the monitor 19. For example, as shown in FIG. 9, the control unit 16 displays a message “OK to cancel, moving from the current observation field of view and start scanning with the specified field of view? I do.

  In step S <b> 12, the control unit 16 determines whether or not “OK” has been selected via the input unit 18. If the control unit 16 determines that OK has been selected, the process proceeds to step S13. If the control unit 16 determines that OK has not been selected (Cancel has been selected), the control unit 16 ends the Frame scan process.

  In step S13, the control unit 16 changes the Start icon in the area a3 in FIG. 4 to a Stop icon.

  In step S14, the control unit 16 sets the scan start position to the upper left corner, which is a predetermined position.

  In step S15, the control unit 16 sets a scan magnification which is an enlargement magnification at the time of scanning based on a user operation. The control unit 16 sets a magnification based on the number of sheets (any one of 9, 4, 1) selected by the user when executing the frame scan as a scan magnification.

  In step S16, the control unit 16 sets the scan direction to a predetermined direction, and sets the scan speed to a predetermined speed. For example, as illustrated in FIG. 10, the control unit 16 sets a continuous direction from left to right and from top to bottom as the scan direction.

  In step S17, the control unit 16 controls each unit to start scanning. The control unit 16 reads out the partial images Img (1,1) to Img (10,10) from the storage unit 17 and corresponds to the case where the tiling image is enlarged and scanned for 4 × 2 × 2. 3 is displayed in the area A1 in FIG. For example, the control unit 16 generates a reduced image for display (for example, about 300 pixels per side) for each of the partial images Img (1,1) to Img (10,10), and stores it in the memory in the control unit 16. Make a temporary recording. In the subsequent display, display is performed using these reduced images.

  FIG. 11 shows an example of an image that is displayed first in the area A1 of FIG. Immediately after the start of scanning, the upper left four partial images (Img (1,1), Img (1,2), Img (2,1), Img (2,2)) are displayed. Then, by gradually changing the display range in accordance with the scan direction described in step S16, it is possible to display as if the tiling image is enlarged and scanned.

  Note that an overlapping portion may be provided when the display range is changed in the vertical direction. By doing in this way, for example, even when an important specific object exists in the boundary portion between the first line and the second line, it is possible to reliably search for the specific object.

  When the scan is started, the enlarged image displayed in the area A1 in FIG. 3 automatically changes, so that the user may look at the monitor 19 and search for the specific object. When a region of interest is found, by selecting the Stop icon in the region a3 in FIG. 4, the scan can be temporarily stopped to perform more detailed confirmation or the above-described point designation. . When the stop icon is selected during the scan, the control unit 16 changes the Stop icon in the area a3 in FIG. 4 to the Continue icon. The user can resume scanning by selecting this Continue icon (resume updating of partial images from a predetermined position in an unscanned area). Further, the scan can be stopped by selecting the Clear icon in the area a3 in FIG.

  Note that when scanning is paused, instead of selecting the Stop icon in the area a3 in FIG. 4 described above, the scanning is paused by selecting (clicking) any part on the area A1 in FIG. Also good.

  In addition, when scanning is paused, image analysis or the like can be performed in addition to the above-described confirmation and point designation. Details of each process when the scan is suspended will be described later.

  In step S18, the control unit 16 starts indicator display in conjunction with the start of scanning described in step S17. FIG. 12 is an example of indicator display. As shown in FIG. 12, the control unit 16 recognizes the area a <b> 1 in FIG. 4 as visual information that can identify an area where scanning has been completed (scanned area) and an area where scanning has not been completed (unscanned area). Indices that can visually recognize the scanned area and the unscanned area are displayed. In the example of FIG. 12, the scanned area is indicated by diagonal lines. In addition, the scanned area and the unscanned area may be identified by changing the color or the like.

  In step S <b> 19, the control unit 16 determines whether scanning has been completed to the lower right end. The control unit 16 waits until the entire region up to the portion corresponding to the lower right end of the tiling image Img (T) is scanned, and scans the entire region up to the portion corresponding to the lower right end of the tiling image Img (T). When it is determined, the Frame scan process is terminated. At this time, the control unit 16 ends the indicator display by returning the indicator display started in step S18 to the initial state.

  Next, the details of Auto browse described in step S4 of FIG. 6 will be described with reference to the flowchart shown in FIG. As described above, Auto browse is a tiling image Img (T with a magnification and start position based on the partial image displayed in the area A1 of FIG. 3 at the start, a predetermined scanning direction, and a predetermined scanning speed. ) Is a screening method that scans a range corresponding to a part of. Auto browse is started by selecting the Start icon in the area a3 of FIG. 4 as described in FIG. Note that Auto browse can be executed when partial images Img (1,1) to Img (10,10) are displayed in the area A1 of FIG. FIG. 14 shows a display example of the monitor 19 when a part of the partial images Img (1,1) to Img (10,10) is enlarged and displayed in the area A1 of FIG.

  In step S <b> 21, the control unit 16 displays a confirmation message on the monitor 19. For example, as shown in FIG. 15, the control unit 16 displays a message “Start scanning in the current observation field of view. Are you sure?” And OK and Cancel.

  In step S <b> 22, the control unit 16 determines whether OK has been selected via the input unit 18. If the control unit 16 determines that OK has been selected, the process proceeds to step S23. If the control unit 16 determines that OK has not been selected (Cancel has been selected), the control unit 16 ends the Auto browse process.

  In step S23, the control unit 16 changes the Start icon in the area a3 in FIG. 4 to a Stop icon.

  In step S24, the control unit 16 sets a scan start position and a scan magnification that is an enlargement magnification at the time of scanning based on the partial image displayed in the area A1 of FIG. 3 at the start of Auto browse. The control unit 16 sets the scan start position to a position corresponding to the partial image displayed in the area A1 in FIG. 3 at the start of Auto browse, and the scan magnification is displayed in the area A1 in FIG. 3 at the start of Auto browse. Set the magnification based on the range of the partial image.

  In step S25, the control unit 16 sets the scan direction to a predetermined direction, and sets the scan speed to a predetermined speed. For example, as illustrated in FIG. 10, the control unit 16 sets a continuous direction from left to right and from top to bottom as the scan direction.

  In step S26, the control unit 16 controls each unit to start scanning. The control unit 16 reads out the partial images Img (1,1) to Img (10,10) from the storage unit 17, and sets them in step S24 and step S25 corresponding to the case where the tiling image is enlarged and scanned. A partial image in a range corresponding to the display condition is displayed in the area A1 of FIG. For example, the control unit 16 generates a reduced image for display (for example, about 300 pixels per side) for each of the partial images Img (1,1) to Img (10,10), and stores it in the memory in the control unit 16. Record temporarily. In the subsequent display, display is performed using these reduced images.

  Then, by gradually changing the display area in accordance with the scan direction described in step S25, it is possible to perform display as if the tiling image is enlarged and scanned.

  Note that an overlapping portion may be provided when the display range is changed in the vertical direction. By doing in this way, for example, even when an important specific object exists in the boundary portion between the first line and the second line, it is possible to reliably search for the specific object.

  After the scan is started, the user may search for the specific object by looking at the monitor 19 as in the case of the frame scan described with reference to the flowchart of FIG. The same applies to the Stop icon, the Continue icon, and the Clear icon in the area a3 in FIG. Details of each process when scanning is suspended will be described later.

  In step S27, the control unit 16 starts indicator display in conjunction with the start of scanning described in step S26. FIG. 16 is an example of indicator display. As shown in FIG. 16, the control unit 16 recognizes the area a1 in FIG. 4 as visual information that can identify a scanned area (scanned area) and a scanned area (unscanned area). Indices that can visually recognize the scanned area and the unscanned area are displayed. FIG. 16 illustrates an indicator display when the position near the center is the scan start position. Further, in FIG. 16, the scanned area is indicated by diagonal lines as in FIG. 12, but the scanned area may be made visible by changing the color or the like.

  In step S28, the control unit 16 determines whether or not a scan end position is designated. If it is determined that the scan end position has been designated, the control unit 16 proceeds to step S29, and if it is not designated, the control unit 16 proceeds to step S30 described later. The scan end position can be designated in advance by a user operation via the input unit 18. The scan end position may be specified, for example, when an auto browse execution instruction described in step S3 in FIG. 6 is performed, or may be specified during execution of auto browse. In this way, by making it possible to specify the scan end position in addition to the scan start position, it is useful, for example, when scanning only a part of the wells of a plate having a plurality of wells.

  In step S29, the control unit 16 determines whether or not the scan has been completed up to the scan end position. When the control unit 16 waits until the portion corresponding to the scan end position in the tiling image Img (T) is scanned, and determines that the portion corresponding to the scan end position in the tiling image Img (T) is scanned, Auto browse Terminate the process. At this time, the control unit 16 ends the indicator display by returning the indicator display started in step S27 to the initial state.

  In step S <b> 30, the control unit 16 determines whether scanning has been completed to the lower right end. The control unit 16 waits until the portion corresponding to the lower right end of the tiling image Img (T) is scanned, and if it determines that the portion corresponding to the lower right end of the tiling image Img (T) has been scanned, the process of auto browse is performed. Exit. At this time, the control unit 16 ends the indicator display by returning the indicator display started in step S27 to the initial state.

  Next, details of each process when the scan (Frame scan or Auto browse) is temporarily stopped will be described. In the following, an analysis process and a registration process will be described as examples of processes when scanning is temporarily stopped. Note that the user can resume scanning by selecting the Continue icon described in step S17 in FIG. 7 and step S26 in FIG. 13 after finishing the analysis process and the registration process.

  The analysis process is a process of analyzing the partial image displayed in the area A1 in FIG. 3 when scanning is temporarily stopped. This analysis process includes processes such as evaluation and marking, and these analysis processes are executed by the control unit 16. In addition, when the analysis process is executed, the control unit 16 displays an indicator (FIG. 5) indicating to which position on the tiling image (entire image) the partial image that is currently the target of the analysis process. Is displayed in the area a1 in FIG.

  When performing the evaluation, for example, the control unit 16 displays a message such as “Specify a desired cell” on the monitor 19 and prompts the user to specify a cell to be analyzed. For example, the user designates a desired cell by selecting and operating the desired cell on the partial image displayed in the area A1 of FIG. When a desired cell is designated by the user, the control unit 16 determines the cell shape information designated by the user based on the partial image displayed in the area A1 in FIG. 3 when the scan is temporarily stopped. And brightness information are measured, and the state of the cell is evaluated from the measured cell shape information and brightness information. Note that the measurement of morphological information and brightness information and the evaluation of the state of cells are performed in the same manner as in the known technique. Further, when the observation apparatus has a configuration such as a fluorescence observation unit that can acquire an image having a characteristic different from that of the image stored in the storage unit 17, morphological information and brightness are obtained from an image having a different characteristic such as a fluorescence image. Information may be measured.

  When performing marking, for example, when the scanning is temporarily stopped, the control unit 16 determines, based on the partial image displayed in the area A1 in FIG. 3, each cell included in the partial image It is determined whether the cell is a desired cell, and the cell determined to be a desired cell is marked. Note that the above-described determination and marking processing are performed in the same manner as in the known technology.

  Furthermore, the control unit 16 may selectively read out only the partial image including the cell that has undergone the marking process and display it on the monitor 19. At the time of display, so-called skip display may be performed in which only partial images including cells that have undergone marking processing are displayed like frame advance, or marking processing is performed while gradually changing the display range. The partial image including the cells may be displayed after being temporarily stopped for a predetermined time.

  In addition, when the result of determination prior to the above-described marking (determination of whether each cell is a user's desired cell) is not correct, the control unit 16 gives a user instruction to correct the marking content to the input unit 18. You may accept by user operation via. When receiving the correction instruction, the control unit 16 executes the correction instruction using the partial image displayed in the area A1 in FIG. 3 when the scan is temporarily stopped.

  In addition, as desired cells in the above-described analysis processing, not only normal cells but also cell colonies and the like can be considered, and specifically, stem cells, cardiomyocytes, nerve cells and the like can be considered.

  Each process described in the analysis process described above may be performed before the scan is executed. For example, after the partial images Img (1,1) to Img (10,10) are captured, each process described in the analysis process described above is performed in advance, and the analysis results are displayed as partial images Img (1,1) to Img ( 10 and 10) are recorded in association with each other. Then, when a scan for the partial images Img (1,1) to Img (10,10) is executed, display reflecting the analysis result may be performed. For example, the above-described marking result may be displayed so as to be visible.

  The registration process is a process of registering an arbitrary point in the partial image displayed in the area A1 in FIG. 3 as an observation point for time lapse observation when scanning is temporarily stopped. This registration process is executed by the control unit 16. In addition, when the registration process is executed, the control unit 16 displays an indicator (FIG. 5) indicating to which position on the tiling image (entire image) the partial image that is currently subjected to the registration process. Is displayed in the area a1 in FIG. In addition, you may add the display which can identify the point by which the coordinate was simply memorize | stored (what is called point designation | designated) and the point registered as an observation point of time lapse observation to the indicator display mentioned above. FIG. 17 illustrates an indicator display during the registration process. In the example of FIG. 17, a cross pointer Pa is displayed on an indicator for points where coordinates are simply stored, and a pointer Pb in which a cross is surrounded by a square is used for points registered as observation points for time-lapse observation. Each indicator is displayed so that it can be identified.

  When performing the registration process, the control unit 16 time-lapses a point corresponding to a predetermined position (for example, the center position) of the partial image displayed in the area A1 of FIG. 3 when the scan is temporarily stopped. Register as an observation point for observation. Further, the control unit 16 may display a map of the registered observation points on the indicator described above. For example, the control unit 16 displays the pointer Pb shown in FIG. 17 as an indicator for the registered observation point.

  Further, the control unit 16 may set observation conditions for time lapse observation in addition to registration of observation points for time lapse observation using the partial image described above. The observation conditions include, for example, a time interval and a photographing range. When the shooting range is determined, a combination of the magnification at the time of shooting and the number of shots (however, the number of shots is an integer of 1 or more) indicating how many images the shot range is divided into is taken is determined. be able to.

  As described above, according to the above-described embodiment, a recording unit that records a captured image and a monitor unit that displays the captured image are provided, and a partial image is read from the captured image under a predetermined scanning condition. A first display image including indicator information indicating the image position of the middle partial image and a second display image corresponding to the partial image are generated and displayed on the monitor unit. Therefore, when searching and selecting a specific object from an image obtained by imaging a wide area, it is possible to display efficiently with a simple operation.

  Further, according to the above embodiment, there is further provided a stop switch unit, which interrupts the reading of the partial image based on the stop signal of the stop switch unit, and the scanned area designated as the partial image read in the past and Visual information that makes it possible to identify an unscanned region other than is added to the first display image. Therefore, since the user can grasp the entire region in the tiling image from a bird's-eye view, problems such as screening omission and duplicate screening can be avoided.

  Further, according to the above embodiment, the image processing apparatus further includes a restart switch unit, and updates a partial image from a predetermined position in an unscanned area on the first display image based on an input signal from the restart switch unit. Resume specification. Therefore, the user can stop scanning and perform a desired process, and then continue the scan immediately.

  Further, according to the above embodiment, the image processing apparatus further includes the area designation input unit, and designates the partial image based on the input value from the area designation input unit at the time of the first time or the update designation at the time of restart. Therefore, the user can execute screening by a simple operation (at least one instruction).

  Further, according to the above embodiment, in an observation apparatus having an observation optical system for observing a specimen, and an observation section that images the specimen through the observation optical system and outputs the image data of the specimen to the image display apparatus, scanning is performed. A stop switch unit that stops reading of a partial image according to conditions, and an image analysis unit that analyzes a second display image when reading of the partial image is stopped by the stop switch unit. Therefore, the user can perform image analysis in the region of interest simply by operating the stop switch unit.

  Further, according to the embodiment, the image analysis unit is configured to designate a desired cell in the second display image, and to measure the shape information or brightness information of the cell designated by the designation operation unit. And an evaluation unit that evaluates the state of the cell from the cell shape information or brightness information measured by the measurement unit. Therefore, the user can obtain the evaluation result regarding the desired cell only by operating the stop switch unit and designating the desired cell.

  Moreover, according to the said embodiment, an image analysis part analyzes the 1st analysis part which analyzes whether the cell which is a sample is a desired cell based on a 2nd display image, and the cell of a 2nd display image And a first processing unit that performs a marking process on a cell determined as a desired cell by the first analysis unit. And the selective reading of the partial image containing the desired cell marking-processed by the 1st process part is performed. Therefore, the user can selectively read out only a partial image including a desired cell only by operating the stop switch unit.

  Moreover, according to the said embodiment, an image analysis part analyzes the 1st analysis part which analyzes whether the cell which is a sample is a desired cell based on a 2nd display image, and the cell of a 2nd display image A first processing unit that performs marking processing on a cell that has been determined to be a desired cell by the first analysis unit, and whether the cell is a desired cell that has been marked or not marked An evaluation correction instruction unit for instructing correction of the evaluation of the analysis result of the first analysis unit, and the first processing unit uses the second display image displayed on the monitor unit to Execute correction instructions. Therefore, when the result of the analysis and evaluation performed automatically is incorrect, the user can make corrections with a simple operation.

  Further, according to the above embodiment, in an observation apparatus having an observation optical system for observing a specimen, and an observation section that images the specimen through the observation optical system and outputs the image data of the specimen to the image display apparatus, scanning is performed. A stop switch unit that stops reading of a partial image according to conditions, an observation unit including a time-lapse observation unit that images a sample at a predetermined time interval via an observation optical system, and outputs sample image data to an image display device; And a registration unit that registers an imaging point on the specimen corresponding to the second display image as an observation point for time-lapse observation when reading of the partial image is stopped by the stop switch unit. Therefore, the user can register the observation point for time-lapse observation only by operating the stop switch unit.

  Moreover, according to the said embodiment, the map display which shows the observation point of the time lapse observation registered in the registration part is displayed in the 1st display image of an image display apparatus. Therefore, the user can grasp the observation point of time-lapse observation from a bird's-eye view.

<Supplementary items of the embodiment>
In the above-described embodiment, a plurality of partial images and a tiling image generated by synthesizing the plurality of partial images have been described as examples. You may use the image etc. which were produced | generated by imaging.

  In addition, the setting conditions and the icons described in the above embodiment are examples, and the present invention is not limited to this example. For example, in the above embodiment, an example has been shown in which any of 9, 4 and 1 can be selected as the number of displayed sheets during Frame Scan, but other numbers such as 16 and 25 may be selectable. Moreover, it is good also as an area | region where the number of length and width differs. Further, for example, in the above embodiment, the scanning direction (see FIG. 10) that is continuous from left to right and from top to bottom is illustrated, but the direction may be continuous from top to bottom and from left to right. Further, it may be a direction continuously spiraling from the vicinity of the center toward the outer periphery.

  In the above embodiment, an example in which the display range is gradually changed when performing the scan has been described. However, the scan range may be changed like a frame advance by changing the display range in steps. good. When frame-by-frame advancement is performed, a message or the like for confirming to the user whether or not to move to the next display range may be displayed every time the display is updated. By adopting such a configuration, the user can perform more careful and reliable screening. Furthermore, when performing a scan, the user may be able to select whether the display range is gradually changed or the display range is changed stepwise.

  In the above-described embodiment, an example in which the scan speed is a predetermined speed has been described. However, the scan speed may be variable by designation by the user or automatic setting by the control unit 16. For example, when the enlargement magnification at the time of scanning is high, the scan speed may be set faster, and when the enlargement magnification is low, the scan speed may be set slower.

  Moreover, in the said embodiment, you may perform indicator display other than having mentioned above. For example, when starting the frame scan, when the user sets the magnification, an index indicating how much range is enlarged on the indicator may be displayed as a preview. Further, for example, when the user manually screens, an indicator that can be visually recognized as the screened region may be displayed on the indicator each time the user manually performs the enlarged display.

  In the above-described embodiment, an example in which a user instruction regarding the suspension of the scan and the resumption of the scan is received during the execution of the scan has been described. Further, the scan stop, the scan rewind, the scan fast forward, etc. It is good also as a structure which receives a user instruction | indication. By adopting such a configuration, it is possible to respond flexibly even when the user looks away or misses by screening.

  In the above embodiment, the entire scan may be generated and recorded as a moving image. That is, the scan movie may be generated without displaying on the display unit, or the scan movie may be generated or recorded while displaying on the display unit.

  In the above embodiment, the control unit 16 may be configured to store the result of screening, the date and time of execution of screening, and the like in association with the sample.

  In the observation apparatus of the above embodiment, an example has been described in which each function of the control unit 16 is realized by software using a program. However, these processes may be realized by hardware using an ASIC or the like.

  In the example of the embodiment, the example in which the control unit 16 of the observation apparatus performs overall control of various processes in the observation apparatus has been described. However, the terminal device 22 may perform overall control of various processes in the observation apparatus. . The monitor 19 may be installed outside the observation apparatus. Moreover, it is good also as a structure which performs the various displays performed on the monitor 19 in the said embodiment on the display part of the terminal device 22. FIG. The same applies to the input unit 18, and the input unit 18 may be installed outside the observation apparatus or may be the input unit of the terminal device 22.

  Further, in the example of the above embodiment, the description has been given by taking the observation apparatus provided with the display device of the present invention as an example, but various devices other than the observation device may be configured to include the display device of the present invention. For example, an imaging device such as a culture device, a microscope device, or an electronic camera provided with the display device of the present invention is also effective as a specific aspect of the present invention.

  In addition, the program memorize | stored in the memory | storage part 17 in the said embodiment may be a firmware program updated by version upgrade. That is, the function of the observation apparatus of the present invention may be provided afterwards by updating the firmware of the existing observation apparatus.

  From the above detailed description, features and advantages of the embodiments will become apparent. It is intended that the scope of the claims extend to the features and advantages of the embodiments as described above without departing from the spirit and scope of the right. Further, any person having ordinary knowledge in the technical field should be able to easily come up with any improvements and modifications, and there is no intention to limit the scope of the embodiments having the invention to those described above. It is also possible to use appropriate improvements and equivalents within the scope disclosed in.

DESCRIPTION OF SYMBOLS 15 ... Imaging part, 16 ... Control part, 17 ... Memory | storage part, 18 ... Input part, 19 ... Monitor, 20 ... Communication part, 21 ... Communication line, 22 ... Terminal device

Claims (15)

A recording unit for recording captured images;
A monitor unit for displaying the captured image;
A first display image that reads out a partial image from the captured image under a predetermined scanning condition and includes indicator information indicating an image position of the partial image in the captured image; and a second display image corresponding to the partial image; Display control means for generating and displaying on the monitor unit;
An image display device comprising: The image display device according to claim 1,
Furthermore, it has a stop switch part,
The display control means interrupts reading of the partial image based on a stop signal of the stop switch unit, and can identify a scanned area designated as a partial image read in the past and other unscanned areas. An image display device characterized in that visual information to be added is added to the first display image. The image display device according to claim 2,
Furthermore, it has a restart switch part,
The display control means restarts the update designation of the partial image from a predetermined position in the unscanned area on the first display image based on an input signal from the restart switch unit. Display device. The image display device according to claim 3,
Furthermore, it has an area designation input part,
The display control unit designates the partial image based on an input value from the region designation input unit at the time of the update designation at the first time or at the time of restart by the display control unit. apparatus. The image display device according to any one of claims 1 to 4,
The recording unit records a plurality of partial images as the captured image,
The display control unit generates the first display image by combining the plurality of partial images. An image display device according to claim 1, an observation optical system for observing a sample, and an observation unit that images the sample through the observation optical system and outputs image data of the sample to the image display device. In an observation device having
A stop switch unit for stopping reading of the partial image according to the scan condition;
An observation apparatus comprising: an image analysis unit that analyzes the second display image when reading of the partial image is stopped by the stop switch unit. The observation device according to claim 6,
The image analysis unit
A designation operation unit for designating a desired cell in the second display image;
A measuring unit that measures the morphological information or brightness information of the cell specified by the specifying operation unit;
An observation apparatus comprising: an evaluation unit that evaluates the state of the cell from the shape information or brightness information of the cell measured by the measurement unit. The observation device according to claim 6,
The image analysis unit analyzes a first analysis unit that analyzes whether the sample cell is a desired cell based on the second display image, and the first analysis among the cells of the second display image. A first processing unit that performs marking processing on the cells determined by the unit as the desired cells,
The said display control part performs selective reading of the said partial image containing the said desired cell marked by the said 1st process part. The observation apparatus characterized by the above-mentioned. The observation device according to claim 6,
The image analysis unit analyzes a first analysis unit that analyzes whether the sample cell is a desired cell based on the second display image, and the first analysis among the cells of the second display image. A first processing unit that performs a marking process on a cell that has been determined to be the desired cell by the unit, and whether the cell is the desired cell with respect to the desired cell that has been marked or not subjected to the marking process. An evaluation correction instruction unit for instructing correction of the evaluation of the analysis result of the first analysis unit of whether or not
The observation apparatus, wherein the first processing unit executes a correction instruction of the evaluation correction instruction unit using the second display image displayed on the monitor unit. An image display device according to claim 1, an observation optical system for observing a sample, and an observation unit that images the sample through the observation optical system and outputs image data of the sample to the image display device. In an observation device having
A stop switch unit for stopping reading of the partial image according to the scan condition;
The observation unit comprising a time-lapse observation unit that images the sample at a predetermined time interval via the observation optical system and outputs image data of the sample to the image display device;
A registration unit that registers an imaging point on the sample corresponding to the second display image as an observation point for the time-lapse observation when reading of the partial image is stopped by the stop switch unit. Observation device. The observation device according to claim 10,
An observation apparatus characterized by displaying a map indicating observation points of the time-lapse observation registered by the registration unit in a first display image of the image display apparatus. In a display program for executing processing of an image display device having a recording unit that records a captured image and a monitor unit that displays the captured image on a computer,
Reading a partial image from the captured image under a predetermined scanning condition;
Generating a first display image obtained by adding indicator information indicating an image position of the partial image to the captured image, and a second display image corresponding to the partial image;
Displaying the first display image and the second display image on the monitor unit;
A display program characterized by comprising: In an image display method of an image display device having a recording unit that records a captured image and a monitor unit that displays the captured image,
Reading a partial image from the captured image under a predetermined scanning condition;
Generating a first display image obtained by adding indicator information indicating an image position of the partial image to the captured image, and a second display image corresponding to the partial image;
Displaying the first display image and the second display image on the monitor unit;
An image display method comprising: An image display device according to claim 1, an observation optical system for observing a sample, and an observation unit that images the sample through the observation optical system and outputs image data of the sample to the image display device. In the observation control method of the observation apparatus having,
Stop reading the partial image according to the scan condition,
An observation control method, comprising: analyzing the second display image when reading of the partial image is stopped. The observation control method according to claim 14, wherein
Time-lapse observation of the specimen at predetermined time intervals through the observation optical system,
An observation control method comprising: registering an imaging point on the specimen corresponding to the second display image as an observation point for the time-lapse observation when reading of the partial image is stopped.

Images