JP2012150335A - Confocal microscope system, image processing method and image processing program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a confocal microscope system, image processing method, and an image processing program, which are able to display an image on a surface, as an observation object, without wasting time even if a user specifies a wide range as an observation range.SOLUTION: If the total of the pixels of pixel data within an acquired range specified by a user exceeds the number of displayable pixels of an operation memory 230, the acquisition rate of plural pixel data in unit areas is adjusted. Based on an output signal from a light receiving element 30, the plural pixel data in the unit areas are sequentially obtained. Confocal image data are created based on the plural pixel data, and surface image data are created based on the confocal image data. Surface image data created for the set plural unit areas are connected. Based on the connected surface image data, an image on the surface, as an observation object S, is displayed on a display part 400 by using an operation memory 230.

Description

  The present invention relates to a confocal microscope system, an image processing method, and an image processing program.

  In the confocal microscope, laser light emitted from a laser light source is condensed on a measurement object by an objective lens. The reflected light from the measurement object is collected by the light receiving lens and enters the light receiving element through the pinhole (see, for example, Patent Document 1). The laser beam is scanned two-dimensionally on the surface of the measurement object. Further, the distribution of the amount of light received by the light receiving element is changed by changing the relative distance between the measurement object and the objective lens. A peak in the amount of received light appears when the surface of the measurement object is focused. An ultra-deep image having a very high depth of focus can be obtained based on the peak intensity of the received light amount distribution. Moreover, a height image showing the height distribution of the surface of the measurement object can be obtained based on the peak position of the received light amount distribution.

JP 2008-83601A

  According to the confocal microscope, confocal image data of a certain region of the measurement object placed on the stage is generated, and ultra-depth image data or height image data is generated based on the generated confocal image data. Is done. An ultra-deep image or a height image is displayed on the display unit based on the ultra-deep image data or the height image data. The user can specify the observation range of the measurement object. When the user specifies a wider range than the observable area of the confocal microscope, the ultra-depth image data or height image data of multiple areas of the measurement object is moved by moving the stage of the confocal microscope. Each is generated. Thereafter, the ultra-depth image data or height image data of a plurality of regions are connected to display a wide-range ultra-depth image or height image of the measurement object on the display unit using the work memory.

  However, when the user designates a wider range as the observation range of the measurement object, the capacity of the ultra-depth image data or the height image data in a plurality of areas exceeds the usable capacity of the work memory. In such a case, the ultra-depth image data or height image data before or after the connection is thinned out to reduce the capacity of the image data. It is necessary to enable processing for display using the memory for the image.

  On the other hand, the confocal microscope scans light two-dimensionally while changing the relative distance between the measurement object and the objective lens, and generates ultra-depth image data and height image data of one region. . Therefore, when the user has specified a wider range as the observation range of the measurement object and it is necessary to link the image data of a plurality of regions, it takes a very long time to generate the linked image data. .

  An object of the present invention is to provide a confocal microscope system, an image processing method, and an image processing program capable of displaying an image of the surface of an observation object without wasting time even when a user designates a wide range as an observation range. Is to provide.

  (1) A confocal microscope system according to a first aspect of the present invention is a confocal microscope system that displays an image of the surface of an observation object, and that receives light in a light receiving element and a unit region set on the surface of the observation object. The confocal optical system that guides the light irradiated to the unit area to the light receiving element, and the light irradiation by the confocal optical system in the unit area at a plurality of positions along the direction perpendicular to the unit area. Control the relative distance changing unit that changes the relative distance between the confocal optical system and the observation object, the confocal optical system and the relative distance changing unit, and based on the output signal of the light receiving element. A control unit that sequentially acquires a plurality of pixel data corresponding to a plurality of pixels arranged in the first and second directions orthogonal to each other in the unit region, and the unit region based on the plurality of pixel data acquired by the control unit Confocal image de An image data generation unit that generates surface image data for displaying an image of the surface of the observation target based on the confocal image data, and an observation range instruction reception unit that receives an instruction of the observation range from the user And an observation range setting unit that sets a plurality of unit areas as an observation range based on an instruction received by the observation range instruction reception unit, and a plurality of unit areas set by the observation range setting unit are generated by the image data generation unit The image data processing unit that connects the surface image data that has been made, and the number of pixels included in the plurality of unit areas set by the observation range setting unit and a preset threshold value are compared, and each based on the comparison result A data acquisition amount adjusting unit that adjusts the number of pixel data acquired by the control unit for a unit area, and a plurality of units connected by the image data processing unit. In which a display unit for displaying an image of the surface of the observation object based on the surface image data of the area.

  In this confocal microscope system, light is irradiated to a unit region set on the surface of an observation object, and the light irradiated to the unit region is guided to a light receiving element. Here, the relative distance between the confocal optical system and the observation object changes so that light is emitted from the confocal optical system in the unit area at a plurality of positions along the direction perpendicular to the unit area. Is done. Based on the output signal of the light receiving element, a plurality of pixel data corresponding to the plurality of pixels arranged in the first and second directions orthogonal to each other in the unit region are sequentially acquired. Confocal image data is generated based on the acquired plurality of pixel data, and surface image data for displaying an image of the surface of the observation object is generated based on the confocal image data.

  When an instruction for the observation range is received from the user, a plurality of unit areas are set as the observation range based on the instruction. The surface image data generated for a plurality of set unit areas is connected. An image of the surface of the observation object is displayed based on the surface image data of the plurality of unit regions that are connected.

  In this case, the number of pixels included in the plurality of set unit areas is compared with a preset threshold value, and the number of pixel data acquired for each unit area is adjusted based on the comparison result. Therefore, even when the user designates a wide range as the observation range, it is possible to prevent the amount of the connected surface image data from exceeding the capacity of the working memory that can be used for display. Thereby, the image of the surface of the observation object can be displayed regardless of the observation range. In addition, the time required to acquire pixel data can be shortened.

  Further, after the surface image data for the plurality of unit areas is generated, the user can check that the amount of the surface image data for the plurality of unit areas does not exceed the capacity of the working memory that can be used for display. There is no need to thin out image data. Therefore, it is not necessary to spend a great deal of time in generating the amount of surface image data that cannot be displayed. As a result, an image of the surface of the observation object can be displayed without wasting time.

  (2) The confocal microscope system further includes a non-confocal image acquisition unit that acquires a non-confocal image of the observation object, and the observation range instruction reception unit is a non-confocal image acquired by the non-confocal image acquisition unit An image may be displayed on the display unit, and an instruction for an observation range by the user may be received on the displayed non-confocal image.

  In this case, a non-confocal image of the observation object is acquired, and the acquired non-confocal image is displayed on the display unit. An instruction of the observation range by the user is received on the displayed non-confocal image. Thereby, the user can easily indicate the observation range.

  (3) The observation range instruction receiving unit divides the displayed non-confocal image into a plurality of regions that can be designated by the user, and excludes the region designated as an acquisition exclusion target by the user from the observation range. Good.

  In this case, the displayed non-confocal image is divided into a plurality of areas that can be designated by the user. The user can designate any of a plurality of areas as an acquisition exclusion target. The area designated as the acquisition exclusion target is excluded from the observation range. Therefore, it is not necessary to scan the laser beam in the designated area. Thereby, the time required for obtaining pixel data can be shortened.

  (4) The confocal microscope system supports the object to be observed and is provided so as to be movable in the first and second directions, and an instruction to enlarge the range of the non-confocal image displayed on the display unit An enlargement instruction accepting unit that accepts an image, and a display range enlarging unit that enlarges the range of the non-confocal image displayed on the display unit by moving the support unit when an enlargement instruction is accepted by the enlargement instruction accepting unit; May be further provided.

  In this case, the observation object is supported by the support part. When the user instructs to expand the range of the non-confocal image to be displayed, the range of the non-confocal image to be displayed is expanded by the support unit moving in the first and second directions. Thereby, the user can specify the observation range on the non-confocal image of the enlarged range. As a result, pixel data within a wider observation range can be acquired.

  (5) An image processing method according to a second aspect of the present invention is an image processing method for displaying an image of the surface of an observation object, and applies light to a unit area set on the surface of the observation object by a confocal optical system. Irradiating and guiding the light irradiated to the unit area to the light receiving element, and irradiating the light by the confocal optical system in the unit area at a plurality of positions along the direction perpendicular to the unit area Corresponding to a step of changing the relative distance between the confocal optical system and the observation object, and a plurality of pixels arranged in the first and second directions orthogonal to each other in the unit area based on the output signal of the light receiving element. Sequentially acquiring a plurality of pixel data, generating confocal image data of a unit region based on the acquired plurality of pixel data, and displaying an image of the surface of the observation object based on the confocal image data of A step of generating plane image data; a step of receiving an instruction of an observation range by a user; a step of setting a plurality of unit areas as an observation range based on the received instruction; and generation of the set unit areas Pixel data acquired for each unit region based on the comparison result by comparing the step of connecting the surface image data, the number of pixels included in the set plurality of unit regions and a preset threshold value And a step of displaying an image of the surface of the object to be observed based on the surface image data of the plurality of connected unit regions.

  In this image processing method, light is irradiated to a unit region set on the surface of the observation object, and the light irradiated to the unit region is guided to the light receiving element. Here, the relative distance between the confocal optical system and the observation object changes so that light is emitted from the confocal optical system in the unit area at a plurality of positions along the direction perpendicular to the unit area. Is done. Based on the output signal of the light receiving element, a plurality of pixel data corresponding to the plurality of pixels arranged in the first and second directions orthogonal to each other in the unit region are sequentially acquired. Confocal image data is generated based on the acquired plurality of pixel data, and surface image data for displaying an image of the surface of the observation object is generated based on the confocal image data.

  When an instruction for the observation range is received from the user, a plurality of unit areas are set as the observation range based on the instruction. The surface image data generated for a plurality of set unit areas is connected. An image of the surface of the observation object is displayed based on the surface image data of the plurality of unit regions that are connected.

  In this case, the number of pixels included in the plurality of set unit areas is compared with a preset threshold value, and the number of pixel data acquired for each unit area is adjusted based on the comparison result. Therefore, even when the user designates a wide range as the observation range, it is possible to prevent the amount of the connected surface image data from exceeding the capacity of the working memory that can be used for display. Thereby, the image of the surface of the observation object can be displayed regardless of the observation range. In addition, the time required to acquire pixel data can be shortened.

  Further, after the surface image data for the plurality of unit areas is generated, the user can check that the amount of the surface image data for the plurality of unit areas does not exceed the capacity of the working memory that can be used for display. There is no need to thin out image data. Therefore, it is not necessary to spend a great deal of time in generating the amount of surface image data that cannot be displayed. As a result, an image of the surface of the observation object can be displayed without wasting time.

  (6) An image processing program according to a third invention is an image processing program for causing a processing device to execute image processing for displaying an image of the surface of an observation object, and is applied to the surface of the observation object by a confocal optical system. A process for irradiating the set unit area with light and guiding the laser light irradiated to the unit area to the light receiving element, and a confocal optical system in the unit area at a plurality of positions along a direction perpendicular to the unit area First and second orthogonal to each other in the unit area based on the process of changing the relative distance between the confocal optical system and the observation object so that the light is irradiated by the light source and the output signal of the light receiving element A process of sequentially acquiring a plurality of pixel data corresponding to a plurality of pixels arranged in a direction, generating confocal image data of a unit region based on the acquired plurality of pixel data, and viewing based on the confocal image data Processing for generating surface image data for displaying an image of the surface of an object, processing for receiving an instruction for an observation range by a user, and processing for setting a plurality of unit areas as an observation range based on the received instructions And the process of connecting the surface image data generated for the set unit areas, the number of pixels included in the set unit areas and a preset threshold value are compared, and a comparison result is obtained. Processing for adjusting the number of pixel data acquired for each unit region based on the processing unit, and processing for displaying an image of the surface of the observation object based on the surface image data of a plurality of unit regions connected to the processing device. To be executed.

  In this image processing program, light is irradiated to a unit area set on the surface of the observation object, and the light irradiated to the unit area is guided to the light receiving element. Here, the relative distance between the confocal optical system and the observation object changes so that light is emitted from the confocal optical system in the unit area at a plurality of positions along the direction perpendicular to the unit area. Is done. Based on the output signal of the light receiving element, a plurality of pixel data corresponding to the plurality of pixels arranged in the first and second directions orthogonal to each other in the unit region are sequentially acquired. Confocal image data is generated based on the acquired plurality of pixel data, and surface image data for displaying an image of the surface of the observation object is generated based on the confocal image data.

  When an instruction for the observation range is received from the user, a plurality of unit areas are set as the observation range based on the instruction. The surface image data generated for a plurality of set unit areas is connected. An image of the surface of the observation object is displayed based on the surface image data of the plurality of unit regions that are connected.

  In this case, the number of pixels included in the plurality of set unit areas is compared with a preset threshold value, and the number of pixel data acquired for each unit area is adjusted based on the comparison result. Therefore, even when the user designates a wide range as the observation range, it is possible to prevent the amount of the connected surface image data from exceeding the capacity of the working memory that can be used for display. Thereby, the image of the surface of the observation object can be displayed regardless of the observation range. In addition, the time required to acquire pixel data can be shortened.

  Further, after the surface image data for the plurality of unit areas is generated, the user can check that the amount of the surface image data for the plurality of unit areas does not exceed the capacity of the working memory that can be used for display. There is no need to thin out image data. Therefore, it is not necessary to spend a great deal of time in generating the amount of surface image data that cannot be displayed. As a result, an image of the surface of the observation object can be displayed without wasting time.

  According to the present invention, even when the user designates a wide range as the observation range, it is possible to display an image of the surface of the observation object without wasting time.

It is a block diagram which shows the structure of the confocal microscope system which concerns on one embodiment of this invention. It is a figure for defining an X direction, a Y direction, and a Z direction. It is a figure which shows the relationship between the position of the Z direction of an observation target object, and the light reception intensity | strength of a light receiving element in one pixel. It is a figure which shows the example of a display of a display part at the time of the setting of the acquisition range of pixel data. It is a figure which shows the acquisition range setting screen for performing the 1st instruction | indication method of the acquisition range of pixel data. It is a figure which shows the example of a display of a display part at the time of execution of the 1st instruction | indication method of the acquisition range of pixel data. It is a figure which shows the acquisition range setting screen for performing the 2nd instruction | indication method of the acquisition range of pixel data. It is a figure which shows the example of a display of a display part at the time of execution of the 2nd instruction | indication method of the acquisition range of pixel data. It is a figure which shows the acquisition range setting screen for performing the 3rd instruction | indication method of the acquisition range of pixel data. It is a figure which shows the example of a display of a display part at the time of execution of the 3rd instruction | indication method of the acquisition range of pixel data. It is a figure which shows the example of a display of a display part in the case of acquiring the pixel data of the range wider than the imaging | photography area | region of a color CCD camera. It is a figure which shows a navigation display screen. It is a figure which shows a navigation display screen. It is a figure which shows a navigation display screen. It is a figure which shows a navigation display screen. It is a figure which shows the confocal image of the observation target object S contained in a some imaging | photography area | region. It is a flowchart which shows the image processing method of the confocal image data in a confocal microscope system. It is a flowchart which shows the image processing method of the confocal image data in a confocal microscope system. It is a figure which shows the confocal image data produced | generated when the acquisition rate of pixel data is 1. It is a figure which shows the confocal image data produced | generated when the acquisition rate of pixel data is 1/4. It is a figure which shows the confocal image data produced | generated when the acquisition rate of pixel data is 1/16. It is a figure which shows the ultra-deep image of the observation target object when the acquisition rate of pixel data is 1. It is a figure which shows the ultra-deep image of the observation target object when the acquisition rate of pixel data is 1/4. It is a figure which shows the ultra-deep image of the observation target object when the acquisition rate of pixel data is 1/16. It is a timing chart of parallel processing of image processing. It is a figure for demonstrating the relationship between the position of the Z direction of the objective lens about one pixel, and the value of effective pixel data. It is a figure for demonstrating the setting method of the upper limit position and the minimum position of the Z direction of an objective lens at the time of the production | generation of height image data and ultra-deep image data. It is a figure for demonstrating the peak position search process of an evaluation value.

  Hereinafter, a confocal microscope system according to an embodiment of the present invention will be described with reference to the drawings.

(1) Basic Configuration of Confocal Microscope System FIG. 1 is a block diagram showing a configuration of a confocal microscope system 500 according to an embodiment of the present invention. As shown in FIG. 1, the confocal microscope system 500 includes a measurement unit 100, a PC (personal computer) 200, a control unit 300, and a display unit 400. The measurement unit 100 includes a laser light source 10, an XY scan optical system 20, a light receiving element 30, an illumination white light source 40, a color CCD (charge coupled device) camera 50, and a stage 60. An observation object S is placed on the stage 60.

  The laser light source 10 is a semiconductor laser, for example. The laser light emitted from the laser light source 10 is converted into parallel light by the lens 1, passes through the half mirror 4, and enters the XY scan optical system 20. Instead of the laser light source 10, another light source such as a mercury lamp may be used. In this case, a band pass filter is disposed between the light source such as a mercury lamp and the XY scan optical system 20. Light emitted from a light source such as a mercury lamp passes through a band-pass filter, becomes monochromatic light, and enters the XY scan optical system 20.

  The XY scan optical system 20 is, for example, a galvanometer mirror. The XY scan optical system 20 has a function of scanning a laser beam in the X direction and the Y direction on the surface of the observation object S on the stage 60. The definitions of the X direction, the Y direction, and the Z direction will be described later. The laser beam scanned by the XY scanning optical system 20 is reflected by the half mirror 5, then passes through the half mirror 6, and is focused on the observation object S on the stage 60 by the objective lens 3. Instead of the half mirrors 4 to 6, a polarization beam splitter may be used.

  The laser light reflected by the observation object S passes through the objective lens 3 and the half mirror 6, then is reflected by the half mirror 5, and passes through the XY scan optical system 20. The laser beam that has passed through the XY scan optical system 20 is reflected by the half mirror 4, collected by the lens 2, and transmitted through the pinhole of the pinhole member 7 and the ND (Neutral Density) filter 8 to receive the light receiving element 30. Is incident on. As described above, the reflection type confocal microscope system 500 is used in the present embodiment, but when the observation object S is a transparent body such as a cell, a transmission type confocal microscope system is used. Also good.

  The pinhole of the pinhole member 7 is disposed at the focal position of the lens 2. The ND filter 8 is used to attenuate the intensity of laser light incident on the light receiving element 30. Therefore, when the intensity of the laser beam is sufficiently attenuated, the ND filter 8 may not be provided.

  In the present embodiment, the light receiving element 30 is a photomultiplier tube. A photodiode and an amplifier may be used as the light receiving element 30. The light receiving element 30 outputs an analog electric signal (hereinafter referred to as a light receiving signal) corresponding to the amount of received light. The control unit 300 includes two A / D converters (analog / digital converter), a FIFO (First In First Out) memory, and a CPU (Central Processing Unit). The light reception signal output from the light receiving element 30 is sampled at a constant sampling period by one A / D converter of the control unit 300 and converted into a digital signal. Digital signals output from the A / D converter are sequentially stored in the FIFO memory. The digital signals stored in the FIFO memory are sequentially transferred to the PC 200 as pixel data.

  The illumination white light source 40 is, for example, a halogen lamp or a white LED (light emitting diode). White light generated by the illuminating white light source 40 is reflected by the half mirror 6 and then condensed by the objective lens 3 onto the observation object S on the stage 60.

  White light reflected by the observation object S passes through the objective lens 3, the half mirror 6 and the half mirror 5 and enters the color CCD camera 50. An imaging element such as a CMOS (complementary metal oxide semiconductor) image sensor may be used in place of the color CCD camera 50. The color CCD camera 50 outputs an electrical signal corresponding to the amount of received light. The output signal of the color CCD camera 50 is sampled at a constant sampling period by another A / D converter of the control unit 300 and converted into a digital signal. Digital signals output from the A / D converter are sequentially transferred to the PC 200 as camera data.

  The control unit 300 supplies pixel data and camera data to the PC 200 and controls the light receiving sensitivity (gain) of the light receiving element 30 and the color CCD camera 50 based on a command from the PC 200. Further, the control unit 300 controls the XY scanning optical system 20 based on a command from the PC 200 to scan the laser light on the observation object S in the X direction and the Y direction.

  The objective lens 3 is provided so as to be movable in the Z direction by the lens driving unit 63. The control unit 300 can move the objective lens 3 in the Z direction by controlling the lens driving unit 63 based on a command from the PC 200. Thereby, the position of the observation target S relative to the objective lens 3 in the Z direction can be changed.

  The PC 200 includes a CPU (Central Processing Unit) 210, a ROM (Read Only Memory) 220, a work memory 230, and a storage device 240. The ROM 220 stores a system program. The working memory 230 includes a RAM (Random Access Memory), and is used for processing various data. The storage device 240 is composed of a hard disk or the like. The storage device 240 stores an image processing program and is used for storing various data such as pixel data and camera data given from the control unit 300. Details of the image processing program will be described later.

  The CPU 210 generates image data based on the pixel data given from the control unit 300. Hereinafter, the image data generated based on the pixel data is referred to as confocal image data. An image displayed based on the confocal image data is referred to as a confocal image.

  The CPU 210 generates image data based on the camera data given from the control unit 300. Hereinafter, the image data generated based on the camera data is referred to as camera image data. An image displayed based on the camera image data is called a camera image.

  The CPU 210 performs various processes on the generated confocal image data and camera image data using the work memory 230, and displays a confocal image based on the confocal image data and a camera image based on the camera image data on the display unit 400. Let Further, the CPU 210 gives a driving pulse to the stage driving unit 62 described later.

  The display unit 400 is configured by, for example, a liquid crystal display panel or an organic EL (electroluminescence) panel.

  The stage 60 has an X direction moving mechanism, a Y direction moving mechanism, and a Z direction moving mechanism. Stepping motors are used for the X direction moving mechanism, the Y direction moving mechanism, and the Z direction moving mechanism.

  The X direction moving mechanism, the Y direction moving mechanism, and the Z direction moving mechanism of the stage 60 are driven by a stage operation unit 61 and a stage driving unit 62. The user can move the stage 60 in the X direction, the Y direction, and the Z direction relative to the objective lens 3 by manually operating the stage operation unit 61.

  The stage drive unit 62 moves the stage 60 relative to the objective lens 3 in the X direction, the Y direction, or the Z direction by supplying a current to the stepping motor of the stage 60 based on the drive pulse given from the PC 200. be able to.

(2) Confocal Image, Ultra-Deep Image, and Height Image FIG. 2 is a diagram for defining the X direction, the Y direction, and the Z direction. As shown in FIG. 2, the observation object S is irradiated with the laser light condensed by the objective lens 3. In the present embodiment, the direction of the optical axis of the objective lens 3 is defined as the Z direction. Further, two directions orthogonal to each other on the surface orthogonal to the Z direction are defined as an X direction and a Y direction, respectively. The X, Y, and Z directions are indicated by arrows X, Y, and Z, respectively.

  The relative position of the surface of the observation object S with respect to the objective lens 3 in the Z direction is referred to as the position of the observation object S in the Z direction. The confocal image data is generated for each unit area. The unit area is determined by the magnification of the objective lens 3.

  While the position of the observation object S in the Z direction is constant, the XY scanning optical system 20 scans the laser beam in the X direction at the end in the Y direction in the unit region. When the scanning in the X direction is completed, the laser beam is shifted by the XY scanning optical system 20 at a constant interval in the Y direction. In this state, the laser beam is scanned in the X direction. By repeating the scanning of the laser beam in the X direction and the shifting in the Y direction within the unit region, the scanning of the unit region in the X direction and the Y direction is completed. Next, the objective lens 3 is moved in the Z direction. Thereby, the scanning in the X direction and the Y direction of the unit area is performed in a constant state where the position of the objective lens 3 in the Z direction is different from the previous time. The unit region is scanned in the X and Y directions at a plurality of positions in the Z direction of the observation object S.

  Confocal image data is generated by scanning in the X and Y directions for each position of the observation object S in the Z direction. As a result, a plurality of confocal image data having different positions in the Z direction within the unit region is generated.

  Here, the number of pixels in the X direction of the confocal image data is determined by the scanning speed of the laser beam in the X direction by the XY scanning optical system 20 and the sampling period of the control unit 300. The number of samples in one X-direction scan (one scan line) is the number of pixels in the X direction. Further, the number of pixels in the Y direction of the confocal image data in the unit area is determined by the amount of shift in the Y direction of the laser beam by the XY scan optical system 20 at the end of scanning in the X direction. The number of scanning lines in the Y direction is the number of pixels in the Y direction. Furthermore, the number of confocal image data in the unit area is determined by the number of movements of the observation object S in the Z direction. Based on the plurality of confocal image data in the unit area, ultra-depth image data and height image data are generated by a method described later.

  In the example of FIG. 2, first, a plurality of confocal image data of the observation object S in the unit region s1 is generated at the initial position of the stage 60, and ultra-depth image data and height image data of the unit region s1 are generated. Is done. Subsequently, the stage 60 is sequentially moved to generate a plurality of confocal image data of the observation object S in the unit regions s2 to s4 and to generate ultra-depth image data and height image data of the unit regions s2 to s4. Is done. In this case, the unit regions s1 to s4 may be set so that adjacent unit regions partially overlap each other. Thereby, by performing pattern matching, the ultra-depth image data and the height image data of the plurality of unit regions s1 to s4 can be connected with high accuracy. In particular, when the total area of the plurality of unit regions is larger than the pixel data acquisition range described later, a portion corresponding to the area of the portion that protrudes from the acquisition range is set as an overlapping portion.

  FIG. 3 is a diagram showing the relationship between the position of the observation object S in the Z direction and the light receiving intensity of the light receiving element 30 in one pixel. As shown in FIG. 1, the pinhole of the pinhole member 7 is disposed at the focal position of the lens 2. Therefore, when the surface of the observation object S is at the focal position of the objective lens 3, the laser light reflected by the observation object S is condensed at the pinhole position of the pinhole member 7. Thereby, most of the laser beam reflected by the observation object S passes through the pinhole of the pinhole member 7 and enters the light receiving element 30. In this case, the light receiving intensity of the light receiving element 30 is maximized. As a result, the voltage value of the light reception signal output from the light receiving element 30 is maximized.

  On the other hand, when the observation object S is at a position out of the focal position of the objective lens 3, the laser light reflected by the observation object S is condensed at a position before or after the pinhole of the pinhole member 7. . Thereby, most of the laser light reflected by the observation object S is blocked by the portion around the pinhole of the pinhole member 7, and the light receiving intensity of the light receiving element 30 is reduced. As a result, the voltage value of the light reception signal output from the light receiving element 30 decreases.

  Thus, a peak appears in the light reception intensity distribution of the light receiving element 30 in a state where the surface of the observation object S is at the focal position of the objective lens 3. The received light intensity distribution in the Z direction is obtained for each pixel from a plurality of confocal image data of each unit region. Thereby, the peak position and peak intensity (peak received light intensity) of the received light intensity distribution are obtained for each pixel.

  Data representing peak positions in the Z direction for a plurality of pixels in each unit area is referred to as height image data, and an image displayed based on the height image data is referred to as a height image. The height image represents the surface shape of the observation object S. In addition, data representing the peak intensity for a plurality of pixels in each unit area is referred to as ultra-deep image data, and an image represented based on the ultra-depth image data is referred to as an ultra-depth image. The ultra-deep image is an image obtained in a state where all parts of the surface of the observation object S are in focus. The PC 200 generates a plurality of confocal image data of the unit region based on the plurality of pixel data of the unit region given from the control unit 300, and the height image data and the super image of the unit region based on the plurality of confocal image data. Generate depth image data. Hereinafter, the height image data and the ultradeep image data are collectively referred to as surface image data, and the height image and the ultradeep image are collectively referred to as a surface image.

(3) Setting of Acquisition Range (Observation Range) of Pixel Data FIG. 4 is a diagram illustrating a display example of the display unit 400 before setting of the acquisition range of pixel data. As shown in FIG. 4, an image display area 410 and a condition setting area 420 are displayed on the screen of the display unit 400. In the image display area 410, a confocal image based on the confocal image data or a camera image based on the camera image data is displayed. In the condition setting area 420, a range setting button 421, an acquisition start button 422, and a navigation display button 423 are displayed.

  The user places the observation object S on the stage 60 of the confocal microscope system 500 of FIG. Thereby, the control part 300 gives camera data to PC200 sequentially. The CPU 210 of the PC 200 generates camera image data based on the camera data given by the control unit 300 and displays the camera image of the observation object S in the image display area 410 of the display unit 400. In this state, the user can easily specify the acquisition range of the pixel data by various methods described below. In this case, the user can change the range of the camera image of the observation object S displayed in the image display area 410 by moving the stage 60 in the X direction or the Y direction.

(3-1) First Instructing Method for Obtaining Range of Pixel Data FIG. 5 is a diagram illustrating an obtaining range setting screen for executing the first instructing method for the obtaining range of pixel data. FIG. 6 is a diagram illustrating a display example of the display unit 400 at the time of executing the first instruction method of the acquisition range of pixel data.

  The user operates the range setting button 421 in the condition setting area 420 of FIG. 4 using a pointing device such as a mouse connected to the PC 200. Accordingly, the acquisition range setting screen 430a of FIG. 5 is displayed on the display unit 400 so as to be aligned with the image display area 410. The acquisition range setting screen 430a includes a designation method setting column 431, a designation range setting column 432, a setting value display column 433, a next button 434a, and an end button 434b. The designation method setting field 431 includes check boxes 431a, 431b, and 431c. The designated range setting field 432 includes an upper end setting unit 432a, a left end setting unit 432b, a right end setting unit 432c, and a lower end setting unit 432d.

  When the user designates the check box 431b, the display unit 400 displays an acquisition range setting screen 430b of FIG. 7 described later instead of the acquisition range setting screen 430a. This makes it possible to execute a second instruction method for the acquisition range of pixel data, which will be described later. When the user specifies the check box 431c, the display unit 400 displays an acquisition range setting screen 430c of FIG. 9 described later instead of the acquisition range setting screen 430a. This makes it possible to execute a third instruction method for the acquisition range of pixel data, which will be described later.

  In a state where the check box 431a is designated, as shown in FIG. 6, the user designates a desired point p1 on the image display area 410 of the display unit 400. Next, the user operates the upper end setting unit 432a and the left end setting unit 432b of FIG. Thus, the CPU 210 determines the point p1 as the start point of the pixel data acquisition range. In this example, the CPU 210 recognizes the point p1 as the upper end and the left end of the pixel data acquisition range.

  Thereafter, as shown in FIG. 6, the user designates a desired point p2 different from the point p1 on the image display area 410 of the display unit 400. Next, the user operates the right end setting unit 432c and the lower end setting unit 432d of FIG. As a result, the CPU 210 determines the point p2 as the end point of the pixel data acquisition range. In this example, the CPU 210 recognizes the point p2 as the right end and the lower end of the pixel data acquisition range.

  Subsequently, the user operates the next button 434a in FIG. Thereby, as shown in FIG. 6, the CPU 210 displays in the image display area 410 a rectangular acquisition range display frame R having the points p <b> 1 and p <b> 2 as diagonal vertices. In FIG. 6, the acquisition range display frame R is indicated by a one-dot chain line. The pixel data portion corresponding to the camera image inside the acquisition range display frame R is set as the acquisition range. In the set value display field 433 of the acquisition range setting screen 430a, the coordinates of the start and end points on the image display area 410 and the number of unit areas constituting the acquisition range display frame R are displayed.

(3-2) Second Indication Method of Pixel Data Acquisition Range FIG. 7 is a diagram illustrating an acquisition range setting screen based on the second instruction method of the acquisition range of pixel data. FIG. 8 is a diagram illustrating a display example of the display unit 400 at the time of executing the second instruction method of the acquisition range of pixel data.

  The user designates the check box 431b on the acquisition range setting screen 430a in FIG. 5 or the acquisition range setting screen 430c in FIG. Accordingly, the acquisition range setting screen 430b of FIG. 7 is displayed in place of the acquisition range setting screens 430a and 430c so as to be aligned with the image display area 410 on the display unit 400. The acquisition range setting screen 430b has the same configuration as the acquisition range setting screen 430a except that it includes length designation units 432e and 432f that can input numerical values instead of the right end setting unit 432c and the lower end setting unit 432d of FIG. Have

  When the user designates the check box 431a, the acquisition range setting screen 430a of FIG. 5 is displayed on the display unit 400 instead of the acquisition range setting screen 430b. This makes it possible to execute the first instruction method for the acquisition range of the pixel data described above. When the user specifies the check box 431c, the display unit 400 displays an acquisition range setting screen 430c of FIG. 9 described later instead of the acquisition range setting screen 430b. This makes it possible to execute a third instruction method for the acquisition range of pixel data, which will be described later.

  With the check box 431b designated, the user designates a desired point p1 on the image display area 410 of the display unit 400 as shown in FIG. Next, the user operates the upper end setting unit 432a and the left end setting unit 432b of FIG. Thus, the CPU 210 determines the point p1 as the start point of the pixel data acquisition range. In this example, the CPU 210 recognizes the point p1 as the upper end and the left end of the pixel data acquisition range.

  Thereafter, the user inputs a numerical value into the length designation section 432e of the designated range setting field 432. As a result, the CPU 210 determines the length of the line segment Lx in the X direction starting from the point p1 on the image display area 410 as shown in FIG. In addition, the user inputs a numerical value into the length designation part 432 f of the designated range setting field 432. As a result, the CPU 210 determines the length of the line segment Ly in the Y direction starting from the point p1 on the image display area 410, as shown in FIG.

  Subsequently, the user operates the next button 434a in FIG. As a result, as shown in FIG. 8, the CPU 210 displays in the image display area 410 a rectangular acquisition range display frame R having the line segments Lx, Ly as two orthogonal sides. In FIG. 8, the acquisition range display frame R is indicated by a one-dot chain line. The pixel data portion corresponding to the camera image inside the acquisition range display frame R is set as the acquisition range. In the set value display field 433 of the acquisition range setting screen 430b, the coordinates of the start and end points on the image display area 410 and the number of unit areas constituting the acquisition range display frame R are displayed. In this example, the right end and lower end point of the acquisition range display frame R is the end point.

(3-3) Third Instruction Method for Obtaining Range of Pixel Data FIG. 9 is a diagram showing an obtaining range setting screen based on a third instruction method for the obtaining range of pixel data. FIG. 10 is a diagram illustrating a display example of the display unit 400 at the time of execution of the third instruction method of the acquisition range of pixel data.

  The user designates the check box 431c on the acquisition range setting screen 430a in FIG. 5 or the acquisition range setting screen 430b in FIG. Accordingly, the acquisition range setting screen 430c of FIG. 9 is displayed in place of the acquisition range setting screens 430a and 430b so as to be aligned with the image display area 410 on the display unit 400. The acquisition range setting screen 430c is the same as the acquisition range setting screen 430a except that the acquisition range setting screen 430c includes unit area number designation units 432g and 432h that can input numerical values instead of the right end setting unit 432c and the lower end setting unit 432d of FIG. It has a configuration.

  When the user specifies the check box 431a, the display unit 400 displays the acquisition range setting screen 430a in FIG. 5 instead of the acquisition range setting screen 430c. This makes it possible to execute the first instruction method for the acquisition range of the pixel data described above. When the user designates the check box 431b, the display unit 400 displays the acquisition range setting screen 430b of FIG. 7 instead of the acquisition range setting screen 430c. This makes it possible to execute a second instruction method for the acquisition range of pixel data, which will be described later.

  With the check box 431c designated, the user designates a desired point p1 on the image display area 410 of the display unit 400, as shown in FIG. Next, the user operates the upper end setting unit 432a and the left end setting unit 432b of FIG. Thereby, the CPU 210 determines the unit area including the point p1 as the start area. In this example, the CPU 210 recognizes the point p1 as the upper end and the left end of the pixel data acquisition range. In this case, the CPU 210 determines a point at the upper end and the left end of the start area as the start point.

  Thereafter, the user inputs a numerical value into the unit area number designation section 432g of the designated range setting field 432. Thereby, as shown in FIG. 10, the CPU 210 determines the acquisition number Nx of unit areas in the X direction including the start area on the image display area 410. In addition, the user inputs a numerical value into the unit area number designation section 432 h of the designated range setting field 432. Thereby, as shown in FIG. 10, the CPU 210 determines the acquisition number Ny of the unit area in the Y direction including the start area on the image display area 410.

  Subsequently, the user operates the next button 434a in FIG. As a result, as shown in FIG. 10, the CPU 210 has a rectangular shape so as to include the unit area of the acquisition number Nx in the X direction including the start area and include the unit area of the acquisition number Ny in the Y direction including the start area. The acquisition range display frame R is displayed in the image display area 410. In FIG. 10, the acquisition range display frame R is indicated by a bold line. The pixel data portion corresponding to the camera image inside the acquisition range display frame R is set as the acquisition range. In the set value display field 433 of the acquisition range setting screen 430c, the coordinates of the start point and end point on the image display area 410 and the number of unit areas constituting the acquisition range display frame R are displayed. In this example, the right end and lower end point of the acquisition range display frame R is the end point.

(4) Method for Enlarging Pixel Data Acquisition Range The shooting range of the color CCD camera 50 with the stage 60 in FIG. 1 fixed is called a shooting area. The user can acquire pixel data corresponding to a wider range than the photographing region of the color CCD camera 50 by the following operation.

  FIG. 11 is a diagram illustrating a display example of the display unit 400 when acquiring pixel data in a wider range than the imaging region of the color CCD camera 50. 12 to 15 are diagrams showing navigation display screens. The user operates the shooting range setting button 423 in the condition setting area 420 of FIG. 11 using a pointing device such as a mouse connected to the PC 200. As a result, the navigation display screen 440 is displayed on the display unit 400 so as to be aligned with the image display area 410.

  As shown in FIG. 12, the navigation display screen 440 includes a camera image display area 441, a shooting range determination button 442, and a shooting range enlargement button 443. In the camera image display area 441, a camera image based on the camera image data is displayed. In the example of FIG. 12, the camera image display area 441 displays a camera image of the observation object S included in one shooting area r1.

  In this state, the user operates the shooting range determination button 442, and the first, second, or third instruction method for the above-described pixel data acquisition range in the shooting area r1 displayed in the camera image display area 441 is performed. By executing, it is possible to indicate the acquisition range of the pixel data.

  On the other hand, when the user operates the shooting range enlargement button 443 on the navigation display screen 440 in FIG. 12, the CPU 210 causes the stage drive unit to take a picture around the shooting area r1 with the color CCD camera 50 in FIG. The stage 60 is moved by controlling 62. Thereby, as shown in FIG. 13, the CPU 210 acquires the camera image data of the shooting areas r2 to r9 around the shooting area r1, and based on the camera image data of the shooting areas r1 to r9, A camera image is displayed in the camera image display area 441.

  In this state, the user operates the shooting range determination button 442, and the first, second, or third instruction of the pixel data acquisition range in the shooting areas r1 to r9 displayed in the camera image display area 441 is displayed. By executing the method, the acquisition range of the pixel data can be indicated. When acquiring the entire pixel data of the observation object S displayed in the camera image display area 441, the user can obtain a rectangular area including the observation object S (in the example of FIG. 13, the imaging areas r1 to r1). The acquisition range display frame R can be designated within r9).

  On the other hand, when the user operates the shooting range expansion button 443 on the navigation display screen 440 in FIG. 13, the CPU 210 causes the stage around the shooting areas r1 to r9 to be shot by the color CCD camera 50 in FIG. The stage 60 is moved by controlling the drive unit 62. Thereby, as shown in FIG. 14, the CPU 210 acquires the camera image data of the shooting areas shooting areas r10 to r25 around the shooting areas r1 to r9 displayed in the camera image display area 441, and the shooting areas r1 to r1. Based on the camera image data of r25, the camera images of the shooting areas r1 to r25 are displayed in the camera image display area 441.

  In this state, the user operates the shooting range determination button 442, and the first, second, or third instruction of the pixel data acquisition range in the shooting areas r1 to r25 displayed in the camera image display area 441 is displayed. By executing the method, the acquisition range of the pixel data can be indicated. When acquiring the entire pixel data of the observation object S displayed in the camera image display area 441, the user can obtain a rectangular area including the observation object S (in the example of FIG. 14, the imaging areas r1 to r1). r9, r11 to r13, r22 to r24) can indicate the acquisition range display frame R.

  The user can set a mask in an arbitrary shooting area of the camera image display area 441. In FIG. 14, there are imaging regions (imaging regions r5, r13, r24 in the example of FIG. 14) that do not include the observation object S. In such a case, the user can set the mask M in the imaging regions r5, r13, r24 as shown in FIG. 15 by designating the imaging regions r5, r13, r24. In the imaging regions r5, r13, and r24 in which the mask M is set, the laser beam is not scanned when the pixel data is acquired. As a result, the imaging region in which the mask M is set is excluded from the pixel data acquisition range.

  FIG. 16 is a diagram illustrating a confocal image of the observation object S included in a plurality of imaging regions. An image processing method to be described later is executed in the acquisition range of the pixel data displayed in the camera image display area 441 in FIG. Thereby, as shown in FIG. 16, the pixel data of the observation object S included in the plurality of imaging regions can be acquired. In this case, it is possible to reduce the time required for acquiring the pixel data by setting the mask M in the imaging region where the pixel data is not acquired.

(5) Image Processing Method FIGS. 17 and 18 are flowcharts showing an image processing method in the confocal microscope system 500. 1 executes an image processing method according to an image processing program stored in the storage device 240. Hereinafter, an image processing method by the CPU 210 will be described. In steps S15 and S16 of the flowchart, the confocal image data and the surface image data are abbreviated as image data.

  CPU210 produces | generates camera image data based on the camera data given by the control part 300, and displays the camera image of the observation target S on the image display area 410 of the display part 400 of FIG. 4 based on camera image data (FIG. Step S1).

  Next, the user instructs the pixel data acquisition range by the first, second, or third instruction method of the pixel data acquisition range. CPU210 sets the acquisition range of pixel data based on a user's instruction (step S2). In this case, the position of each unit area within the acquisition range and the number of unit areas are stored in the work memory 230.

  Subsequently, the user instructs the movement range of the objective lens 3 in FIG. The CPU 210 sets a movement range in the Z direction of the objective lens 3 based on a user instruction (step S3). The instruction screen for the movement range of the objective lens 3 in the Z direction is displayed when the next button 434a is operated on the acquisition range setting screens 430a to 430c of FIG. 5, FIG. 7, or FIG.

  Thereafter, the user operates the acquisition start button 422 in the condition setting area 420 on the display unit 400 of FIG. 6, FIG. 8, or FIG. When the CPU 210 detects that the acquisition start button 422 has been operated by the user, the CPU 210 calculates the total number of pixels in the acquisition range display frame R (step S4).

  Next, the CPU 210 compares the calculated total number of pixels with a threshold value stored in advance in the storage device 240 of FIG. 1 (step S5). The threshold value is set to a capacity (number of display processable pixels) that can be used for the display process of image data in the work memory 230. The threshold value is, for example, 5000 × 5000 pixels or 10000 × 10000 pixels. If the calculated total number of pixels is equal to or less than the threshold value, the CPU 210 determines the pixel data acquisition rate to be 1 (step S6), and proceeds to the process of step S10.

  On the other hand, when the calculated total number of pixels is larger than the threshold value in step S5, the CPU 210 compares the calculated total number of pixels with twice the threshold value (step S7). If the calculated total number of pixels is less than or equal to twice the threshold value, the CPU 210 determines the pixel data acquisition rate to be ¼ (step S8), and proceeds to the process of step S10.

  On the other hand, if the calculated total number of pixels is larger than twice the threshold value in step S7, the CPU 210 determines the pixel data acquisition rate to be 1/16 (step S9), and proceeds to the processing in step S10. .

  Next, the CPU 210 stores the pixel data acquisition rate determined in step S6, step S8, or step S9 in the work memory 230 of FIG. 1 (step S10). Based on the stored acquisition rate, the number of samplings and the number of scanning lines (the number of scans in the X direction) in scanning in the X direction are calculated (step S11).

  Subsequently, the CPU 210 moves the stage 60 by applying a drive pulse to the stage drive unit 62 of FIG. 1 based on the position of the unit area stored in the work memory 230 (step S12). Thereafter, the CPU 210 notifies the control unit 300 of the number of samplings in the scanning in the X direction, the number of scanning lines, and the movement range in the Z direction, and commands the acquisition of the pixel data of the unit area (step S13).

  The control unit 300 controls the XY scan optical system 20 of FIG. 1 and controls the observation object S based on the number of samplings in the X direction, the number of scanning lines, and the movement range in the Z direction notified from the CPU 210. The position in the Z direction is moved, and pixel data is given to the CPU 210 of the PC 200 based on the light reception signal output from the light receiving element 30. CPU210 acquires the pixel data of a unit area from the control part 300 (step S14). The CPU 210 generates a plurality of confocal image data of the unit area based on the acquired pixel data and also generates surface image data (step S15) and stores it in the storage device 240.

  Here, the CPU 210 determines whether or not the confocal image data and the surface image data of all the unit areas have been generated (step S16). When the confocal image data and the surface image data of all the unit areas have not been generated, the CPU 210 returns to the process of step S12. The CPU 210 moves the stage 60 to a position where the confocal image data of the next unit area can be generated, and repeats the processes of steps S13 to S16.

  On the other hand, when the confocal image data and the surface image data of all the unit areas are generated in step S <b> 16, the CPU 210 connects the surface image data of the plurality of unit areas stored in the storage device 240 using the storage device 240. (Step S17). Thereby, surface image data of the designated acquisition range is generated. When only one unit region is included in the acquisition range display frame R, the surface image data is not connected in step S17.

  Thereafter, the CPU 210 causes the display unit 400 to display an image of the surface of the observation object S based on the connected surface image data using the work memory 230 (step S18). Finally, the CPU 210 stores the connected surface image data in the storage device 240 and ends the image processing method.

  FIG. 19 is a diagram illustrating the confocal image data generated when the pixel data acquisition rate is 1. FIG. FIG. 20 is a diagram illustrating the confocal image data generated when the pixel data acquisition rate is 1/4. FIG. 21 is a diagram illustrating the confocal image data generated when the pixel data acquisition rate is 1/16. Here, pixel data generated at one position in the Z direction is shown.

  In FIG. 19 to FIG. 21, the pixel data constituting the confocal image data of one or a plurality of unit areas is illustrated by dots in order to visually express the acquisition rate of the confocal image data.

  In the example of FIG. 19, confocal image data of one unit region s1 is generated. When the pixel data acquisition rate is determined to be 1 in step S6, the sampling number in the X-direction scanning is set to the maximum sampling number in the X-direction scanning in the acquisition range display frame R, and the scanning line number is displayed in the acquisition range display. The maximum number of scanning lines in the frame R is set. In this case, as shown in FIG. 19, pixel data corresponding to the maximum number of pixels in the unit region s1 is acquired without being thinned out in the X direction and the Y direction.

  In the example of FIG. 20, the confocal image data of the four unit regions s1 to s4 is generated. When the pixel data acquisition rate is determined to be ¼ in step S8, the number of samplings in the X-direction scanning is set to ½ of the maximum number of samplings in the X-direction scanning in the acquisition range display frame R, and scanning is performed. The number of lines is set to ½ of the maximum number of scanning lines in the acquisition range display frame R. In this case, as shown in FIG. 20, pixel data corresponding to the maximum number of pixels in the unit regions s1 to s4 is acquired by being thinned out in half in the X direction and the Y direction.

  In the example of FIG. 21, confocal image data of 16 unit regions s1 is generated. When the pixel data acquisition rate is determined to be 1/16 in step S9, the sampling number in the X-direction scanning is set to ¼ of the maximum sampling number in the X-direction scanning in the acquisition range display frame R. The number of lines is set to ¼ of the maximum number of scanning lines in the acquisition range display frame R. In this case, as shown in FIG. 21, pixel data corresponding to the maximum number of pixels in the unit areas s1 to s16 is acquired by being thinned out to 1/4 in the X direction and the Y direction.

  FIG. 22 is a diagram illustrating an ultra-deep image of the observation object S when the pixel data acquisition rate is 1. FIG. FIG. 23 is a diagram illustrating an ultra-deep image of the observation target S when the pixel data acquisition rate is ¼. FIG. 24 is a diagram illustrating an ultra-deep image of the observation target S when the pixel data acquisition rate is 1/16.

  As illustrated in FIGS. 22 to 24, the CPU 210 uses the work memory 230 to display an ultra-depth image corresponding to the ultra-depth image data stored in the storage device 240 based on a user instruction. It can be displayed in the image display area 410. Similarly, the CPU 210 displays a height image corresponding to the height image data stored in the storage device 240 on the image display area 410 of the display unit 400 using the work memory 230 based on a user instruction. be able to.

  In the present embodiment, the number of samples in the X direction and the scanning are determined according to the comparison result between the total number of pixels in the pixel data acquisition range designated by the user and the threshold value stored in the storage device 240. Although both of the number of lines are thinned out, the present invention is not limited to this. Only the number of scanning lines may be thinned out according to the comparison result between the total number of pixels within the acquisition range designated by the user and the threshold value stored in the storage device 240.

  Even if the number of samplings is thinned out, the number of scans in the X direction and the scan interval do not change, so that the time required for acquiring pixel data of the unit region is substantially shortened. On the other hand, when the number of scanning lines is thinned, the number of scans in the Y direction of the XY scan optical system 20 decreases, so that the time required to acquire pixel data of the unit region is greatly shortened.

  Thus, the CPU 210 calculates the amount of pixel data corresponding to the total number of pixels within the acquisition range specified by the user. The calculated amount of pixel data is compared with the capacity of the working memory 230 that can be used for display. When the calculated amount of pixel data is larger than the capacity of the working memory 230 that can be used for display, the mechanical scanning of the XY scan optical system 20, that is, the number of samplings and the number of scanning lines in the X direction is large. Thinned out. Thereby, the amount of surface image data to be generated can be reduced. Further, unlike the case where thinning is performed after the generation of surface image data, it is not necessary to spend a great deal of time in generating the amount of surface image data that cannot be displayed. As a result, the time required for displaying the image of the surface of the observation object S can be shortened.

  If the amount of surface image data is larger than the capacity of the work memory 230 that can be used for display even if the amount of surface image data is reduced by thinning out the number of scanning lines, the generated data is generated. By thinning out the surface image data, the amount of the surface image data may be made smaller than the capacity of the working memory 230 that can be used for display.

(6) Parallel processing of image processing In the image processing, the CPU 210 sequentially executes the processing of steps S12 to S16 in Fig. 18, but is not limited thereto. The CPU 210 may execute part of the processes of steps S12 to S16 in parallel.

  FIG. 25 is a timing chart of parallel processing of image processing acquisition processing. The horizontal axis of FIG. 25 is time.

  The image processing of one unit area includes processing before scanning (hereinafter referred to as preprocessing), processing during scanning (hereinafter referred to as processing during scanning), and processing after scanning (hereinafter referred to as postprocessing). .

  The preprocessing includes securing a confocal image data storage area in the storage device 240, setting parameters such as the position of the unit area, moving the stage 60, and waiting for stability of the XY scan optical system 20. The processing during scanning includes scanning of laser light by the XY scanning optical system 20, movement of the objective lens 3 in the Z direction, transfer of pixel data from the control unit 300 to the PC 200, and storage of pixel data in the PC. The post-processing includes generation of confocal image data on the PC 200, generation of surface image data, and storage of the surface image data.

  As shown in FIG. 25, after the preprocessing for the first unit area and the processing during scanning are performed, the preprocessing for the second unit area starts simultaneously with the start of the postprocessing for the first unit area. Is done. After the pre-processing for the second unit area is completed, the in-scan process and the post-process for the second unit area are sequentially performed. Similarly, pre-processing for the third unit region is started simultaneously with the start of post-processing for the second unit region. After the preprocessing for the third unit area is completed, the in-scan process and the post-process for the third unit area are sequentially performed.

  In this way, the post-processing of each unit area can be performed in parallel with the in-scan processing and post-processing of the next unit area. Thereby, it is possible to reduce the time required to acquire pixel data of a plurality of unit areas.

(7) Automatic setting of the Z-direction movement range of the objective lens In step S3 of FIG. 17, the Z-direction movement range of the objective lens 3 is manually set by the user, but is not limited thereto. The movement range of the objective lens 3 in the Z direction may be automatically set by the CPU 210.

  In the manual setting, the movement range of the objective lens 3 in the Z direction is set so that pixel data corresponding to the lowest and highest portions of the surface of the observation object S is appropriately acquired. Therefore, when the surface of the observation object S includes large unevenness or large inclination, it is necessary to increase the movement range of the objective lens 3 in the Z direction. Therefore, the capacity of the acquired pixel data and the time required for acquiring the pixel data increase.

  On the other hand, in the automatic setting, the movement range in the Z direction of the objective lens 3 is set for each unit area corresponding to the height difference of the surface of the observation object S. Hereinafter, details of the upper and lower limit automatic setting processing for automatically setting the movement range of the objective lens 3 in the Z direction will be described.

  FIG. 26 is a diagram for explaining the relationship between the position in the Z direction of the objective lens 3 for one pixel and the value of effective pixel data. In FIG. 26, the vertical axis represents pixel data values, and the horizontal axis represents the position of the objective lens 3 in the Z direction. In the horizontal axis of FIG. 26, the position of the objective lens 3 in the Z direction increases from left to right.

  The pixel data is a digital signal corresponding to the light reception signal output from the light receiving element 30. For this reason, the value of the pixel data increases as the gain of the light receiving element 30 increases and decreases as the gain of the light receiving element 30 decreases. Pixel data is output from the A / D converter. Therefore, the upper limit value of the pixel data is the upper limit value of the output range of the A / D converter (hereinafter referred to as the output upper limit value).

  When the pixel data value is saturated at the output upper limit value max, the pixel data value corresponding to the light receiving intensity of the light receiving element 30 cannot be obtained. If the pixel data value is less than or equal to the noise level nl, the peak of the pixel data cannot be clearly identified from the noise of the light receiving element 30. Hereinafter, pixel data values that are smaller than the output upper limit value max and larger than the noise level nl are referred to as effective pixel data values.

  In FIG. 26, a range of valid pixel data values is indicated by an arrow HL. In this case, as indicated by the curve l1, when an arbitrary first gain is set in the light receiving element 30, the pixel data value for one pixel is a position where the objective lens 3 is lower than the peak position z0. It is effective in a state where it is in a range from ma1 to a position mb1 higher than the peak position z0.

  Therefore, the upper limit position and the lower limit position in the Z direction of the objective lens 3 for obtaining a valid pixel data value for one pixel are determined according to the output upper limit value max and the noise level nl.

  As indicated by a curve l2 in FIG. 26, when a second gain smaller than the first gain is set in the light receiving element 30, the pixel data value is set to the first gain in the light receiving element 30. Compared to the case, it becomes smaller overall. In this case, the value of the pixel data for one pixel is effective when the objective lens 3 is in the range from the position ma2 higher than the position ma1 to the position mb2 lower than the position mb1. As described above, when the gain set in the light receiving element 30 is changed, the upper limit position and the lower limit position in the Z direction of the objective lens 3 capable of obtaining effective pixel data values are also changed.

  FIG. 27 is a diagram for explaining a method of setting the upper limit position and the lower limit position in the Z direction of the objective lens 3 when generating the height image data and the ultra-depth image data.

  First, pixel data corresponding to all the pixels in the unit region is acquired by performing scanning in the X direction and Y direction of the unit region with the objective lens 3 held at an arbitrary position zs1 in the Z direction. In this state, when the values of all the pixel data are not equal to or greater than a predetermined value, the gain of the light receiving element 30 is increased by a certain amount.

  Next, as shown by a thick solid line arrow na in FIG. 27, the evaluation value is calculated by moving the objective lens 3 in the Z direction by a certain amount, for example, by a peak position search process of the evaluation value described later. The evaluation peak position Ez0 is searched based on the evaluation value. Here, the evaluation value is the sum of the values of pixel data corresponding to all the pixels in the unit area. The position in the Z direction of the objective lens 3 when the evaluation value shows the peak value is referred to as an evaluation peak position Ez0. During this search, the gain is decreased by a certain amount each time the calculated evaluation value reaches the output upper limit Emax. The output upper limit Emax is a product of the output upper limit max of the A / D converter 30 and the total number of pixels in the unit area. Thereby, the upper limit position UP and the lower limit position BP of the objective lens 3 can be searched with the detected evaluation peak position Ez0 as a reference.

  Subsequently, the objective lens 3 is moved to the evaluation peak position Ez0. In this state, pixel data corresponding to all the pixels in the unit region is acquired by scanning the unit region in the X direction and the Y direction.

  When the value of any pixel data is the output upper limit value max, the value of the pixel data is not valid. Therefore, the gain of the light receiving element 30 is decreased by a certain amount. Thereafter, by scanning the unit region in the X direction and the Y direction again, pixel data corresponding to all the pixels in the unit region is acquired.

  Whether the values of all the pixel data are equal to or less than the noise level nl when the values of all the pixel data become smaller than the output upper limit value max by repeating the above gain adjustment and the acquisition of the pixel data. judge.

  If all the pixel data values are not less than or equal to the noise level nl, the objective lens 3 is moved upward by a certain amount, as shown by a thick solid arrow nb in FIG.

  Subsequently, the above gain adjustment, pixel data acquisition, pixel data determination operation, and upward movement of the objective lens 3 are repeated. Thereby, finally, the position in the Z direction of the objective lens 3 when it is determined that the values of all the pixel data are equal to or less than the noise level nl is set as the upper limit position UP.

  After the upper limit position UP is set, the objective lens 3 is moved in the Z direction to the position when the gain of the light receiving element 30 is finally decreased. In this state, pixel data corresponding to all the pixels in the unit region is acquired by scanning the unit region in the X direction and the Y direction.

  Also in this case, similarly to the above, when the value of any pixel data is the output upper limit value max, the gain of the light receiving element 30 is decreased by a certain amount. Thereafter, by scanning the unit region in the X direction and the Y direction again, pixel data corresponding to all the pixels in the unit region is acquired.

  If the values of all the pixel data become smaller than the output upper limit value max by repeating the above gain adjustment and pixel data acquisition, whether or not the values of all the pixel data are equal to or less than the noise level nl Determine.

  If all the pixel data values are not less than or equal to the noise level nl, the objective lens 3 is moved downward by a certain amount, as indicated by the thick solid arrow nc in FIG. Subsequently, the above gain adjustment, pixel data acquisition, pixel data determination operation, and downward movement of the objective lens 3 are repeated. Finally, the position in the Z direction of the objective lens 3 when it is determined that all the pixel data values are equal to or less than the noise level nl is set as the lower limit position BP.

  Here, when the gain of the light receiving element 30 changes, the range in the Z direction of the objective lens 3 in which effective pixel data values can be obtained also changes. Therefore, when the gain of the light receiving element 30 is reduced when the lower limit position BP is set after the upper limit position UP is set, the objective lens 3 is moved to the position at the time when the gain of the light receiving element 30 is finally reduced, As shown by a thick solid line arrow nd in FIG. 27, the upper limit position UP is searched again.

  When searching for the upper limit position UP again, the objective lens 3 moves in the Z direction within substantially the same range as when searching for the first upper limit position UP. Therefore, the gain of the light receiving element 30 is hardly reduced. As a result, the upper limit position UP is reset while the light receiving element 30 is adjusted to substantially the same gain as that when the lower limit position BP is set.

  As described above, when the upper and lower limit automatic setting process is executed, the peak position z0 of the objective lens 3 is searched while the gain of the light receiving element 30 is adjusted by the peak position search process. In this case, the value of the pixel data is not saturated at the output upper limit value max. Further, the value of the pixel data does not become smaller than the noise level nl. Thereby, the peak position in the change in the value of the pixel data due to the movement of the objective lens 3 is detected, and the peak position z0 is automatically detected.

  Thereafter, the objective lens 3 is moved to the detected peak position z0, and the upper limit position UP and the lower limit position BP are searched. Thereby, the upper limit position UP and the lower limit position BP of the objective lens 3 when the height image data and the ultra-depth image data are generated are set.

  Next, evaluation value peak position search processing will be described. FIG. 28 is a diagram for explaining an evaluation value peak position search process. FIG. 28A shows the relationship between the position of the objective lens 3 in the Z direction and the evaluation value. In FIG. 28A, the vertical axis represents the evaluation value, and the horizontal axis represents the position of the objective lens 3 in the Z direction. On the horizontal axis in FIG. 28A, the position of the objective lens 3 in the Z direction increases from left to right.

  At the start of the peak position search process, the gain of the light receiving element 30 is set so that the calculated evaluation value is smaller than the output upper limit Emax and sufficiently larger than the noise level nl. The output upper limit Emax is a product of the output upper limit max of the A / D converter and the number of pixels in the unit area. The gain of the light receiving element 30 is set such that the calculated evaluation value is, for example, 1/2 of the output upper limit value Emax.

  Thereafter, as shown by a thick solid arrow in FIG. 28A, the objective lens 3 is set while being gradually moved upward from the current position zs1 in the Z direction (position lower than the evaluation peak position z0). The pixel data corresponding to all the pixels in the unit area is acquired with the gain obtained and the evaluation value is calculated.

  In this case, the evaluation value increases exponentially until the objective lens 3 reaches the peak position z0. Therefore, in this peak position search process, the gain is decreased by a certain amount every time the evaluation value reaches the output upper limit value Emax.

  This prevents the value of the pixel data that increases exponentially from being saturated at the output upper limit max until the objective lens 3 moves from the position zs1 in the Z direction to the evaluation peak position Ez0. As a result, in the vicinity of the evaluation peak position Ez0, the gain of the light receiving element 30 is finally set to an appropriate value.

  The evaluation value decreases as the gain of the light receiving element 30 decreases. Therefore, in the peak position search process, the calculated evaluation value is corrected based on the number of times the gain of the light receiving element 30 is decreased.

  This correction is performed, for example, by adding a multiplication value of the gain reduction count of the light receiving element 30 after the start of the peak position search process and the output upper limit value Emax to the calculated evaluation value. Thereby, it is possible to approximately associate the corrected evaluation value with the light receiving intensity of the light receiving element 30 over the entire range of the objective lens 3 in the Z direction.

  FIG. 28B shows the relationship between the position of the objective lens 3 in the Z direction and the corrected evaluation value. In FIG. 28B, the vertical axis represents the evaluation value after correction, and the horizontal axis represents the position of the objective lens 3 in the Z direction. On the horizontal axis in FIG. 28B, the position of the objective lens 3 in the Z direction increases from left to right.

  As shown in FIG. 28B, the corrected evaluation value increases exponentially to the peak value and then decreases exponentially from the peak value, similarly to the light receiving intensity of the light receiving element 30. Thereby, it is detected that the increase or decrease in the evaluation value after correction is switched by moving the objective lens 3 in one direction (upward direction) in the Z direction. It is detected that the evaluation value after correction has decreased by a predetermined value pi2, and it is detected that the evaluation value after correction has increased by a predetermined value pi1 before switching the increase / decrease in the evaluation value after correction. In this case, the position of the objective lens 3 when the evaluation value shows the peak value can be detected as the evaluation peak position Ez0.

  The value of the pixel data changes due to the influence of noise of the light receiving element 30. Therefore, the above values pi1 and pi2 are set to a value larger than, for example, a product of the noise level nl and the number of pixels in the unit area. Thereby, even when the corrected evaluation value changes due to the influence of noise of the light receiving element 30, it is possible to prevent the evaluation peak position Ez0 from being erroneously detected. The values pi1 and pi2 correspond to first and second identification values described later, respectively. The values pi1 and pi2 may be the same value or different values.

  As described above, the evaluation peak position Ez0 is searched while the gain of the light receiving element 30 is adjusted by the peak position search process.

  Thus, since the upper and lower limit automatic setting processing is performed for each unit region, useless scanning of laser light is reduced, and the generation time of height image data and ultra-depth image data is shortened.

  Thereafter, the objective lens 3 is moved within the movement range in the Z direction set for each unit region, whereby pixel data is acquired for each unit region. In this case, the gain of the light receiving element 30 is set to be the same in the plurality of unit regions.

(8) Effects of the Embodiment In the confocal microscope system 500 according to the present embodiment, when the acquisition range of the pixel data is instructed by the user, a plurality of unit areas are set based on the instruction. Then, the total number of pixels of the pixel data within the acquisition range is calculated. When the calculated total number of pixels is compared with the display processable pixel number of the work memory 230 and the calculated total pixel number exceeds the display processable pixel number of the work memory 230, a plurality of pixel data in the unit area The acquisition rate is adjusted. Therefore, even when the user designates a wide range as the pixel data acquisition range, it is possible to prevent the amount of the connected surface image data from exceeding the capacity of the work memory 230 that can be used for display. . Thereby, the image of the surface of the observation object S can be displayed regardless of the acquisition range of the pixel data. In addition, the time required to acquire pixel data can be shortened.

  Further, after the surface image data for the plurality of unit areas is generated, the user can prevent the amount of the surface image data for the plurality of unit areas from exceeding the capacity of the work memory 230 that can be used for display. There is no need to thin out the surface image data. Therefore, it is not necessary to spend a great deal of time in generating the amount of surface image data that cannot be displayed. As a result, an image of the surface of the observation object S can be displayed without wasting time.

(9) Other Embodiments (9-1) In the above embodiment, the laser beam is scanned in the X direction and the Y direction on the observation object S by controlling the XY scan optical system 20. However, it is not limited to this. The laser beam may be scanned on the observation object S in the X direction and the Y direction by moving the stage 60.

  Further, line light (for example, elongated light extending in the X direction) may be used as the laser light. In this case, instead of the XY scan optical system 20, a Y scan optical system that does not perform scanning in the X direction is used. Further, instead of the light receiving element 30, a line CCD camera or the like including a plurality of light receiving elements arranged in a direction corresponding to the X direction is used.

  The size of the light receiving surface in the direction corresponding to the Y direction of each light receiving element of the line CCD camera is generally several tens of μm. In this case, the light receiving surface of the line CCD camera is disposed at the focal position of the lens 2. When the surface of the observation object S is at the focal position of the objective lens 3, the line light reflected by the observation object S is collected on the light receiving surface of the line CCD camera. Thereby, most of the line light reflected by the observation object S enters the light receiving surface of the line CCD camera.

  On the other hand, when the observation object S is at a position out of the focal position of the objective lens 3, the line light reflected by the observation object S is condensed at a position before or after the light receiving surface of the line CCD camera. Thereby, only part of the line light reflected by the observation object S enters the light receiving surface of the line CCD camera. Therefore, it is not necessary to arrange the pinhole member 7 in front of the line CCD camera.

  (9-2) In the above embodiment, the relative position of the observation object S with respect to the objective lens 3 in the Z direction is changed by moving the objective lens 3 in the Z direction. However, the present invention is not limited to this. The relative position of the observation object S with respect to the objective lens 3 in the Z direction may be changed by moving the stage 60 in the Z direction.

  (9-3) In the above embodiment, the CPU 210 of the PC 200 may have the function of the control unit 300. In this case, the control unit 300 may not be provided.

(10) Correspondence between each component of claim and each part of embodiment The following describes an example of a correspondence between each component of the claim and each part of the embodiment. It is not limited.

  The observation object S is an example of the observation object, the ultra-depth image or the height image is an example of the surface image, the confocal microscope system 500 is an example of the confocal microscope system, and the light receiving element 30 is the light receiving element. It is an example. The lenses 1 and 2, the objective lens 3, the pinhole member 7, and the laser light source 10 are examples of confocal optical systems. The unit areas s1 to s16 are examples of unit areas, the imaging areas r1 to r25 are examples of areas, the X direction is an example of the first direction, the Y direction is an example of the second direction, and Z The direction is an example of the third direction.

  The lens driving unit 63 is an example of a relative distance changing unit, the control unit 300 is an example of a control unit, the display unit 400 is an example of a display unit, and the camera image is an example of a non-confocal image. The color CCD camera 50, the control unit 300, and the PC 200 are examples of non-confocal image acquisition units, and the stage 60 is an example of a support unit. The PC 200 is an example of an image data generation unit, an observation range instruction reception unit, an observation range setting unit, an image data processing unit, a data acquisition amount adjustment unit, an enlargement instruction reception unit, a display range expansion unit, and a processing device.

  As each constituent element in the claims, various other elements having configurations or functions described in the claims can be used.

  The present invention can be effectively used for various confocal microscope systems, image processing methods, and image processing programs.

DESCRIPTION OF SYMBOLS 1, 2 Lens 3 Objective lens 4-6 Half mirror 7 Pinhole member 8 ND filter 10 Laser light source 20 XY scan optical system 30 Light receiving element 40 White light source for illumination 50 Color CCD camera 60 Stage 61 Stage operation part 62 Stage drive Unit 63 Lens drive unit 100 Measurement unit 200 PC
210 CPU
220 ROM
230 Working Memory 240 Storage Device 300 Control Unit 400 Display Unit 410 Image Display Area 420 Condition Setting Area 421 Range Setting Button 422 Acquisition Start Button 423 Navigation Display Buttons 430a to 430c Acquisition Range Setting Screen 431 Specification Method Setting Field 431a to 431c Check Box 432 designated range setting field 432a upper end setting part 432b left end setting part 432c right end setting part 432d lower end setting part 432e, 432f length designation part 432g, 432h unit area number designation part 433 setting value display field 434a next button 434b end button 440 navigation Display screen 441 Camera image display area 442 Shooting range determination button 443 Shooting range expansion button 500 Confocal microscope system BP Lower limit position Emax, max Output upper limit value Ez0 Evaluation peak position fp Focus position HL, na to nd Arrow Lx, Ly Line segment l1, l2 Curve M Mask ma1, ma2, mb1, mb2, zs1 Position Nx, Ny Acquisition number nl Noise level p1, p2 Points pi1, pi2 Value R Acquisition range display frame r1 to r25 Shooting area S Observation object s1 to s16 Unit area UP Upper limit position z0 Peak position

Claims (6)

A confocal microscope system that displays an image of the surface of an observation object,
A light receiving element;
Irradiating the unit area set on the surface of the observation object with light, and confocal optical system for guiding the light irradiated to the unit area to the light receiving element;
The relative distance between the confocal optical system and the observation object is changed so that light is emitted from the confocal optical system in the unit region at a plurality of positions along a direction perpendicular to the unit region. A relative distance change part to be
Controlling the confocal optical system and the relative distance changing unit, and a plurality of pixels corresponding to a plurality of pixels arranged in the first and second directions orthogonal to each other in the unit region based on the output signal of the light receiving element A control unit for sequentially acquiring pixel data;
Generate confocal image data of a unit area based on a plurality of pixel data acquired by the control unit, and generate surface image data for displaying an image of the surface of the observation object based on the confocal image data An image data generation unit to perform,
An observation range instruction receiving unit for receiving an instruction of an observation range by a user;
An observation range setting unit that sets a plurality of unit regions as an observation range based on an instruction received by the observation range instruction reception unit;
An image data processing unit for linking the surface image data generated by the image data generation unit for a plurality of unit areas set by the observation range setting unit;
The number of pixels included in the plurality of unit regions set by the observation range setting unit is compared with a preset threshold value, and pixel data acquired by the control unit for each unit region based on the comparison result A data acquisition amount adjustment unit for adjusting the number;
A confocal microscope system comprising: a display unit configured to display an image of a surface of an observation target based on surface image data of a plurality of unit regions connected by the image data processing unit. A non-confocal image acquisition unit that acquires a non-confocal image of the observation object;
The observation range instruction receiving unit displays the non-confocal image acquired by the non-confocal image acquisition unit on the display unit, and instructs the observation range on the displayed non-confocal image by a user. The confocal microscope system according to claim 1, wherein the confocal microscope system is received. The observation range instruction reception unit divides the displayed non-confocal image into a plurality of regions that can be designated by the user, and excludes the region designated as an acquisition exclusion target by the user from the observation range. The confocal microscope system according to claim 2. A support portion provided to support the observation object and to be movable in the first and second directions;
An enlargement instruction receiving unit that receives an instruction to enlarge the range of the non-confocal image displayed on the display unit;
A display range enlarging unit for enlarging a range of a non-confocal image displayed on the display unit by moving the support unit when the enlargement instruction accepting unit is accepted by the enlargement instruction accepting unit; The confocal microscope system according to claim 2 or 3. An image processing method for displaying an image of the surface of an observation object,
Irradiating the unit region set on the surface of the observation object with the confocal optical system, and guiding the light irradiated to the unit region to the light receiving element;
The relative distance between the confocal optical system and the observation object is changed so that light is emitted from the confocal optical system in the unit region at a plurality of positions along a direction perpendicular to the unit region. Steps,
Sequentially obtaining a plurality of pixel data corresponding to a plurality of pixels arranged in first and second directions orthogonal to each other in a unit region based on an output signal of the light receiving element;
Generating the confocal image data of the unit region based on the acquired plurality of pixel data and generating the surface image data for displaying the image of the surface of the observation object based on the confocal image data;
Receiving an instruction of an observation range by a user;
Setting a plurality of unit areas as observation ranges based on received instructions;
Concatenating surface image data generated for a plurality of set unit areas;
Comparing the number of pixels included in a plurality of set unit areas with a preset threshold value, and adjusting the number of pixel data acquired for each unit area based on the comparison results;
And a step of displaying an image of the surface of the observation object based on the surface image data of the plurality of connected unit regions. An image processing program for causing a processing device to execute image processing for displaying an image of a surface of an observation object,
Irradiating light to the unit region set on the surface of the observation object by the confocal optical system, and guiding the laser light irradiated to the unit region to the light receiving element;
The relative distance between the confocal optical system and the observation object is changed so that light is emitted from the confocal optical system in the unit region at a plurality of positions along a direction perpendicular to the unit region. Processing,
A process of sequentially obtaining a plurality of pixel data corresponding to a plurality of pixels arranged in the first and second directions orthogonal to each other in the unit region based on an output signal of the light receiving element;
A process of generating confocal image data of a unit region based on a plurality of acquired pixel data and generating surface image data for displaying an image of the surface of the observation object based on the confocal image data;
A process of accepting an instruction of the observation range by the user;
A process of setting a plurality of unit areas as an observation range based on an accepted instruction;
A process of connecting the surface image data generated for a plurality of set unit areas;
A process of comparing the number of pixels included in a plurality of set unit areas with a preset threshold value and adjusting the number of pixel data acquired for each unit area based on the comparison result;
A process of displaying an image of the surface of the observation object based on the surface image data of a plurality of connected unit areas,
An image processing program that is executed by the processing apparatus.

Images