JP6009288B2 - Measurement microscope apparatus, measurement microscope apparatus operation program, and computer-readable recording medium - Google Patents

Description

  The present invention relates to a microscope apparatus having a measurement function, an operation program for operating the microscope apparatus, and a computer-readable recording medium.

As a measuring device for measuring an object, a measuring device using triangulation has been developed. As shown in FIG. 6, such an apparatus is provided between the optical axis of the measurement light emitted from the light projecting unit 110 and the optical axis of the measurement light incident on the light receiving unit 120 (the optical axis of the light receiving unit 120). The angle α is preset. Here, when the object S is not placed on the stage 140, the measurement light emitted from the light projecting unit 110 is reflected by the point O on the placement surface of the stage 140 and enters the light receiving unit 120. On the other hand, when the object S is placed on the stage 140, the measurement light emitted from the light projecting unit 110 is reflected by the point A on the surface of the object S and enters the light receiving unit 120.
Then, the distance d in the X direction between the point O and the point A is measured, and the height h of the point A on the surface of the object S is calculated based on the distance d. By calculating the heights of all points on the surface of the object S, the three-dimensional shape of the object S is measured. In order to irradiate all the points on the surface of the object S with the measurement light, the light projecting unit 110 emits the measurement light according to a predetermined structured measurement light pattern, and by a fringe projection method using the striped measurement light. The three-dimensional shape of the object S is efficiently measured.

  In measurement using such a fringe projection method, the object is irradiated with measurement light from an obliquely upward direction to project the fringe, so that shadows and saturation points are generated depending on the shape and orientation of the object. The “shadow” refers to a part where the measurement light is not reflected, and measurement is impossible at this part. On the other hand, in the case of “saturated”, the measurement result can be obtained as long as the light and darkness by the measurement light can be discriminated, but the reliability of the data is low compared to the point that is not saturated.

  In this way, when the measurement light is projected onto the measurement object and a shadow is generated, the measurement cannot be performed. Therefore, it is necessary to adjust the position, posture, and the like of the object so as not to make a shadow as much as possible. Similarly, when the reflected light is saturated, it is desirable that such a saturation point does not occur because accurate measurement cannot be performed.

  Therefore, it is conceivable to reduce the shadow by preparing a plurality of measurement light projecting portions and arranging them at different positions and irradiating the measurement light from different directions.

  However, even in this case, it is not always possible to eliminate shadows and saturation points in all the parts, and there are cases where such unmeasurable points and parts where measurement is insufficient remain. In this case, since shadows and saturation points cannot be measured correctly, the measurement image acquisition conditions are adjusted by changing the position, angle, and posture of the object so as to reduce these parts as much as possible. There is a need.

  However, when a plurality of measurement lights are provided, the shadow or saturation point is a part that cannot be measured by any measurement light, or cannot be measured by the currently selected measurement light, but can be measured by switching to another measurement light. There was a problem that it could not be determined. In other words, when there is a single measurement light, the part where the shadow is generated is displayed in real time on the display unit. The measurement image acquisition conditions can be changed by adjusting so that shadows or the like are not generated or reduced.

  However, when there are a plurality of measurement beams, the current measurement beam is shaded, but there are cases in which measurement can be performed without being shaded by switching to another measurement beam. In this case, measurement is possible even if the current measurement light is in shadow, so there is no need to worry about it. On the other hand, it is necessary to adjust the imaging condition of the measurement image for a portion that is also shadowed even when switched to the other measurement light. However, with respect to the shadow part that can be confirmed in the measurement image obtained with the current measurement light, is it a part that is really not measurable (whether it is a shadow even when switched to another measurement light) or is a measurable part? It is difficult to determine whether it is a shadow when switching to another measurement light, and as a result, there is a problem that it is difficult to set an imaging condition for an appropriate measurement image.

Toni F. Schenk, "Remote Sensing and Reconstruction for Three-Dimensional Objects and Scenes", Proceedings of SPIE, Volume 2572, pp. 1-9 (1995) Sabry F. El-Hakim and Armin Gruen, "Videometrics and Optical Methods for 3D Shape Measurement", Proceedings of SPIE, Volume 4309, pp. 219-231 (2001)

  The present invention has been made in view of such conventional problems. The main object of the present invention is to easily set the measurement image acquisition conditions in advance so as to reduce the number of measurement failure portions when acquiring a measurement image in a measurement microscope apparatus including a plurality of measurement light projecting units. Another object of the present invention is to provide a measurement microscope apparatus, a measurement microscope apparatus operation program, and a computer-readable recording medium.

Means for Solving the Problems and Effects of the Invention

  To achieve the above object, according to the measurement microscope apparatus of the first aspect of the present invention, measurement for projecting measurement light as a structured illumination of a predetermined pattern from an oblique direction to an object As the light projection means, the first measurement light projection means capable of irradiating the object with the first measurement light from the first direction, and the object from the second direction different from the first direction. The second measurement light projecting means capable of irradiating the second measurement light, the observation illumination light source for capturing the observation image, and the first measurement light projecting means or the second measurement light projecting means. In order to display the measurement image or the observation image by acquiring the measurement light reflected by the object and capturing the measurement image, and for imaging the observation image using the observation illumination light source Display means, and display the measurement image or observation image on the display means. State, it is possible and a highlight unit for displaying overlapping the measurement abnormality region either measurement results becomes abnormal in the first measuring light projecting means and the second measurement light projecting means. With the above configuration, when using a plurality of measurement light projecting means, while reducing the area where the measurement is abnormal, specifically which part can be measured, which part is still abnormal measurement, Since it can be confirmed on one screen on the measurement image or the observation image, it becomes easy to adjust the measurement image imaging condition so that the measurement abnormal region is reduced, and the usability for the user is drastically improved.

  Further, according to the measurement microscope apparatus according to the second aspect, the first measurement image acquired by the imaging unit using the first measurement light projecting unit, and the second measurement target, A measurement image synthesizing unit that synthesizes the second measurement image acquired by the imaging unit using a measurement light projecting unit and generates one synthesized measurement image, and the highlight unit includes the measurement image synthesis unit. In the state in which the composite measurement image generated in (1) is displayed on the display means, it is possible to display the measurement abnormal region in an overlapping manner. With the above configuration, when using a plurality of measurement light projecting means, while reducing the area where the measurement is abnormal, specifically which part can be measured, which part is still abnormal measurement, Since it can be confirmed on one screen on the composite measurement image, it becomes easy to adjust the measurement image imaging condition so that the measurement abnormal region is reduced, and the usability for the user is dramatically improved.

  Furthermore, according to the measurement microscope apparatus according to the third aspect, the measurement abnormal region is a non-measurable region in which the measurement light is shaded by any measurement light projecting unit and data cannot be acquired by the imaging unit. Can be included. With the above configuration, an area that cannot be measured by any of the measurement light projecting means can be visually recognized separately from the measurement area by any of the measurement light projecting means, and the user can set measurement image acquisition conditions. Can help.

  Furthermore, according to the measurement microscope apparatus according to the fourth aspect, the measurement abnormal region is saturated with the reflected light of the measurement light detected by the imaging means saturated by any measurement light projecting means. The highlight means may be configured to highlight the saturation area in a different manner from the non-measurable area and to display the area on the display means. With the above configuration, it is possible to visually distinguish and grasp a region where measurement is not possible and a region where measurement is possible but saturated and low accuracy, which contributes to the setting operation of the acquisition condition of the measurement image by the user. it can.

  Furthermore, according to the measurement microscope apparatus according to the fifth aspect, when the measurement image combining unit generates the first measurement image by the first measurement image and the second measurement image, the first measurement image When one of the corresponding pixels in the measurement image and the second measurement image is a normally measured value and the other includes a measurement abnormal region, the composite measurement image can be configured to be generated using the normal value. . With the above configuration, even if a measurement abnormal region is included in one of the measurement images, it can be supplemented with the other measurement image to obtain a composite measurement image.

  Furthermore, according to the measurement microscope apparatus according to the sixth aspect, the synthesized measurement image is obtained by using a pixel having a higher pixel value among pixels corresponding to the first measurement image and the second measurement image. Can be composed image.

  Furthermore, according to the measurement microscope apparatus according to the seventh aspect, the composite measurement image is configured using an average of pixel values among pixels corresponding to the first measurement image and the second measurement image. Can be an image.

  Furthermore, according to the measurement microscope apparatus according to the eighth aspect, the composite measurement image is obtained by using a pixel having a lower pixel value among pixels corresponding to the first measurement image and the second measurement image. Can be composed image.

  Furthermore, according to the measurement microscope apparatus according to the ninth aspect, the display unit divides the image display region for displaying the measurement image, and displays the composite measurement image, the first divided display region, A second divided display area for displaying a second measurement image and a third divided display area for displaying the first measurement image can be included. With the above configuration, it is possible to display a measurement measurement area and a measurement measurement image that is based on the combined measurement image on a single screen, and display the measurement measurement area with the highlighting means. There is an advantage that the operation of setting the measurement image acquisition condition so as to be reduced can be easily performed.

  Furthermore, the measurement microscope apparatus according to the tenth aspect further includes measurement light brightness individual adjustment means capable of individually adjusting the brightness of the first measurement image and the brightness of the second measurement image. be able to. With the above configuration, the brightness of the first measurement image and the second measurement image can be individually adjusted, and the brightness can be individually adjusted to an appropriate brightness so that there are fewer areas where measurement results are abnormal in both measurement images. Is possible.

  Furthermore, according to the measurement microscope apparatus according to the eleventh aspect, the composite measurement image, the second measurement image, and the first measurement image are acquired in a time division manner, and the first division display region and the second division image are obtained. In the display area and the third divided display area, the display area can be updated and displayed at regular intervals. With the above configuration, the combined measurement image, the second measurement image, and the first measurement image can be confirmed as a live image on one screen, so the state after the measurement image acquisition conditions change, such as the position change of the object and the change in the amount of measurement light, The images can be checked in real time while contrasting. In particular, the result of adjusting the brightness or the like of each measurement image can be confirmed by sequentially reflecting it on the display means, so that there is an advantage that adjustment work can be easily performed.

  Furthermore, according to the measurement microscope apparatus according to the twelfth aspect, as a setting screen for setting measurement image acquisition conditions for acquiring a measurement image with the imaging unit using the measurement light projecting unit, Measurement image acquisition mode selection means capable of switching between a simple mode that enables easy setting and an application mode that enables detailed setting. With the above configuration, the measurement image acquisition condition setting screen can be changed according to the user's skill level, providing an easy-to-use environment for beginners, and capable of handling detailed settings for skilled users, making measurement easy to use Images can be acquired.

  Furthermore, according to the measurement microscope apparatus according to the thirteenth aspect, the observation image captured using the observation illumination light source and the measurement image captured using the measurement light projecting unit are further combined. A three-dimensional image synthesizing unit for generating a three-dimensional synthesized image can be provided. With the above configuration, it is possible to synthesize a fine three-dimensional image using the texture information of the observation image and the height information of the measurement image, and obtain an advantage that detailed measurement can be performed.

Furthermore, according to the measurement microscope apparatus according to the fourteenth aspect, as the measurement light projecting means for projecting the measurement light as the structured illumination of the predetermined pattern from the oblique direction to the object, the first The first measurement light projecting means capable of irradiating the object with the first measurement light from the direction, and the second measurement light can be applied to the object from a second direction different from the first direction. Imaging for capturing a measurement image by acquiring measurement light projected by the second measurement light projection means and the first measurement light projection means or the second measurement light projection means and reflected by the object An operation program for operating a measurement microscope apparatus comprising: a first measurement acquired by the imaging unit using the first measurement light projecting unit on the same object on a computer Acquired by the imaging means using the image and the second measuring light projection means The first measurement light projecting means and the second measurement are added to the image composition function for synthesizing the second measurement image thus generated and generating one composite measurement image, and the measurement image or the observation image displayed on the display means. In any of the light projecting means, it is possible to realize a highlight function for displaying a measurement abnormal region in which the measurement result is abnormal in an overlapping manner. With the above configuration, when using a plurality of measurement light projecting means, while reducing the area where the measurement is abnormal, specifically which part can be measured, which part is still abnormal measurement, Since it can be confirmed on one screen on the measurement image or the observation image, it becomes easy to adjust the imaging condition so that the measurement abnormal region is reduced, and the usability for the user is dramatically improved.

  A fifteenth computer-readable recording medium stores the above program. CD-ROM, CD-R, CD-RW, flexible disk, magnetic tape, MO, DVD-ROM, DVD-RAM, DVD-R, DVD + R, DVD-RW, DVD + RW, Blu-ray (product) Name), HD DVD (AOD), and other magnetic disks, optical disks, magneto-optical disks, semiconductor memories, and other media that can store programs. The program includes a program distributed in a download manner through a network line such as the Internet, in addition to a program stored and distributed in the recording medium. Further, the recording medium includes a device capable of recording the program, for example, a general purpose or dedicated device in which the program is implemented in a state where the program can be executed in the form of software, firmware, or the like. Furthermore, each process and function included in the program may be executed by computer-executable program software, or each part of the process or function may be executed by hardware such as a predetermined gate array (FPGA, ASIC, DSP), or a program. You may implement | achieve in the format with which the partial hardware module which implement | achieves some elements of software and hardware is mixed.

It is a block diagram which shows the measurement microscope apparatus which concerns on one embodiment of this invention. It is a block diagram which shows the structure of the imaging means of FIG. 3A to 3D are schematic views of the object in a state where light is irradiated. 4A to 4D are schematic views of the object in a state where light is irradiated respectively. It is an image figure which shows an example of GUI of a measurement microscope apparatus operation program. It is a figure for demonstrating the principle of a triangulation system. FIG. 7A is a perspective view for explaining a first pattern of measurement light, and FIG. 7B is a plan view of FIG. 7A. 8A to 8D are diagrams for explaining the second pattern of the measurement light, respectively. 9A to 9C are diagrams for explaining the third pattern of the measurement light, respectively. It is a figure which shows the relationship between the pixel data (the intensity | strength of the light received) in the specific part of a target object, and the order (what number) of the image from which pixel data was obtained. 11A to 11D are diagrams for explaining a fourth pattern of measurement light, respectively. It is a table | surface which shows the example of handling when a measurement abnormal point is contained in any measurement image. It is an image figure which shows the state which selected simple mode by GUI of the measurement microscope apparatus operation program. It is an image figure which shows the state which pressed the "measurement image" button from the state of FIG. It is an image figure which shows the state which selected the application mode from the state of FIG. It is an image figure which shows the state which pressed the "measurement image" button from the state of FIG. It is an image figure which shows the state which divided and displayed the image display area from the state of FIG. It is an image figure which shows the state which selected the measurement direction "only the left side" from the state of FIG. It is a flowchart which shows the procedure which acquires a measurement image using a measurement microscope apparatus operation program. It is an image figure which shows the example which displayed the synthesized image with the ratio of the observation image of 100%. It is an image figure which shows the example which displayed the synthesized image with the ratio of 100% of the measurement image. It is an image figure which shows the state switched to the display of a measurement image from the state of FIG. It is an image figure which shows the example which displayed the image improvement panel. It is an image figure which shows a mode that the image quality of a measurement image is selected in the "measurement mode" selection column. It is an image figure which shows a mode that measurement light is selected in the "measurement direction" selection column. It is an image figure which shows a mode that the imaging condition of an observation image is set with an observation image imaging condition setting means.

Hereinafter, embodiments of the present invention will be described with reference to the drawings. However, the embodiment described below exemplifies a measurement microscope apparatus, a measurement microscope apparatus operation program, and a computer-readable recording medium for embodying the technical idea of the present invention. The microscope apparatus and measurement microscope apparatus operation program and the computer-readable recording medium are not specified as follows. Further, the present specification by no means specifies the members shown in the claims to the members of the embodiments. In particular, the dimensions, materials, shapes, relative arrangements, and the like of the component parts described in the embodiments are not intended to limit the scope of the present invention unless otherwise specified, and are merely explanations. It is just an example. Note that the size, positional relationship, and the like of the members shown in each drawing may be exaggerated for clarity of explanation. Furthermore, in the following description, the same name and symbol indicate the same or the same members, and detailed description thereof will be omitted as appropriate. Furthermore, each element constituting the present invention may be configured such that a plurality of elements are constituted by the same member and the plurality of elements are shared by one member, and conversely, the function of one member is constituted by a plurality of members. It can also be realized by sharing.
(Embodiment 1)

FIG. 1 is a block diagram showing the configuration of the measurement microscope apparatus according to Embodiment 1 of the present invention. As shown in FIG. 1, the measurement microscope apparatus 500 includes an imaging unit 100, a control unit 200, a light source unit 300, and a display unit 400.
(Imaging means 100)

The configuration of the imaging means 100 of the measurement microscope apparatus 500 of FIG. 1 is shown in the block diagram of FIG. The imaging unit 100 is, for example, a microscope, and includes a light projecting unit 110, a light receiving unit 120, an illumination light output unit 130, a stage 140, and a measurement control unit 150. The light projecting unit 110 includes a measurement light source 111, a pattern generation unit 112, and a plurality of lenses 113, 114, and 115. The light receiving unit 120 includes a camera 121 and a plurality of lenses 122 and 123. An object S is placed on the stage 140.
(Light Projecting Unit 110)

The light projecting unit 110 is disposed obliquely above the stage 140. The imaging unit 100 may include a plurality of light projecting units 110. In the example of FIG. 2, the imaging unit 100 includes two light projecting units 110. Here, the first measurement light projector 110A (right side in FIG. 2) that can irradiate the measurement illumination light to the object S from the first direction and the second direction different from the first direction. Second measurement light projectors 110 </ b> B (left side in FIG. 2) that can irradiate the object S with measurement illumination light are respectively disposed. The first measuring light projecting unit 110A and the second measuring light projecting unit 110B are arranged symmetrically across the optical axis of the light receiving unit. In addition, it is also possible to provide three or more light projecting units, or to move the light projecting unit and the stage relative to each other and use the common light projecting unit while projecting light with different illumination directions. Furthermore, in this example, the irradiation angle of the illumination light with respect to the vertical direction in which the light projecting unit projects is fixed, but this can also be made variable.
(Measurement light source 111)

  The measurement light sources 111 of each of the first measurement light projectors 110A and the second measurement light projectors 110B are, for example, halogen lamps that emit white light. The measurement light source 111 may be another light source such as a white LED (light emitting diode) that emits white light. Light emitted from the measurement light source 111 (hereinafter referred to as “measurement light”) is appropriately condensed by the lens 113 and then enters the pattern generation unit 112.

The pattern generation unit 112 is a DMD (digital micromirror device), for example. The pattern generation unit 112 may be an LCD (Liquid Crystal Display), LCOS (Liquid Crystal on Silicon), or a mask. The measurement light incident on the pattern generation unit 112 is converted into a preset pattern and a preset intensity (brightness) and emitted. The measurement light emitted by the pattern generation unit 112 is converted into light having a diameter larger than the field of view that can be observed and measured by the light receiving unit 120 by the plurality of lenses 114 and 115, and then is applied to the object S on the stage 140. Irradiated.
(Light receiving unit 120)

The light receiving unit 120 is disposed above the stage 140. The measurement light reflected above the stage 140 by the object S is condensed and imaged by the plurality of lenses 122 and 123 of the light receiving unit 120 and then received by the camera 121.
(Camera 121)

  The camera 121 is, for example, a CCD (charge coupled device) camera including an image sensor 121a and a lens. The image sensor 121a is, for example, a monochrome CCD (charge coupled device). The image sensor 121a may be another image sensor such as a CMOS (complementary metal oxide semiconductor) image sensor. The color image sensor requires each pixel to receive light for red, green, and blue, so the measurement resolution is lower than that of a monochrome image sensor, and a color filter must be provided for each pixel. Sensitivity decreases. Therefore, in this embodiment, a monochrome CCD is used as the image sensor, and a color image is acquired by illuminating the illumination light output unit 130 described later with illumination corresponding to RGB in a time-sharing manner. Yes. With such a configuration, it is possible to acquire a color image of the measurement object without reducing the measurement accuracy.

However, it goes without saying that a color image sensor may be used as the image sensor 121a. In this case, although the measurement accuracy and sensitivity are reduced, it is not necessary to irradiate illumination corresponding to RGB from the illumination light output unit 130 in a time-sharing manner, and a color image can be obtained simply by irradiating white light. The optical system can be configured simply . From each pixel of the image sensor 121 a, an analog electrical signal (hereinafter referred to as “light reception signal”) corresponding to the amount of received light is output to the measurement control unit 150.
(Measurement control unit 150)

The measurement control unit 150 includes an A / D converter (analog / digital converter) and a FIFO (First In First Out) memory (not shown). The light reception signal output from the camera 121 is sampled at a constant sampling period and converted to a digital signal by the A / D converter of the measurement control unit 150 based on control by the light source unit 300. Digital signals output from the A / D converter are sequentially stored in the FIFO memory. The digital signal stored in the FIFO memory is sequentially transferred to the control means 200 as pixel data.
(Control means 200)

  As shown in FIG. 1, the control means 200 includes a CPU (Central Processing Unit) 210, a ROM (Read Only Memory) 220, a work memory 230, a storage device 240 and an operation unit 250. As the control means 200, a PC (personal computer) or the like can be used. The operation unit 250 includes a keyboard and a pointing device. As a pointing device, a mouse or a joystick is used.

  The ROM 220 stores a system program. The working memory 230 includes a RAM (Random Access Memory), and is used for processing various data. The storage device 240 is composed of a hard disk or the like. The storage device 240 stores an image processing program and a shape measurement program. The storage device 240 is used for storing various data such as pixel data provided from the measurement control unit 150.

The CPU 210 generates image data based on the pixel data given from the measurement control unit 150. The CPU 210 performs various processes on the generated image data using the work memory 230 and causes the display unit 400 to display an image based on the image data. Further, the CPU 210 gives a driving pulse to a stage driving unit 145 described later. Further, this CPU realizes functions of a measurement image composition unit 211, a highlight unit 212, and a three-dimensional image composition unit 213, which will be described later.
(Display unit 400)

The display unit 400 is a member for displaying a measurement image acquired by the imaging unit 100 and a captured observation image. The display unit 400 is configured by, for example, an LCD panel or an organic EL (electroluminescence) panel.
(Stage 140)

  In FIG. 2, two directions orthogonal to each other within a plane (hereinafter referred to as “mounting surface”) on the stage 140 on which the object S is placed are defined as an X direction and a Y direction. Y. A direction orthogonal to the mounting surface of the stage 140 is defined as a Z direction and is indicated by an arrow Z. A direction rotating around an axis parallel to the Z direction is defined as a θ direction, and is indicated by an arrow θ.

  The stage 140 includes an XY stage 141, a Z stage 142, and a θ stage 143. The XY stage 141 has an X direction moving mechanism and a Y direction moving mechanism. The Z stage 142 has a Z direction moving mechanism. The θ stage 143 has a θ direction rotation mechanism. The XY stage 141, the Z stage 142, and the θ stage 143 constitute a stage 140. The stage 140 further includes a fixing member (clamp) (not shown) that fixes the object S to the mounting surface. The stage 140 may further include a tilt stage having a mechanism that can rotate around an axis parallel to the placement surface.

  Stepping motors are used for the X direction moving mechanism, Y direction moving mechanism, Z direction moving mechanism, and θ direction rotating mechanism of the stage 140, respectively. The X direction moving mechanism, Y direction moving mechanism, Z direction moving mechanism, and θ direction rotating mechanism of the stage 140 are driven by the stage operation unit 144 or the stage driving unit 145 of FIG.

  The user manually operates the stage operation unit 144 to move the mounting surface of the stage 140 in the X direction, the Y direction, the Z direction, or rotate in the θ direction relative to the light receiving unit 120. be able to. The stage driving unit 145 supplies the current to the stepping motor of the stage 140 based on the driving pulse supplied from the control unit 200, thereby moving the stage 140 relative to the light receiving unit 120 in the X direction, Y direction, or Z direction. It can be moved or rotated in the θ direction.

Here, as shown in FIG. 2, the light receiving unit 120 and the light projecting unit are arranged such that the central axis of the left and right light projecting units 110 and the center axis of the light receiving unit 120 intersect each other on the focus plane where the focus of the stage 140 is best. 110 and the relative position of the stage 140 are determined. Since the center of the rotation axis in the θ direction coincides with the center axis of the light receiving unit 120, when the stage 140 rotates in the θ direction, the object S does not deviate from the field of view and is centered on the rotation axis. It is designed to rotate within the field of view. Further, the XYθ and tilt moving mechanisms are supported with respect to the Z direction moving mechanism. In other words, even if the stage is rotated in the θ direction or tilted, the center axis of the light receiving unit 120 and the movement axis in the Z direction are not displaced. With such a stage mechanism, even when the position and orientation of the object S are changed, it is possible to move the stage 140 in the Z direction and capture and combine a plurality of images at different focal positions. . In this embodiment, an electric stage that can be driven by a stepping motor has been described as an example. However, a manual stage that can be moved only manually may be used.
(Light source unit 300)

The light source unit 300 includes a control board 310 and an observation illumination light source 320. A CPU (not shown) is mounted on the control board 310. The CPU of the control board 310 controls the light projecting unit 110, the light receiving unit 120, and the measurement control unit 150 based on a command from the CPU 210 of the control unit 200. This configuration is an example, and other configurations may be used. For example, the control board may be omitted by controlling the light projecting unit 110 and the light receiving unit 120 by the measurement control unit 150 or by controlling the light projecting unit 110 and the light receiving unit 120 by the control unit 200. Alternatively, the light source unit 300 can be provided with a power supply circuit for driving the imaging unit 100.
(Light source for observation 320)

  The observation illumination light source 320 includes, for example, three color LEDs that emit red light, green light, and blue light. By controlling the luminance of the light emitted from each LED, light of an arbitrary color can be generated from the observation illumination light source 320. Light generated from the observation illumination light source 320 (hereinafter referred to as “illumination light”) is output from the illumination light output unit 130 of the imaging unit 100 through a light guide member (light guide).

  The illumination light output from the illumination light output unit 130 irradiates the object S by switching red light, green light, and blue light in a time-sharing manner. Thereby, the observation images captured with these RGB lights can be synthesized to obtain a color observation image, which can be displayed on the display unit 400.

  When the color observation image is displayed in this manner, if the switching frequency for switching the color of the illumination light is made to coincide with the frame rate when the display contents are updated (rewrite the screen), the frame rate is low. In such a case (for example, about several Hz), the flicker becomes remarkable. In particular, when color switching by RGB primary colors is conspicuous, the user may be uncomfortable. Therefore, such a problem can be avoided by setting the switching frequency for switching the RGB illumination light to a high speed (for example, several hundred Hz) that the user cannot recognize. The color of the illumination light is switched by the illumination light output unit 130 or the like. The timing at which the imaging unit 100 actually images the object S while switching RGB of illumination light at high speed is the timing for updating the display content of the display unit 400. In other words, it is not necessary that the observation image capturing timing and the illumination light switching timing coincide completely. In other words, the RGB switching period of the illumination light is captured to the extent that an RGB observation image can be captured by the image sensor. This can be handled by linking so as to be a multiple of the period. With this method, the illumination light switching timing can be increased, and the discomfort given to the user can be reduced without improving the frame rate that can be processed by the image sensor 121a.

  In the example of FIG. 1, the observation illumination light source 320 is externally attached to the imaging unit 100, and the observation illumination light source 320 is disposed in the light source unit 300. By doing so, it is possible to avoid a situation in which the heat generated by the observation illumination light source 320 affects the optical system of the imaging unit 100. However, the observation illumination light source can be provided in the imaging means by using an observation illumination light source with a small calorific value or by providing a corresponding heat radiation mechanism on the imaging means side. In this case, a light guide member for optically connecting the light source unit and the imaging unit and an illumination light output unit can be eliminated, and the observation illumination light source can be used as the illumination light output unit.

  In the example of FIG. 1, the measurement light projecting means is integrated with the measurement light source. However, the light source of the measurement light projecting means is not limited to being provided in the imaging means, and may be externally attached. it can. For example, the light source of the measurement light projecting unit and the illumination light source for observation can be collectively arranged in the light source unit.

  The illumination light output unit 130 of FIG. 2 has an annular shape and is disposed above the stage 140 so as to surround the light receiving unit 120. Thereby, illumination light is irradiated to the target object S from the illumination light output unit 130 so that no shadow is generated. 3A to 3D and 4A to 4D are schematic views of the object S in a state where light is irradiated. In the examples of FIGS. 3A to 3D and FIGS. 4A to 4D, the object S has a hole Sh at the approximate center of the upper surface. 3A, 3C, and 4A, the shadow Ss is represented by hatching.

  3A is a plan view of the object S in a state irradiated with measurement light from the first measurement light projecting unit 110A of FIG. 2, and FIG. 3B is a cross-sectional view taken along line AA of FIG. 3A. As shown in FIGS. 3A and 3B, when the measurement light is irradiated from the first measurement light projecting unit 110A to the object S, the measurement light does not reach the bottom of the hole Sh depending on the depth of the hole Sh. , A shadow Ss is generated. Therefore, a part of the object S cannot be observed.

  3C is a plan view of the object S in the state irradiated with the measurement light from the second measurement light projecting unit 110B of FIG. 2, and FIG. 3D is a cross-sectional view taken along line BB of FIG. 3C. As shown in FIGS. 3C and 3D, when the measurement light is irradiated from the second measurement light projecting unit 110B to the object S, the measurement light does not reach the bottom of the hole Sh depending on the depth of the hole Sh. , A shadow Ss is generated. Therefore, a part of the object S cannot be observed.

  4A is a plan view of the object S in a state in which measurement light from both the first measurement light projection unit 110A and the second measurement light projection unit 110B is irradiated, and FIG. 4B is a CC view of FIG. 4A. It is line sectional drawing. As shown in FIGS. 4A and 4B, when the measurement light is irradiated from both the first measurement light projection unit 110A and the second measurement light projection unit 110B, the first measurement light projection unit 110A, Compared with the case where the object S is irradiated with the measurement light from one of the second measurement light projectors 110B, the measurement light that does not reach the bottom of the hole Sh is reduced, and thus the generated shadow Ss is reduced. Therefore, the portion of the object S that can be observed increases.

4C is a plan view of the object S in a state where the illumination light from the illumination light output unit 130 of FIG. 2 is irradiated, and FIG. 4D is a cross-sectional view taken along the line DD in FIG. 4C. As shown in FIG. 4C and FIG. 4D, the illumination light is irradiated from directly above the object S, so that the illumination light reaches the bottom of the hole Sh regardless of the depth of the hole Sh. Therefore, most of the object S can be observed.
(Example of GUI)

In the measurement microscope apparatus, an operation program for operating the measurement microscope apparatus 500 is installed in a PC that is the control means 200. The display unit 400 displays a GUI (Graphical User Interface) for operating the measurement microscope apparatus operation program. An example of such a GUI screen is shown in FIG. In this example, on the display unit 400, the first measurement image S1 of the object S irradiated with the first measurement light from the first measurement light projecting unit 110A and the second measurement light from the second measurement light projecting unit 110B. It can be displayed so that the second measurement image S2 of the object S irradiated with light is aligned. In this example, the first display area 416 is provided on the right side and the second display area 417 is provided on the left side of the image display area 410 provided on the left side of the display unit 400. By using such a two-screen display, it is possible to confirm while comparing the state of the measurement image obtained with each measurement light, particularly the shadowed area. In addition, the division example of the image display area is not limited to the configuration in which the image display areas are arranged side by side in this manner, and an arbitrary configuration such as an arrangement in the vertical direction or a separate screen can be used as appropriate.
(Measurement light brightness individual adjustment means 442)

In the operation area 420 of the display unit 400, two brightness adjustment sliders 444 and 446 are provided as the measurement light brightness individual adjustment means 442. The brightness adjustment sliders 444 and 446 are sliders that can move in the horizontal direction, and adjust the brightness of each measurement light projecting unit. Here, the brightness adjustment slider 446 can individually adjust the brightness of the second measurement light projector 110B, and the brightness adjustment slider 444 can adjust the brightness of the first measurement light projector 110A.
The position of the brightness adjustment slider 444 corresponds to the brightness of the measurement light emitted from the first measurement light projector 110A or the camera exposure time when an image is taken with the measurement light from the first measurement light projector 110A. Correspond. The position of the brightness adjustment slider 446 is determined by the brightness of the measurement light emitted from the second measurement light projector 110B or the camera exposure time when an image is taken with the measurement light from the second measurement light projector 110B. Corresponding to The user operates the operation area 420 provided in the GUI with the operation unit 250 of the control unit 200 in FIG. 1 to move the brightness adjustment slider 444 in the horizontal direction, whereby the first measurement light projecting unit 110A. The brightness of the measurement light emitted from the camera or the camera exposure time corresponding to 110A can be changed. Similarly, by operating the operation unit 250 and moving the brightness adjustment slider 446 in the horizontal direction, the brightness of the measurement light emitted from the second measurement light projector 110B or the second measurement light projector 110B. The camera exposure time corresponding to can be changed.

  As described above, in the image display area 410, images of the object S when the measurement light is irradiated by each of the first measurement light projection unit 110A and the second measurement light projection unit 110B are displayed so as to be aligned. it can. Accordingly, the user moves the positions of the brightness adjustment sliders 444 and 446 while looking at the image of the object S displayed in the image display area 410, whereby the first measurement light projecting unit 110A and the second measurement light projecting unit 110A. The brightness of the measurement light emitted from each of the light projectors 110B or the camera exposure time corresponding to each of the light projectors can be appropriately adjusted.

  In addition, the appropriate brightness of the measurement light emitted from the first measurement light projection unit 110A and the second measurement light projection unit 110B and the appropriate brightness of the illumination light emitted from the illumination light output unit 130 or the respective illuminations There may be a correlation with the camera exposure time corresponding to. In this case, the brightness of the measurement light emitted from each of the first measurement light projection unit 110A and the second measurement light projection unit 110B or the camera exposure time corresponding to each projection unit is the illumination light output unit 130. May be automatically adjusted based on the brightness of the illumination light emitted from the camera or the camera exposure time corresponding to the illumination light.

  Alternatively, the light is emitted from each of the first measurement light projection unit 110A and the second measurement light projection unit 110B based on the brightness of the illumination light emitted from the illumination light output unit 130 or the camera exposure time corresponding to the illumination light. An adjustment guide for adjusting the brightness of the measured light or the camera exposure time corresponding to each light projecting unit may be displayed on the display unit 400. In this case, the user moves the positions of the brightness adjustment sliders 444 and 446 based on the adjustment guide, thereby emitting the light from each of the first measurement light projection unit 110A and the second measurement light projection unit 110B. The brightness of the measurement light or the camera exposure time corresponding to each light projecting unit can be adjusted appropriately.

If the light irradiation direction is different, the light reflection direction is also different, so that the brightness of the resulting image varies depending on the light irradiation direction even in the same region. That is, the brightness of the measurement light suitable for measurement and the exposure time of the image sensor vary depending on the irradiation direction. In the present embodiment, the brightness of each image captured by irradiating light from the plurality of first measurement light projection units 110A and the second measurement light projection unit 110B can be individually adjusted. An appropriate measurement light brightness or exposure time can be set for each irradiation direction. In addition, since the image whose brightness is being adjusted is displayed in the image display area 410 while being updated, the brightness can be adjusted while checking the adjusted image. At this time, in the image displayed in the image display area 410, a portion that is too bright and overexposed, or a portion that is too dark and obscured by black is displayed in an identifiable manner, so that the brightness is improved for the user. It is also possible to display whether or not can be adjusted appropriately .
(1) Shape measurement by triangulation

  In the imaging unit 100, the shape of the object S is measured by a triangulation method. FIG. 6 is a diagram for explaining the principle of the triangulation system. As shown in FIG. 6, an angle α between the optical axis of the measurement light emitted from the light projecting unit 110 and the optical axis of the measurement light incident on the light receiving unit 120 (the optical axis of the light receiving unit 120) is set in advance. The The angle α is larger than 0 degree and smaller than 90 degrees.

  When the object S is not placed on the stage 140, the measurement light emitted from the light projecting unit 110 is reflected by the point O on the placement surface of the stage 140 and enters the light receiving unit 120. On the other hand, when the object S is placed on the stage 140, the measurement light emitted from the light projecting unit 110 is reflected by the point A on the surface of the object S and enters the light receiving unit 120.

  When the distance in the X direction between the point O and the point A is d, the height h of the point A of the object S with respect to the mounting surface of the stage 140 is given by h = d ÷ tan (α). The CPU 210 of the control means 200 in FIG. 1 measures the distance d between the point O and the point A in the X direction based on the pixel data of the object S given by the measurement control unit 150. Further, the CPU 210 calculates the height h of the point A on the surface of the object S based on the measured distance d. By calculating the heights of all points on the surface of the object S, the three-dimensional shape of the object S is measured.

In order to irradiate all the points on the surface of the object S with measurement light, measurement light having various patterns is emitted from the light projecting unit 110 in FIG. The pattern of the measurement light is controlled by the pattern generation unit 112 in FIG. Hereinafter, the pattern of the measurement light will be described.
(2) First pattern of measurement light

  7A to 7B are diagrams for explaining a first pattern of measurement light. FIG. 7A shows a state in which the measuring light is irradiated from the light projecting unit 110 onto the object S on the stage 140. FIG. 7B shows a plan view of the object S irradiated with the measurement light. As shown in FIG. 7A, measurement light having a linear cross section parallel to the Y direction (hereinafter referred to as “line measurement light”) is emitted from the light projecting unit 110 as the first pattern. In this case, as shown in FIG. 7B, the portion of the line-shaped measurement light irradiated on the stage 140 and the portion of the line-shaped measurement light irradiated on the surface of the object S are the height h of the surface of the object S. Are shifted from each other in the X direction by a distance d corresponding to. Therefore, the height h of the object S can be calculated by measuring the distance d.

  When a plurality of portions along the Y direction on the surface of the object S have different heights, the height h of the plurality of portions along the Y direction is determined by measuring the distance d for each portion. Can be calculated.

Further, the CPU 210 in FIG. 1 measures the distance d for a plurality of portions along the Y direction at one position in the X direction, and then scans the line-shaped measurement light parallel to the Y direction in the X direction. The distance d is measured for a plurality of portions along the Y direction at other positions in the direction. Thereby, the height h of the several part of the target object S along the Y direction in the several position of a X direction is calculated. The height h of all points on the surface of the object S can be calculated by scanning the line-shaped measurement light in the X direction over a range wider than the field of view that can be observed and measured by the light receiving unit 120. Thereby, the three-dimensional shape of the target object S can be measured.
(3) Second pattern of measurement light

  8A to 8D are diagrams for explaining the second pattern of the measurement light. As shown in FIGS. 8A to 8D, as the second pattern, measurement light having a linear cross section parallel to the Y direction and a pattern whose intensity changes in a sine wave shape in the X direction (hereinafter referred to as “sinusoidal wave shape”). (Referred to as “measurement light”) is emitted from the light projecting unit 110 a plurality of times (in this example, four times).

  FIG. 8A shows sinusoidal measurement light emitted for the first time. The intensity of the sinusoidal measurement light emitted for the first time has an initial phase φ at an arbitrary position P0 on the surface of the object S. When the sinusoidal measurement light is emitted, the light reflected by the surface of the object S is received by the light receiving unit 120. The intensity of the received light is measured based on the pixel data of the object S. The intensity of light reflected by an arbitrary part P0 on the surface of the object S is assumed to be I1.

  FIG. 8B shows the sinusoidal measurement light emitted for the second time. The intensity of the sinusoidal measurement light emitted for the second time has a phase (φ + π / 2) at an arbitrary position P0 on the surface of the object S. When the sinusoidal measurement light is emitted, the light reflected by the surface of the object S is received by the light receiving unit 120. The intensity of the received light is measured based on the pixel data of the object S. The intensity of the light reflected by the portion P0 on the surface of the object S is I2.

  FIG. 8C shows sinusoidal measurement light emitted for the third time. The intensity of the sinusoidal measurement light emitted for the third time has a phase (φ + π) at an arbitrary position P0 on the surface of the object S. When the sinusoidal measurement light is emitted, the light reflected by the surface of the object S is received by the light receiving unit 120. The intensity of the received light is measured based on the pixel data of the object S. The intensity of the light reflected by the portion P0 on the surface of the object S is I3.

  FIG. 8D shows sinusoidal measurement light emitted for the fourth time. The intensity of the fourth sinusoidal measurement light has a phase (φ + 3π / 2) at an arbitrary position P0 on the surface of the object S. When the sinusoidal measurement light is emitted, the light reflected by the surface of the object S is received by the light receiving unit 120. The intensity of the received light is measured based on the pixel data of the object S. The intensity of the light reflected by the portion P0 on the surface of the object S is assumed to be I4.

The initial phase φ is given by φ = tan ⁻¹ [(I1−I3) / (I2−I4)]. A height h of an arbitrary portion of the object S is calculated from the initial phase φ. According to this method, the initial phase φ of all portions of the object S can be calculated quickly and easily by measuring the intensity of light four times. The initial phase φ can be calculated by irradiating measurement light with different phases at least three times and measuring the intensity of the received light. In other words, the initial phase when the object S does not exist is φo, the initial phase of the stripe on the object shifted by the presence of the object S is φs, and the phase difference between φo and φs is obtained (see FIG. The height h is calculated based on this. By calculating the height h of all portions on the surface of the object S, the three-dimensional shape of the object S can be measured.
(4) Third pattern of measurement light

  9A to 9C are diagrams for explaining the third pattern of the measurement light. As shown in FIGS. 9A to 9C, measurement light having a linear cross section parallel to the Y direction and aligned in the X direction (hereinafter referred to as “striped measurement light”) is used as the third pattern. The light is emitted from the light projecting unit 110 a plurality of times (in this example, 16 times).

  That is, in the striped measurement light, a linear bright portion parallel to the Y direction and a linear dark portion parallel to the Y direction are periodically arranged in the X direction. Here, when the pattern generation unit 112 is a DMD, the dimension of the micromirror is set to one unit. The width of each bright portion of the striped measurement light in the X direction is, for example, 3 units, and the width of each dark portion of the striped measurement light in the X direction is, for example, 13 units. In this case, the period of the striped measurement light in the X direction is 16 units. The unit of the bright part and the dark part differs depending on the configuration of the pattern generation unit 112 in FIG. For example, when the pattern generation unit 112 is a liquid crystal, one unit is the size of one pixel.

  When the first striped measurement light is emitted, the light reflected by the surface of the object S is received by the light receiving unit 120. The intensity of the received light is measured based on the pixel data of the first captured image of the object S. FIG. 9A is a first captured image of the object S corresponding to the first striped measurement light.

  The second striped measurement light has a pattern in which the bright portion and the dark portion are moved by one unit in the X direction from the first striped measurement light. When the second striped measurement light is emitted, the light reflected by the surface of the object S is received by the light receiving unit 120. The intensity of the received light is measured based on the pixel data of the second captured image of the object S.

  The third striped measurement light has a pattern in which the bright and dark portions are moved by one unit in the X direction from the second striped measurement light. By the third striped measurement light being emitted, the light reflected by the surface of the object S is received by the light receiving unit 120. The intensity of the received light is measured based on the pixel data of the third captured image of the object S.

  By repeating the same operation, the light intensities corresponding to the 4th to 16th striped measurement lights are respectively measured based on the pixel data of the 4th to 16th captured images of the object S. When the striped measurement light having a period of 16 units in the X direction is emitted 16 times, the entire surface of the object S is irradiated with the striped measurement light. FIG. 9B is a seventh captured image of the object S corresponding to the seventh striped measurement light. FIG. 9C is a thirteenth captured image of the object S corresponding to the thirteenth striped measurement light.

  FIG. 10 is a diagram illustrating the relationship between the timing (numbered number) at which an image is captured and the intensity of received light in a specific part of the object S. The horizontal axis in FIG. 10 indicates the number of the captured image, and the vertical axis indicates the intensity of the received light. As described above, the first to sixteenth captured images are generated for the object S. In addition, the intensity of light corresponding to each pixel of the generated first to sixteenth captured images is measured.

As shown in FIG. 10, a scatter diagram is obtained by illustrating the light intensity of each part in the image corresponding to the number of the captured image. By fitting, for example, a Gaussian curve, a spline curve, or a parabola to the obtained scatter diagram, it is possible to estimate the number (number) of the captured image when the light intensity becomes maximum with an accuracy of less than 1. . In the example of FIG. 10, the 9.38th photographed image between the ninth and tenth images (there is no such photographed image actually and is used for calculation estimation only by the curve indicated by the fitted dotted line. It is estimated that the light intensity is maximized.
(5) Fourth pattern of measurement light

  11A to 11D are diagrams for explaining the fourth pattern of the measurement light. As shown in FIG. 11A to FIG. 11D, as the fourth pattern, measurement light having a linear cross section parallel to the Y direction and having a bright portion and a dark portion aligned in the X direction (hereinafter referred to as “code-like measurement light”). Is emitted from the light projecting unit 110 a plurality of times (in this example, four times). The ratio of the bright part and the dark part of the code-like measurement light is 50%.

  In this example, the surface of the object S is divided into a plurality of regions (16 in the examples of FIGS. 11A to 11D) in the X direction. Hereinafter, the area | region of the target object S in the X direction divided | segmented into plurality is each called the 1st-16th area | region.

  FIG. 11A shows the code-like measurement light emitted for the first time. The code-shaped measurement light emitted for the first time has a bright portion that is irradiated onto the first to eighth regions of the object S. Moreover, the code-shaped measurement light emitted for the first time has dark portions that are irradiated to the ninth to sixteenth regions of the object S. Thereby, in the code-like measurement light emitted for the first time, the bright portion and the dark portion are parallel to the Y direction and aligned in the X direction. Further, the ratio of the bright part and the dark part of the code-like measurement light emitted for the first time is 50%.

  FIG. 11B shows the coded measurement light emitted for the second time. The code-shaped measurement light emitted for the second time has a bright portion that is irradiated onto the fifth to twelfth regions of the object S. Moreover, the code-like measurement light emitted for the second time has dark portions that are irradiated to the first to fourth and thirteenth to sixteenth regions of the object S. Thereby, in the code-like measurement light emitted for the second time, the bright part and the dark part are parallel to the Y direction and aligned in the X direction. Further, the ratio of the bright part and the dark part of the code-like measurement light emitted for the second time is 50%.

  FIG. 11C shows the code-like measurement light emitted for the third time. The code-shaped measurement light emitted for the third time has bright portions that are irradiated on the first, second, seventh to tenth, fifteenth and sixteenth regions of the object S. Moreover, the code-like measurement light emitted for the third time has dark portions that are irradiated to the third to sixth and eleventh to fourteenth regions of the object S. Thereby, in the code-like measurement light emitted for the third time, the bright part and the dark part are parallel to the Y direction and aligned in the X direction. Moreover, the ratio of the bright part and the dark part of the code-like measurement light emitted for the third time is 50%.

  FIG. 11D shows the coded measurement light emitted for the fourth time. The code-like measurement light emitted for the fourth time has bright portions that are irradiated on the first, fourth, fifth, eighth, ninth, twelfth, thirteenth, and sixteenth regions of the object S. In addition, the coded measurement light emitted for the fourth time is a dark portion irradiated on the second, third, sixth, seventh, tenth, eleventh, fourteenth and fifteenth regions of the object S. Have. Thereby, in the code-like measurement light emitted for the fourth time, the bright part and the dark part are parallel to the Y direction and aligned in the X direction. Further, the ratio of the bright part and the dark part of the code-like measurement light emitted for the fourth time is 50%.

  Logic “1” is assigned to the bright part of the code-like measurement light, and logic “0” is assigned to the dark part of the code-like measurement light. In addition, the logic arrangement of the first to fourth code-like measurement lights irradiated on each area of the object S is referred to as a code. In this case, the first region of the object S is irradiated with the code-shaped measurement light with the code “1011”. Thereby, the first region of the object S is encoded to the code “1011”.

  The second region of the object S is irradiated with code-shaped measurement light with a code “1010”. As a result, the second region of the object S is encoded into the code “1010”. The third region of the object S is irradiated with code-shaped measurement light with a code “1000”. Thereby, the third region of the object S is encoded to the code “1000”. Similarly, the sixteenth region of the object S is irradiated with the code-shaped measurement light with the code “0011”. As a result, the sixteenth region of the object S is encoded with the code “0011”.

  In this way, between the adjacent areas of the object S, the code-shaped measurement light is irradiated onto the object S a plurality of times so that any digit of the code differs by “1”. That is, the code-shaped measurement light is irradiated on the object S a plurality of times so that the bright portion and the dark portion change into a gray code shape.

  Light reflected by each region on the surface of the object S is received by the light receiving unit 120. The code-shaped measurement light image is generated by the received light (four in this example), and by measuring the sign of each area from these images, the object S exists for each area of the object. A changed sign is obtained. A distance corresponding to d in FIG. 6 is obtained by obtaining a difference between this code and the code when the object S does not exist for each region. At this time, the absolute value of d is obtained from the characteristic of the coding method that the above-mentioned code appears only once in the X-axis direction in the image. From here, the absolute height (the absolute value of the height) of the region of the object S is calculated. By calculating the height of all regions on the surface of the object S, the three-dimensional shape of the object S can be measured.

In the above description, the surface of the object S is divided into 16 regions in the X direction, and the code-shaped measurement light is emitted from the light projecting unit 110 four times. However, the present invention is not limited to this. The surface of the object S may be divided into 2 ^N regions (N is a natural number) in the X direction, and the code-shaped measurement light may be emitted from the light projecting unit 110 N times. In the above description, N is set to 4 for easy understanding. In the shape measurement process in the present embodiment, N is set to 8, for example. Therefore, the surface of the object S is divided into 256 regions in the X direction.

  In the shape measurement of the object S using the code-shaped measurement light, the minimum resolution at which the stripe can be separated as a code, that is, a distance corresponding to one pixel is the minimum resolution. Therefore, when the number of pixels in the field of view in the X direction of the light receiving unit 120 is 1024 pixels, the object S having a height of, for example, 10 mm can be measured with a resolution of 10 mm ÷ 1024≈10 μm. Shape measurement using code-like measurement light (absolute value is obtained but resolution is insufficient) using the above sine wave-like measurement light or striped measurement light (these can be obtained only relative values but high resolution) By combining with the shape measurement, the absolute value of the height of the object S can be calculated with higher resolution.

  In particular, in the shape measurement of the object S using the striped measurement light in FIGS. 9A to 9C, the resolution can be 1/100 pixels. The resolution of 1/100 pixels is that the surface of the object S is divided into about 100,000 areas in the X direction when the number of pixels of the field of view in the X direction of the light receiving unit 120 is 1024 pixels (that is, N≈17). ). Therefore, the absolute value of the height of the object S can be calculated with higher resolution by combining the shape measurement using the code-shaped measurement light and the shape measurement using the striped measurement light.

The method of scanning the above-described line-shaped measurement light on the object is generally called a light cutting method. On the other hand, a method of irradiating sinusoidal measurement light, a method of irradiating striped measurement light, or a method of irradiating code-like measurement light is classified as a pattern projection method. Among pattern projection methods, the method of irradiating sinusoidal measurement light and the method of irradiating striped measurement light are classified as phase shift methods, and the method of irradiating code-like measurement light is classified as a spatial code method. Is done. The phase shift method is a phase calculated based on the amount of received light reflected from the reference height position when there is no target when irradiating a sine wave or a plurality of slit lights that are periodic projection patterns, The height of the object is obtained from the phase difference calculated based on the amount of received light reflected from the surface of the object when the object exists. Although the phase shift method cannot distinguish individual periodic fringes and has an uncertainty corresponding to an integral multiple of one fringe period (2π), there is a drawback that the absolute phase cannot be obtained. Compared to the cutting method, the number of images to be acquired is small, so that the measurement time is relatively short and the measurement resolution is high. On the other hand, the spatial code method obtains a code that changes due to the presence of an object for each area of the object, and obtains the difference between this code and the code when there is no object for each area. The absolute height of an object can be determined. The spatial code method can measure with a relatively small number of images and has an advantage that the absolute height can be obtained, but has a limit in measurement resolution compared to the phase shift method. Each of these projection methods has disadvantages and advantages, but both use the principle of triangulation. Therefore, it is impossible to measure a shadow portion where the measurement light does not reach by any measurement method.
(Measurement image composition means 211)

Returning to FIG. 1, the measurement image synthesis unit 211 of the control unit 200 performs the first measurement image acquired by the imaging unit 100 using the first measurement light projecting unit 110 </ b> A on the same object S, and the first measurement image. The second measurement light projecting unit 110B is used to synthesize the second measurement image acquired by the imaging unit 100 to generate one combined measurement image. As a method for generating the composite measurement image, for example, a pixel having a higher pixel value among the corresponding pixels in the first measurement image and the second measurement image can be used (max measurement image). Or you may comprise using the average of a pixel value among the pixels corresponding by a 1st measurement image and a 2nd measurement image (average measurement image). Or it can also comprise using the pixel with a lower pixel value among the pixels corresponding by a 1st measurement image and a 2nd measurement image (minimum measurement image).
(Highlight 212)

The highlighting unit 212 displays the combined measurement image generated by the measurement image combining unit 211 on the display unit 400, and the highlight unit 212 uses either the first measurement light projection unit 110A or the second measurement light projection unit 110B. A measurement abnormal region where the measurement result is abnormal is displayed in an overlapping manner (see FIG. 17 and the like described later).
(Abnormal measurement area)

Here, the measurement abnormality region includes a measurement impossible region where measurement itself cannot be performed, and a saturation region where measurement is possible but the obtained data is saturated and inaccurate. The non-measurable area refers to an area where the measurement light is shaded by any of the measurement light projecting means and data cannot be acquired by the imaging means 100. By doing so, it is possible to visually grasp the area that cannot be measured by any of the measurement light projecting means separately from the measurement area that can be measured by any of the measurement light projecting means. It can contribute to the setting work of measurement image acquisition conditions.
(Saturation region)

  The saturated region refers to a region where the reflected light of the measurement light detected by the imaging unit 100 is saturated by any measurement light projecting unit. Even if the level of the reflected light is saturated, if the light ON / OFF can be discriminated, the measurement result can be obtained as it is. However, the reliability of the data is lower than the point that is not saturated. Further, the non-measurable area is not limited to these, and may be an area including, for example, multiple reflections and light penetration. In the present specification, the “region” is not necessarily limited to a linear shape or a planar shape having a certain area, but is used to include a point or a set of points.

  Further, the highlight unit 212 highlights these non-measurable areas and saturated areas in different manners so that they can be displayed on the display unit 400 in an overlapping manner. As a result, it is possible to visually distinguish and grasp a region where measurement is impossible and a region where measurement is possible but is saturated and low accuracy, which can contribute to the user's setting work of measurement image acquisition conditions. Conventionally, even if a measurement abnormal point that is shadowed or saturated with one measuring light projecting means can be confirmed, it can be measured correctly with other measuring light projecting means. It was not possible to distinguish whether there was no problem even if the measurement abnormal point remained, or whether it could not be measured by other measurement light projection means, and could not be measured correctly even in the composite measurement image. On the other hand, according to the present embodiment, when a plurality of measurement light projecting units are used, it is possible to measure which part can be specifically measured and which part can be measured while reducing the area where measurement is impossible. Is still unmeasurable on a single screen on the composite measurement image, making it easier to adjust the measurement image acquisition conditions so that the non-measurable area is reduced, which dramatically improves user convenience. . The highlight unit 212 measures the first measurement light projector 110A or the second measurement light projector 110B in a state where the combined measurement image generated by the measurement image combiner 211 is displayed on the display unit 400. An area where the data that has been saturated is saturated can be displayed as a saturated area in a state of being highlighted in a manner different from that of the unmeasurable area.

  As described above, information related to areas that can be measured by a plurality of measuring light projecting means and areas that cannot be measured can be collected on a single screen and displayed in a manner that is easy to grasp visually. This can be helpful when setting or adjusting image acquisition conditions.

  Here, the example in which the measurement abnormality region is superimposed and displayed on the composite measurement image SG by the highlight unit 212 has been described, but not limited thereto, the first measurement image and the second measurement image are also respectively displayed. It is also possible to display the measurement abnormal region in a superimposed manner. For example, in the example of FIG. 18 to be described later, the non-measurable region and the saturation region are superimposed and displayed on the first measurement image S1 obtained only for the second measurement light projecting unit 110B. In this way, in addition to simply displaying the non-measurable region and saturation region that occur for the measurement light projecting unit alone, as in the case of the above-described synthetic measurement image, taking into account the measurement result of the other measurement light projecting unit, By highlighting only the area where measurement cannot be performed by any of the measurement light projecting means or saturation, it is possible to easily adjust the setting of the measurement image acquisition condition appropriately.

  Further, in the above example, the example in which the highlighting unit 212 displays the non-measurable area or the saturated area superimposed on the measurement image has been described. However, the measurement is not limited to the measurement image, and the observation image is similarly measured. It is also possible to display the impossible area and the saturated area in a superimposed manner.

Furthermore, in the above example, the areas that have become non-measurable points or saturation points by any of the measuring light projecting means are displayed as non-measurable areas and saturation points, respectively. there there unmeasurable point unit may also be displayed to distinguish what was the saturation point. For example, the first non-measurable area that is not measurable by the first measurement light projecting unit 110A is displayed in light red, and the second non-measurable area that is not measurable by the second measurement light projecting unit 110B is displayed in dark red. To do. Similarly, highlight means is used so that the first measurement light projector 110A displays a saturated first saturated region in light yellow, and the second measurement light projector 110B displays a saturated second saturated region in dark yellow. In 212, the measurement image is colored. In addition, the highlight processing that distinguishes the measurement light projecting means can be performed on the composite measurement image SG as well as the first measurement image and the second measurement image. By doing so, it is possible to visually distinguish a region where measurement is impossible or saturated, and it can be used as a reference when adjusting the method of placing the object S or applying measurement light.

  For example, when adjusting the brightness of the first measurement image by the first measurement light projector 110A, the first measurement light projector 110A sets the first non-measurable region that becomes a shadow and the first saturation region that becomes saturated. The second measurement light projection unit 110B displays the second non-measurable area and the second saturation area in light red and light yellow, respectively, while displaying them in dark red and dark yellow. In consideration of the area that can be measured by the means (here, the second measuring light projecting unit 110B), in other words, the area that compensates for the deficiency of the first measurement image, the area that cannot be measured or becomes saturated is reduced. Thus, an environment that can be easily adjusted to the optimum measurement image acquisition condition is provided.

Further, in the above-described embodiment, the color colored by the highlight unit 212 is an example, and it is needless to say that other colors can be used as appropriate. The above-described measurement image synthesizing unit 211, highlight unit 212, and three-dimensional image synthesizing unit 213 to be described later are the CPU of the control unit 200 in the example of FIG. You can also
(Treatment of abnormal measurement points in composite measurement images)

  FIG. 12 shows an example of handling when a measurement abnormal point is included in any of the measurement images when the measurement image combining unit 211 combines the measurement images. As shown in this figure, when there is some abnormality in the reflected light acquired for each point, that is, for each pixel, in the first measurement light projection unit 110A and the second measurement light projection unit 110B, specifically, measurement is performed. for inability when saturated, for each combination, it should decide whether to use previously which data the composite measurement image. Here, when a normal point is acquired by any of the light projecting means, this normal point is used. In addition, when a saturation point and an unmeasurable point are included, the saturation point is used. By doing in this way, the precision of the synthetic | combination measurement image obtained can be improved by employ | adopting the more highly accurate one as data.

  Further, such a treatment can be used not only when generating a composite measurement image, but also when performing some measurement processing using the obtained composite measurement image. For example, when measuring the distance or height difference between arbitrary positions of the composite measurement image, it was determined in advance about which data to be used as the basis of such measurement, the data obtained by which light projecting means is adopted. Processing is performed based on certain rules. At this time, different rules can be set in addition to making the rule at the time of generating the composite measurement image the same as the rule at the time of measurement.

  In the above example, the handling of the unmeasurable point and the saturation point has been described, but the measurement abnormal point is not limited to the unmeasurable point and the saturation point, for example, the point where multiple reflection or light penetration occurs, It goes without saying that the treatment can be decided in the same way.

In addition, for example, a method for providing such data selection criteria is a method in which a table as shown in FIG. 12 is held in a storage device or the like in advance, and the measurement image composition unit 211 refers to the table for processing. Alternatively, the processing may be performed based on a procedure (for example, normal point> saturation point> unmeasurable point) in which high priority data is designated. Furthermore, in addition to the method of setting the rules as shown in FIG. 12, the user can arbitrarily set which data is used in each combination.
(Measurement microscope operation program)

As described above, in the example of FIG. 1, an operation program for operating the measurement microscope apparatus 500 is installed in the PC that is the control means 200. While the measurement microscope apparatus operation program is executed and the GUI screen is displayed on the display unit 400, the user operates the mouse and keyboard as the operation unit, sets various conditions, and acquires the measurement image. it can. User interface screens of the measurement microscope apparatus operation program are shown in FIGS. 5 and 13 to 18. In these drawings, FIG. 13 is an image diagram showing a state where the simple mode is selected in the GUI of the measurement microscope apparatus operation program, FIG. 14 is an image diagram showing a state where the “measurement image” button 428 is pressed from the state of FIG. 13 is an image diagram showing a state where an application mode is selected from the state of FIG. 13, FIG. 16 is an image diagram showing a state where the “measurement image” button 428 is pressed from the state of FIG. 15, and FIG. 17 is an image display area 410 from the state of FIG. FIG. 18 is an image diagram showing a state in which the measurement direction “only the left side” is selected from the state of FIG. 16. Each GUI screen is provided with an image display area 410 for displaying a measurement image and an observation image, and an operation area 420 including buttons for performing various operations is provided at the right end of the image display area 410. ing.
(Image display area 410)

In the image display area 410, an observation image and a measurement image can be displayed. In particular, in addition to displaying acquired and imaged high-resolution measurement images and observation images, measurement images obtained when the object S to be imaged is imaged under the currently set measurement image acquisition conditions and imaging conditions, By simply obtaining the observation image and updating it in real time on the display unit 400, the user compares and refers to changes in the measurement image and the observation image displayed in the image display area 410 before and after the setting. The measurement image acquisition conditions and the imaging conditions can be set. That is, since it is possible to check in real time the parameters that are currently set and the image that will be obtained when the image is taken at the position of the object S, the imaging conditions and measurement image acquisition that are visually in line with the image image desired by the user It is easy to set the conditions.
(Split display function)

  Further, the display unit 400 has a split display function, and the image display area 410 can be divided into two or more screens in addition to a mode of displaying one image. For example, in the example of FIG. 17, the first divided display area 411 is provided slightly larger on the left side of the image display area 410, and the slightly narrow right side thereof is divided into two in the vertical direction, the second divided display area 412 and the third divided display area. 413. The vertical and horizontal ratios of the first divided display area 411 to the third divided display area 413 are preferably the same. At this time, the first divided display area 411 is adjusted to have the same ratio as the second divided display area 412 and the third divided display area 413 by providing masks on the upper and lower sides.

  In order to perform split display of the image display area 410, for example, as shown in FIG. 16, the “3D scan” tab 421 is selected by the image mode switching means, the “expert” button 425 is selected by the measurement image acquisition mode selection means, In the state where “both sides” is selected in the “measurement direction” selection field 470, the “manual” button in the “brightness adjustment for measurement” field 440 is selected. As a result, the screen is divided and displayed on three screens as shown in FIG. If the “Auto” button is selected in the “Brightness adjustment for measurement” field 440, the split display is canceled and the image display area 410 returns to the one-screen display as shown in FIG. A type display field 415 for displaying the type of the displayed image can be added to each divided area. In the example of FIG. 17, types such as “left and right composite”, “left floodlight”, and “right floodlight” are displayed as characters in the upper left of each divided display area as the type display field 415 to identify each image. It is easy. In addition to or instead of the character string, an icon indicating the direction of the measurement light may be displayed in the type display field 415. In the example of FIG. 17, the display contents of each divided display area are visually shown to the user by displaying the measurement light projecting unit and the icon illustrating the spread of the measurement light projected from the left side of the character string. It is easy to understand.

  The aspect of dividing the image display area into three parts is not limited to the above-described example. For example, the image display area is equally divided into three parts to display the composite measurement image, the first measurement image, and the second measurement image, Alternatively, various modes such as displaying each measurement image in a separate window can be used as appropriate.

In the example of FIG. 17, the composite measurement image SG of the object S in the first divided display area 411, the second measurement image S <b> 2 of the same object S in the second divided display area 412, and the first measurement in the third divided display area 413. Each image S1 is displayed. In order to display the images while updating each image in real time, the measurement light is alternately irradiated to the object S from the first measurement light projection unit 110A and the second measurement light projection unit 110B. In the second divided display area 412, an image of the object S when the measurement light is irradiated from the second measurement light projecting unit 110B is displayed. In the third divided display area 413, an image of the object S when the measurement light is irradiated from the first measurement light projecting unit 110A is displayed. Thereby, the user can distinguish and recognize the image of the object S when the measurement light is irradiated by each of the first measurement light projection unit 110A and the second measurement light projection unit 110B. The frequency of measurement light switching is, for example, about several Hz to several tens Hz.
(Operation area 420)

The operation area 420 is provided with buttons, slide bars, input fields, etc. for performing various settings and operations. Also, by selecting and changing various modes, the buttons displayed in accordance with this can be changed. In addition, arrangement | positioning of the buttons shown below is an illustration, Comprising: It can arrange | position in arbitrary aspects.
(Image mode switching means)

The measurement microscope apparatus operation program can switch between an observation image mode for capturing an observation image of the object S and a measurement image mode for acquiring a measurement image of the object S by an image mode switching unit. In this example, a “microscope” tab 422 that collects buttons related to an observation mode and a “3D scan” tab 421 that collects buttons related to a measurement image acquisition mode are provided as image mode switching means. By selecting a tab, the image mode can be switched between the observation image mode and the measurement image mode.
(Measurement image acquisition mode selection means)

This measurement microscope device operation program can be switched between a simple mode that makes it easy for beginners to set measurement image acquisition conditions and an application mode that allows users to set more detailed measurement image acquisition conditions. . Therefore, in the operation area 420, each image mode tab is provided with measurement image acquisition mode selection means for selecting the simple mode and the application mode in the upper column. In the example of FIG. 13, a “1shot-3D” button 424 for selecting a simple mode and an “expert” button 425 for selecting an application mode are provided as measurement image acquisition mode selection means.
(Image switching means)

Further, below the measurement image acquisition mode selection means, there is provided an image switching means capable of switching the displayed image between the observation image and the measurement image. In this example, when an “observation image” button 427 is pressed as an image switching unit, an observation image captured using an observation illumination light source is displayed in the image display area 410, and when a “measurement image” button 428 is pressed, A measurement image acquired using the measurement light projecting unit is displayed in the image display area 410. Here, the parameter that changes the brightness of the measurement light is the exposure time of the camera.
(Measurement image acquisition procedure)

  Next, a procedure for acquiring a measurement image using an operation program of the measurement microscope apparatus will be described based on the flowchart of FIG. First, in step S1, the object S is set on the stage 140, and an initial image is displayed. At this stage, since the measurement image has not been acquired yet, for example, an observation image is used as the initial image. Here, the brightness of the illumination light when capturing the observation image is automatically adjusted. In the example shown in FIG. 13, the observation image SO is displayed in real time in the image display area 410. Further, as the initial image, instead of the observation image, the structured illumination pattern of the measurement light projected from the measurement light projecting unit is obtained by projecting from all points, and is a full projection image of structured illumination. You can also. The brightness of the measurement light in this case is also automatically adjusted.

Next, in step S2, the measurement image acquisition mode is selected from the measurement image acquisition mode selection means. Here, either the simple mode or the application mode can be selected by the measurement image acquisition mode selection means. In the example of FIG. 13, the simple mode is selected when the “1shot-3D” button 424 is pressed, and the application mode is selected when the “expert” button 425 is pressed.
(Easy mode)
(Measurement light brightness adjustment means)

  When the simple mode is selected in step S2, the process proceeds to step S3, and an observation image as shown in FIG. 13 is displayed. Here, when the “Measure” button 430 provided at the lower part of the operation area 420 on the right side is pressed on the screen of FIG. 13, the process proceeds to Step 4 to automatically adjust the brightness of the measurement light (camera exposure time or light amount). After that, the measurement is started.

When the “measurement image” button 428 is pressed in the state of FIG. 13, a measurement image is simply acquired and displayed in the image display area 410 as shown in FIG. In this state, the brightness of the measurement image is automatically adjusted, but the user can manually adjust the brightness of the measurement light (camera exposure time or light amount) using the measurement light brightness adjustment means. (Step S4).
(Three-dimensional image composition means 213)

When the “measurement” button 430 provided in the lower part of the operation area 420 is pressed in the state where the brightness is adjusted in this way, a normal measurement image is acquired (step S5). Further, a composite image ST in which the observation image SO is combined with the measurement image is generated by the three-dimensional image combining unit 213 and displayed on the display unit 400. The three-dimensional image combining unit 213 combines the observation image captured using the observation illumination light source and the measurement image captured using the measurement light projecting unit to generate a three-dimensional composite image ST. That is, it is possible to generate a three-dimensional image in which texture information obtained from the observation image is provided with unevenness using the height information of the measurement image. In the example illustrated in FIG. 20, a synthesized image ST obtained by synthesizing the observation image as a texture image using the height information of the measurement image is displayed in a three-dimensional manner on the image display area 410. The composite image ST has a three-dimensional shape, and its position, posture, and angle can be arbitrarily changed. For example, the synthesized image ST can be moved and rotated by dragging the synthesized image ST on the image display area 410 with a mouse or the like.
(Texture ratio adjusting means 452)

  The ratio of the measurement image to the observation image of the composite image ST is adjusted by the texture ratio adjustment unit 452. The texture ratio adjusting unit 452 is configured, for example, in a slider shape, and can continuously change the ratio between the measurement image (height image) and the observation image (texture image) by moving the slider left and right. You can also input the ratio numerically, or a specified numerical value (for example, 0%, 25%, 50%, 75%, 100%; or 0: 1, 0.5: 1, 1: 1, 2: 1, (3: 1, 4: 1, etc.) can be selected using a drop box or combo button. In the example of FIG. 20, the ratio between the measurement image (height) and the observation image (texture) of the composite image ST is expressed as a percentage occupied by the observation image (texture). The ratio of (texture) is set to 100%. For example, when the ratio of the observed image (texture) is adjusted to 0%, that is, the measurement image (height) is adjusted to 100% by the texture ratio adjusting unit 452, the display is switched to the display shown in FIG. The display of the composite image ST in the image display area 410 is updated in real time in response to the adjustment of the texture ratio adjusting unit 452. The user can adjust the ratio of the measurement image and the observation image to a desired value by using the texture ratio adjusting unit 452 while referring to the composite image ST displayed in the image display area 410. In this example, the initial value of the texture ratio adjusting unit 452 after generation of the composite image ST is 100% of the observed image (texture), but the default value is an arbitrary value such as 50%, for example 50%. May be set.

  In addition, the measurement image can be displayed with different heights. For example, in a contour line, the low area is blue, the high area is red, and the middle area is colored so as to change continuously from blue → green → yellow → orange → red, etc. Can be easily recognized. The color to be colored and the height separation that makes the colors different can be arbitrarily set. In this example, a scale that is color-coded for each height is displayed at the upper left of the image display area 410 so that the user can easily grasp the relationship between the color and the height visually.

  Furthermore, buttons for performing various processes on the composite image ST are provided in the operation area 420. For example, if the height magnification slide bar 453 is adjusted, the magnification in the height direction of the composite image ST can be adjusted. As a result, it is possible to highlight and display fine unevenness, or to smooth the fine unevenness and to help grasp the overall shape. In addition, measurement abnormal points can be superimposed on the composite image ST, the light source can be placed at an arbitrary position, the stereoscopic effect can be emphasized by changes in shadows, scales can be displayed in a grid, and simple dimensions can be displayed. Various operations such as measurement can be performed from the operation area 420.

  Furthermore, even after the composite image ST is generated, the display on the display unit 400 can be switched to the measurement image and the observation image. In the example of FIGS. 20 and 21, the display of the image display area 410 can be switched with one touch by the image display switching means 454 provided in the upper stage of the operation area 420. 20 and 21, the “3D” button 455 of the image display switching unit 454 is selected. When the “texture” button 456 is pressed in this state, the screen is switched to the screen of FIG. An observation image is displayed on the top. Similarly, when the “height” button 457 is pressed by the image display switching means 454, the display of the image display area 410 is switched to the measurement image. The user can perform various operations on the synthesized image ST obtained in this way as necessary. In addition, in order to switch to an analysis program for the composite image ST or the measurement image, a “to analysis application” button 450 provided at the top of the operation area 420 is pressed. This switches to the analysis program.

As described above, according to the simple mode, the three-dimensional composite image ST can be almost automatically acquired by pressing the “Measure” button without being particularly aware of the setting items related to the three-dimensional measurement.
(Application mode)

On the other hand, when the application mode is selected in step S2, the process proceeds to step S6, and the measurement light is manually adjusted. Here, as shown in FIG. 15, the observation image is displayed in the image display area 410 as the initial image as in FIG. 13. On this screen, a texture image to be pasted as a composite image ST can be selected for a measurement image acquired later. When the “image improvement” button 481 is further pressed, an image improvement panel 480 is displayed in the operation area 420 as shown in FIG. From the image improvement panel 480, edge enhancement, offset, gamma correction, white balance, and the like of the observation image can be adjusted.
(Texture image)

  The texture image is selected by the texture image selection means 460. In the example of FIG. 15, in addition to a normal observation image, either an HDR image or a depth composite image can be selected with a radio button. Here, an HDR (High Dynamic Range) image is generated by imaging a plurality of observation images while changing the camera exposure time, and then combining these images with a high dynamic range (HDR). Depth composite image, if the difference in height of the measurement target part of the object S exceeds the depth of field, only the in-focus part is extracted and synthesized from the observation images taken individually with different height directions It is an image.

When the texture image is selected in this way, the “measurement image” button 428 is pressed from the image switching means provided in the operation area 420 in FIG. 15 to switch to the screen in FIG. This screen is a measurement image acquisition condition setting screen, and various members for setting the measurement image acquisition condition are arranged in the operation area 420. In this example, an “e preview” button, a “measurement mode” selection field 472, a “measurement direction” selection field 470, and a “brightness adjustment for measurement” field 440 are provided in order from the top. On this screen, the brightness of the measurement light is adjusted while checking the measurement image acquisition conditions.
("Measurement mode" selection field 472)

The “measurement mode” selection field 472 can select a measurement method (stripe pattern). In this example, “standard” is selected, and “fine mode” for removing indirect light and “halation removal” can also be selected. When “Hallation Removal” is selected, the camera exposure time can be changed to capture multiple images, and these can be combined to compensate for parts that are overexposed or underexposed from other images. It becomes possible. Furthermore, “super fine” can be measured while removing indirect light and removing halation. In the example of FIG. 24, any of standard, fine, halation removal, and super fine can be selected from the “measurement mode” selection field 472 using a pull-down menu.
("Measurement direction" selection field 470)

In the “measurement direction” selection column 470, a measurement light projecting unit is selected. Here, either the first measurement light projector 110A or the second measurement light projector 110B can be selected. In the screen example of FIG. 25, when “only the left side” is selected from the pull-down menu of the “measurement direction” selection field 470, the second measurement light projector 110B is selected as the measurement light projector, and the left side of the object S The second measurement image S2 irradiated with the second measurement light is displayed in the image display area 410. Similarly, when “only the right side” is selected, each first measurement light projecting unit 110A is selected, and the image display region 410 is displayed on the first measurement image S1 irradiated with the first measurement light from the right side of the object S. The contents are switched. When “both sides” is further selected, a combined measurement image SG obtained by combining the second measurement image and the first measurement image is displayed in the image display area 410.
(Measurement light brightness adjustment means)

  Further, a “measurement brightness adjustment” column 440 is provided in the middle of the operation area 420 on the right side of FIG. The brightness of the measurement light is adjusted by the camera exposure time and the amount of light. Here, when “Auto” is selected in the “Brightness adjustment for measurement” column 440, the brightness of the measurement light can be continuously varied by adjusting the slider provided below it to the left and right. This slider displays the brightness of the measurement light as a numerical value at the top. It is also possible to directly input the brightness of the measurement light as a numerical value. When the measurement light brightness is adjusted by the measurement light brightness adjustment unit in this way, the brightness of the measurement image displayed in the image display area 410 is updated, and the user adjusts the brightness. Adjustments can be made while checking the results in real time.

In the above example, the brightness in the composite measurement image SG is adjusted by the measurement light brightness adjusting means. That is, as shown in FIG. 16, “both sides” is selected in the “measurement direction” selection field 470, and the composite measurement image SG is displayed in the image display area 410, and the measurement light brightness adjusting means is displayed in the operation area 420. A “measurement brightness adjustment” column 440 is displayed. The “measurement brightness adjustment” column 440 similarly adjusts the light amounts of the first measurement light projection unit 110A and the second measurement light projection unit 110B, which are measurement light projection units. If “left side only” or “right side only” is selected in the “measurement direction” selection field 470, the measurement image captured by each selected measurement light projecting means is displayed in the image display area 410 as described above. Therefore, the amount of light of the second measurement light projection unit 110B or the first measurement light projection unit 110A can be adjusted in the “brightness adjustment for measurement” column 440, respectively.
(Measurement light brightness individual adjustment means 442)

On the other hand, the light amounts of the first measuring light projecting unit 110A and the second measuring light projecting unit 110B can be individually adjusted. As shown in FIG. 16, when “both” is selected in the “measurement direction” selection field 470 and “manual” is selected in the “brightness adjustment for measurement” field 440, the screen shown in FIG. The measurement light brightness individual adjusting means 442 capable of individually adjusting the brightness of the light projecting unit 110A and the second measurement light light projecting unit 110B is displayed in the operation area 420. Here, the measurement light brightness individual adjusting means 442 is configured in a slider shape capable of adjusting the brightness for each measurement light projecting means. In this example, the brightness adjustment slider 446 for the second measurement light projector 110B and the brightness adjustment slider 444 for the first measurement light projector 110A are arranged vertically. By individually moving these brightness adjustment sliders 444 and 446 to the left and right, the brightness level of each measurement image can be individually adjusted. In addition, as described above, the display of the measurement image in the image display area 410 is updated according to the value adjusted by the measurement light brightness individual adjustment unit 442, and the user adjusts the desired brightness while checking the measurement image in real time. Is possible. In addition, although it demonstrated that the light quantity of the measurement light projector part was adjusted here for convenience of explanation, since it is the purpose of adjusting the brightness of the first measurement image and the second measurement image, the measurement light projector part is actually used. As described above, the brightness can be adjusted not only by adjusting the amount of light but also by adjusting the camera exposure time.
(Image connection mode)

  In the lower part of the operation area 420, an “image connection mode” selection field is provided. When this “image connection mode” selection field is turned ON, the image connection mode is selected, data can be measured continuously while moving the stage vertically and horizontally, and the data can be combined as one measurement data.

  In the “measurement direction” selection column 470, the direction of measurement light can be selected as described above. In this example, as shown in FIG. 25, any of “both”, “left side only”, and “right side only” can be selected, and the display content of the image display area 410 is switched to the corresponding content according to the selected item. It is done. For example, in the example of FIG. 18, “left side only” is selected, and the second measurement image S2 obtained by the second measurement light projector 110B that is the left measurement light projector is displayed in the image display area 410. Is displayed. Further, at this time, the second measurement image S2 is displayed by the highlighting unit 212 by the second measurement light projecting unit 110B in which the measurement light is shaded and the measurement impossible region that cannot be measured is red and the saturation region is yellow. Has been.

  When switching to “both” in this state, the screen is switched to the screen of FIG. 16, and the first measurement light projection unit 110A and the first measurement light projection unit 110B obtained by both measurement light projection units. A combined measurement image SG obtained by combining the image S1 and the second measurement image S2 is displayed in the image display area 410. Further, at this time, in the combined measurement image SG, the highlight unit 212 causes the measurement light to be shaded by any of the first measurement light projecting unit 110A and the second measurement light projecting unit 110B. And the saturated regions are displayed in yellow, respectively.

  As is apparent from a comparison between FIG. 16 and FIG. 18, it can be seen that both the unmeasurable region and the saturated region have fewer composite measurement images SG. That is, in the obtained composite measurement image SG, the measurement abnormal region is actually much narrower than the non-measurable region and the saturation region that are apparent from one of the measurement light projecting units. It can be understood that it is more appropriate and easy to adjust the measurement image acquisition condition so that the measurement abnormal region is narrowed on an SG basis.

  Furthermore, if necessary, the image display area 410 can be divided so that the combined measurement image SG and each measurement image that is the source thereof are displayed simultaneously on one screen. That is, when “manual” is selected in the “brightness adjustment for measurement” field 440 of the operation area 420 on the screen of FIG. 16, the image display area 410 is divided into three parts as shown in FIG. The combined measurement image SG is displayed in 411, the second measurement image S2 is displayed in the second divided display area 412, and the first measurement image S1 is displayed in the third divided display area 413. As a result, the measurement abnormal areas by the respective measurement light projecting means can be confirmed while contrasting with each other. Therefore, the measurement image acquisition conditions such as the position and orientation of the object S, the brightness of the measurement light, and the like are excellent. Can be adjusted. In addition, on the screen of FIG. 17, the brightness of each measurement light can be individually adjusted using the measurement light brightness individual adjusting means 442 as described above.

  Even in the simple mode described above, it is possible to check such a measurement inability point and a saturation point. For example, if “measurement image” is displayed in step S 3, it is possible to display a measurement impossible point and a saturation point in a two-sided composite image.

  In this manner, measurement image acquisition conditions are set and adjusted in the application mode. Then, in step S7 in the flowchart of FIG. 19, it is determined whether or not the brightness of the measurement light is appropriate. On the other hand, if the brightness of the measurement light is not yet appropriate, the process proceeds to step S8 to select the measurement mode and adjust the measurement brightness.

When the measurement light is appropriately set in this way, the process proceeds to step S9 to determine whether or not the texture image needs to be set. If necessary, a texture image is set in step S10. Here, in the screen of FIG. 15, the texture image selection means 460 is used to select the texture image.
(Observation Image Imaging Condition Setting Unit 490)

  Moreover, the imaging condition of an observation image is set as needed. In the upper part of the image display area 410 in FIG. 15, observation image imaging condition setting means 490 for setting the imaging conditions for such an observation image is provided. The observation image imaging condition setting means 490 includes settings such as shutter speed switching for imaging an observation image, imaging magnification, and focus adjustment. In the example shown in FIG. 26, the brightness of the imaging means is selected from “auto” or “manual”. When “Manual” is selected, the brightness of the imaging means is adjusted by the camera brightness adjustment slider 492. Further, such setting of the observation image capturing condition can be performed even in the simple mode. For example, also in FIG. 13, the observation image imaging condition setting means 490 is provided in the upper stage of the image display area 410 as described above, and magnification, focus adjustment, shutter speed switching, and the like can be performed from here.

  In obtaining the measurement image, the observation image can be taken arbitrarily. For example, when the composite measurement image or the observation image is not necessary, steps S9 and S10 can be omitted in the flowchart of FIG.

  When all the imaging conditions have been set in this way, the process proceeds to step S11 to obtain a measurement image. Here, when the “Measure” button 430 is pressed from the screen of FIG. 17 or the like, a measurement image is acquired, and a composite image obtained by adding a texture image to the measurement image is displayed in the image display area 410 (step S9). . Subsequently, the user performs a measurement operation as necessary. In order to switch to the measurement program, the “to analysis application” button 450 provided at the top of the operation area 420 is pressed to switch to the analysis program.

  As described above, in the application mode, the user can adjust more detailed conditions regarding the acquisition of the measurement image. As a result, a user who is familiar with the operation can set a desired condition. On the other hand, for a user who is not familiar with the operation, the simple mode is provided as described above so that a single setting can be automatically performed. In this way, by changing the setting items to be provided in the simple mode and the application mode and changing the parameters that can be set by the user, it is possible to provide an operating environment according to the user's proficiency level and requirements.

  The measurement microscope apparatus, the measurement microscope apparatus operation program, and the computer-readable recording medium of the present invention can be suitably used for inspection apparatuses and digitizers that use the principle of triangulation.

DESCRIPTION OF SYMBOLS 100 ... Imaging means 110 ... Light projection part; 110A ... First measurement light light projection part; 110B ... Second measurement light light projection part 111 ... Measurement light source 112 ... Pattern generation parts 113-115, 122, 123 ... Lens 120 ... Light reception Unit 121 ... Camera 121a ... Image sensor 130 ... Illumination light output unit 140 ... Stage 141 ... XY stage 142 ... Z stage 143 ... [theta] stage 144 ... Stage operation unit 145 ... Stage drive unit 150 ... Measurement control unit 200 ... Control means 210 ... CPU
211 ... Measured image composition means 212 ... Highlight means 213 ... 3D image composition means 220 ... ROM
230 ... Working memory 240 ... Storage device 250 ... Operating unit 300 ... Light source unit 310 ... Control board 320 ... Observation illumination light source 400 ... Display unit 410 ... Image display area 411 ... First divided display area 412 ... Second divided display area 413 ... Third divided display area 415 ... Type display field 416 ... First display area 417 ... Second display area 420 ... Operation area 421 ... "3D scan" tab 422 ... "Microscope" tab 424 ... "1shot-3D" button 425 ... "Expert" button 427 ... "Observation image" button 428 ... "Measurement image" button 430 ... "Measurement" button 440 ... "Measurement brightness adjustment" column 442 ... Measurement light brightness individual adjustment means 444 ... Brightness adjustment Slider 446 ... Brightness adjustment slider 450 ... "To analysis application" button 452 ... Texture ratio adjustment means 453 ... Height Rate slide bar 454 ... Image display switching means 455 ... "3D" button 456 ... "Texture" button 457 ... "Height" button 460 ... Texture image selection means 470 ... "Measurement direction" selection field 472 ... "Measurement mode" selection field 480 ... Image improvement panel 481 ... "Image improvement" button 490 ... Observation image imaging condition setting means 492 ... Camera brightness adjustment slider 500 ... Measuring microscope device Sh ... Hole Ss ... Shadow S1 ... First measurement image S2 ... Second measurement image SG ... Composite measurement image SO ... Observation image ST ... Composite image

Claims (15)

As measurement light projecting means for projecting measurement light as a structured illumination of a predetermined pattern from an oblique direction to the object,
A first measuring light projecting means capable of irradiating the object with the first measuring light from the first direction;
Second measurement light projecting means capable of irradiating the object with the second measurement light from a second direction different from the first direction;
An illumination light source for observation for capturing an observation image;
The measurement light projected by the first measurement light projection means or the second measurement light projection means and reflected by the object is acquired to take a measurement image, and the observation image is obtained using the observation illumination light source. Imaging means for imaging
Display means for displaying the measurement image or the observation image;
In a state where the measurement image or the observation image is displayed on the display unit, a measurement abnormality region in which the measurement result is abnormal in both the first measurement light projection unit and the second measurement light projection unit is displayed in an overlapping manner. A measuring microscope apparatus comprising: highlight means for performing the operation. The measurement microscope apparatus according to claim 1, wherein the first measurement image acquired by the imaging unit using the first measurement light projecting unit and the second measurement are further performed on the same object. A measurement image combining unit that combines the second measurement image acquired by the imaging unit using a light projecting unit and generates one combined measurement image,
A measurement microscope apparatus characterized in that the highlight means can display a measurement abnormal region in an overlapping state in a state where the composite measurement image generated by the measurement image composition means is displayed on the display means. A measuring microscope apparatus according to claim 1 or 2,
The measurement microscope apparatus characterized in that the measurement abnormal region includes an unmeasurable region in which measurement light is shaded by any measurement light projecting unit and data cannot be acquired by the imaging unit. A measurement microscope apparatus according to claim 3, wherein
The measurement abnormal region includes a saturated region where the reflected light of the measurement light detected by the imaging unit is saturated by any measurement light projecting unit,
The measurement microscope apparatus is characterized in that the highlighting unit is configured to highlight the saturation region in a mode different from the non-measurable region and to display the saturation region on the display unit. A measuring microscope apparatus according to claim 2 ,
When the measurement image synthesizing unit generates one composite measurement image by the first measurement image and the second measurement image, one of the pixels corresponding to the first measurement image and the second measurement image is normal. A measurement microscope apparatus configured to generate a composite measurement image using a normal value when the other includes a measurement abnormal region. A measurement microscope apparatus according to any one of claims 1 to 5,
The measurement microscope apparatus, wherein the composite measurement image is an image configured by using a pixel having a higher pixel value among pixels corresponding to the first measurement image and the second measurement image. A measurement microscope apparatus according to any one of claims 1 to 5,
The measurement microscope apparatus, wherein the composite measurement image is an image configured by using an average of pixel values among pixels corresponding to the first measurement image and the second measurement image. A measurement microscope apparatus according to any one of claims 1 to 5,
The measurement microscope apparatus, wherein the composite measurement image is an image configured by using a pixel having a lower pixel value among pixels corresponding to the first measurement image and the second measurement image. A measurement microscope apparatus according to any one of claims 1 to 8,
An image display area for displaying a measurement image is divided on the display means,
A first divided display area for displaying the composite measurement image;
A second divided display area for displaying the second measurement image;
A third divided display area for displaying the first measurement image;
A measuring microscope apparatus comprising: The measurement microscope apparatus according to claim 9, further comprising measurement light brightness individual adjustment means capable of individually adjusting the brightness of the first measurement image and the brightness of the second measurement image. A measuring microscope apparatus characterized by that. A measuring microscope apparatus according to claim 9 or 10,
The composite measurement image, the second measurement image, and the first measurement image are acquired in a time-sharing manner, and are updated at regular intervals in the first split display region, the second split display region, and the third split display region, respectively. A measuring microscope apparatus characterized by being capable of being displayed. The measurement microscope apparatus according to claim 1, further comprising: a setting screen for setting measurement image acquisition conditions for acquiring a measurement image with the imaging unit using the measurement light projecting unit A measurement microscope apparatus comprising measurement image acquisition mode selection means capable of switching between a simple mode enabling simple setting and an application mode enabling detailed setting. The measurement microscope apparatus according to any one of claims 1 to 12, further comprising combining an observation image captured using the observation illumination light source and a measurement image captured using a measurement light projecting unit. And a three-dimensional image composition means for generating a three-dimensional composite image. As measurement light projecting means for projecting measurement light as a structured illumination of a predetermined pattern from an oblique direction to the object,
A first measuring light projecting means capable of irradiating the object with the first measuring light from the first direction;
Second measurement light projecting means capable of irradiating the object with the second measurement light from a second direction different from the first direction;
A measurement microscope apparatus comprising an imaging unit for acquiring measurement light that is projected by the first measurement light projecting unit or the second measurement light projecting unit and reflected by an object, and capturing a measurement image, An operating program for operating a computer,
For the same object, the first measurement image acquired by the imaging unit using the first measurement light projecting unit and the first measurement image acquired by the imaging unit using the second measurement light projecting unit. An image composition function for synthesizing two measurement images and generating one composite measurement image;
A highlight function for displaying a measurement image or an observation image displayed on the display unit with a measurement abnormal region in which the measurement result is abnormal in both the first measurement light projection unit and the second measurement light projection unit. A measurement microscope apparatus operation program characterized by realizing the above.   The computer-readable recording medium which recorded the program of Claim 14.