JP6667394B2 - Magnifying observation device - Google Patents

Description

  The present invention relates to a magnifying observation apparatus for magnifying and observing an observation target.

  In order to magnify and observe an observation target, a magnifying observation apparatus may be used (for example, see Patent Document 1). In the microscope system described in Patent Literature 1, bright-field illumination light and dark-field illumination light are applied to a specimen on a stage through an objective lens. Here, the bright-field illumination light is illumination light emitted in a direction parallel to the optical axis of the objective lens, and the dark-field illumination light is illumination light emitted in a direction inclined with respect to the optical axis of the objective lens. . The sample is imaged by the observation light reflected by the sample being incident on the imaging device through the imaging lens.

JP 2013-72971 A

  In the microscope system described in Patent Literature 1, the illumination intensity of one illumination light is relatively reduced according to the ratio of the exposure time to the bright-field illumination light and the exposure time to the dark-field illumination light. Thereby, the intensity of the bright-field illumination light and the intensity of the dark-field illumination light are made uniform. As a result, Patent Literature 1 describes that it is possible to observe a sample with the most suitable illumination intensity when observing the sample that has been simultaneously irradiated with bright-field illumination light and dark-field illumination light. . Further, the user can intuitively adjust the ratio of the illumination intensity between the bright-field illumination light and the dark-field illumination light from the image in the state where the illumination intensity is optimized.

  However, appropriate imaging conditions such as illumination intensity vary depending on the shape and material of the observation target. Therefore, it is difficult for an unskilled user to acquire an image captured under appropriate imaging conditions. In some cases, it may be determined after the fact that the imaging conditions were not appropriate. In such a case, it is necessary to re-image the observation target under different imaging conditions, and the burden on the user increases.

  An object of the present invention is to provide a magnifying observation apparatus capable of easily acquiring an image of an observation target according to a user's request.

(1) A magnifying observation apparatus according to the present invention is a magnifying observation apparatus capable of displaying an image of an observation target imaged according to set imaging conditions, wherein first and second emission beams different from each other are displayed on the observation target object. a first light projecting portion for selectively irradiating direction of the light, receiving the light from the observation object, an image of the observation object when the light of the first and second emission direction is irradiated respectively An imaging unit for generating a plurality of original image data including the first and second original image data shown, an illumination condition setting unit for setting an illumination condition including a specific emission direction input by a user, and an illumination condition setting The ratio of the combination of the first original image data and the second original image data is determined based on the lighting conditions set by the section, and the first original image data and the second original image data are determined at the determined ratio. By combining, a specific emission direction A data generating unit that generates display image data indicating an image of the observed object to be obtained when it is assumed that light is irradiated on the observation target object, the stored display image data in a general-purpose image file formats One area and a second area for storing illumination conditions and a plurality of original image data as metadata in a non-generic image file format, and associating the illumination conditions, the plurality of original image data and the display image data with each other. And a editing unit that updates the display image data stored in the first area by editing the metadata stored in the second area of the storage unit.

  In this magnifying observation apparatus, a plurality of original images showing an image of the observation target are obtained by irradiating the observation target with light from the first light projecting unit and receiving light from the observation target by the imaging unit. Data is generated. The illumination condition set by the illumination condition setting unit and the plurality of original image data generated by the imaging unit are stored in the second area of the storage unit as metadata in a non-generic image file format.

  Display image data indicating an image of the observation target that should be obtained when it is assumed that light is irradiated on the observation target under the illumination condition set by the illumination condition setting unit is one or a plurality of display image data generated by the imaging unit. Generated based on the original image data. The generated display image data is stored in the first area of the storage unit in a general-purpose image file format. The display image data of the first area, the illumination condition of the second area, and the plurality of original image data are associated with each other. When the metadata stored in the second area is edited by the editing unit, the display image data stored in the first area is updated.

  According to this configuration, the display image data stored in the first area is updated by editing the metadata stored in the second area of the storage unit. Therefore, it is possible to generate display image data indicating an image of the observation target under appropriate imaging conditions by editing the metadata so that appropriate illumination conditions according to the shape and material of the observation target are obtained. it can. Thereby, it is possible to easily obtain an image of the observation target object according to the request of the user.

  Further, since the display image data is stored in a general-purpose image file format, the user can observe an image based on the display image data using a general display device different from the magnifying observation device. it can. Furthermore, since the display image data can be generated using the first and second original image data in a non-general image file format that has already been generated, it is not necessary to take an image of the observation target again. Therefore, the burden on the user can be reduced.

( 2 ) The data generation unit may generate the display image data by combining the first and second original image data stored in the second area of the storage unit. In this case, the first and second original image data respectively corresponding to the light in the first and second emission directions different from each other are combined. This makes it possible to easily generate display image data indicating an image of the observation target that should be obtained when it is assumed that light in an arbitrary emission direction has been irradiated on the observation target.

( 3 ) The data generation unit may determine the combination ratio of the first and second original image data based on the illumination condition stored in the second area of the storage unit. In this case, display image data can be more easily generated based on the first and second original image data.

( 4 ) The magnifying observation apparatus further includes a display unit that displays an index indicating a light emission position, and a first operation unit that is operated by a user to specify the position of the index displayed on the display unit. In addition, when the position of the index is designated by the first operation unit, the editing unit may determine the specific emission direction based on the position of the designated index. In this case, the user can easily specify a specific emission direction by operating the position of the indicator on the display unit using the first operation unit.

( 5 ) The first operation unit is operated by the user to change the position of the index displayed on the display unit, and when the position of the index is changed by the first operation unit, the editing unit performs the change. The illumination condition stored in the second area of the storage unit may be edited based on the position of the index thus set. In this case, the user can easily change the specific emission direction.

( 6 ) The editing unit stores the lighting conditions before and after editing in the second area of the storage unit, and the first operation unit stores the plurality of lighting conditions stored in the second area of the storage unit. If the user operates to specify one of them and the lighting condition is specified by the first operation unit, the data generation unit generates display image data corresponding to the specified lighting condition, May display an index at a position corresponding to the lighting condition specified by the first operation unit.

  In this case, the user specifies the desired lighting condition previously stored in the second area of the storage unit, thereby reacquiring the display image data corresponding to the lighting condition, and setting the position of the index. Can be returned to the position corresponding to the lighting condition. Thereby, the user can determine whether or not the specified lighting condition is appropriate. Alternatively, the user can search for a more appropriate lighting condition based on the designated lighting condition.

( 7 ) The metadata further includes an imaging condition at the time of generating the display image data, and the magnifying observation apparatus is used to specify any one of the plurality of imaging conditions stored in the second area of the storage unit. The image processing apparatus may further include a second operation unit operated by a user, and an imaging condition setting unit that updates the set imaging condition to an imaging condition specified by the second operation unit.

  In this case, the user can reset the imaging condition set by the imaging condition setting unit to the specified imaging condition by designating the desired imaging condition previously stored in the second area of the storage unit. As a result, the original image data can be generated again under the same imaging conditions as those used when the desired original image data was previously generated.

( 8 ) The imaging condition set by the imaging condition setting unit includes the light receiving time of the imaging unit, and the imaging unit may generate a plurality of original image data based on the light receiving time set by the imaging condition setting unit. Good. In this case, the original image data can be generated again in the same light receiving time as the light receiving time when the desired original image data was previously generated.

( 9 ) The magnifying observation apparatus is configured such that the objective lens arranged to condense light from the observation target and guide the light to the imaging unit, and a focal position of the light passing through the objective lens relative to the observation target. Further comprising: a focus position changing unit that changes the focal position of light in the optical axis direction of the objective lens. Based on the data, display image data indicating an image focused on each part of the observation target is generated, and the imaging condition set by the imaging condition setting unit is such that the data generation unit focuses on each part of the observation target. May further include generating display image data indicating an image that matches.

  In this case, under the same imaging conditions as the previously generated desired display image data, display image data indicating an image focused on each part of the observation target object can be generated again. Further, the user can observe an image in which each part of the observation object is focused.

( 10 ) The imaging condition set by the imaging condition setting unit further includes a range of the focal position of the light in the optical axis direction of the objective lens, and the focal position changing unit sets the focal position of the light in the range set by the imaging condition setting unit. The focal position may be changed in the direction of the optical axis of the objective lens.

  In this case, the plurality of original image data can be generated again while changing the focal position of the light in the same range as when the desired plurality of original image data was previously generated. Further, since the focal position changing unit does not need to change the focal position of light within an excessively large range, a plurality of original image data can be generated at high speed.

( 11 ) The imaging unit generates original image data in a state where a plurality of light reception times are changed, and the data generation unit is based on a plurality of original image data generated by the imaging unit in a state where a plurality of light reception times are changed. Generating the display image data whose dynamic range has been adjusted by the control unit, and the imaging condition set by the imaging condition setting unit may further include that the data generation unit generates the display image data whose dynamic range has been adjusted. .

  In this case, the display image data whose dynamic range has been adjusted can be generated under the same imaging conditions as the previously generated desired display image data. Further, when the dynamic range of the display image data is adjusted so as to be expanded, the user can observe an image in which blackout and halation (whiteout) are reduced. When the dynamic range of the display image data is adjusted so as to be reduced, the user can observe an image with improved resolution.

( 12 ) The magnifying observation apparatus further includes a second light projecting unit arranged so that the observation target can be irradiated with coaxial incident light, and the imaging condition set by the imaging condition setting unit is the second light projecting unit. May further include irradiating the observation object with coaxial incident light.

  In this case, the observation object is irradiated with coaxial incident light from the second light projecting unit under the same imaging conditions as the previously generated desired display image data, and light from the observation object is received by the imaging unit. Is done. This makes it possible to generate display image data based on the original image data obtained when the observation object is irradiated with the coaxial incident light again. Further, in the original image data when the observation object is irradiated with the coaxial incident light, irregularities on the surface of the observation object are more clearly shown. Therefore, the user can observe an image showing the surface irregularities of the observation target more clearly.

( 13 ) The magnifying observation apparatus further includes a third light projecting unit arranged to face the imaging unit with the observation target interposed therebetween, and the imaging condition set by the imaging condition setting unit is the third projection condition. The light unit may further include irradiating the observation target with light.

  In this case, under the same imaging conditions as the previously generated desired display image data, light is emitted from the third light projecting unit to the observation target, and light transmitted through the observation target is received by the imaging unit. You. This makes it possible to generate display image data again based on the original image data when the light is emitted from the third light emitting unit to the observation target. In addition, the original image data obtained when the observation target is irradiated with light from the third light projecting unit indicates the internal structure of the observation target. Therefore, the user can observe an image showing the internal structure of the observation target.

( 14 ) The magnifying observation apparatus may further include a report creation unit that creates a report in which an observation result of an image based on the display image data is displayed. In this case, the user can easily report the observation result of the observation target object to another worker using the report created by the report creation unit.

( 15 ) The magnifying observation device further includes a communication unit that is communicably connected to an external device having a display function and causes the external device to display an image based on the display image data stored in the first area of the storage unit. You may.

  In this case, the user can acquire display image data from the magnifying observation device via the communication unit. Here, since the display image data is stored in a general-purpose image file format, the user can observe an image based on the display image data on a display device different from the magnifying observation device.

  ADVANTAGE OF THE INVENTION According to this invention, the image of the observation target object according to a user's request can be easily acquired.

1 is a schematic diagram illustrating a configuration of a magnifying observation apparatus according to a first embodiment of the present invention. FIG. 2 is a block diagram illustrating a configuration of a control device in FIG. 1. It is the perspective view and top view which show the structure of a light projection part. It is the external appearance perspective view of a measuring head, and the schematic diagram which shows the structure of a lens-barrel part. FIG. 3 is a schematic diagram illustrating a configuration example of a focus driving unit. FIG. 4 is a diagram for describing an area of a file stored in a storage unit. FIG. 3 is a block diagram illustrating a configuration of an arithmetic processing unit in FIG. 2. It is a figure showing an example of a report created by a report preparation part. FIG. 3 is a diagram illustrating a polar coordinate system defined on a mounting surface of a stage. It is a schematic diagram for demonstrating the basic operation | movement of the magnifying observation apparatus at the time of instruction | indication of multiple illumination imaging. It is a figure showing an example of a display of an observation screen. FIG. 9 is a schematic diagram for explaining processing when an image of an observation target is updated in response to designation of a virtual light emission direction. It is a figure showing other examples of a display of an observation screen. FIG. 11 is a diagram showing still another display example of the observation screen. 9 is a flowchart illustrating an example of a multiple illumination imaging process. 9 is a flowchart illustrating an example of a display image generation process. 9 is a flowchart illustrating an example of a display image generation process. It is a conceptual diagram of a depth synthesis process. It is a figure which shows mask image data visually. 9 is a flowchart illustrating an example of a depth synthesis process. 9 is a flowchart illustrating an example of a depth synthesis process. 9 is a flowchart illustrating an example of a depth synthesis process. It is a flow chart which shows other examples of depth synthesis processing. It is a flow chart which shows other examples of depth synthesis processing. It is a flow chart which shows other examples of depth synthesis processing. It is a flowchart which shows an example of DR adjustment processing. It is a flowchart which shows an example of DR adjustment processing. It is a flow chart which shows other examples of DR adjustment processing. It is a flow chart which shows other examples of DR adjustment processing. It is a schematic diagram which shows the structure of the magnification observation apparatus which concerns on 2nd Embodiment of this invention. FIG. 14 is a diagram illustrating a display example of an observation screen after a multiple illumination imaging process according to the second embodiment. FIG. 11 is a schematic diagram illustrating a configuration of a magnifying observation apparatus according to a third embodiment of the present invention. FIG. 14 is a diagram illustrating a display example of an observation screen after a multiple illumination imaging process according to the third embodiment.

[1] First Embodiment (1) Configuration of Magnifying Observation Device (a) Measuring Head Hereinafter, a magnifying observation device according to a first embodiment of the present invention will be described with reference to the drawings. FIG. 1 is a schematic diagram illustrating a configuration of a magnifying observation apparatus according to the first embodiment of the present invention. As shown in FIG. 1, the magnification observation device 1 includes a measurement head 100 and a processing device 200. The measurement head 100 is, for example, a microscope, and includes a stand unit 110, a stage device 120, a lens barrel unit 130, a light emitting unit 140, and a control board 150.

  The stand section 110 has an L-shaped vertical section, and includes an installation section 111, a holding section 112 and a focus driving section 113. The installation section 111 and the holding section 112 are formed of, for example, a resin. The installation part 111 has a horizontal flat plate shape, and is installed on an installation surface. The holding section 112 is provided to extend upward from one end of the installation section 111.

  The stage device 120 includes a stage 121 and a stage driving unit 122. The stage 121 is provided on the upper surface of the installation unit 111. The observation target S is placed on the stage 121. Two directions orthogonal to each other in a plane on the stage 121 on which the observation target S is mounted (hereinafter, referred to as a mounting surface) are defined as an X direction and a Y direction, and are indicated by arrows X and Y, respectively. The direction of a normal line perpendicular to the mounting surface of the stage 121 is defined as a Z direction, and is indicated by an arrow Z. The direction of rotation about an axis parallel to the Z direction is defined as the θ direction, and is indicated by an arrow θ.

  The stage drive unit 122 includes an actuator (not shown) such as a stepping motor. The stage driving section 122 moves the stage 121 in the X direction, the Y direction or the Z direction, or rotates the stage 121 in the θ direction based on a driving pulse given by the control board 150. Further, the user can manually move the stage 121 in the X direction, the Y direction or the Z direction, or rotate the stage 121 in the θ direction.

  The lens barrel unit 130 includes a lens unit 131 and an imaging unit 132, and is disposed above the stage 121. The lens unit 131 can be replaced with another lens unit according to the type of the observation target S. The lens unit 131 includes an objective lens 131a and a plurality of lenses (not shown). The optical axis A1 of the objective lens 131a is parallel to the Z direction. The imaging unit 132 includes, for example, a CMOS (complementary metal oxide semiconductor) camera. The imaging unit 132 may include another camera such as a CCD (charge coupled device) camera.

  The lens barrel unit 130 is attached to the holding unit 112 by the focus driving unit 113 of the stand unit 110. The focus driving unit 113 includes an actuator (not shown) such as a stepping motor. The focus driving unit 113 moves the lens unit 131 in the direction of the optical axis A1 (Z direction) of the objective lens 131a based on a driving pulse given by the control board 150. Thereby, the focal position of the light passing through the lens unit 131 changes in the Z direction. Further, the user can manually move the lens unit 131 in the direction of the optical axis A1 of the objective lens 131a.

  The light projecting unit 140 is integrally attached to the lens unit 131 so as to surround the optical axis A1 of the objective lens 131a. Thus, the positional relationship between the light projecting unit 140 and the lens unit 131 can be uniquely determined. Further, since it is not necessary to add a member for holding the light projecting unit 140 to the magnifying observation apparatus 1, the magnifying observation apparatus 1 can be made compact. The optical axis A2 (FIG. 3 described later) of the light projecting unit 140 is substantially the same as the optical axis A1 of the objective lens 131a.

  The light projecting unit 140 irradiates the observation object S on the stage 121 with light in a plurality of emission directions. The light reflected by the observation target S above the stage 121 is condensed and imaged by the lens unit 131, and then received by the imaging unit 132. The imaging unit 132 generates image data based on pixel data corresponding to the amount of light received by each pixel. Each of the plurality of image data generated by the imaging unit 132 when light in a plurality of emission directions is irradiated on the observation target S by the light projecting unit 140 is referred to as original image data. The imaging unit 132 supplies the generated plurality of original image data to the control device 400.

  The control board 150 is provided, for example, in the holding section 112 of the stand section 110 and is connected to the focus driving section 113, the stage driving section 122, and the imaging section 132. The control board 150 controls the operations of the focus driving unit 113 and the stage driving unit 122 based on the control by the processing device 200. A control signal is input from the control device 400 to the imaging unit 132. The plurality of original image data generated by the imaging unit 132 are sequentially provided to the processing device 200 via the cable 203.

(B) Processing Device The processing device 200 includes a housing 210, a light generation unit 300, and a control device 400. The housing 210 houses the light generation unit 300 and the control device 400. The light generating unit 300 is optically connected to the light emitting unit 140 of the measuring head 100 by the fiber unit 201. The fiber unit 201 includes a plurality of optical fibers (not shown).

  The light generating unit 300 includes a light source 310 and a light blocking unit 320. The light source 310 is, for example, an LED (light emitting diode). The light source 310 may be another light source such as a halogen lamp. The light blocking unit 320 is arranged between the light source 310 and the fiber unit 201 so as to partially block light emitted by the light source 310. The light emitted by the light source 310 passes through the light blocking unit 320 and enters the fiber unit 201. Thereby, light is emitted from the light emitting unit 140 of the measuring head 100 through the fiber unit 201.

  FIG. 2 is a block diagram showing a configuration of the control device 400 of FIG. As shown in FIG. 2, the control device 400 includes a control unit 410, a storage unit 420, a display unit 430, an operation unit 440, and a communication unit 450. Control unit 410 includes, for example, a CPU (Central Processing Unit). The storage unit 420 includes, for example, a ROM (Read Only Memory), a RAM (Random Access Memory), or an HDD (Hard Disk Drive). In the present embodiment, control unit 410 and storage unit 420 are realized by a personal computer.

  Control unit 410 includes a drive control unit 500 and an arithmetic processing unit 600. The storage unit 420 stores a system program. The storage unit 420 is used for processing various data and storing various data provided from the control unit 410. The functions of the drive control unit 500 and the arithmetic processing unit 600 are realized by the control unit 410 executing a system program stored in the storage unit 420.

  The drive control unit 500 includes a light emission control unit 510, an imaging control unit 520, a focus control unit 530, and a stage control unit 540. The light emission control unit 510 is connected to the light generation unit 300 of FIG. 1 through the cable 202 and controls the operation of the light generation unit 300. The imaging control unit 520, the focus control unit 530, and the stage control unit 540 are connected to the control board 150 of the measurement head 100 in FIG.

  The imaging control unit 520, the focus control unit 530, and the stage control unit 540 control the operations of the imaging unit 132, the focus driving unit 113, and the stage driving unit 122 through the control board 150, respectively. Further, the imaging control unit 520 sequentially supplies the plurality of original image data generated by the imaging unit 132 to the arithmetic processing unit 600.

  The arithmetic processing unit 600 is based on at least one of the plurality of acquired original image data, and the observation target to be obtained when it is assumed that the light in the emission direction specified by the user is irradiated on the observation target S. Display image data indicating the image of S can be generated. The details of the arithmetic processing unit 600 will be described later. The plurality of original image data and the generated display image data acquired by the arithmetic processing unit 600 are stored in the storage unit 420.

  The display unit 430 is configured by, for example, an LCD (liquid crystal display) panel. The display unit 430 may be configured by another display unit such as an organic EL (electroluminescence) panel. The display unit 430 displays an image based on the image data stored in the storage unit 420 or the image data generated by the arithmetic processing unit 600. The operation unit 440 includes a mouse, a touch panel, a pointing device such as a trackball or a joystick, and a keyboard, and is operated by a user to give instructions and the like to the control device 400. The operation unit 440 may include a jog shuttle in addition to the pointing device and the keyboard, and includes a dial-shaped operation unit whose rotation center for moving the lens barrel unit 130 and the stage 121 in the vertical direction is oriented in the horizontal direction. May be.

  Communication unit 450 includes an interface for connecting control device 400 to a network. In the example of FIG. 1, an external device 2 having a display function is connected to a network. The control device 400 can transmit image data to the external device 2 having a display function via the communication unit 450. The user of the external device 2 can acquire the image data stored in the general-purpose image file format from the control device 400 via the communication unit 450, and display an image based on the image data on the external device 2. .

(C) Light Projecting Section FIGS. 3A and 3B are a perspective view and a plan view showing the configuration of the light projecting section 140, respectively. As shown in FIG. 3A, the light projecting unit 140 includes a holding member 141 and a plurality of optical fibers 142. The holding member 141 is formed of, for example, a resin and has a cylindrical shape. The outer diameter of the holding member 141 in a plan view is smaller than the dimension of the stage 121 in FIG. The holding member 141 is arranged so as to surround the optical axis A1 of the objective lens 131a in FIG.

  A plurality of through holes 141a penetrating from the upper surface to the lower surface are formed in the holding member 141. The plurality of through holes 141a are arranged at substantially equal intervals, and are rotationally symmetric about the optical axis A1 of the objective lens 131a. The plurality of optical fibers 142 are respectively inserted into the plurality of through holes 141a. Thereby, the plurality of optical fibers 142 are integrally held by the holding member 141. The light entrance and exit of each optical fiber 142 are located on the upper and lower surfaces of the holding member 141, respectively. Thus, the light emitting portion 140o is formed on the lower surface of the holding member 141.

  The plurality of optical fibers 142 are arranged on one circumference centered on the optical axis A1 of the objective lens 131a. Therefore, the distance from the optical axis A1 of the objective lens 131a to the emission portions of the plurality of optical fibers 142 is substantially equal. The angle formed by the line connecting the output part of each optical fiber 142 and the center of the stage 121 with respect to the optical axis A1 of the objective lens 131a is an acute angle. In the present embodiment, since the holding member 141 integrally holds the plurality of optical fibers 142, the positional relationship among the plurality of optical fibers 142 is easily maintained.

  As shown in FIG. 3B, the annular light emitting portion 140o of the light projecting portion 140 is substantially equally divided into a plurality (four in this example) of regions 140A, 140B, 140C, and 140D. The plurality of regions 140A to 140D are arranged rotationally symmetric around the optical axis A1 of the objective lens 131a. The plurality of regions 140 </ b> A to 140 </ b> D include substantially the same number of emission portions of the optical fibers 142.

  The incident portions of the plurality of optical fibers 142 are optically connected to the light generation unit 300 of the processing device 200 by the fiber unit 201 of FIG. As a result, the light emitted from the light generation unit 300 enters the incident portions of the plurality of optical fibers 142 from the upper surface of the holding member 141, and passes through the emitting portions of the plurality of optical fibers 142, and the light emitting portions on the lower surface of the holding member 141. It is emitted from 140o. That is, light is emitted from each of the regions 140A to 140D when the optical fiber 142 included in each of the regions 140A to 140D emits light from the light emitting portion 140o.

  1 includes a mask having a plurality of opening patterns respectively corresponding to regions 140A to 140D of light projecting unit 140. The light emitted from the light source 310 in FIG. 1 passes through one of the opening patterns of the light shielding unit 320 and enters the fiber unit 201. The light projection control unit 510 in FIG. 2 switches the light emitting regions 140A to 140D in the light projection unit 140 by switching the opening pattern of the light blocking unit 320 that allows light to pass. Thus, the light projecting unit 140 can emit light from the whole of the regions 140A to 140D and can selectively emit light from any of the regions 140A to 140D.

  As described above, the light projecting unit 140 can irradiate the observation target S with lights having different emission directions. Light emitted simultaneously from all of the regions 140A to 140D is called ring illumination, and light emitted from any one of the regions 140A to 140D is called directional illumination. In the present embodiment, light projecting section 140 can selectively emit any of ring illumination and four directional illuminations. Therefore, the imaging unit 132 in FIG. 1 can generate five original image data indicating the observation target S when the ring illumination and the four directional illuminations are respectively radiated to the observation target S.

  The four directional illuminations are lights emitted from four positions (regions 140A to 140D) different from each other by about 90 ° in the θ direction about the optical axis A1 of the objective lens 131a. It is rotationally symmetric about the center. Therefore, each directional illumination has a bias with respect to the optical axis A1 of the objective lens 131a. The four directional illuminations are emitted in directions different from each other with respect to the optical axis A1 of the objective lens 131a. The light amounts of the four directional illuminations are substantially equal to each other. The angles of irradiation of the four directional illuminations with respect to the optical axis A1 of the objective lens 131a are not uniform in the θ direction.

  On the other hand, the ring illumination is light that is not deviated with respect to the optical axis A1 of the objective lens 131a, and its center substantially coincides with the optical axis A1 of the objective lens 131a. Therefore, the ring illumination is substantially emitted in the direction of the optical axis A1 of the objective lens 131a. The ring illumination has a substantially uniform light quantity distribution around the optical axis A1 of the objective lens 131a, and the light quantity of the ring illumination is substantially equal to the sum of the light quantities of the four directional illuminations. That is, the light quantity of the ring illumination is about four times the light quantity of each directional illumination. The irradiation angle of the ring illumination with respect to the optical axis A1 of the objective lens 131a is more uniform in the θ direction.

  As described above, in the present embodiment, the plurality of regions 140A to 140D are arranged rotationally symmetric around the optical axis A1 of the objective lens 131a. Accordingly, when display image data is generated by calculation based on a plurality of original image data, the calculation can be simplified.

  In the present embodiment, the optical fiber 142 is provided as a light emitting member in each of the regions 140A to 140D of the light projecting unit 140, but the present invention is not limited to this. A light source such as an LED may be provided as a light emitting member in each of the regions 140A to 140D of the light emitting unit 140. In this case, the light generating unit 300 is not provided in the processing device 200. In this configuration, light is emitted from each of the regions 140A to 140D when one or a plurality of light sources provided in each of the regions 140A to 140D emits light.

  Furthermore, in the present embodiment, light emitting section 140 is provided with regions 140A to 140D for emitting four lights, but the present invention is not limited to this. The light emitting unit 140 may be provided with a region that emits three or less or five or more lights.

  Further, in the present embodiment, the plurality of light emitting members (optical fibers 142) are arranged on one circumference centered on the optical axis A1 of the objective lens 131a, but the present invention is not limited to this. The plurality of light emitting members may be arranged on two or more concentric circles around the optical axis A1 of the objective lens 131a. Furthermore, in the present embodiment, a plurality of light emitting members are arranged in each of the regions 140A to 140D, but the present invention is not limited to this. One light emitting member may be arranged in each of the regions 140A to 140D.

(D) Barrel Section FIGS. 4A and 4B are an external perspective view of the measuring head 100 and a schematic diagram showing the configuration of the barrel section 130, respectively. As shown in FIG. 4A, the measuring head 100 includes a tilting mechanism 101 for tilting the lens barrel 130 with respect to the stage 121. The tilt mechanism 101 supports the upper part of the holding part 112 with respect to the lower part of the holding part 112 in a plane orthogonal to the Y direction. Thus, the tilt mechanism 101 can tilt the lens barrel 130 with respect to the stage 121 around the tilt center 130c. In FIG. 4B, the lens barrel 130 after the inclination is indicated by a chain line.

  The stage 121 moves in the Z direction under the control of the stage control unit 540 in FIG. 2 so that the surface of the observation target S is located at substantially the same height as the tilt center 130c of the lens barrel 130. Therefore, even when the lens barrel unit 130 is tilted, a eucentric relationship in which the field of view of the imaging unit 132 does not move is maintained, and a desired observation region of the observation target S is prevented from deviating from the field of view of the imaging unit 132. Can be.

  As shown in FIG. 4B, the lens barrel unit 130 includes a lens unit 131, an imaging unit 132, and an inclination sensor 133. The imaging unit 132 receives light from the observation target S mounted on the mounting surface of the stage 121 via the lens unit 131 based on control by the imaging control unit 520 in FIG. 2 to generate original image data. I do.

  The imaging control unit 520 controls the light receiving time, gain, timing, and the like of the imaging unit 132. For example, the imaging control unit 520 adjusts the light reception time at the time of irradiation of each directional illumination based on the light reception time at the time of irradiation of ring illumination. In this example, as described above, since the light amount of the ring illumination is about four times the light amount of each directional illumination, the imaging control unit 520 sets the light receiving time at the time of irradiation of each directional illumination at the time of irradiation of the ring illumination. Adjust so as to be four times the light receiving time.

  According to this control, the imaging unit 132 can generate the original image data at a higher speed than in the case where the light receiving time at the time of irradiating each directional illumination is independently adjusted. Further, the brightness of the image at the time of irradiation with the ring illumination and the image at the time of irradiation with each directional illumination can be easily made substantially equal. Note that, in this example, the control contents of the imaging unit 132 during the irradiation of the plurality of directional illuminations are the same.

  Further, the imaging unit 132 can generate a plurality of original image data in a state where a plurality of light receiving times are changed by the imaging control unit 520. The arithmetic processing unit 600 in FIG. 2 generates image data with a dynamic range adjusted by selectively combining a plurality of original image data generated in a state where a plurality of light reception times of the imaging unit 132 are changed. (DR (dynamic range) adjustment processing).

  The tilt angle of the optical axis A1 of the objective lens 131a with respect to the Z direction (hereinafter, referred to as the tilt angle of the lens barrel 130) is detected by a tilt sensor 133, and an angle signal corresponding to the tilt angle is transmitted to the control board 150 of FIG. Is output to The control board 150 supplies the angle signal output from the tilt sensor 133 to the arithmetic processing unit 600 via the cable 203 and the imaging control unit 520 in FIG. The arithmetic processing unit 600 calculates the inclination angle of the lens barrel 130 based on the angle signal. The tilt angle calculated by the arithmetic processing unit 600 can be displayed on the display unit 430 in FIG.

  According to the above configuration, the planar observation and the tilt observation of the observation target S placed on the placement surface of the stage 121 can be selectively performed. During planar observation, the optical axis A1 of the objective lens 131a is parallel to the Z direction. That is, the inclination angle of the lens barrel 130 becomes 0 °. On the other hand, during oblique observation, the optical axis A1 of the objective lens 131a is inclined with respect to the Z direction. Further, the user can observe the observation target S in a state where the lens barrel unit 130 is detached from the stand unit 110 in FIG. 1 and fixed by hand or another fixing member. In the following description, planar observation of the observation target S is performed.

(E) Focus Drive Unit The focus control unit 530 in FIG. 2 is configured so that the focal position of light from the observation target S that has passed through the lens unit 131 changes in the Z direction relative to the observation target S. One focus driving unit 113 is controlled. Thereby, the imaging unit 132 in FIG. 1 can generate a plurality of original image data indicating the observation target S at different positions in the Z direction.

  In this processing, the user can specify a range in which the focus driving unit 113 moves in the Z direction. When the movement range is specified, the focus control unit 530 controls the focus driving unit 113 so that the focal position of light changes in the Z direction in the specified movement range. Thereby, the imaging unit 132 can generate a plurality of original image data indicating the observation target S at different positions in the Z direction in a short time.

  In addition, the arithmetic processing unit 600 can determine the degree of focus for each pixel for each of the plurality of pieces of the generated original image data indicating the observation target S having different positions in the Z direction. The focus control unit 530 can adjust the focus driving unit 113 based on the determination result of the degree of focus by the arithmetic processing unit 600 so that the focus of the imaging unit 132 is focused on a specific portion of the observation target S ( Auto focus processing). Further, the arithmetic processing unit 600 generates image data focused on all parts of the observation target S by selectively combining a plurality of original image data for each pixel based on the determination result of the degree of focus. (Depth composition processing).

  FIG. 5 is a diagram illustrating a configuration example of the focus driving unit 113. In the present embodiment, light emitting section 140 is attached to lens unit 131. As shown by the dotted line in FIG. 5, the lens unit 131 is moved in the Z direction by the focus driving unit 113 integrally with the light projecting unit 140. Further, as shown by a dashed line in FIG. 5, the stage 121 is moved in the Z direction by the stage driving unit 122 in FIG. Thus, the lens unit 131, the light projecting unit 140, and the stage 121 are relatively movable in the Z direction.

  When the positional relationship between the observation target S, the lens unit 131, and the light projecting unit 140 in the Z direction changes, the elevation angle (see FIG. 9 described later) of the light source that irradiates the observation target S with illumination changes.

  In the example of FIG. 5, the light projecting unit 140 is provided integrally with the lens barrel unit 130, but the present invention is not limited to this. The light emitting unit 140 may be detachably attached to the lens barrel unit 130 as a unit. In this case, it is preferable that a positioning mechanism for maintaining a constant angle relationship between the light projecting unit 140 and the lens unit 131 in the θ direction be provided in the lens unit 131 or the light projecting unit 140.

(F) Storage Unit The storage unit 420 of FIG. 1 stores a plurality of data items related to each other as files. FIG. 6 is a diagram for explaining an area of a file stored in the storage unit 420. As shown in FIG. 6, the file FI includes a header area 421 and a data area 422. The header area 421 generally stores information indicating the vertical and horizontal widths of the image data. The header area 421 includes a user setting area in which the user can write other information.

  The data area 422 includes a general-purpose data area 422A and a metadata area 422B. Display image data generated by the arithmetic processing unit 600 is stored in the general-purpose data area 422A. The metadata about the image data for display of the general-purpose data area 422A is stored in the metadata area 422B. That is, the display image data in the general-purpose data area 422A and the metadata in the metadata area 422B are stored in the data area 422 in a state where they are associated with each other.

  The metadata area 422B includes a plurality of sub data areas. In the example of FIG. 6, sub data areas SUB1, SUB2, and SUB3 are illustrated. In the sub-data area SUB1, a plurality (five in this example) of original image data is stored, and imaging information indicating imaging conditions described later and illumination information indicating illumination conditions are stored. The sub data area SUB2 stores lens information indicating the type, magnification, and the like of the lens unit 131. In the sub-data area SUB3, a plurality of thumbnail image data respectively corresponding to the plurality of original image data in the sub-data area SUB1 are stored.

  The image data stored in the metadata area 422B has a general-purpose image file format such as a JPEG (Joint Photographic Experts Group) file or a non-general-purpose image file format. Here, the non-general-purpose image file format refers to an image file format that has not been subjected to compression processing (an uncompressed image file format) or an image file format that has been subjected to lossless compression processing. The image data stored in the metadata area 422B can be displayed on the display unit 430 in FIG. 2 by using a dedicated application program included in the control device 400. In addition, the arithmetic processing unit 600 edits the metadata stored in the metadata area 422B according to a user's instruction. As a result, the display image data stored in the general-purpose data area 422A is updated.

  The dedicated application program for displaying an image based on the image data stored in the metadata area 422B may be configured to be installable on an external commercially available personal computer or the like. In this case, an image based on the image data stored in the metadata area 422B can be displayed on the personal computer or the like.

  The display image data stored in the general data area 422A has a general image file format. Here, the general-purpose image file format includes a JPEG file format or TIFF (Tagged Image File Format). The image based on the general-purpose image file format can be displayed not only on the display unit 430 but also on another general display device using a general-purpose application program. Thereby, the user of the external device 2 of FIG. 2 can display an image based on the display image data on the external device 2 having no dedicated application program.

(G) Arithmetic Processing Unit FIG. 7 is a block diagram showing a configuration of the arithmetic processing unit 600 in FIG. As shown in FIG. 7, the arithmetic processing unit 600 includes a data generation unit 610, a focus determination unit 620, a calculation unit 630, a condition setting unit 640, an editing unit 650, and a report creation unit 660.

  The data generation unit 610 generates display image data based on at least one of the plurality of original image data generated by the imaging unit 132 in FIG. Further, the data generation unit 610 performs a DR adjustment process or a depth synthesis process on the image data according to a user's instruction.

  When the focus control unit 530 in FIG. 2 performs the autofocus process, the focus determination unit 620 determines the degree of focus for each pixel for each of a plurality of original image data generated by moving the focus drive unit 113 in the Z direction. Is determined. In addition, when the data generation unit 610 performs the depth synthesis process, the focus determination unit 620 determines the focus degree of each of the plurality of original image data for each pixel.

  The calculating unit 630 calculates the tilt angle of the lens barrel 130 in FIG. 4 based on the angle signal output from the tilt sensor 133 in FIG. In addition, the calculation unit 630 causes the display unit 430 of FIG. 2 to display the calculated inclination angle of the lens barrel 130 in accordance with a user's instruction.

  The condition setting unit 640 includes an imaging condition setting unit 641 and an illumination condition setting unit 642. The imaging condition setting unit 641 sets an imaging condition according to a user's instruction. In addition, the condition setting unit 640 stores the imaging information indicating the set imaging conditions as metadata in the metadata area 422B of FIG. The imaging conditions include, for example, the light receiving time of the imaging unit 132 in FIG. 1, whether or not the DR adjustment process is performed, whether or not the depth combining process is performed, and the range of the focal position of the light in the Z direction. The drive control unit 500 of FIG. 2 controls the operations of the measurement head 100 and the light generation unit 300 of FIG. 1 based on the imaging conditions set by the imaging condition setting unit 641.

  According to this configuration, even when the imaging conditions are variously changed when the original image data is generated, the imaging information corresponding to each original image data is stored in the metadata area 422B. The user can specify any one of the plurality of pieces of imaging information stored in the metadata area 422B. When any of the imaging information in the metadata area 422B is specified by the user, the imaging condition setting unit 641 updates the set imaging condition to the imaging condition indicated by the specified imaging information. This makes it possible to easily reproduce the imaging conditions when the desired original image data is generated.

  By reading the imaging information stored in the metadata area 422B by the imaging condition setting unit 641, the observation target S can be imaged under the imaging conditions when the original image data was generated. For example, when five pieces of original image data are stored in the metadata area 422B, the imaging information indicated when the five pieces of original image data are generated is stored in the metadata area 422B. The observation target S can be imaged under the conditions.

  The lighting condition setting unit 642 sets lighting conditions according to a user's instruction. The lighting condition setting unit 642 stores lighting information corresponding to the set lighting condition as metadata in the metadata area 422B of FIG. The illumination condition includes a virtual emission direction of light with respect to the observation target S. The data generation unit 610 generates display image data based on the illumination conditions set by the illumination condition setting unit 642, and stores the display image data in the general-purpose data area 422A of FIG. A method of instructing lighting conditions by the user will be described later.

  The editing unit 650 edits illumination information (virtual emission direction of light) stored as metadata in the metadata area 422B according to a user's instruction. The data generation unit 610 generates display image data based on the illumination information edited by the editing unit 650, and stores the display image data in the general-purpose data area 422A. As a result, the display image data stored in the general-purpose data area 422A is updated. When editing the lighting information, the editing unit 650 stores the lighting information before editing in the metadata area 422B.

  The report creation unit 660 creates a report in which an observation result of an image based on the display image data is displayed according to a user's instruction. Thus, the user can easily report the observation result of the observation target S to another worker using the report. The report is prepared in accordance with a predetermined description format. FIG. 8 is a diagram illustrating an example of a report created by the report creating unit 660. 8, the report 10 includes a name display column 11, an image display column 12, a status display column 13, an affiliation display column 14, and a result display column 15.

  The name of the report 10 is displayed in the name display column 11. The image display column 12 displays an image based on the display image data stored in the general-purpose data area 422A of FIG. The status display column 13 displays a model name of the magnifying observation apparatus 1, a name of display image data indicating an image displayed in the image display column 12, a date when the observation target S was imaged, and the like. The affiliation display column 14 describes the name of the image displayed in the image display column 12, the user name, the affiliation of the user, and the like. The user can also add a comment to the affiliation display field 14. In the result display column 15, the observation result of the observation target S is displayed.

(2) Basic Operation of Magnification Observation Apparatus (a) Contents of Basic Operation The position where the optical axis A1 of the objective lens 131a intersects on the mounting surface of the stage 121 is referred to as a reference point. The observation target S is placed on the stage 121 such that the observation target portion is located on the reference point. In this state, the position of the lens unit 131 (FIG. 1) in the Z direction is adjusted so that at least a part of the observation target S is focused on the objective lens 131a. The stage 121 is adjusted in the X direction and the Y direction so that a desired portion of the observation target S can be observed. Further, imaging conditions such as the light receiving time and white balance of the imaging unit 132 are adjusted.

  In the following description, light emitted from each of the regions 140A, 140B, 140C, and 140D of the light projecting unit 140 will be referred to as a first directional illumination and a second directional illumination, respectively, in order to distinguish the above four directional illuminations. These are referred to as directional illumination, third directional illumination, and fourth directional illumination. In the following description, the traveling direction of the combined light beam when a plurality of light beams forming the ring illumination are combined vectorwise is referred to as a ring emission direction. The ring emission direction is a direction perpendicular to the mounting surface of the stage 121. In addition, when a plurality of light beams forming the first directional illumination are vector-wise synthesized, a traveling direction of the synthesized light beam is referred to as a first emission direction, and a plurality of light beams forming the second directional illumination are formed. The traveling direction of the combined light beam when the light beams are combined vectorwise is referred to as a second emission direction. Further, when a plurality of light beams forming the third directional illumination are vector-wise synthesized, the traveling direction of the synthesized light beam is referred to as a third emission direction, and the plurality of light beams forming the fourth directional illumination are referred to as a third emission direction. The traveling direction of the combined light beam when the light beams are combined vectorwise is referred to as a fourth emission direction.

  Here, the reference point is set as the origin on the mounting surface of the stage 121 so that the emission direction or the emission position of the light when irradiating the observation target S mounted on the reference point with light can be specified. Define a polar coordinate system. FIG. 9 is a diagram illustrating a polar coordinate system defined on the mounting surface of the stage 121. As shown in FIG. 9, a reference point on the mounting surface of the stage 121 is defined as an origin O. As shown by the thick solid line in FIG. 9, the azimuth is defined in a counterclockwise direction about the origin O when the mounting surface of the stage 121 is viewed from above. In this example, a direction from the origin O to one side of the magnifying observation apparatus 1 is defined as a reference angle (0 °) of the azimuth.

  Also, a point Q is assumed at an arbitrary position on the mounting surface or above the mounting surface. In this case, as shown by a thick dashed line in FIG. 9, an angle between a straight line connecting the point Q and the origin O and the mounting surface is defined as an elevation angle of the point Q. When the point Q is on the mounting surface, the elevation angle of the point Q is 0 °. When the point Q is above the mounting surface and on the optical axis A1, the elevation angle of the point Q is 90 °. In FIG. 9, the azimuth on the mounting surface of the stage 121 is shown for each 90 °. In the following description, when describing an arbitrary position above the mounting surface of the stage 121, the direction of light applied to the observation target S, and the like, the “elevation angle” and “ Azimuth "is used as appropriate.

  In this example, the central portions of the regions 140A, 140B, 140C, and 140D of the light projecting unit 140 are arranged at azimuths of 45 °, 135 °, 225 °, and 315 ° around the optical axis A1. Note that the arrangement of the light projecting unit 140 is not limited to the above example. For example, the central portions of the regions 140A, 140B, 140C, 140D may be arranged at azimuths of 0 °, 90 °, 180 °, and 270 ° around the optical axis A1. Sequential imaging of the observation target S using the ring illumination and the first to fourth directional illuminations is referred to as multiple illumination imaging.

  Each component of the magnifying observation apparatus 1 performs the following basic operation in response to an instruction for multiple illumination imaging. FIG. 10 is a schematic diagram for explaining the basic operation of the magnifying observation apparatus 1 when multiple illumination imaging is instructed. FIGS. 10A to 10E show changes in illumination applied to the observation target S in a time-series manner. 10A to 10E, the region of the light projecting unit 140 that emits light is indicated by a thick solid line, and the ring emission direction and the first to fourth emission directions are each indicated by a thick solid line arrow. . FIGS. 10F to 10J show images SI of the observation target S when the illuminations of FIGS. 10A to 10E are applied to the observation target S. In the following description, the display portion of the observation target S in the image in which the observation target S is displayed is referred to as a target partial image sp.

  As shown in FIG. 10A, first, the observation target S is irradiated with ring illumination, and the observation target S is imaged. In this case, the first to fourth directional illuminations are simultaneously and uniformly applied to the respective portions of the observation target S from all the regions 140A to 140D surrounding the optical axis A1 of the objective lens 131a. Thereby, as shown in FIG. 10F, in the image SI of the observation target S captured by the ring illumination, a shadow due to the shape of the observation target S hardly occurs in the target partial image sp. Therefore, it is possible to observe the surface state of the upwardly facing portion of the observation target S almost entirely.

  Next, as shown in FIG. 10B, only the first directional illumination is applied to the observation target S, and the observation target S is imaged. As shown in FIG. 10G, in the image SI of the observation target S captured by the first directional illumination, the first emission direction and the shape of the observation target S are included in a part of the target partial image sp. , A shadow SH is generated from the azimuth angle of 45 ° to the azimuth angle of 225 °. Thereby, the uneven portion in the observation target S is strongly emphasized in the first emission direction.

  Next, as shown in FIG. 10C, the observation target S is irradiated with only the second directional illumination, and the observation target S is imaged. As shown in FIG. 10H, in the image SI of the observation target S captured by the second directional illumination, the second emission direction and the irregularities of the observation target S are included in a part of the target partial image sp. , A shadow SH is generated from the 135 ° azimuth position to the 315 ° azimuth position. Thereby, the uneven portion in the observation target S is strongly emphasized in the second emission direction.

  Next, as shown in FIG. 10D, only the third directional illumination is applied to the observation target S, and the observation target S is imaged. As shown in FIG. 10 (i), in the image SI of the observation target S captured by the third directional illumination, the third emission direction and the irregularities of the observation target S are included in a part of the target partial image sp. , A shadow SH is generated from the azimuth angle of 225 ° to the azimuth angle of 45 °. Thereby, the uneven portion in the observation target S is strongly emphasized in the third emission direction.

  Next, as shown in FIG. 10E, only the fourth directional illumination is applied to the observation target S, and the observation target S is imaged. As shown in FIG. 10J, in the image SI of the observation target S captured by the fourth directional illumination, the fourth emission direction and the irregularities of the observation target S are included in a part of the target partial image sp. , A shadow SH is generated from the azimuth angle of 315 ° to the azimuth angle of 135 °. Thereby, the uneven portion in the observation target S is strongly emphasized in the fourth emission direction.

  The above-described series of operations is automatically performed by the control unit 410 of FIG. 1 executing a system program stored in the storage unit 420 as described later, but is manually performed based on a user operation. You may.

  When the multiple illumination imaging is completed, an observation screen is displayed on the display unit 430 of FIG. FIG. 11 is a diagram illustrating a display example of an observation screen. As shown in FIG. 11, a function display area 431 is set above the observation screen 430A. The function display area 431 displays a depth composition button b1, a DR adjustment button b2, and a save button b7.

  The user can operate each button displayed in the function display area 431 using the operation unit 440 of FIG. The details of the processing executed by operating the depth synthesizing button b1, the DR adjustment button b2, and the save button b7 will be described later.

  As shown in FIG. 11, a main display area 432 and a sub display area 433 are set below the function display area 431 so as to be arranged side by side. The main display area 432 has a larger area than the function display area 431 and the sub display area 433. In the initial state, one of the plurality of images SI based on the plurality of original image data generated by the immediately preceding plurality of illumination imaging is displayed over substantially the entire main display area 432. In this example, the image SI of the observation target S when the first directional illumination is emitted (see FIGS. 10B and 10G) is displayed in the main display area 432.

  In the sub display area 433, an emission direction designation field 433a and an emission direction display field 433b are displayed. An object position image ss0 indicating the position of the observation object S on the mounting surface is displayed in the emission direction designation field 433a. In addition, when the observation target S is viewed from a position above the light projecting unit 140, a light icon ss1 indicating a light emission position with respect to the observation target S is superimposed on the target position image ss0.

  In this case, the relative positional relationship between the target partial image sp of the observation target S on the target position image ss0 and the light icon ss1 is determined by obtaining the observation target S in order to obtain the image SI displayed in the main display area 432. Corresponds to the emission direction of the light to be irradiated to the light beam (hereinafter referred to as a virtual emission direction of light). As the target position image ss0, for example, a thumbnail image corresponding to ring illumination can be used.

  The user can instruct the editing unit 650 in FIG. 7 to edit the illumination information by operating the operation unit 440 in FIG. Specifically, the user moves the light icon ss1 in FIG. 11 relative to the target partial image sp of the observation target S on the target position image ss0, thereby setting the virtual emission position of light. , It is possible to easily specify a virtual light emitting direction.

  When the virtual emission direction of the light is designated by the user, the image SI of the observation target S displayed in the main display area 432 is such that the light in the designated emission direction is applied to the observation target S. It is updated to the image SI of the observation target S that should be obtained when it is assumed. The update processing of the image SI is executed by the data generation unit 610 in FIG.

  In the emission direction display field 433b, an image indicating a reference point on the mounting surface is displayed as a reference point image ss2, and an image of a virtual hemisphere covering the reference point on the stage 121 is three-dimensionally displayed as a hemisphere image ss3. Will be displayed. On the hemisphere image ss3, an image indicating the light emission position corresponding to the virtual emission direction of the light specified by the light icon ss1 is displayed as the emission position image ss4.

  Further, a straight line is displayed so as to connect the emission position image ss4 on the hemisphere image ss3 and the reference point image ss2. In this case, the direction from the emission position image ss4 to the reference point image ss2 on the straight line indicates the emission direction of the light specified by the light icon ss1. By visually recognizing the reference point image ss2, the hemispherical image ss3, and the emission position image ss4 displayed in the emission direction display field 433b, the user can easily and accurately determine the virtual emission direction of the light specified by the light icon ss1. Can be recognized.

  The magnifying observation apparatus 1 may be configured so that only a part of the ring illumination and the first to fourth directional illuminations can be selected as illumination used for multiple illumination imaging. When multiple illumination imaging is performed using only a part of the ring illumination and the first to fourth directional illuminations, the range of the emission direction that can be specified by the light icon ss1 may be limited.

  In this case, in the hemispherical image ss3, the range of the emission direction that can be specified by the light icon ss1 in the emission direction specification column 433a may be displayed so as to be identifiable from the range of the emission direction that cannot be specified. For example, when the range of the specifiable emission direction is limited to the range of a specific azimuth, the color etc. of the portion corresponding to the range of the specifiable azimuth and the portion corresponding to the range of the non-specifiable azimuth can be specified. May be made different. Thereby, the user can easily recognize the range of the emission direction that can be specified by the light icon ss1. Alternatively, instead of the above example, the hemispherical image ss3 may display only the range of the emission direction that can be specified by the light icon ss1.

  In this example, a virtual hemisphere covering the reference point is stereoscopically displayed as the hemisphere image ss3 in the emission direction display field 433b, but the present invention is not limited to this. The emission direction display field 433b may display a planar hemisphere image in which the virtual hemisphere covering the reference point is viewed from above, and a side hemisphere image in which the virtual hemisphere is viewed from one side. In this case, the reference point image ss2 and the emission position image ss4 may be displayed on the plane hemisphere image. Further, the reference point image ss2 and the emission position image ss4 may be displayed on the side hemisphere image.

  FIG. 12 is a schematic diagram for explaining the processing content when the image SI of the observation target S is updated in response to the designation of the virtual emission direction of light.

  In the magnifying observation apparatus 1, a predetermined plane coordinate system is defined for the object position image ss0 displayed in the sub-display area 433 of FIG. Further, as shown in FIG. 12A, on the target position image ss0 in FIG. 11, the above-described ring illumination and the emission of light corresponding to the first, second, third, and fourth emission directions, respectively. Points PA, PB, PC, PD, and PE indicating positions are set in advance.

  The positions of the points PA to PE are set, for example, based on the relative positional relationship between the light projecting unit 140 and the stage 121. In this example, the point PA is located at the center of the object position image ss0, and the points PB, PC, PD, and PE are arranged at equal angular intervals on a concentric circle centered on the point PA.

  The control unit 410 detects the position (coordinates) of the light icon ss1 on the target position image ss0 at a predetermined cycle, and displays the image SI of the observation target S corresponding to the position of the light icon ss1 in FIG. It is displayed in the display area 432.

  For example, when the light icon ss1 is located on the point PA in FIG. 12A, the control unit 410 displays the image SI in FIG. 10F corresponding to the ring emission direction in the main display area 432 in FIG. . In addition, when the light icon ss1 is located on the point PB in FIG. 12A, the control unit 410 displays the image SI in FIG. 10G corresponding to the first emission direction in the main display area 432 in FIG. When the light icon ss1 is positioned on the point PC in FIG. 12A, the image SI in FIG. 10H corresponding to the second emission direction is displayed in the main display area 432 in FIG. Further, when the light icon ss1 is located on the point PD in FIG. 12A, the control unit 410 displays the image SI in FIG. 10I corresponding to the third emission direction in the main display area 432 in FIG. When the light icon ss1 is positioned on the point PE in FIG. 12A, the image SI in FIG. 10J corresponding to the fourth emission direction is displayed in the main display area 432 in FIG.

  When the light icon ss1 is located at a position different from the points PA to PE, the control unit 410 generates an image SI of the observation target S to be displayed in the main display area 432 according to the following procedure.

  As shown in FIG. 12A, when the light icon ss1 is at a position different from the points PA to PE, the control unit 410 calculates the distance between each of the points PA to PE and the light icon ss1. The control unit 410 extracts a predetermined number of points (three points in this example) from the plurality of points PA to PE in ascending order of the calculated distance. In the example of FIG. 12A, the distance d1 between the light icon ss1 and the point PB is the shortest, the distance d2 between the light icon ss1 and the point PA is the second shortest, and the distance between the light icon ss1 and the point PC is short. Is the third shortest. Therefore, control unit 410 extracts points PA, PB, and PC.

  Subsequently, based on the distances d1, d2, and d3, the control unit 410 calculates a synthesis ratio of the original image data corresponding to the point PA, the original image data corresponding to the point PB, and the original image data corresponding to the point PC. To determine.

  The combination ratio is, for example, a ratio of reciprocals of values of the distances d1, d2, and d3. In this case, the composition ratio increases as the original image data corresponds to a point that is shorter from the light icon ss1, and decreases as the original image data corresponds to a point that is farther from the light icon ss1. In the example of FIG. 12B, the synthesis ratios of the original image data corresponding to the point PA, the original image data corresponding to the point PB, and the original image data corresponding to the point PC are 30%, 50%, and 20%, respectively. Has been determined.

  The control unit 410 combines three original image data corresponding to the points PA, PB, and PC based on the determined combination ratio. Specifically, the control unit 410 multiplies the values (pixel values) of all the pixel data of the original image data by the combination ratio for each original image data, and combines the multiplied three original image data to display. Generate image data for use. After that, the control unit 410 causes the main display area 432 of FIG. 11 to display the image SI based on the generated display image data.

  FIG. 12C shows an example of the image SI displayed in the main display area 432 corresponding to the position of the light icon ss1 in FIG. In the image SI of FIG. 12C, the three original image data respectively corresponding to the points PA, PB, and PC are combined, so that the uneven portion on the observation target S becomes the light icon ss1 of FIG. Are strongly emphasized in the virtual emission direction of the light designated by.

  As described above, the image data of the image SI displayed in the main display area 432 includes a plurality of generated original image data and a virtual emission direction of light designated by the user after completion of the multiple illumination imaging. Generated based on Therefore, even when the virtual emission direction of the light designated by the user continuously changes, a plurality of image data corresponding to the designated emission direction is determined according to the processing capability of the control unit 410. Generated almost continuously at speed. Further, the images SI based on the plurality of generated image data are continuously displayed. Therefore, an image substantially the same as an image (moving image) obtained when imaging is continuously performed while changing the position of the illumination is pseudo-reproduced on the main display area 432. Thus, the user visually recognizes the image SI on the main display area 432 while specifying the virtual emission direction of the light, so that the light in the designated emission direction is irradiated on the observation target S in real time. Feel like you are.

  In the above example, the control unit 410 extracts three points out of the points PA to PE corresponding to the ring illumination and the first to fourth directional illuminations, and three sources corresponding to the extracted three points. Although the composition ratio of the image data is determined, the present invention is not limited to this. The control unit 410 may determine the combining ratio for the five original image data respectively corresponding to all the points PA to PE, and combine the five original image data based on the determined combining ratio.

  Note that, on the observation screen 430A in FIG. 11, a combination ratio input field in which a combination ratio of a plurality of original image data obtained by multiple illumination imaging by the operation unit 440 in FIG. 1 may be displayed. In this case, the control unit 410 may combine some or all of the plurality of original image data based on the value input to the combination ratio input field by the user.

  When the virtual emission direction of the light is changed by the user, the editing unit 650 of FIG. 7 edits the illumination information, and converts the illumination information before editing into the metadata of the data area 422 in the file FI of FIG. It is stored in the area 422B. In this case, the user can specify any of the plurality of pieces of illumination information stored in the metadata area 422B by operating the operation unit 440 of FIG.

  When any of the lighting information is specified, the data generation unit 610 in FIG. 7 generates display image data corresponding to the specified lighting information. Thereby, the image SI of the observation target S based on the generated display image data is displayed in the main display area 432 of the display unit 430. The light icon ss1 is displayed at the position of the emission direction designation field 433a corresponding to the designated lighting information, and the emission position image ss4 is displayed at the position of the emission direction display field 433b corresponding to the designated lighting information.

  Thereby, the display on the display unit 430 is reset to the display on the display unit 430 corresponding to the illumination information before editing. The user can determine whether or not the illumination condition (virtual emission direction of light) indicated by the illumination information before editing is appropriate. Alternatively, the user can again give an instruction to change the virtual emission direction of light while visually checking the display on the display unit 430 corresponding to the illumination information before editing.

  A specific example will be described of a change in the observation screen 430A when the virtual emission direction of light is designated by the light icon ss1 while the observation screen 430A of FIG. 11 is displayed on the display unit 430.

  FIG. 13 is a diagram showing another display example of the observation screen 430A. In FIG. 13, five points PA to PE in FIG. 12A are shown on the target position image ss0 displayed in the emission direction designation field 433a for easy understanding of the following description.

  As indicated by a dotted line in the emission direction designation field 433a of FIG. 13, the light icon ss1 is moved from the position of the point PB to a position between the points PB and PE on a concentric circle of the point PA where the points PB to PE are arranged. In this case, the control unit 410 generates display image data by combining some of the plurality of original image data corresponding to the points PA to PE, as in the above example. Further, control unit 410 causes main display area 432 to display image SI of observation target S based on the generated display image data.

  In the image SI displayed in the main display area 432 in FIG. 13, the uneven portion of the observation target S is strongly emphasized in the virtual emission direction of the light designated by the light icon ss1 in the emission direction designation field 433a. .

  The virtual light emission direction includes azimuth and elevation components. In this example, the light icon ss1 is moved on a concentric circle of the point PA where the points PB to PE are arranged. The user can specify the azimuth of the virtual light emission direction by moving the light icon ss1 on the object position image ss0 so as to rotate with respect to the center of the object position image ss0. Thereby, the azimuth of the virtual emission direction of the light is changed to the azimuth specified by the light icon ss1. As described above, the user can specify the azimuth of the virtual emission direction of light in a desired direction by operating the light icon ss1. As a result, in the image SI displayed in the main display area 432, the direction in which the uneven portion of the observation target S is emphasized can be easily changed in the θ direction.

  FIG. 14 is a diagram showing still another display example of the observation screen 430A. In FIG. 14, as in the example of FIG. 13, five points PA to PE of FIG. 12A are shown on the target position image ss0 displayed in the emission direction designation field 433a. As indicated by a dotted line in the emission direction designation field 433a of FIG. 14, the light icon ss1 is moved from the position indicated in the emission direction designation field 433a of FIG. 13 to approach the point PA. In this case, the control unit 410 generates display image data by combining some of the plurality of original image data corresponding to the points PA to PE, as in the above example. Further, control unit 410 causes main display area 432 to display image SI of observation target S based on the generated display image data.

  In the image SI displayed in the main display area 432 in FIG. 14, the uneven portion of the observation target S is weakly emphasized compared to the image SI displayed in the main display area 432 in FIG.

  In this example, the light icon ss1 is moved from a concentric circle on which the points PB to PE are arranged around the point PA toward the point PA. The user can specify the elevation angle of the virtual light emission direction from the mounting surface by moving the light icon ss1 closer to or farther from the center of the target position image ss0 on the target position image ss0. . Thus, the elevation angle of the light in the virtual emission direction from the mounting surface is changed to the elevation angle specified by the light icon ss1. As described above, the user can specify the desired elevation angle of the virtual light emission direction by operating the light icon ss1. As a result, in the image SI displayed in the main display area 432, it is possible to easily change the degree of emphasis of the uneven portion of the observation target S.

  In the above example, a main display area 432 for displaying the image SI based on the display image data and a sub-display area 433 for operating the light icon ss1 are set on the observation screen 430A of the display unit 430. You. Thus, the image SI of the observation target S and the light icon ss1 do not overlap. Therefore, the image SI and the light icon ss1 of the observation target S are easily visually recognized.

  The user operates the save button b7 in FIG. 11 by using the operation unit 440 in FIG. 1 in a state where the virtual emission direction of the light with respect to the image SI of the observation target S is specified in a desired direction. In this case, the display image data of the image SI displayed in the main display area 432 is stored in the storage unit 420 together with a plurality of data relating to the display image data. The plurality of data includes a plurality of original image data related to the display image data, a plurality of thumbnail image data, imaging information, illumination information, lens information, and the like.

  In the above example, in the initial state after completion of the multiple illumination imaging, one of the multiple images SI based on the multiple original image data generated by the immediately preceding multiple illumination imaging is displayed in the main display area 432 of the observation screen 430A. Although shown, the invention is not so limited. In the magnifying observation apparatus 1, for example, before a plurality of illumination imaging is started, a virtual light emission direction corresponding to the image SI to be displayed in the initial state may be specified in advance. Alternatively, the virtual light emission direction corresponding to the image SI to be displayed in the initial state may be specified in advance by the manufacturer when the magnifying observation apparatus 1 is shipped from the factory. In addition, until the operation of the user is received, the emission direction of the virtual light corresponding to the image SI is gradually changed to the main display area 432 of the observation screen 430A in the initial state after the completion of the multiple illumination imaging. An image (moving image) in which the emission direction with respect to the image SI changes smoothly may be displayed.

  In this case, when the multiple illumination imaging is completed, the control unit 410 sets the emission direction of the virtual light specified in advance and the emission direction specified based on the multiple original image data generated by the multiple illumination imaging. Generate the corresponding display image data. Further, control unit 410 causes main display area 432 to display image SI based on the generated display image data.

(B) Example of Multiple Illumination Imaging Process The system program stored in the storage unit 420 in FIG. 2 includes a multiple illumination imaging program and a display image generation program. The control unit 410 in FIG. 2 performs the multiple illumination imaging process and the display image generation process by executing the multiple illumination imaging program and the display image generation program. The series of basic operations described above is realized by the multiple illumination imaging process and the display image generation process. Thereby, even when the user is not skilled, the original image data can be easily generated.

  FIG. 15 is a flowchart illustrating an example of the multiple illumination imaging process. The multiple illumination imaging process is started in response to a multiple illumination imaging instruction from the user. When the multiple illumination imaging process is started, the control unit 410 irradiates the observation target S with ring illumination according to a preset imaging condition, and images the observation target S with the imaging unit 132 (step S101). Original image data generated by imaging is stored in the storage unit 420.

  Next, the control unit 410 sets i to 1 (step S102). Here, i indicates a number of a plurality of directional lights. Subsequently, the control unit 410 irradiates the i-th directional illumination to the observation target S and images the observation target S with the imaging unit 132 (step S103). Original image data obtained by imaging is stored in the storage unit 420.

  Next, the control unit 410 determines whether or not i is 4 (Step S104). If i is not 4, the control unit 410 updates i to i + 1 (step S105), and returns to the process of step S103.

  If i is 4 in step S104, control unit 410 generates a plurality of thumbnail image data respectively corresponding to the plurality of original image data (step S106). The plurality of generated thumbnail image data are stored in the storage unit 420. Thereby, the multiple illumination imaging process ends.

  In the above description, when it is not necessary to display the thumbnail image on the display unit 430, the process of step S106 may be omitted. Thereby, the processing time is shortened.

  Also, in the above description, some processes may be performed at other times. For example, the processing in step S101 may be executed after the processing in steps S102 to S105.

(C) Example of Display Image Generation Process FIGS. 16 and 17 are flowcharts illustrating an example of the display image generation process. In the present embodiment, control unit 410 starts the display image generation process after the end of the multiple illumination imaging process.

  The control unit 410 first causes the main display area 432 of the display unit 430 to display an image SI of the observation target S based on arbitrary original image data among a plurality of original image data generated by the multiple illumination imaging process (step S201). ). In addition, the control unit 410 causes the target position image ss0 and the light icon ss1 to be displayed in the emission direction designation column 433a of the display unit 430 (Step S202). Further, control unit 410 causes reference point image ss2, hemispherical image ss3, and emission position image ss4 to be displayed in emission direction display field 433b of display unit 430 (step S203).

  Thereafter, control unit 410 determines whether light icon ss1 has been operated (step S204). When the light icon ss1 is not operated, the control unit 410 proceeds to the process of step S210 described below.

  When the light icon ss1 is operated, the control unit 410 updates the display of the light icon ss1 and the emission position image ss4 in response to the operation of the light icon ss1 (step S205). Further, the control unit 410 recognizes that the virtual emission direction of the light has been designated by operating the light icon ss1 (step S206), and the designated emission direction is the ring emission direction or the first to fourth emission directions. It is determined whether it is any of them (step S207). The determination process in step S207 is executed based on the positional relationship between the points PA to PE set on the target object position image ss0 and the light icon ss1.

  If the designated emission direction is any of the ring emission direction or the first to fourth emission directions, control unit 410 sets the original image data corresponding to the designated emission direction as display image data, and displays the original image data. The image SI of the observation target S based on the image data for use is displayed on the main display area 432 of the display unit 430 (step S208). Thereafter, control unit 410 proceeds to the process of step S210 described below.

  In step S207, when the designated emission direction is not any of the ring emission direction or the first to fourth emission directions, the control unit 410 determines the combination ratio of the plurality of original image data based on the designated emission direction. It is calculated (step S209). The calculation processing in step S209 is executed based on the positional relationship between the points PA to PE set on the target object position image ss0 and the light icon ss1, similarly to the processing in step S207.

  Thereafter, the control unit 410 generates display image data by synthesizing the plurality of original image data based on the synthesis ratio calculated in step S209, and generates the image SI of the observation target S based on the display image data. Is displayed in the main display area 432 of the display unit 430 (step S210).

  In the present embodiment, the user can instruct the end of the observation of the observation target S by operating the operation unit 440 of FIG. After the process in step S209, the control unit 410 determines whether or not an instruction to end the observation of the observation target S has been issued (step S211). When the end of the observation of the observation target S is instructed, the control unit 410 ends the display image generation processing. On the other hand, when the end of the observation of the observation target S is not instructed, the control unit 410 returns to the process of step S204.

  In the display image generation processing of FIGS. 16 and 17, the control unit 410 calculates at least one of the azimuth and the elevation of the designated virtual light emitting direction during or after the processing of step S209. You may. In this case, control unit 410 may cause display unit 430 to display at least one of the calculated azimuth and elevation. Thereby, the user can easily recognize the information about the virtual emission direction of the designated light.

  In the display image generation process described above, the image SI of the observation target S based on arbitrary original image data among the plurality of original image data generated by the multiple illumination imaging process in the process of step S201 is displayed on the display unit 430. However, the present invention is not limited to this. In the process of step S201, control unit 410 may cause display unit 430 to display image SI based on original image data corresponding to predetermined illumination (for example, ring illumination), instead of arbitrary original image data. . Alternatively, control unit 410 may omit the process in step S201.

  In the above example, the display image generation processing is executed after the multiple illumination imaging processing ends, but the present invention is not limited to this. When the multiple illumination imaging process is performed continuously or intermittently at a constant cycle, the display image generation process may be performed in parallel with the multiple illumination imaging process. In this case, the display image generation process can be executed based on the latest plurality of original image data stored in the storage unit 420 by the immediately preceding multiple illumination imaging process.

  In the magnifying observation apparatus 1 according to the present embodiment, the user can designate a part of a plurality of original image data stored in the storage unit 420 using the operation unit 440 of FIG. In this case, the control unit 410 may read the designated original image data in response to the designation of the original image data by the user, and perform a display image generation process based on the read original image data.

(3) Depth synthesis processing (a) Processing contents The user operates the depth synthesis button b1 in FIG. 11 using the operation unit 440 in FIG. 1 to instruct the control unit 410 in FIG. 1 to perform the depth synthesis processing. Can be given.

  In the depth synthesis processing, it is preferable that the range of the focus position of the light in the Z direction and the movement pitch of the focus position be set in advance as imaging conditions for the depth synthesis processing. In this case, the focus driving unit 113 in FIG. 1 does not need to change the focal position of the light within an excessively large range, so that a plurality of original image data can be generated at high speed. The imaging condition is set based on, for example, the operation of the operation unit 440 by the user. The range of the focal position of the light in the Z direction and the movement pitch of the focal position may be automatically set according to the magnification of the objective lens 131a used for imaging. In the following description, it is assumed that the imaging conditions for the depth synthesis processing are set in advance.

  FIG. 18 is a conceptual diagram of the depth synthesis processing. FIG. 18A illustrates a positional relationship among the lens unit 131, the light projecting unit 140, and the stage 121. In this example, the lens unit 131 is moved in the Z direction integrally with the light projecting unit 140 while the stage 121 is stationary. In this case, the positions H1 to Hj (j is a natural number) in the Z direction to which the lens unit 131 (objective lens 131a) should move based on the range of the focal position of the light in the Z direction and the moving pitch of the focal position in the designated Z direction. Is determined.

  In the depth synthesis processing, the observation target S is imaged using the ring illumination and the first to fourth directional illuminations in a state where the lens unit 131 is positioned at each of the positions H1 to Hj. Thereby, a plurality of (j) original image data respectively corresponding to the positions H1 to Hj are generated using the ring illumination and the first to fourth directional illuminations. FIG. 18B shows a plurality of images SI of the observation target S corresponding to the positions H1 to Hj for each illumination.

  For each of a plurality of original image data obtained by imaging using ring illumination, the degree of focus for each pixel is determined. A plurality of original image data are selectively combined based on the determination result of the degree of focus. Thereby, depth-combined image data focused on the entire portion of the observation target S irradiated with the ring illumination is generated. Further, depth composite image data corresponding to each directional illumination is generated based on a plurality of original image data corresponding to each directional illumination and mask image data described later. An image based on the depth-combined image data is called a depth-combined image. FIG. 18B illustrates a plurality of depth-combined images SF of the observation target S corresponding to the ring illumination and the first to fourth directional illuminations, respectively.

  The control unit 410 executes a display image generation process based on a plurality of depth synthesized image data generated by the depth synthesis process instead of the plurality of original image data. Thus, the user can easily display the depth composite image SF of the observation target S that should be obtained when it is assumed that the observation target S is irradiated with light from a desired direction, on the display unit 430.

  In the above-described depth synthesis processing, a plurality of original image data respectively corresponding to the positions H1 to Hj are generated for each illumination. Operation buttons for instructing only generation of a plurality of original image data may be displayed on the observation screen 430A of FIG. When only the generation of a plurality of original image data is instructed, the control unit 410 performs only the generation of the plurality of original image data, receives the designation of the position of the focal point in the Z direction by the user, and The display image generation process may be performed using a plurality of original image data corresponding to the position in the direction.

  In the above-described depth synthesis processing, when depth synthesis image data corresponding to ring illumination is generated, mask image data is generated, and the generated mask image data is used to correspond to the first to fourth directional illuminations, respectively. Are generated. The mask image data will be described.

  Each of the plurality of original image data generated in the process of imaging the observation target S while changing the position of the lens barrel unit 130 and the stage 121 in the Z direction includes focal positions H1 to Hj of light in the Z direction. A corresponding number is assigned. The data generation unit 610 in FIG. 7 generates mask image data indicating the correspondence between each pixel of the depth-combined image data corresponding to ring illumination and the number of the original image data.

  FIG. 19 is a schematic diagram visually illustrating the mask image data. Each small square in FIG. 19 corresponds to each pixel data of the depth-combined image data corresponding to the ring illumination. In addition, the number given to each square indicates which of the focal points H1 to Hj of the light in the Z direction at the pixel corresponding to the square has the optimal pixel value of the pixel data generated, that is, the saturated value. Instead, the highest luminance value indicates the number of the original image data from which the data indicating the focal position of the light is extracted. That is, in the example of FIG. 19, the upper left pixel data is extracted from the original image data at the light focal position H12, and the lower right pixel data is extracted from the original image data at the light focal position H85. Show.

  The data generation unit 610 generates depth-combined image data corresponding to each of the first to fourth directional illuminations based on the generated mask image data. In this case, the focus determination unit 620 of FIG. 7 determines the degree of focus for each pixel of the original image data when generating the depth-combined image data corresponding to each of the first to fourth directional illuminations. No need. This makes it possible to speed up the depth synthesis processing.

  In the above description, depth synthesized image data is generated first for each of the ring illumination and the first to fourth directional illuminations, and display image data is generated based on the generated plurality of depth synthesized image data. However, the present invention is not limited to this. A plurality of display image data respectively corresponding to a plurality of positions H1 to Hj in the Z direction are generated based on the virtual emission direction of the light specified by the user, and the plurality of display image data are generated. Based on this, the depth-combined image data for display may be generated. In this case, the mask image data is unnecessary.

  In the above description, the mask image data is generated based on the image data when the ring illumination is emitted, but the present invention is not limited to this. Mask image data may be generated based on each of the image data when the ring illumination is emitted and the image data when the directional illumination is emitted. Further, the mask image data when the directional illumination is emitted may be generated for each of the plurality of directional illuminations. This configuration is useful when the optimal position in the Z direction is different due to a difference in light amount between the ring illumination and the directional illumination.

(B) Example of Depth Composition Process The system program stored in the storage unit 420 of FIG. 2 includes a depth composition program. The control unit 410 in FIG. 2 performs a depth synthesis process by executing a depth synthesis program.

  FIGS. 20, 21, and 22 are flowcharts illustrating an example of the depth combining process. The control unit 410 moves the lens unit 131 to the lower limit position (Step S301). Next, the control unit 410 irradiates the observation target S with ring illumination by the light projecting unit 140 and captures an image of the observation target S by the imaging unit 132 (step S302). Further, the control unit 410 assigns a number corresponding to the position of the lens unit 131 in the Z direction to the generated original image data (step S303). Subsequently, the control unit 410 determines whether the lens unit 131 has moved to the upper limit position (Step S304).

  If the lens unit 131 has not moved to the upper limit position in step S304, the control unit 410 moves the lens unit 131 upward by a predetermined amount (predetermined movement pitch) (step S305). Thereafter, control unit 410 returns to the process of step S302. Until the lens unit 131 moves to the upper limit position, the control unit 410 repeats the processing of steps S302 to S305.

  If the lens unit 131 has moved to the upper limit position in step S304, the control unit 410 determines the degree of focus for each pixel for each piece of original image data corresponding to ring illumination (step S306). Next, the control unit 410 generates depth-combined image data corresponding to ring illumination by combining pixel data of a plurality of original image data based on the determination result of the degree of focus (step S307). In addition, the control unit 410 generates mask image data indicating the correspondence between each pixel of the synthesized image data and the number of the original image data, and stores the mask image data in the storage unit 420 (step S308).

  Thereafter, the control unit 410 moves the lens unit 131 to the lower limit position (Step S309). Next, control unit 410 sets i to 1 (step S310). Here, i indicates a number of a plurality of directional lights. Subsequently, the control unit 410 irradiates the i-th directional illumination to the observation target S by the light projecting unit 140 and captures an image of the observation target S by the imaging unit 132 (step S311). In addition, the control unit 410 assigns a number corresponding to the position of the lens unit 131 in the Z direction to the generated original image data (step S312). Subsequently, the control unit 410 determines whether the lens unit 131 has moved to the upper limit position (Step S313).

  If the lens unit 131 has not moved to the upper limit position in step S313, the control unit 410 moves the lens unit 131 upward by a predetermined amount (predetermined movement pitch) (step S314). Thereafter, control unit 410 returns to the process of step S311. Until the lens unit 131 moves to the upper limit position, the control unit 410 repeats the processing of steps S311 to S314.

  In step S313, when the lens unit 131 has moved to the upper limit position, the control unit 410 synthesizes pixel data of a plurality of original image data based on the mask image data stored in the storage unit 420, thereby forming an i-th pixel. The depth synthesis image data corresponding to the directional illumination is generated (step S315).

  Next, the control unit 410 determines whether or not i is 4 (step S316). If i is not 4 in step S316, control unit 410 updates i to i + 1 (step S317). Thereafter, control unit 410 moves lens unit 131 to the lower limit position (step S318), and returns to the process of step S311. Until i becomes 4, control unit 410 repeats the processing of steps S311 to S318. Thereby, a plurality of original image data corresponding to each of the first to fourth directional illuminations is generated, and depth-combined image data corresponding to each of the first to fourth directional illuminations is generated. . If i is 4 in step S316, control unit 410 ends the process.

  In the above description, some processes may be performed at other times. For example, the processing of steps S306 to S308 may be executed in parallel with steps S309 to S314. Further, the processing of step S315 corresponding to the i-th directional illumination may be executed in parallel with steps S311 to S314 corresponding to the (i + 1) th directional illumination. In these cases, the speed of the depth synthesis processing can be increased.

  Alternatively, the processing of steps S306 to S308 may be executed after the processing of steps S309 to S314. Further, the processing of step S315 corresponding to the i-th directional illumination may be executed after the processing of steps S311 to S314 corresponding to the (i + 1) th directional illumination.

(C) Another Example of Depth Composition Processing FIGS. 23, 24, and 25 are flowcharts illustrating another example of the depth composition processing. The control unit 410 moves the lens unit 131 to the lower limit position (Step S321). Next, the control unit 410 irradiates the observation target S with ring illumination by the light projecting unit 140 and captures an image of the observation target S by the imaging unit 132 (step S322). Further, the control unit 410 assigns a number corresponding to the position of the lens unit 131 in the Z direction to the generated original image data (step S323).

  Subsequently, control unit 410 sets i to 1 (step S324). Here, i indicates a number of a plurality of directional lights. After that, the control unit 410 irradiates the i-th directional illumination to the observation target S by the light projecting unit 140 and captures an image of the observation target S by the imaging unit 132 (step S325). Further, the control unit 410 assigns a number corresponding to the position of the lens unit 131 in the Z direction to the generated original image data (step S326). Next, the control unit 410 determines whether or not i is 4 (Step S327).

  If i is not 4 in step S327, control unit 410 updates i to i + 1 (step S328). Thereafter, control unit 410 returns to the process of step S325. Until i becomes 4, control unit 410 repeats the processing of steps S325 to S328. Thereby, a plurality of original image data corresponding to each of the first to fourth directional illuminations is generated. If i is 4 in step S327, control unit 410 determines whether or not lens unit 131 has moved to the upper limit position (step S329).

  If the lens unit 131 has not moved to the upper limit position in step S329, the control unit 410 moves the lens unit 131 upward by a predetermined amount (step S330). Thereafter, control unit 410 returns to the process of step S322. Until the lens unit 131 moves to the upper limit position, the control unit 410 repeats the processing of steps S322 to S330.

  When the lens unit 131 has moved to the upper limit position in step S329, the control unit 410 determines the degree of focus for each pixel for each piece of original image data corresponding to ring illumination (step S331). Next, the control unit 410 generates depth-combined image data corresponding to ring illumination by combining pixel data of a plurality of original image data based on the determination result of the degree of focus (step S332). In addition, the control unit 410 generates mask image data indicating the correspondence between each pixel of the synthesized image data and the number of the original image data, and causes the storage unit 420 to store the mask image data (step S333).

  Subsequently, control unit 410 sets i to 1 again (step S334). Thereafter, the control unit 410 generates depth-combined image data corresponding to the i-th directional illumination by combining pixel data of a plurality of original image data based on the mask image data stored in the storage unit 420. (Step S335).

  Next, control unit 410 determines whether i is 4 or not (step S336). If i is not 4 in step S316, control unit 410 updates i to i + 1 (step S337). Thereafter, control unit 410 returns to the process of step S335. Until i becomes 4, control unit 410 repeats the processing of steps S335 to S337. Thereby, depth-combined image data corresponding to each of the first to fourth directional illuminations is generated. If i is 4 in step S336, control unit 410 ends the process.

  In the above description, some processes may be performed at other times. For example, part of the processing of steps S331 to S337 may be executed in parallel with steps S321 to S330. In this case, the speed of the depth synthesis processing can be increased. Further, the processing of steps S322 and S323 may be executed after the processing of steps S324 to S328.

  In one example and other examples of the above-described depth synthesis processing, the lens unit 131 is moved to the lower limit position as an initial position and then is moved upward by a predetermined amount to the upper limit position, but the present invention is not limited to this. In the depth synthesis process, after the lens unit 131 is moved to the upper limit position as the initial position, it may be moved downward by a predetermined amount to the lower limit position.

  Further, in one example of the above-described depth synthesis processing and other examples, the lens unit 131 is moved in the Z direction integrally with the light projecting unit 140 as in the example of FIG. It is not limited to. In the depth synthesis processing, as in the example of FIGS. 5B, 6A, and 6B or FIGS. 7A and 7B, the stage 121, the lens unit 131, and the light projecting unit 140 are used. One or two members may be moved in the Z direction with respect to the other members.

  In the above description, a plurality of original image data respectively corresponding to the positions H1 to Hj are synthesized as the depth synthesis processing, but the present invention is not limited to this. A plurality of original image data respectively corresponding to the positions H1 to Hj may be used independently without being combined.

  For example, based on the determination result by the focus determination unit 620 in FIG. 7, the original image data in which the focus of the imaging unit 132 most matches the specific part of the observation target S is a plurality of original image data respectively corresponding to the positions H1 to Hj. It may be extracted from image data. In this case, based on the plurality of original image data extracted corresponding to the ring illumination and the first to fourth directional illuminations, display image data indicating an image having a large degree of focus as a whole at high speed. Can be generated.

(4) DR Adjustment Process (a) Process Content The user gives an instruction of the DR adjustment process to the arithmetic processing unit 600 by operating the DR adjustment button b2 in FIG. 11 using the operation unit 440 in FIG. Can be.

  In the DR adjustment processing, the observation target when each of the ring illumination and the first to fourth directional illuminations is irradiated with the light receiving time of the imaging unit 132 changed to a plurality of predetermined values. S is imaged. Thereby, a plurality of original image data corresponding to each of the ring illumination and the first to fourth directional illuminations is generated by the data generation unit 610 of FIG. The overall pixel value of each original image data generated when the light receiving time of the imaging unit 132 is short is relatively small, and the overall pixel value of each original image data generated when the light receiving time of the imaging unit 132 is long Is relatively large.

  A plurality of original image data corresponding to the ring illumination is synthesized by the data generation unit 610. Thereby, the dynamic range of the original image data corresponding to the ring illumination can be adjusted. Similarly, a plurality of original image data corresponding to each directional illumination is synthesized by the data generator 610. Thereby, the dynamic range of the original image data corresponding to each directional illumination can be adjusted.

  Here, the adjustment of the dynamic range includes enlargement and reduction of the dynamic range. By combining a plurality of original image data so as to expand the dynamic range, it is possible to reduce blackout and halation (overexposure) from the image. On the other hand, by combining a plurality of original image data so as to reduce the dynamic range, the difference in shading of the image is increased. Thereby, the unevenness of the observation target S having a smooth surface can be precisely observed.

  In the above description, the original image data is first synthesized so that the dynamic range is adjusted for each of the ring illumination and the first to fourth directional illuminations, and the original image data is displayed based on the synthesized plurality of original image data. Although image data is generated, the present invention is not limited to this. Display image data is first generated based on a plurality of original image data for each light receiving time of the imaging unit 132, and display image data generated for each light receiving time of the imaging unit 132 is adjusted so that the dynamic range is adjusted. It may be synthesized.

(B) Example of DR Adjustment Process The system program stored in the storage unit 420 in FIG. 2 includes a DR adjustment program. The control unit 410 in FIG. 2 performs a DR adjustment process by executing the DR adjustment program.

  FIGS. 26 and 27 are flowcharts illustrating an example of the DR adjustment processing. The control unit 410 sets the light receiving time of the imaging unit 132 to a predetermined initial value (step S401). In this state, the control unit 410 irradiates the observation target S with ring illumination by the light projecting unit 140 and captures an image of the observation target S by the imaging unit 132 (step S402). Subsequently, the control unit 410 determines whether or not the observation target S has been imaged in all desired light receiving times of the imaging unit 132 in a state where the ring illumination is emitted (Step S403).

  In Step S403, when the observation target S is not imaged in all the desired light receiving times of the imaging unit 132, the control unit 410 sets the light receiving time of the imaging unit 132 to the next predetermined value (Step S403). S404). Thereafter, control unit 410 returns to the process of step S402. The control unit 410 repeats the processing of steps S402 to S404 until the observation target S is imaged in all the desired light receiving times of the imaging unit 132.

  In step S403, when the observation target S is imaged in all desired light receiving times of the imaging unit 132, the control unit 410 combines a plurality of original image data corresponding to the generated ring illumination (step S405). . Thereby, the dynamic range of the original image data corresponding to the ring illumination is adjusted.

  After that, the control unit 410 sets i to 1 (Step S406). Here, i indicates a number of a plurality of directional lights. Next, the control unit 410 sets the light receiving time of the imaging unit 132 to a predetermined initial value (Step S407). In this state, the control unit 410 irradiates the i-th directional illumination to the observation target S by the light projecting unit 140 and captures an image of the observation target S by the imaging unit 132 (step S408). Subsequently, the control unit 410 determines whether or not the observation target S has been imaged during all desired light receiving times of the imaging unit 132 in a state where the i-th directional illumination is emitted (Step S409).

  In step S409, when the observation target S is not imaged in all the desired light receiving times of the imaging unit 132, the control unit 410 sets the light receiving time of the imaging unit 132 to the next predetermined value (step S409). S410). Thereafter, control unit 410 returns to the process of step S408. The control unit 410 repeats the processing of steps S408 to S410 until the observation target S is imaged in all the desired light receiving times of the imaging unit 132.

  In step S409, when the observation target S is imaged in all the desired light receiving times of the imaging unit 132, the control unit 410 determines the plurality of original image data corresponding to the generated i-th directional illumination. Are combined (step S411). Thereby, the dynamic range of the original image data corresponding to the i-th directional illumination is adjusted.

  Next, control unit 410 determines whether or not i is 4 (step S412). If i is not 4 in step S412, the control unit 410 updates i to i + 1 (step S413). Thereafter, control unit 410 returns to the process of step S407. Until i becomes 4, control unit 410 repeats the processing of steps S407 to S413. Thereby, a plurality of original image data corresponding to each of the first to fourth directional illuminations is generated and synthesized so that the dynamic range is adjusted. If i is 4 in step S412, control unit 410 ends the process.

  In the above description, some processes may be performed at other times. For example, the processing of step S405 may be executed in parallel with the processing of steps S406 to S413. Further, the process of step S411 corresponding to the i-th directional illumination may be executed in parallel with steps S407 to S410 corresponding to the (i + 1) th directional illumination. In these cases, the speed of the DR adjustment processing can be increased.

  Alternatively, the processing of steps S401 to S405 may be executed after the processing of steps S406 to S413. Further, the processing in step S411 corresponding to the i-th directional illumination may be executed after the processing in steps S407 to S410 corresponding to the (i + 1) th directional illumination.

(C) Another Example of DR Adjustment Process FIGS. 28 and 29 are flowcharts illustrating another example of the DR adjustment process. The control unit 410 sets the light receiving time of the imaging unit 132 to a predetermined initial value (Step S421). In this state, the control unit 410 irradiates the observation target S with the ring illumination by the light projecting unit 140 and captures an image of the observation target S by the imaging unit 132 (step S422).

  Subsequently, control unit 410 sets i to 1 (step S423). Here, i indicates a number of a plurality of directional lights. After that, the control unit 410 irradiates the i-th directional illumination to the observation target S by the light projecting unit 140 and images the observation target S by the imaging unit 132 (step S424).

  Next, control unit 410 determines whether or not i is 4 (step S425). If i is not 4 in step S425, control unit 410 updates i to i + 1 (step S426). Thereafter, control unit 410 returns to the process of step S424. Until i becomes 4, control unit 410 repeats the processing of steps S424 to S426. Thereby, a plurality of original image data corresponding to each of the first to fourth directional illuminations is generated. Subsequently, the control unit 410 determines whether or not the observation target S has been imaged in all the desired light receiving times of the imaging unit 132 (Step S427).

  In step S427, when the observation target S is not imaged in all the desired light receiving times of the imaging unit 132, the control unit 410 sets the light receiving time of the imaging unit 132 to the next predetermined value (step S427). S428). Thereafter, control unit 410 returns to the process of step S422. The control unit 410 repeats the processing of steps S422 to S428 until the observation target S is imaged in all the desired light receiving times of the imaging unit 132.

  In step S427, when the observation target S is imaged in all the desired light receiving times of the imaging unit 132, the control unit 410 adjusts the dynamic range of the plurality of original image data corresponding to the generated ring illumination. (Step S429). Thereby, the dynamic range of the original image data corresponding to the ring illumination is adjusted.

  After that, the control unit 410 sets i to 1 again (step S430). Next, the control unit 410 combines a plurality of original image data corresponding to the generated i-th directional illumination (step S431). Thereby, the dynamic range of the original image data corresponding to the i-th directional illumination is adjusted. Subsequently, control unit 410 determines whether or not i is 4 (step S432).

  If i is not 4 in step S432, control unit 410 updates i to i + 1 (step S433). Thereafter, control unit 410 returns to the process of step S431. Until i becomes 4, control unit 410 repeats the processing of steps S431 to S433. Thereby, a plurality of original image data corresponding to each of the first to fourth directional illuminations is synthesized such that the dynamic range is adjusted. If i is 4 in step S432, control unit 410 ends the process.

  In the above description, some processes may be performed at other times. For example, part of the processing of steps S429 to S433 may be performed in parallel with the processing of steps S421 to S428. In this case, the speed of the DR adjustment processing can be increased. Further, the processing of step S422 may be executed after the processing of steps S423 to S426. Further, the processing of step S429 may be executed after the processing of steps S430 to S433.

(5) Effect In the magnifying observation apparatus 1 according to the present embodiment, the display image data stored in the general-purpose data area 422A is edited by editing the metadata stored in the metadata area 422B of the file FI. Be updated. Therefore, by editing the metadata so as to obtain appropriate lighting conditions according to the shape and material of the observation target S, display image data indicating an image of the observation target S under appropriate imaging conditions is generated. be able to. Thereby, it is possible to easily obtain an image of the observation target S according to a request of the user.

  Further, since the display image data is stored in a general-purpose image file format, the user can use an external device 2 having a general display function different from the magnifying observation device 1 to use an image based on the display image data. Can be observed. Furthermore, since the display image data can be generated using the previously generated original image data in a non-general image file format, there is no need to re-capture the observation target S. Therefore, the burden on the user can be reduced.

[2] Second Embodiment The following describes a magnifying observation apparatus according to a second embodiment of the present invention, which is different from the magnifying observation apparatus 1 according to the first embodiment. FIG. 30 is a schematic diagram illustrating a configuration of a magnifying observation apparatus according to the second embodiment of the present invention. As shown in FIG. 30, in the magnifying observation apparatus 1 according to the present embodiment, the measurement head 100 further includes a light projecting unit 160. The light projecting unit 160 has a half mirror 161.

  The lens unit 131 is configured to hold the light projecting unit 160 therein. The light projecting unit 160 is disposed in the lens unit 131 in a state where it is inclined at about 45 ° with respect to the optical axis A1 of the objective lens 131a so that the reflection surface of the half mirror 161 faces obliquely downward. The light projecting unit 160 is optically connected to the light generating unit 300 of the processing device 200 by a part of optical fibers (not shown) of the fiber unit 201.

  The light shielding unit 320 of the light generation unit 300 has a plurality of opening patterns respectively corresponding to the regions 140A to 140D of the light projection unit 140 in FIG. 3 and has an opening pattern corresponding to the light projection unit 160. The light projection control unit 510 of FIG. 2 can switch light into the light projection unit 140 as in the first embodiment by switching the aperture pattern of the light shielding unit 320 that allows light to pass therethrough. Light can be incident on 160. The behavior of the light incident on the light emitting unit 140 is the same as the behavior of the light incident on the light emitting unit 140 in the first embodiment.

  The light that has entered the light projecting unit 160 is reflected by the half mirror 161 and is emitted downward along the optical axis A1 of the objective lens 131a to irradiate the observation target S. The light emitted from the light projecting unit 160 is referred to as coaxial incident illumination. The light applied to the observation target S is reflected upward, transmitted through the half mirror 161 and the lens unit 131 of the light projecting unit 160, and guided to the imaging unit 132.

  According to the above configuration, the light projecting unit 160 can irradiate the observation target S with light from a position closer to the optical axis A1 of the objective lens 131a than the light projecting unit 140. Therefore, the coaxial epi-illumination is a bright-field illumination emitted in a direction parallel to the optical axis A1 of the objective lens 131a, and the ring illumination is a dark-field illumination illuminated in a direction inclined with respect to the optical axis A1 of the objective lens 131a. Become. By irradiating the observation target S with the coaxial incident illumination, it is possible to more clearly image unevenness on the surface of the observation target S and a difference in material. In addition, it is also possible to irradiate the observation target S with light simultaneously from the light projecting units 140 and 160.

  The imaging control unit 520 in FIG. 2 controls the light receiving time, the gain, the timing, and the like of the imaging unit 132 during the irradiation of the coaxial epi-illumination. In the present embodiment, the irradiation of the coaxial epi-illumination is performed after the irradiation of the ring illumination and the first to fourth directional illuminations. The imaging control unit 520 adjusts the light receiving time at the time of irradiation of the coaxial epi-illumination by performing automatic exposure based on the average luminance value of the original image data corresponding to the previously generated ring illumination. The light receiving time during the irradiation of the coaxial epi-illumination may be adjusted to a desired value by the user.

  The imaging unit 132 further generates original image data indicating the observation target S when the coaxial incident illumination is applied to the observation target S. The generated original image data corresponding to the coaxial epi-illumination is stored as metadata in the sub-data area SUB1 of the data area 422 in the file FI in FIG. In addition, imaging information that further indicates whether or not the irradiation of the coaxial epi-illumination is performed and the imaging conditions such as the light receiving time during the irradiation of the co-axial epi-illumination is stored as metadata in the sub-data area SUB1 of the data area 422 in the file FI.

  The data generation unit 610 in FIG. 7 further includes a display image based on the original image data corresponding to the coaxial incident illumination in addition to the original image data corresponding to the ring illumination and the directional illumination stored in the sub-data area SUB1. Generate data. Specifically, the data generation unit 610 combines a part or all of a plurality of original image data respectively corresponding to the ring illumination, the directional illumination, and the coaxial epi-illumination at a ratio determined by an illumination condition designated by a user. I do. The generated display image data is stored in the general-purpose data area 422A of the data area 422 in the storage unit 420.

  In the multiple illumination imaging process according to the present embodiment, the observation target S is imaged using coaxial epi-illumination after imaging the observation target S using the ring illumination and the first to fourth directional illuminations. Thereby, a plurality of (six in this example) original image data respectively corresponding to the ring illumination, the first to fourth directional illuminations, and the coaxial incident illumination are generated.

  When the multiple illumination imaging process is completed, an observation screen 430A is displayed on the display unit 430. FIG. 31 is a diagram illustrating a display example of the observation screen 430A after the multiple illumination imaging processing according to the second embodiment. As shown in FIG. 31, in the function display area 431, an incident light button b11 is displayed in addition to the plurality of buttons b1 to b7.

  The user operates the epi-illumination button b11 using the operation unit 440. Thereby, the user can instruct that the display image data should be generated using the original image data corresponding to the coaxial incident illumination.

  When the use of the original image data corresponding to the coaxial epi-illumination is instructed, a combination ratio of the original image data corresponding to the co-axial epi-illumination with other original image data (hereinafter, referred to as an epi-image ratio) is displayed in the sub display area 433. .) Are displayed, and a bar 433c and a slider 433d are displayed.

  The user can specify the incident image ratio by operating the slider 433d. In the example of FIG. 31, the incident image ratio is specified to be higher as the slider 433d approaches the left end of the bar 433c, and the incident image ratio is specified to be lower as the slider 433d approaches the right end of the bar 433c.

  When the incident-light image ratio is specified, the control unit 410 determines a plurality of original images based on the specified incident-light image ratio in addition to the specified emission direction in the process of step S209 in FIG. Calculate the data composition ratio.

  For example, the control unit 410 first calculates the combination ratio of the original image data corresponding to each of the ring illumination and the first to fourth directional illuminations based on the designated emission direction. Thereafter, when the specified incident image ratio is t (t is a number from 0 to 100)%, the control unit 410 determines the total of the plurality of combined ratios of the ring illumination and the first to fourth directional illuminations. Is corrected to (100-t)%, the synthesis ratio of the original image data corresponding to each of the ring illumination and the first to fourth directional illuminations is corrected.

  The display object image data is generated by combining the plurality of original image data based on the plurality of combination ratios calculated as described above, and the observation target S including the components of the original image data corresponding to the coaxial incident illumination. Is displayed in the main display area 432.

  According to the image SI of the original image data corresponding to the coaxial incident illumination, compared to the image SI of the original image data corresponding to each of the ring illumination and the first to fourth directional illuminations, Irregularities and differences in materials can be observed more accurately.

  Therefore, the user can easily acquire the image SI of the observation target S according to the observation purpose by operating the falling-light button b11 and the slider 433d in FIG.

[3] Third Embodiment The following describes a magnifying observation apparatus according to a third embodiment of the present invention, which is different from the magnifying observation apparatus 1 according to the first embodiment. FIG. 32 is a schematic diagram illustrating a configuration of a magnifying observation apparatus according to the third embodiment of the present invention. As shown in FIG. 32, in magnifying observation apparatus 1 according to the present embodiment, measurement head 100 further includes a light projecting unit 170. The light projecting unit 170 has a mirror 171. The magnifying observation apparatus 1 further includes a fiber unit 204. The fiber unit 204 includes a plurality of optical fibers (not shown). The light emitting section 170 may be built in the installation section 111 of the stand section 110.

  The installation section 111 of the stand section 110 is configured to be able to hold the light emitting section 170 inside. The light projecting unit 170 is arranged in the installation unit 111 in a state in which the reflecting surface of the mirror 171 is inclined approximately 45 ° with respect to the optical axis A1 of the objective lens 131a so that the reflecting surface faces obliquely upward. Accordingly, the light projecting unit 170 faces the lens barrel unit 130 with the observation target S and the stage 121 interposed therebetween. The light projecting unit 170 is optically connected to the light generating unit 300 of the processing device 200 by a part of optical fibers (not shown) of the fiber unit 201.

  The light blocking section 320 of the light generating section 300 has a plurality of opening patterns respectively corresponding to the regions 140A to 140D of the light projecting section 140 in FIG. 3 and has an opening pattern corresponding to the light projecting section 170. The light projection control unit 510 of FIG. 2 can switch light into the light projection unit 140 as in the first embodiment by switching the aperture pattern of the light shielding unit 320 that allows light to pass therethrough. Light can be incident on 170. The behavior of the light incident on the light emitting unit 140 is the same as the behavior of the light incident on the light emitting unit 140 in the first embodiment.

  The light that has entered the light projecting unit 170 is reflected by the mirror 171 and emitted upward along the optical axis A1 of the objective lens 131a, and irradiates the observation target S on the stage 121. Light emitted from the light projecting unit 170 is referred to as transmitted illumination. The light applied to the observation target S is transmitted upward, transmitted through the lens unit 131, and guided to the imaging unit 132. By irradiating the observation target S with the transmitted illumination, the internal structure of the observation target S can be imaged.

  The imaging unit 132 further generates original image data indicating the observation target S when the transmission illumination is applied to the observation target S. The generated original image data corresponding to the transmitted illumination is stored as metadata in the sub-data area SUB1 of the data area 422 in the file FI in FIG. In addition, imaging information further indicating imaging conditions such as whether or not transmission illumination is performed and the light receiving time at the time of transmission illumination is stored as metadata in the sub-data area SUB1 of the data area 422 in the file FI.

  The data generation unit 610 in FIG. 7 further displays the display image data based on the original image data corresponding to the transmission illumination in addition to the original image data corresponding to the ring illumination and the directional illumination stored in the sub-data area SUB1. Generate Specifically, the data generation unit 610 combines a part or all of the plurality of original image data respectively corresponding to the ring illumination, the directional illumination, and the transmitted illumination at a ratio determined by the illumination condition specified by the user. . The generated display image data is stored in the general-purpose data area 422A of the data area 422 in the file FI.

  The measuring head 100 in FIG. 32 does not include the light projecting unit 160 that emits coaxial incident illumination, but the present invention is not limited to this. The measuring head 100 of FIG. 32 may include a light emitting unit 160 similar to the light emitting unit 160 of FIG. In this case, display image data is generated based on original image data respectively corresponding to the ring illumination, the directional illumination, the coaxial incident illumination, and the transmitted illumination.

  In the multiple illumination imaging process according to the present embodiment, the observation target S is imaged using transmission illumination after imaging the observation target S using ring illumination and the first to fourth directional illuminations. Thereby, a plurality of (six in this example) original image data respectively corresponding to the ring illumination, the first to fourth directional illuminations, and the transmitted illuminations are generated.

  When the multiple illumination imaging process is completed, an observation screen 430A is displayed on the display unit 430. FIG. 33 is a diagram illustrating a display example of the observation screen 430A after the multiple illumination imaging processing according to the third embodiment. As shown in FIG. 33, in the function display area 431, a transparent button b12 is displayed in addition to the plurality of buttons b1 to b7.

  The user operates the transmission button b12 using the operation unit 440. Thereby, the user can instruct that display image data should be generated using the original image data corresponding to the transmitted illumination.

  When the use of the original image data corresponding to the transmitted illumination is instructed, a combination ratio of the original image data corresponding to the transmitted illumination to other original image data (hereinafter, referred to as a transmitted image ratio) is displayed in the sub display area 433. Is displayed, and a bar 433e and a slider 433f for designating are displayed.

  The user can specify the transmission image ratio by operating the slider 433f. In the example of FIG. 33, the transmission image ratio is specified to be higher as the slider 433f approaches the left end of the bar 433e, and the transmission image ratio is specified to be lower as the slider 433f approaches the right end of the bar 433e.

  When the transmission image ratio is specified, the control unit 410 determines a plurality of original images based on the specified transmission direction and the specified transmission image ratio in the process of step S209 in FIG. 17 in the display image generation process. Calculate the data composition ratio.

  For example, the control unit 410 first calculates the combination ratio of the original image data corresponding to each of the ring illumination and the first to fourth directional illuminations based on the designated emission direction. After that, when the designated transmission image ratio is u (u is a number from 0 to 100)%, the control unit 410 determines the sum of the plurality of combined ratios of the ring illumination and the first to fourth directional illuminations. Is corrected to (100-u)%, the synthesis ratio of the original image data corresponding to each of the ring illumination and the first to fourth directional illuminations is corrected.

  The display image data is generated by combining the plurality of original image data based on the plurality of combination ratios calculated as described above, and the observation target S including the component of the original image data corresponding to the transmitted illumination is generated. Image SI is displayed in main display area 432. When the observation target S is formed of a material that transmits light, the internal structure of the observation target S clearly appears in the image SI of the original image data corresponding to the transmitted illumination.

  Therefore, the user can easily obtain the image SI of the observation target S according to the observation purpose by operating the transmission button b12 and the slider 433f in FIG.

  As described above, when the measurement head 100 in FIG. 32 includes the light projecting unit 160 in FIG. 30, in the multiple-illumination imaging process, a plurality of light sources corresponding to ring illumination, directional illumination, Original image data may be generated. In this case, on the observation screen 430A, the incident button b11, bar 433c, and slider 433d of FIG. 31 and the transmission button b12, bar 433e, and slider 433f of FIG. 33 may be displayed at the same time. Thereby, the degree of freedom of adjustment of the image SI of the observation target S displayed in the main display area 432 is improved.

[4] Other Embodiments (1) In the above embodiment, it is preferable that the regions 140A to 140D of the light projecting unit 140 are rotationally symmetric about the optical axis A1 of the objective lens 131a. Is not limited to this. The regions 140A to 140D of the light projecting unit 140 need not be arranged rotationally symmetrically about the optical axis A1 of the objective lens 131a.

  (2) In the above embodiment, when display image data is generated by combining a plurality of original image data, one of the plurality of original image data is original image data corresponding to ring illumination. However, the present invention is not limited to this. The original image data corresponding to the ring illumination is not used for synthesis, and a part or all of the plurality of original image data corresponding to the directional illumination, the coaxial epi-illumination, and the transmitted illumination is synthesized, so that display image data is obtained. May be generated.

  (3) In the above embodiment, display image data is generated by combining a plurality of original image data, but the present invention is not limited to this. The display image data may be generated by selecting one of the plurality of generated original image data. In this configuration, it is preferable that more original image data is generated. In this case, more accurate display image data can be generated. Therefore, in order to be able to generate more original image data, more light emitting regions may be provided, or light emitting members may be provided so as to emit light from more positions. Alternatively, a single light emitting member may be provided to be movable to a plurality of emitting positions.

[5] Correspondence relationship between each component of the claims and each unit of the embodiment Hereinafter, an example of correspondence between each component of the claims and each unit of the embodiment will be described. Not limited.

  In the above embodiment, the observation target S is an example of the observation target, the magnifying observation device 1 is an example of the magnifying observation device, and the light projecting units 140, 160, and 170 are the first, second, and third, respectively. , And the imaging unit 132 is an example of an imaging unit. The illumination condition setting unit 642 is an example of an illumination condition setting unit, the data generation unit 610 is an example of a data generation unit, and the general data area 422A and the metadata area 422B are examples of first and second areas, respectively. , The storage unit 420 is an example of a storage unit.

  The editing unit 650 is an example of the editing unit, the light icon ss1 or the emission position image ss4 is an example of the index, the display unit 430 is an example of the display unit, and the operation unit 440 is the first and second operation units. In this example, the imaging condition setting unit 641 is an example of the imaging condition setting unit, and the objective lens 131a is an example of the objective lens. The optical axis A1 is an example of an optical axis, the focus driving unit 113 is an example of a focus position changing unit, the report 10 is an example of a report, and the report creating unit 660 is an example of a report creating unit. , The external device 2 is an example of an external device, and the communication unit 450 is an example of a communication unit.

  Various other elements having the configuration or function described in the claims may be used as the constituent elements in the claims.

  INDUSTRIAL APPLICATION This invention can be utilized effectively for various magnification observation apparatuses.

REFERENCE SIGNS LIST 1 magnification observation device 2 external device 10 report 11 name display column 12 image display column 13 status display column 14 affiliation display column 15 result display column 100 measuring head 101 tilting mechanism 110 stand unit 111 installation unit 112 holding unit 113 focus driving unit 120 Stage device 121 Stage 122 Stage drive unit 130 Lens barrel unit 130c Center of tilt 131 Lens unit 131a Objective lens 132 Imaging unit 133 Tilt sensor 140, 160, 170 Light emitting units 140A to 140D Area 140o Light emitting unit 141 Holding member 141a Through hole 142 Optical fibers 142a, 310 Light source 150 Control board 161 Half mirror 171 Mirror 200 Processing unit 201, 204 Fiber unit 202, 203 Cable 210 Housing 300 Light generation unit 320 Light shielding unit 4 0 Control device 410 Control unit 420 Storage unit 421 Header area 422 Data area 422A General-purpose data area 422B Metadata area 430 Display unit 430A Observation screen 431 Function display area 432 Main display area 432a, 432b Position adjustment bar 433 Secondary display area 433a Emission direction Designation field 433b Emission direction display field 433c, 433e Bar 433d, 433f Slider 440 Operation part 500 Drive control part 510 Light emission control part 520 Imaging control part 530 Focus control part 540 Stage control part 600 Arithmetic processing part 610 Data generation part 620 Focusing Judgment unit 630 Calculation unit 640 Condition setting unit 641 Imaging condition setting unit 642 Illumination condition setting unit 650 Editing unit 660 Report creation unit A1, A2 Optical axis b1 Depth synthesis button b2 DR adjustment button b7 Save button b11 falling button b12 transmission button d1 to d3 distance FI file H1 to Hj position PA to PE point S observation target SF depth composite image SH shadow SI image sp target partial image ss0 target position image ss1 light icon ss2 reference point image ss3 hemisphere Image ss4 Emission position image SUB1 to SUB3 Sub data area

Claims (15)

An enlarged observation device capable of displaying an image of the observation target imaged according to the set imaging conditions,
A first light projecting unit for selectively irradiating the observation target with light in different first and second emission directions ,
A plurality of original images including first and second original image data respectively showing images of the observation object when receiving light from the observation object and irradiating the light in the first and second emission directions. An imaging unit that generates data;
An illumination condition setting unit that sets an illumination condition including a specific emission direction input by a user ,
To determine the percentage of the synthesis of the first original image data and the second original image data based on the illumination condition set by the pre-Symbol illumination condition setting unit, the first original image at a rate determined By combining the data and the second original image data, display image data indicating an image of the observation target to be obtained when it is assumed that the light in the specific emission direction is irradiated on the observation target is generated. A data generator to perform
A first area for storing the display image data in a general-purpose image file format and a second area for storing the illumination conditions and a plurality of the original image data as metadata in a non-general-purpose image file format. Including, a storage unit that stores the illumination conditions, a plurality of the original image data and the display image data in association with each other,
An enlargement observation apparatus comprising: an editing unit that updates the display image data stored in the first area by editing the metadata stored in the second area of the storage unit. Wherein the data generating unit, the generating display image data by synthesizing the second region to the stored first and second original image data of the storage unit, magnifying observation of claim 1, wherein apparatus. The enlarged observation according to claim 2 , wherein the data generation unit determines a combination ratio of the first and second original image data based on the illumination condition stored in the second area of the storage unit. apparatus. A display unit that displays an index indicating a light emission position,
A first operation unit operated by a user to specify a position of the index displayed on the display unit,
If the position of the indicator by the first operating unit is specified, the editing unit, wherein determining the specific emission direction based on the position of the specified the index, one of the claims 1 to 3, one The magnifying observation device according to the paragraph. The first operation unit is operated by a user to change a position of the index displayed on the display unit,
When the position of the index is changed by the first operation unit, the editing unit edits the lighting condition stored in the second area of the storage unit based on the changed position of the index. The magnifying observation device according to claim 4 , which performs the observation. The editing unit stores the lighting conditions before and after editing in the second area of the storage unit,
The first operation unit is operated by a user to specify any of the plurality of lighting conditions stored in the second area of the storage unit,
When the lighting condition is specified by the first operation unit, the data generation unit generates the display image data corresponding to the specified lighting condition, and the display unit performs the first operation. The magnifying observation apparatus according to claim 5 , wherein the indicator is displayed at a position corresponding to the illumination condition specified by a unit. The metadata further includes an imaging condition at the time of generating the display image data,
The magnifying observation device,
A second operation unit operated by a user to specify any of the plurality of imaging conditions stored in the second area of the storage unit;
The magnification observation apparatus according to any one of claims 1 to 6 , further comprising: an imaging condition setting unit configured to update the set imaging condition to the imaging condition designated by the second operation unit. The imaging condition set by the imaging condition setting unit includes a light receiving time of the imaging unit,
The magnifying observation apparatus according to claim 7 , wherein the imaging unit generates a plurality of the original image data based on a light receiving time set by the imaging condition setting unit. An objective lens arranged to guide the light from the observation target to the imaging unit while condensing light,
A focus position changing unit that changes a focus position of light passing through the objective lens relative to an observation target in an optical axis direction of the objective lens,
The data generating unit focuses on each part of the observation target based on the plurality of original image data generated by the imaging unit in a state where a plurality of focal positions of light are changed by the focal position changing unit. Generating the display image data indicating an image,
Imaging condition set by the imaging condition setting unit, the further comprising data generating unit generates the display image data indicating an image-focus to each part of the observed object, according to claim 8 Magnifying observation device. The imaging condition set by the imaging condition setting unit further includes a range of a focal position of light in an optical axis direction of the objective lens,
The magnifying observation apparatus according to claim 9 , wherein the focal position changing unit changes a focal position of light in an optical axis direction of the objective lens in a range set by the imaging condition setting unit. The imaging unit generates the original image data in a state where a plurality of light receiving times are changed,
The data generation unit generates the display image data having a dynamic range adjusted based on a plurality of the original image data generated by the imaging unit in a state where a plurality of light reception times are changed,
The imaging condition set by the imaging condition setting unit further includes that the data generation unit generates the display image data whose dynamic range is adjusted, according to any one of claims 7 to 10 . Magnifying observation device. A second light projecting unit arranged so as to be able to irradiate the observation object with coaxial incident light;
The enlargement according to any one of claims 7 to 11 , wherein the imaging condition set by the imaging condition setting unit further includes irradiating the observation target with coaxial incident light from the second light projecting unit. Observation device. A third light-emitting unit disposed so as to face the imaging unit with the observation object interposed therebetween,
The magnifying observation apparatus according to any one of claims 7 to 12 , wherein the imaging condition set by the imaging condition setting unit further includes irradiating the observation target with light by the third light projecting unit. . The magnifying observation apparatus according to any one of claims 1 to 13 , further comprising a report creation unit that creates a report in which an observation result of an image based on the display image data is displayed. 2. The communication apparatus according to claim 1, further comprising: a communication unit connected to the external device having a display function so as to be able to communicate with the external device, the communication unit causing the external device to display an image based on the display image data stored in the first area of the storage unit. The magnifying observation apparatus according to any one of claims 14 to 14 .

Images