KR20000057920A - Microscope image transmitting system, method of processing microscope images, and recording medium for use in processing microscope images - Google Patents

Abstract

PURPOSE: A system, method and recording medium are provided to allow designation for a mesh split of an observation area to be performed with efficiency and accuracy through a simple operation. CONSTITUTION: A system comprises a video camera(6) for obtaining a specimen picture by picking up a specimen, and an observation side personal computer(1) and a client side personal computer(5) for recording a still picture including the picked up specimen picture, detecting a position of the specimen picture from the still picture, splitting an area where the specimen picture exists into blocks of a predetermined unit, and displaying the lines indicative of the blocks onto monitors(2,4) in such a manner as to be superposed onto the still picture.

Description

MICROSCOPE IMAGE TRANSMITTING SYSTEM, METHOD OF PROCESSING MICROSCOPE IMAGES, AND RECORDING MEDIUM FOR USE IN PROCESSING MICROSCOPE IMAGES}

The present invention relates to, for example, a system for remotely observing a microscope image of a pathological specimen, and in particular, an area desired to be enlarged in the specimen is designated as a predetermined block, and further magnification can be observed for the block. A microscope image transmission system, a microscope image processing method, and a recording medium used for microscope image processing.

Conventionally, a specimen placed on a stage of a microscope is photographed with a video camera or the like, captured by an image capturing board of a personal computer, and the image is digitalized and transmitted to a personal computer located remotely, for example, via a public line such as ISDN. Accordingly, various techniques related to a microscope image transmission system for displaying an image on a monitor of the personal computer have been developed. Such a microscope image transmission system is used for pathological diagnosis of pathology, for example, and microscopic manipulation, magnification change, and stage movement according to pathology diagnosis by remote pathology are also possible.

Hereinafter, the precedent example of such a microscope image transmission system is demonstrated.

For example, Japanese Patent Laid-Open Publication No. 9-120031 connects a transmitting terminal personal computer (client side) and a receiving terminal personal computer (observation side) via an ISDN line to change the magnification about the next photographing position and magnification. Positioning is performed remotely (observation side), calculation of absolute coordinates of the microscope stage of this designated shooting position is performed, and stage position control is performed based on the calculation result, and then capturing the image to be observed. A technique characterized by this is disclosed.

In general, pathology observation under a microscope means that the pathologist first visualizes the entire image of the preparat, establishes an approximate observation policy, and subsequently performs microscopic observation at a low magnification to perform initial diagnosis or detailed observation at a higher magnification. In this system, such a series of diagnostic procedures can be realized remotely (observation side).

On the other hand, Japanese Unexamined Patent Publication No. 6-222281 discloses that when performing microscopic observation at low magnification, a sample image is divided into meshes into a predetermined rectangular area, and a desired area is selected from the divided areas. A system is disclosed which automatically inputs an image.

Further, Japanese Laid-Open Patent Publication No. 9-138355 discloses a technique characterized by binarizing a sample image as a predetermined predetermined threshold value and deleting unnecessary blocks in a mesh-divided area.

However, in the technique disclosed by Japanese Patent Laid-Open No. 6-222281 described above, it is possible to mesh divide a sample image and to confirm whether or not to capture an image for each of the divided portions. On the other hand, however, there has been a problem that the operation is complicated, with the task of specifying the start point and the end point of the area in which the worker desires to divide the mesh, and deleting the unnecessary part after the mesh is divided. In particular, it is desirable not to waste time in the diagnosis during the operation, and the improvement is promising.

In addition, in the technique disclosed by Japanese Patent Laid-Open No. 9-138355, unnecessary points can be automatically deleted by setting a threshold value only by specifying the start point and the end point of the mesh input position. However, since block designation is based on fixed coordinates (x, y), there is a problem that optimal block designation cannot be executed.

In addition, there was a possibility that accurate mesh segmentation could not be specified by recognizing unnecessary data such as garbage. That is, in the case of a macro image, since the range that can be captured is large, the end of the preparation may be captured, and even the space between the stage cranmel and the preparat that presses the preparat may be captured. In addition, when a low magnification mesh division designation is executed from a macro image, unnecessary image portions are likely to occur, and if these unnecessary portions are recognized as a sample image, a solution is required because an error is generated in the mesh segmented result.

SUMMARY OF THE INVENTION The present invention has been made in view of the above problem, and its object is to provide a microscope image that can efficiently and accurately perform designation on mesh division of an observation area in an initial observation image by simple operation in remote observation of a microscope image. The present invention provides a transmission system, a microscope image processing method, and a recording medium for use in microscope image processing.

In order to achieve the above object, in the first aspect of the present invention, a microscope image transmission system that designates a desired area to be observed at a high magnification among sample images in a predetermined block unit includes imaging means for imaging a sample to obtain the sample image. Recording means for recording a still image including a sample image picked up by the image pickup means, display means for displaying a still image recorded in the recording means, and a position of the sample image is extracted, And a control means having a function of dividing only an area in which the sample image exists by a predetermined block unit and superimposing the indicator representing the block on the display means.

According to a second aspect, a microscope image transmission system that designates a desired area to be observed at a high magnification among sample images in a predetermined block unit includes: imaging means for picking up a sample to obtain the sample image, and a sample picked up by the imaging means. Recording means for recording a still image including an image; display means for displaying at least a still image recorded in the recording means; and inputting a still image in the absence of the specimen, and displaying the still image in the presence of the specimen. The detection means for detecting the light quantity lacking portion and the light quantity lacking portion detected by the detecting means based on the difference between the luminance information of the two inputs, and then detecting the position of the sample image from the still image, Only the area in which the sample image exists is divided into predetermined block units, and the index of the block is displayed on the display means. Control means for controlling to display in superimposition on the still image.

In the third aspect, a microscope image transmission system is provided in which at least one of the observation side and the requesting side provided with the control means are communicatively connected to each other, and divides a region of the sample image desired to be observed at a high magnification by a predetermined block unit. The requesting side is capable of transmitting and receiving information to and from the imaging unit that captures a sample to obtain the sample image, first display means to display a still image including the sample image picked up by the imaging unit, and the observation side. A first transmission / reception means, wherein the observation side includes second display means for displaying a still image transmitted from the requesting side, and second transmission / reception means for transmitting and receiving information with the requesting side, wherein the control means Extracts the position of the sample image from the still image displayed on the display means on the side using the control means, and A function of dividing only the region in which the sample image exists based on a predetermined block unit on the basis of a predetermined block, and displaying an index indicating the block superimposed on the still image, and stopping the display means on the side not using the control means. Transmitting at least the positional information of the index indicating the block from the side using the control means to the side not in use so that the image is also the same image as the still image displayed on the display means on the side using the control means. It has a function, and the same information is shared between the requesting side and the observation side.

In a fourth aspect, the microscope image processing method includes extracting luminance information of a still image including a sample image, setting a maximum luminance level based on luminance information regarding a background position of the sample image, and Setting a minimum luminance level based on luminance information regarding an unnecessary data position of a sample image, converting the luminance information based on the maximum luminance level and the minimum luminance level, and converting the luminance information into the converted luminance information. Based on the background position, unnecessary data position, and position of the sample image, the position of the sample image is extracted.

In a fifth aspect, the microscope image processing method includes extracting color information of a sample image, recognizing a position at which the color information approaches a maximum value as a background position of the sample image, and displaying the color information at a minimum value. Recognizing the position of the sample image as an unnecessary data position of the sample image, and recognizing the position at which the color information becomes above a predetermined threshold value as the position of the sample image, and extracting the position of the sample image.

In a sixth aspect, the microscope image processing method includes extracting different color information at the same position of a sample image, finding a position at which a difference of color information at the same position of the sample image is maximized, and the color information. Setting a predetermined threshold value based on a value at which the difference is maximized; distinguishing the difference of the color information by the threshold value, and recognizing the position of the sample image as a position above or below a threshold value as the position of the sample image. The position of a sample image is extracted.

In the seventh aspect, the recording medium used for the microscopic image processing extracts the luminance information of the still image including the sample image, sets the maximum luminance level based on the luminance information on the background position of the sample image, A minimum luminance level is set based on luminance information regarding an unnecessary data position of the sample image, the luminance information is converted based on the maximum luminance level and the minimum luminance level, and the sample image is based on the converted luminance information. The program for extracting the position of the sample image is recorded by determining the background position, the unnecessary data position, and the position of the sample image.

In the eighth aspect, the recording medium used for the microscopic image processing extracts the color information of the sample image, recognizes the position where the color information approaches the maximum value as the background position of the sample image, and at the same time, the color information is the minimum value. A program for extracting the position of the sample image is recorded by recognizing the position of the sample image as an unnecessary data position of the sample image and recognizing the position where the color information becomes above a predetermined threshold as the position of the sample image.

In the ninth aspect, the recording medium used for the microscopic image processing extracts different color information at the same position of the sample image, and then obtains a position where the difference of the color information at the same position of the sample image is maximized. By setting a predetermined threshold value based on the value at which the difference is maximized, distinguishing the difference of the color information by the threshold value, and recognizing the position of the sample image above or below the threshold value as the position of the sample image. Record the program to extract the location.

1 is a schematic diagram showing the configuration of a microscope image transmission system of the present invention;

2 is a block diagram embodying a part of the configuration of FIG. 1;

3 is a flowchart showing the operation of the microscope image transmission system of the present invention;

4A to 4D are diagrams for describing a series of processes relating to image recognition of sample image positions by the microscope image transmission system according to the first embodiment;

5 is a flowchart for explaining a mesh data creation process by the microscope image transmission system according to the first embodiment;

6A to 6E are diagrams for explaining a mesh data generation process by the microscope image transmission system according to the first embodiment;

7 is a diagram for explaining a mesh data generation process by the microscope image transmission system according to the second embodiment;

8 is a view for explaining a mesh data generation process by the microscope image transmission system according to the third embodiment;

9 is a flowchart for explaining a mesh data creation process by the microscope image transmission system according to the third embodiment;

10 is a flowchart for explaining a high magnification designation block designation process (hereinafter referred to as an in-block mesh designation) in a block when the area ratio in which the sample image exists in the block used in FIG. 9 is 50% or less;

11 is a flowchart for explaining a movement operation of a mesh frame by the microscope image transmission system according to the fifth embodiment;

12 is a view for explaining a movement operation of a mesh frame by the microscope image transmission system according to the fifth embodiment;

13 is a flowchart for explaining an automatic focus position recognition operation by the microscope image transfer system according to the sixth embodiment;

14 is a view for explaining an automatic focus position recognition operation by the microscope image transmission system according to the sixth embodiment;

15 is a diagram illustrating an example of a macro image that is caused to lack an ambient light amount;

FIG. 16 is a flowchart for explaining the operation of specifying mesh division in response to a lack of ambient light amount by the microscope image transmission system according to the seventh embodiment; FIG.

17A to 17E are schematic diagrams showing a process of detecting a sample image position by color information extraction in the eighth embodiment;

18 is a functional block diagram of a microscope image transmission system according to an eighth embodiment;

19 is a flowchart for explaining the operation of the microscope image transmission system according to the eighth embodiment;

20 is a flowchart for explaining the operation of the microscope image transmission system according to the eighth embodiment;

FIG. 21 is a view showing a state after automatically executing a mesh magnification specifying frame in the entire sample image by the microscope image transmission system according to the ninth embodiment; FIG.

Fig. 22 is a flowchart for explaining in detail the procedure for automatically setting a point for performing an autofocus for the first time after automatically executing a mesh magnification specifying frame on the entire sample image by the microscope image transmission system according to the ninth embodiment;

Fig. 23 is a flowchart for explaining the detailed procedure of pixel number data check in which a sample image exists by the microscope image transfer system according to the ninth embodiment.

EMBODIMENT OF THE INVENTION Hereinafter, with reference to drawings, embodiment of this invention is described.

1 is a schematic diagram of a microscope image transmission system of the present invention.

As shown in Fig. 1, the microscope image transmission system includes a line connection device and a public line (e.g., ISDN) 3, in which the personal computer 1 on the observation side and the personal computer 5 on the request side are not shown. It is connected so that communication is possible. In addition, each personal computer 1, 5 is provided with the monitors 2, 4 provided with the display function for displaying an observation image.

The requesting personal computer 5 is also connected to the microscope 7. This microscope 7 includes a motorized stage 9, an electric revolver 8, and a video camera 6. In addition to these, the microscope 7 can be provided with an auto focus unit, a dimming mechanism, an electric diaphragm mechanism, etc. which are not shown in figure. In addition, although the microscope which has a transmission function is employ | adopted in this example, it is not limited to this, If the microscope is equipped only with the transmission stage at least, this invention can be applied. In this embodiment, although ISDN is used for the public line, the present invention is not limited to this, and may be applied as long as it is possible to transmit information such as a satellite line.

The personal computers 1 and 5 have a video capture (not shown) function and are equipped with various terminals including a terminal for receiving the image output of the video camera 6. In addition, the personal computers 1 and 5 have an external storage unit 31 for storing image information and the like output from the microscope 7 or the like, and the MO as a peripheral device separately from the external storage unit 31. It goes without saying that a recording device such as the above may be included.

In addition to the above configuration, a hand free telephone can also be connected to the line connection device. According to this hand-free telephone, conversation is realized while carrying out personal computer operation or the like without having a handset. In the above example, only the microscope 7 is connected to the personal computer 5, but the macro image pickup device can be further connected to the personal computer 5 via a video selector or the like.

FIG. 2 is a functional block diagram illustrating the configuration of FIG. 1.

As shown in FIG. 2, the present system inputs various control signals to the microscope 7 and the video signal from the imaging unit 21 mounted on the microscope 7 to be described later. It consists of the personal computer 5 which performs a process.

The microscope 7 includes an imaging unit 21, a magnification switching unit 22, a stage driving unit 23, an auto focus unit 24, and an auto iris unit 25. However, the auto focus unit 24 and the auto iris 25 may be omitted.

On the other hand, the personal computer 5 is composed of a microscope control unit 12, an internal processing unit 13 and a user interface unit 14. The microscope control unit 12 includes a frame memory unit 26, a magnification control unit 27, a stage control unit 28, a focus control unit 29, and an iris control unit 30. The internal processing unit 13 includes an external storage unit 31, a sample area recognition unit 32, a block division unit 33, and a correlation information storage unit 34 of magnification information and block size. The user interface section 14 includes a display section 35, a magnification designation section 36, and a block addition / deletion / movement designation section 37.

The external storage unit 31 may be, for example, a hard disk or a floppy disk, a CD-ROM, a MO, or the like as a storage medium for storing the computer program of the flowchart described later.

In FIG. 2, the configuration on the requesting side has been described, but the control system of the personal computer 1 on the observation side is also substantially the same as the control system of the personal computer 5 except for the microscope control unit 12. Therefore, description is omitted.

Hereinafter, with reference to the flowchart of FIG. 3, the operation procedure of the microscope image transmission system of this invention of the said structure is demonstrated in detail.

First, on the requesting side, a sample image desired for inspection by pathology and the like on the observation side is captured as a macro image (step S1). Here, the said macro image is a sample image input by the macrophotographing apparatus which is not shown in figure, or the sample image input by the minimum to equivalent magnification of the microscope 7.

The macro image picked up by the imaging section 21 of the microscope 7 is sequentially transmitted as data to the frame memory section 26 via a video capture board (not shown) in the personal computer 5 on the requesting side, and the display section ( 35) (on the monitor 4). The input of the macro image is started in synchronization with any switch operation. That is, the capture of the macro image can be started by clicking on the operation SW button displayed based on the application software on the display unit 35 (monitor 4) on the requesting side with a mouse or by selecting with an external operation panel switch (not shown). do.

Subsequently, when the macro image is captured as described above, the requesting personal computer 5 issues a line connection request to the personal computer 1 on the observation side (step S 2). This line connection request is executed by transmitting digital data to the personal computer 1 on the observation side via a public line 3 such as a circuit connection device (not shown) or ISDN. In this line connection request, a line connection request command, initial data, macro image information, and the like are transmitted.

When the personal computer 1 on the observation side receives the line connection request command, initial data, macro image information, and the like (step S3), the operation right of the microscope 7 is changed to the observation side. Moreover, about the change of the operation right of the microscope 7, you may perform automatically at the timing with which the line connection was established. In addition, of course, you may switch the operation button displayed on the screen of the monitor 2 which is the display part 35 based on application software at the timing which an operator (pathology) of an observation side clicks with a mouse.

Next, the observation side designates the objective magnification of the microscope 7 (step S4). This magnification designation is performed by clicking on the operation button displayed on the screen of the monitor 2, which is the display unit 35 of the personal computer 1 on the observation side, with the mouse, or by the personal computer 1 on the observation side. It can be executed by operating a predetermined key pre-assigned to the keyboard, which is an operation unit (not shown). The magnification information specified in this way is appropriately transmitted to the requesting side where the microscope 7 is located.

The observer (of pathology) then selects whether to mesh or spot the portion to be enlarged (step S5). Here, the "mesh designation" refers to dividing the macro image into a grid shape (hereinafter referred to as a block) to designate an enlarged position. On the other hand, "spot designation" designates one enlargement position centering on arbitrary positions. Based on the selection of "mesh designation" or "spot designation", since the line overlay is performed on the sample image displayed on the monitor 2 which is the display section 35 on the observation side, the observer (pathology) is easy. You can check the location.

In the present invention, since the positional information is to be communicated with the requesting side at any time in accordance with the mesh designation or the spot designation on the viewing side, the requesting display unit 35 (monitor 4) is the same as the viewing side. You can share your screen.

In step S5, when mesh designation is selected, mesh processing is executed (step S6), and when spot designation is selected, spot processing is executed (step S7). Among them, one of the features of the present invention is that efficiency and high precision are achieved by mesh processing. The mesh processing will be described later using various embodiments.

An image capture request is made to the requesting personal computer 5 at a designated magnification for a position (area) designated based on the spot process or the mesh process (step S8). The specified positional information is recorded in a recording medium (hereinafter, referred to as a data storage device) such as a memory (not shown) inside the requesting personal computer 5 and the observation personal computer 1.

Subsequently, when the requesting personal computer 5 receives the image capture request from the personal computer 1 on the observation side, the requesting personal computer 5 performs the operation of the microscope 7 to observe the observation side. The image suitable for the request from the personal computer 1 is captured (step S9).

That is, in the microscope operation, the position designated by the mesh or the spot is converted into the stage coordinate position, the stage driving unit 23 (electric stage 9) is driven to move the specimen, and the magnification switching unit 22 according to the designated magnification. (Electric Revolver 8) is converted. Thereafter, the sample image information is captured by the imaging unit 21. Then, the captured image information is captured by a video capture board in the personal computer 5 on the requesting side, and the image is converted into a still image. The image converted into this still image may be recorded as it is in an external storage device 31 such as a hard disk or an MO. However, in the present invention, the image is compressed after being compressed in a JPEG format or the like for convenience of image information transfer. Doing. In this way, the image information recorded in the external storage device 31 is transferred from the personal computer 5 on the request side to the personal computer 1 on the observation side via a line connection device (not shown) and a public line 3. (Step S10).

When the observing personal computer 1 receives the image information transmitted from the requesting personal computer 5 (step S11), the observing personal computer 1 displays the received image information on the observing side display unit ( 35) is displayed on the screen of the monitor 2. In addition, as described above, when the image information is compressed by the JPEG method or the like, the display is accompanied by a predetermined stretching process.

In this way, when all the images have been transmitted, the observation side performs remote observation and makes a diagnosis (step S12). At this time, the holiday of the image, the mouse position information, and the like are transmitted from the personal computer 1 on the observation side with the operation right to the requesting personal computer 5 through the line connection device and the public line 3. The requesting personal computer 5 that has received the image consecutive information and the mouse position information displays an image on the requesting unit display unit 35 (monitor 4) based on these information and also displays information such as the mouse position. Holidays In addition, since the remote diagnosis is normally performed from the personal computer 1 on the observation side, the right to operate is on the observation side, but the personal computer 5 on the requesting side can arbitrarily switch this operation right.

As described above, after the remote observation is executed, it is judged whether or not to end (steps S14, S15). Here, if it is desired to further enlarge the objective lens to continue the diagnosis, the process returns to step S4 to repeat the series of processes, and if not desired, all processes are terminated. If there is no termination request from the observation side, the request returns to step S9 to be in a standby state.

In this invention, it aims at the improvement of the mesh process of step S6 of the said process. In other words, it is focused on easily and accurately realizing mesh division designation after data reception of a macro image. Hereinafter, the first to seventh embodiments made in view of these points will be described in detail.

First, with reference to FIG. 4, a series of processes related to image recognition of a sample image position by the microscope image transmission system according to the first embodiment will be described.

In addition, image recognition of the position of the sample image demonstrated below is a technique employ | adopted also in 2nd-6th Example mentioned later.

First, when a pathological specimen is imaged by microscope transmission illumination at the request side, the image shown in FIG. 4A is obtained. In this case, the background portion 51 is white, the sample image 52 is dyed, and has a color, and the noise portion 53 is usually close to black. The noise portion 53 is likely to occur when the image is captured by a macro image, for example, and occurs when the stage portion between the preparat and the cranmel fixing it is photographed. In addition, as other factors, garbage attached to the optical system is also recognized as the noise portion 53 of the still image. In this embodiment, image recognition of a sample image position is performed on the basis of the luminance information of the sample, and the luminance information of the extracted image is stored on a data storage device (not shown) inside the personal computer 5 or 1. do. On the data storage device, for example, when imaging an image of VGA (640 x 480), luminance information of an image having a two-dimensional array of (640, 480) is stored. The image information is divided into three pieces of information of R, G, and B, and stores pixel information as R (640, 480), G (640, 480), and B (640, 480). Among them, information of G (640, 480), which is relatively easy to recognize a shape, to be used as luminance information is preferable.

In the case where the sample image shown in FIG. 4A is arranged two-dimensionally as described above and stored in the data storage device, in the background portion 51, the pixel information of the R, G, and B becomes close to the FF 255, and the noise portion ( 53, R, G, and B all become below an arbitrary threshold value (close to 0), and the location where the sample image 52 exists becomes data converged within a predetermined range.

Here, in the sample image shown in FIG. 4A, the luminance information of arbitrary Y coordinate Yi is extracted, as shown in FIG. 4B. Based on this characteristic, the sample image upper limit luminance level 55 and the sample image lower limit luminance level 56 are set.

The upper limit of this luminance level can be set by recognizing the luminance of the background portion 51.

Specifically, the following method is followed. That is, the operator (observation side or request side) instructs by clicking on the background portion 51 with a mouse, and reads the luminance information of the image corresponding to the position of the mouse pointer from the data storage device, and this is the luminance level of the background. A value slightly lower than this value is set to the sample image upper limit luminance level 55. Alternatively, when storing the sample image on the data storage device, R, G, and B detect a block that continuously repeats a value close to FF 255, and automatically recognizes the block as a background, and pixel information of the block. It is a matter of course that the averaged data is taken as the luminance level of the background, and a value slightly lower than this value may be set as the sample image upper limit luminance level 55.

On the other hand, the sample image lower limit luminance level 56 is similar to the sample image upper limit luminance level 55, and an operator clicks and instructs the noise portion 53 with a mouse, and the luminance information of the image corresponding to the mouse position is recalled. It extracts from a data storage device, makes this a noise level, and sets the sample lower limit luminance level 56 to a value slightly higher than this value. In addition, it is possible for an operator to change both an upper limit and a lower limit arbitrarily.

After the above setting is completed, the luminance level is set to FF 255 in all cases not included in the sample image upper limit luminance level 55 and the sample image lower limit luminance level 56. The result of this data conversion is as shown in Fig. 4C, and only the portion where the sample image exists is at the luminance level zero. Subsequently, when the sample image is displayed again on the basis of the converted data, as shown in FIG. 4D, only the portion of the sample image is displayed in black. The converted luminance information is inputted back into the data storage device as two-dimensional array data, so that only pixels with a sample have a luminance level of O, otherwise the values of luminance level FF (255).

In the first embodiment of the present invention, while detecting the range of the sample image with the upper and lower limit luminance information of the sample image described above, the predetermined information is first referred to while referring to the luminance information stored in the data storage device as two-dimensional array data. Execute the mesh specification of.

Hereinafter, a process of creating mesh data according to the first embodiment will be described with reference to the flowchart of FIG. 5 and the display example of FIG. 6.

First, as shown in Fig. 6A, based on the luminance information arranged in two dimensions, the minimum coordinates (X min, Y min) and the maximum coordinates (X max, Y max) at which the sample image exists are automatically detected. (Step S21).

Subsequently, the width and height of the designated block are calculated from the designated magnification (step S22).

The designated magnification can be determined by mouse click of the button SW displayed based on the application software or assigned by assigning a magnification to a function key such as a keyboard of a personal computer. Regarding the calculation of the width and height of the designated block, the block size is determined here by the ratio of the magnification based on the magnification of the current objective lens. That is, when the magnification of the sample image of the current display is doubled and the next designated magnification is four times, both the width and height of the designated block are half the size (320 x 240) of the block displayed at 640 x 480.

When the size of the designation block is determined in this way, the initial position of the mesh designation is determined as shown in Fig. 6B. That is, the block 63 is created by setting the above-described minimum coordinate value to the upper left of the designated block (step S23). The position of this block 63 is stored in the data storage device as an initial position. The center position may be stored or the upper left coordinate may be stored.

When the initial position is determined, the next mesh position is detected as shown in Fig. 6C (step S24). That is, the X coordinate is fixed, the initial moving position of Y is set to the position in consideration of the height of the stored block, and the designated block is moved in the Y coordinate direction to detect a location where the sample image exists.

When at least one sample existence point is recognized on the X-line on the block, the movement in the Y direction is stopped, the position is made the next mesh position 64, and the coordinates are stored in the data storage device (step S25). ). Further, the block is moved in the Y direction to detect the next mesh position (step S26).

As shown in Fig. 6D, when the center coordinate of the block exceeds Y max (step S28), the X position is moved by the width length of the first block 63 (step S29), and it is the X start position. In the same manner as in step S24, the location (mesh positions 65 to 67) where the sample exists is detected.

The above operation is repeated, and as shown in FIG. 4E, when the center coordinates of the block exceed both (X max, Y max) (steps S28, S30), the mesh designation is finished.

The mesh designation result is a line overlay display of the block of the capture position designation together with the sample image on the monitor 4 on the requesting side and the monitor 2 on the viewing side. The mesh division position data on the monitors 2 and 4 are converted into coordinate data of the transmission stage 9, and the transmission stage 9 is moved to each mesh division designation position based on an image capture request from the observation side. Then, the sample image is captured by the imaging unit 21 (video camera 6) through the microscope 7.

The conversion of the coordinate system on the monitor to the coordinate system of the transmission stage 9 is executed based on the conversion table stored in advance in the data storage device in the personal computer 5. Then, the transmission stage 9 is moved based on the data transmitted from the personal computer 5 to the transmission stage 9. After the movement of the motorized stage 9, an image is captured, and the line connection device and the public line 3 which are not shown in the personal computer 1 on the observation side are captured from the personal computer 5 on the request side. Data transfer through.

As described above, according to the microscope image transmission system according to the first embodiment, it is possible to automatically perform highly accurate mesh segmentation only for sample images, excluding noise parts such as garbage, by simple operations.

Next, a second embodiment of the present invention will be described.

In the above-mentioned first embodiment, the sample image position detection is determined to determine the mesh division designation frame on the basis of the X coordinate. However, in some cases, it is preferable to perform the mesh division designation on the basis of the Y coordinate depending on the sample. have. With this point in mind, the second embodiment is characterized in that mesh division designation is executed on the basis of the Y coordinate. That is, as shown in Fig. 7, mesh division designation is executed in the order of the mesh positions 71 to 76.

Next, a third embodiment of the present invention will be described.

Hereinafter, this third embodiment will be described with reference to the flowcharts of FIGS. 9 and 10 and the display example of FIG. 8. In the third embodiment, a plurality of magnifications can be specified, which enables more efficient mesh division designation.

First, with reference to the flowchart of FIG. 9, the procedure of the expansion block designation process which concerns on 3rd Embodiment is demonstrated.

In this procedure, first, the minimum coordinates (X min, Y min) and the maximum coordinates (X max, Y max) of the position where the sample image exists are automatically detected (step S31).

Subsequently, the width and height of the designated block are calculated from the designated magnification (step S32). This designated magnification is established by clicking the button SW displayed based on the application software, or is assigned by assigning a magnification to a function key such as a keyboard of a personal computer. The calculation of the width and height of the designated block is performed at a ratio of magnification based on the current objective lens magnification.

After determining the size of the designated block, the initial position of the mesh designation is subsequently determined. In other words, the above-described minimum coordinate value is set at the upper left of the designation block (step S33). When this setting is completed, a determination is made as to whether or not a specimen exists at the position initially set to the upper left (step S34).

If it is determined in step S34 that a sample exists, an area ratio in which the block area and the sample image in the block exist is calculated (step S35). Then, it is judged whether or not the area in which the sample in the block area exists is 50% or less (step S36). If it is determined that this area ratio is 50% or less, further high magnification designation block processing (in-block mesh processing) is executed in this block (step S38). In addition, the detailed description regarding this intra-block mesh processing is demonstrated in the flowchart of FIG. 10 mentioned later.

The block position is stored as a temporary coordinate in a data storage device (not shown) inside the personal computer 5 or 1 at the current designated magnification before the in-block mesh processing.

After the high magnification designation block processing ends, the process returns to the designated magnification to detect the next mesh position. That is, the Y coordinate is fixed, and the initial X movement position is moved from the block position of the temporary coordinates described above to the position in consideration of the block width, and then moved in the X coordinate direction (step S39). Next, it is judged whether or not the center X coordinate of the block after the movement exceeds X max (step S41), and if not, the process returns to the determination of whether or not a sample exists in the block after the movement (step S34).

On the other hand, in step S36, when the block ratio and the area ratio in which the sample image exists in the block exceeds 50%, the enlargement designated block at the current designated magnification is displayed in lines, and the coordinates are stored in the data storage device (step S37). .

When the coordinate recording position is determined, the next mesh position is detected as shown in FIG. That is, the Y coordinate is fixed, and the initial position of X is moved to the X coordinate direction with the position taking into account the width of the stored block (step S39).

Next, it is judged whether or not the center X coordinate of the block after the movement exceeds the maximum coordinate X max (step S41). Here, if the maximum coordinate is not exceeded, the process returns to the determination of whether the block position and the sample exist after the movement (step S34).

In step S34, when it is determined that no sample exists at the block position after the movement, the Y coordinate is fixed and the block for one pixel is moved in the X direction (step S40). This movement process is the same even when the sample image does not exist in the initial position set to the upper left.

Next, it is determined whether or not the center X coordinate value of the block after the block movement for one pixel exceeds the maximum X coordinate (X max) (step S41). Here, if it does not exceed, determination of whether a sample exists again is performed (step S34). This continues until a sample image exists or until the block center X coordinate after block movement exceeds the maximum X coordinate (X max).

When the center X coordinate of the block after the movement exceeds the maximum X coordinate (X max) (step S41), the Y position is moved by the length of the block height, and the X position is X min (step S42). . The coordinates specified here are the coordinates of the upper left corner of the block.

If the center Y coordinate of the block after the movement exceeds the maximum Y coordinate Y max (step S43), the process ends. If the maximum Y coordinate is not exceeded, the process returns to the determination process (step S34) of whether a sample exists at the block position after the movement to determine the next mesh position.

Next, referring to the flowchart of FIG. 10, a higher magnification designation block designation process (hereinafter referred to as an in-block mesh designation) in the block when the area ratio in which the sample image exists in the block used in FIG. 9 is 50% or less. Explain.

First, it is judged whether or not an object that can be designated at higher magnification exists in the block (step S51). If the current block designation magnification is not the maximum magnification, the process is executed. Otherwise, the process ends.

In step S51, when it is determined that there is a magnification higher than the current block designation magnification, the minimum maximum coordinates (X min, Y min) and (X max, Y max) of the sample image in the block are detected (step S52).

Next, the width / height of the enlargement / designation block having a higher magnification than the current block designation magnification is calculated (step S53). For example, when a microscope is equipped with objective lenses of 1.25x, 2x, 4x, 10x, 20x, and 40x, and the current block designation magnification is 2x, the higher magnification objective lens is 4x, and the higher magnification in the block is obtained. Calculate the expansion block width and height.

Then, the initial position of the higher magnification designation block in the block is set at the upper left in the block (step S54). Then, it is judged whether or not a specimen exists at this initial position (step S55).

Here, if the sample exists, the ratio of the initial block area and the area in which the sample image in the block exists is calculated (step S56).

Next, it is determined whether the area ratio is 50% or less (step S57). Here, if it is 50% or less, further higher magnification designation block processing (in-block mesh processing) is executed (step S59). In-block meshing is performed until the specified object exceeds the maximum magnification. In this example, the intra-block mesh is continued up to the maximum magnification, but the intra-block mesh processing may be performed at any magnification as an upper limit. .

Also, the block position is stored as a temporary coordinate at the current designated magnification before in-block mesh processing, and stored in the data storage device. After the intra-block mesh processing is completed, the designated magnification is returned to detect the next mesh position. In other words, the X coordinate is fixed, and the Y movement initial position is moved to the Y coordinate direction from the above-described block position of the temporary coordinate in consideration of the block height (step S60). Then, it is judged whether or not the center Y coordinate of the block after the movement exceeds Y max (step S62). If it is not exceeded, the process returns to the determination of whether a sample of the block after the movement exists (step S55).

On the other hand, when the block ratio and the area ratio in which the sample image exists in the block exceed 50%, a higher magnification line is displayed in the block, and the coordinates and the designated objective magnification at this time are stored (step S58). In order to detect the mesh position in the next block, X is fixed and moved in the Y coordinate direction with the position considering the height of the block after Y movement (step S60). If the block center Y coordinate after this movement does not exceed Y max (step S62), it returns to the determination of whether a sample exists in the block after the movement (step S 55).

As a result of judging whether or not a sample exists in step S55, if no sample exists, X is fixed and a block within one pixel block is moved in the Y direction (step S61).

When the center coordinate Y coordinate of the moving block in the block exceeds Y max (step S62), the X position is moved by the length of the width of the moving block in the block, and the Y position is set to Y min (step S63). ).

Further, it is determined whether the center X coordinate of the moving block in the block exceeds the maximum X max (step S64). Here, if the center X coordinate exceeds the maximum X max, the intra-block mesh processing is terminated. If the center X coordinate of the moving block in the block does not exceed the maximum X max, the process returns to the determination process of whether or not a sample exists (step S55), and the above-described process is performed, and the center coordinate of the moving block is X max. Continue until Y max is exceeded.

As described above, according to the microscope image transmission system according to the third embodiment, it is possible to automatically perform further mesh division appropriately based on the occupancy rate of the sample images in the block.

Next, a fourth embodiment of the present invention will be described.

This fourth embodiment is characterized by using color information as a method of recognizing a sample image position. That is, in general, a pathological specimen becomes a red color or a blue color by a dyeing method. In the fourth embodiment, the sample image position is recognized by detecting the dyeing information of the sample image.

In the configuration, the microscope 7 includes a first sensor that extracts only a red spatial frequency region, a second sensor that extracts only a blue spatial frequency region, and a third sensor that extracts only a green spatial frequency region. The point differs from the first embodiment. In each sensor, since the sample image is divided and guided by the beam splitter in the video camera 6, the position of each sensor is arranged at the same optical position as the video camera 6.

The outputs of the first to third sensors are extracted in units of pixels, and these are recorded in the data storage device in the personal computer. The position where the output of the first to third sensors approaches the maximum value is recognized as the background position, and the position where the output of the first to third sensors approaches the minimum value is recognized as an unnecessary data position, and the red spatial frequency Only the input from the first sensor which extracts only the region recognizes the position which becomes more than an arbitrary threshold value as a sample image position. In addition, depending on the dyeing method, the sample image position may be a position where only the input from the second sensor that extracts only the blue spatial frequency region becomes a certain threshold value or more.

In addition, as the sample image position recognition based on the color information, in addition to using the first to third sensors, the optical filter which passes only spatial frequencies corresponding to each of red, green, and blue in the microscope 7 is detected. Is possible. That is, the optical filter of the microscope 7 is switched with respect to one sensor, the image of only the desired spatial frequency region is guided to the video camera 6, and the position where the sample image exists is obtained from the obtained image information. It can also be recognized and recorded in the data storage device.

Moreover, although it is preferable that it is a two-dimensional area sensor as a sensor used by the Example mentioned above, it is not specifically limited. In addition, in the above-mentioned embodiment, although the interval frequencies of red, green, and blue were all seen using the first to third sensors, the present invention is not limited thereto, and the background may be obtained even by a sensor corresponding to a green spatial frequency that is relatively easy to recognize, for example. It is possible to recognize the position, unnecessary data position and sample image position.

As described above, according to the microscope image transmission system according to the fourth embodiment, highly accurate mesh division can be performed even based on the color information of the sample image.

Next, a fifth embodiment of the present invention will be described.

In the above-described first to third embodiments, examples of automatically dividing meshes are shown. On the other hand, the fourth embodiment is characterized in that the mesh division displayed on the monitor of the personal computer can be arbitrarily moved, deleted and added by manipulating the mesh division position once automatically determined by the operator.

Hereinafter, the movement operation of the mesh block according to the fifth embodiment will be described with reference to the flowchart of FIG. 11 and the display example of FIG. 12.

First, any block to be moved among the mesh partitioned blocks is clicked on the screen of the monitor 2 by operating a mouse or the like (step S101).

The personal computer 1 recognizes this mouse position coordinate and stores the coordinate in a data storage device (not shown) inside the personal computer 5 or 1 (step S102). Then, the already recorded mesh division coordinate information is taken out from the data storage device (step S103).

The mouse position and the mesh coordinate position data determine whether the mesh coordinate position exists at the mouse click position (step S104). If the click position of the mouse is within the designated block, it has a judgment function that recognizes even if the area in the block is pressed.

If a mesh block already exists at the current mouse click position (step S105), it is recognized as a mesh to be moved. By moving the mouse while clicking the mouse (step S106), the mesh display position is changed (step S107). The change of the mesh position is repeated by erasing and creating mesh lines.

The deletion of any part of the mesh division position is determined by the deletion means by determining whether the mesh division coordinate information exists at the mouse click position. The deletion means may be any button on the keyboard, for example, the "Delete" key, or may use the delete button displayed by the application software.

Adding the mesh splitting position determines whether mesh splitting coordinate information exists at the mouse click position. If it does not exist, it is possible to specify a new mesh block, and when the mouse is clicked, the new mesh dividing position coordinate is recorded in the data storage device. As described above, after automatically performing mesh division specification once, the operator can arbitrarily move, add, and delete the mesh division specification.

As described above, according to the microscope image transmission system according to the fifth embodiment, after automatically performing mesh division, the block can be appropriately moved to a desired position. That is, the degree of freedom of observation is improved.

Next, a sixth embodiment of the present invention will be described.

In the sixth embodiment, the mesh auto focus position is automatically recognized. When an image is input by mesh division designation, the focus position is fixed at one place and the images of the plurality of mesh division positions are input. I do it.

Hereinafter, the autofocus position recognition operation according to the sixth embodiment will be described with reference to the flowchart in FIG. 13 and the display example in FIG. 14.

When the focus position is determined based on the tone information of the image, the larger the image, the more accurately the auto focus can be achieved. When the mesh dividing position is automatically detected, first, the mesh dividing position information having a sample image near the center coordinates and the maximum luminance level difference in each block are stored in a data storage device (not shown) inside the personal computer 5 or 1. . Whether or not a sample image exists near the center coordinates can be determined by reading the sample image recognition data from the data storage device and using the sample image recognition data.

That is, in the case of VGA display, it can be determined whether or not the luminance information of any pixel block (for example, 50 pixels x 50 pixels) is 0 (sample exists) centering on G (320, 240) (step S51). ). This block may be averaged luminance information. The reason for selecting a block with a sample image near the center of the image is that when the autofocus is performed, the line sensor for autofocus of the microscope takes data near the center.

The mesh block in which the sample image exists near the center of the image is extracted, and the block position is stored on the data storage device.

Next, a block having the maximum luminance difference among all the mesh blocks in which the sample image exists near the center of the image is detected (step S52). Then, the mesh dividing position having the maximum luminance difference is detected, and stored in the data storage device as a location for autofocusing the position (step S53). In this embodiment, the mesh decomposition position 101 is selected as the location to be autofocused.

When actually inputting the mesh segmented image, first, the motorized stage 9 is moved to the place where the autofocus is performed, and the control is performed from the personal computer 5 side so as to execute the autofocus control of the microscope 7 at this position. .

As described above, according to the microscope image transmission system according to the sixth embodiment, the focus position can be automatically recognized with high precision by simple operation based on the result of the mesh division.

Next, a seventh embodiment of the present invention will be described.

In general, a macro image may cause a lack of ambient light (see Fig. 15), but this lack of ambient light lowers the luminance level. Therefore, in the conventional automatic mesh segmentation position recognition algorithm, processing is performed based on the luminance information, and therefore, there is a possibility that accurate mesh segmentation specification cannot be made due to the effect of insufficient amount of ambient light.

That is, when a lack of ambient light occurs, there is a possibility that the information originally to be recognized as the background may be mistakenly recognized as the luminance level of the sample image, and the mesh division position is also overlaid on the background portion. In the seventh embodiment, an appropriate mesh division designation is performed for a sample image that is causing this lack of ambient light.

Hereinafter, with reference to the flowchart of FIG. 16, the operation | movement of mesh division designation with respect to the sample in which the ambient light quantity shortage is occurring is demonstrated in detail.

First, before actually recognizing the sample image as data, the image is captured in the absence of the sample, and the image information is recorded in a recording device such as a data storage device or an MO. In the present embodiment, two-dimensional array data of R0 (640, 480), G0 (640, 480), and B0 (640, 480) is stored in the data storage device (step S61). In this case, since the background information is captured, the image becomes white as an image, and the luminance information of the pixel is usually FF (255), but not at the coordinate position of the peripheral portion.

Therefore, when storing in a two-dimensional array, the following calculation is performed on all the pixel information.

R0 (x, y) = FF (255)-R0 (x, y)

(x = 1 to 640, y = 1 to 480)

G0 (x, y) = FF (255)-G0 (x, y)

(x = 1 to 640, y = 1 to 480)

B0 (x, y) = FF (255)-B0 (x, y)

(x = 1 to 640, y = 1 to 480)

That is, the luminance information to be added to the pixel portion lacking the ambient light amount is stored as R0 (640, 480), G (640, 480), and B (640, 480).

Subsequently, the original image of the macro image is input, and the image information is stored in a two-dimensional array memory of R1 (640, 480), G1 (640, 480), and B1 (640, 480) (step S62). . Image information without a sample image is added from the image information of the original image, and the data is made into R (640, 480), G (640, 480), and B (640, 480). Here, one pixel is added one by one (step S63).

Specifically,

R (x, y) = R1 (x, y) + R0 (x, y)

(x = 1 to 640, y = 1 to 480)

G (x, y) = G1 (x, y) + G0 (x, y)

(x = 1 to 640, y = 1 to 480)

B (x, y) = B1 (x, y) + B0 (x, y)

(x = 1 to 640, y = 1 to 480)

Becomes

As described above, according to the microscope image transmission system according to the seventh embodiment, data can be supplemented for only the portion of the lack of ambient light amount.

That is, in the seventh embodiment, more accurate mesh segmentation specification is automatically enabled by performing the mesh segmentation specification based on the data obtained by supplementing the data of the lack of ambient light amount.

Next, an eighth embodiment of the present invention will be described.

This eighth embodiment relates to a microscope image transmission system characterized by extracting color information and detecting a sample image position using the information.

17 is a schematic diagram showing a process of detecting a sample image position by color information extraction in this embodiment, FIG. 18 is a functional block diagram of a microscope image transmission system according to this embodiment, and FIGS. 19 and 20 are It is a flowchart for explaining the operation of the microscope image transmission system according to the present embodiment.

As shown in FIG. 18, this microscope image transmission system is largely divided into an image input unit 500, an internal processing unit 501, a user operation unit 502, and a microscope 503.

In the image input unit 500, a video camera unit 201 and a video capture unit 202 are disposed. The microscope 503 is provided with an auto focus unit 203, a magnification switching unit 204, and a stage moving unit 205.

The internal processing unit 501 includes an image display memory 206 for storing display images, a video capture control unit 207 for controlling the video capture unit 202, a sample image position recognition processing unit 208, a memory 300, and an external device. External memory memory control unit 214 for controlling the memory memory 218, focus control unit 215 for controlling the auto focus unit 203, magnification control unit 216 for controlling the magnification switching unit 204, and stage moving unit ( The stage control part 217 which controls 205 is arrange | positioned. In the memory 300, each of the luminance memories 209 to 211, the RG luminance difference memory 212, the BG luminance difference memory 213, the RG luminance difference maximum value memory 225, and the BG luminance difference maximum value memory ( 226, RG threshold memory 227, and BG threshold memory 228.

The user operation unit 502 includes an image display unit 219 for displaying an image of the image display memory 206, a sample image position recognition designator 220, an image save / call instruction unit 221, and the focus control unit 215. A focus designation unit 222 for designating focus control by the control panel), a magnification designation unit 223 for designating magnification control by the magnification control unit 216, and a stage position for designating stage control by the stage control unit 217 Designator 224 is disposed, respectively.

Hereinafter, the sample image position detection process realized by such a configuration will be described.

In this embodiment, the sample image under the microscope is obtained through the video camera unit 201, the video capture unit 202 is converted into digital data, and this digital data is output to the image display memory 206. It displays on the image display part 219 as still image information. The shape of this sample image is shown in Fig. 17A.

In addition, in the display example shown in this FIG. 17A, in addition to the area | region of a sample image, the area | regions, such as the shadow of a cover glass on slide glass, garbage, etc. are mixed.

17B shows the color information displayed by separating the luminance information at any Y coordinate position of the sample image shown in FIG. 17A. The background part and the noise part are changed as if the brightness information of red and green changes as the same, but there is a difference in the brightness information of red and green at the position where the sample exists. In general, since the pathological sample image is stained in various ways for observation, the luminance difference can be used to calculate the sample image position. In the example of FIG. 17, pathological specimens that are entirely dyed with a red subject are used. However, since color information characterized by dyeing is different, three pieces of information, red, green, and blue, are obtained as the luminance information.

The difference between the red and green luminance information is taken and displayed as shown in Fig. 17C. Based on the characteristic of FIG. 17, the maximum value RGref of the luminance difference of red and green is computed, and a threshold value is set based on this.

The shape of the data which made FFh the data of the X coordinate value below this threshold value and 0 above the threshold value is shown in FIG. 17D. 17E shows the result of performing such an operation in all the Y coordinates.

In the above example, an example of detecting a sample image position using red and green luminance difference information is shown. However, depending on the sample, blue and red luminance difference information, or red and green luminance difference and blue and green luminance, depending on the sample. It goes without saying that the sample image position may be detected by a combination of differences.

Hereinafter, an operation of recognizing a sample image position using the luminance difference information will be described in detail with reference to the flowcharts of FIGS. 19 and 20.

First, luminance information of red, green, and blue (hereinafter referred to as RGB) of the entire region of the entire sample image is stored in each of the luminance memories 209, 210, and 211 (step S301). In this example, the luminance information of red (R) is represented by Rdata (X, Y), the green (G) luminance information is represented by Gdata (X, Y), and the blue (B) luminance information is represented by Bdata (X, Y). Each luminance memory 209 to 211 is stored in an array of dimensions. X is in the range of 0 to (X number of pixels-1), and Y is in the range of 0 to (Y number of pixels-1).

Subsequently, the luminance difference information in the same pixel is stored in the luminance difference memories 212 and 213 (step S302). That is, the luminance information difference (RGref (X, Y) of R and G is taken, and stored in the RG luminance difference memory 212, and the luminance information difference (BGref (X, Y)) of B and G is taken, and the BG luminance is taken. The difference memory 213 is stored.

RGref (X, Y) = Rdata (X, Y)-Gdata (X, Y)

BGref (X, Y) = Bdata (X, Y)-Gdata (X, Y)

Subsequently, the maximum values RGref max and BGref max of each luminance difference are detected and stored in the RG luminance difference maximum value memory 225 and the BG luminance difference maximum value memory 226 (step S303). From the maximum value of the luminance difference, the threshold for sample image recognition is stored in the RG threshold memory 227 and the BG threshold memory 228 (step S304).

Here, with reference to the flowchart of FIG. 20, the procedure of setting the said threshold value is explained in full detail. To set the threshold value, the maximum value (RGref max, BGref max) of each luminance difference is called from the RG luminance difference maximum value memory 225 and the BG luminance difference maximum value memory 226 (step S401), and the sample dyeing classification is performed. Is selected (step S402). The threshold value setting (RGrefth, BGrefth) is performed according to this sample staining type (step S403).

RGrefth = RGrefmax / n

BGrefth = BGrefmax / m

Here, the values of n and m change according to the sample staining type. In addition, of course, you may change arbitrarily.

For example, n = m = 2, and half of the maximum value of the luminance difference between R and G (RGrefmax) and half of the maximum value of the luminance difference between B and G (BGrefmax) are set to the threshold values RGrefth and BGrefth. You may also

Also, depending on the staining method, there are samples of blue subjects. In this case, the values of n and m are n = RGrefmax and m = 2. When n = 1, RGrefth = RGrefmax, and the threshold value setting based on the information on the luminance difference between R and G is raised to the maximum level. By doing this, it becomes equivalent to detecting the sample image position only by setting the threshold value by the luminance difference between B and G.

Then, the flow returns to the description of FIG. 19 again.

After determining the threshold values RGrefth and BGrefth as described above, an area for performing a sample image recognition check is set (step S305). When the position of the entire sample image is to be recognized, X min = 0, X max = (X number of pixels-1), Y min = 0, and Y max = (Y number of pixels-1).

It is also possible to check a partial region from the whole image by determining the region. When the area to be checked is determined, it is judged whether or not a sample image exists in the area.

First, initial values (X = X min, Y = Y min) for executing a sample image recognition check are set (step S306).

After setting the initial value in this way, the comparison between the luminance difference data RGref and BGref and the sample image recognition thresholds RGrefth and BGrefth is performed to check whether or not the sample image exists. First, a comparison between the red green luminance difference data RGref (X, Y) and the red green luminance difference threshold value RGrefth in arbitrary X and Y coordinates is performed (step S 307).

Here, if RGref (X, Y)> RGrefth, a sample image exists in (X, Y) coordinates (step S310).

Also, RGref (X, Y) If it is RGrefth, a sample image recognition check based on the blue green luminance difference is performed (step S308). Here, if BGref (X, Y)> BGrefth, a sample image exists in (X, Y) coordinates (step S310). In contrast, BGref (X, Y) If it is BGrefth, the sample image does not exist in the (X, Y) coordinates (step S309). In this way, since the sample image presence check in arbitrary (X, Y) coordinates was complete | finished, the sample image check of the next coordinate is performed.

That is, the X coordinate is incremented (X = X + 1) (step S311), and it is checked whether the new X coordinate value does not exceed the maximum value Xmax (step S312). If the X coordinate does not exceed the maximum value X max, the process returns to the process of checking the existence of the sample image in this coordinate (step S307).

If the X coordinate does not exceed the maximum value X max, the X coordinate is made X min and the Y coordinate is incremented (step S313). As a result of incrementing the Y coordinate, if the Y coordinate does not exceed the maximum value (Y max), the process returns to the process of performing a sample image presence check in this coordinate (step S307). If the Y coordinate exceeds the maximum value (Y max), all of the sample image recognition check areas are confirmed, and the process ends.

Next, a ninth embodiment of the present invention will be described.

In the ninth embodiment, after automatically designating the mesh enlargement frame designation, the mesh enlargement frame position to be automatically specified for the first time in the mesh enlargement designation frame is automatically detected.

First, FIG. 21 is a diagram showing a state after automatically performing mesh magnification frame designation on the entire sample image. In FIG. 21, the width of the individual mesh enlargement specifying frame is set to Frame Width and the height is set to Frame Height. In addition, let the center coordinates of an individual mesh enlargement specification frame be (X0c, Y0c)-(X3c, Y3c). The vertex of the coordinate is the upper left of FIG. 21.

Hereinafter, with reference to the flowchart of FIG. 22, the procedure to automatically set the place which performs autofocus for the first time after automatically executing a mesh enlargement specification frame to a sample whole image is demonstrated in detail.

First, the number (Mesh Num) of the mesh enlargement specifying frame is checked (step S601). Subsequently, in order to have the sample image recognition pixel number data of each mesh enlargement specification frame, the counter for sample image presence check is provided as array data (Chk Counter (0) to Chk Counter (Mesh Num-1)) ( Step S602). Further, the sample image existence check of each mesh enlargement specifying frame is checked, and the pixel number data recognized as the sample image is inserted into the Chk Counter (0) to Chk Counter (Mesh Num-1) (step S603).

Here, with reference to the flowchart of FIG. 23, the detailed procedure of the pixel number data check in which a sample image exists is demonstrated.

First, luminance difference thresholds RGrefth and BGrefth for recognition as sample images are set and stored in a memory not shown in the requesting terminal personal computer (step S701). The threshold setting is as described above.

Subsequently, an initial value of coordinates for judging whether or not a sample image exists is set (step S702). Here, Xic and Yic correspond to (X0c, Y0c)-(X3c, Y3c) of FIG. Frame Width and Frame Height are the width and height of the mesh zoom specification frame.

Subsequently, when the initial value of the coordinate is determined, the check counter for determining whether or not the sample image exists is cleared (step S703).

Next, the luminance difference information RGref (X, Y) and BGref (X, Y) in the individual pixels is obtained while changing the XY coordinates, and the luminance difference information and the luminance difference threshold value determined in the above step S701 ( RGrefth and BGrefth) are compared (step S704). Here, if greater than the luminance difference threshold values RGrefth and BGrefth, the check counter is incremented (step S705). Then, the X coordinate is incremented to prepare the next coordinate (step S706). On the other hand, when the X coordinate enters the area within the mesh enlargement specifying range (step S707), the process returns to the step S704. In contrast, if the X coordinate does not fall within the area within the mesh magnification specifying frame range (step S707), the Y coordinate is incremented and the X coordinate is returned to the initial value (step S708). Then, it is checked whether the Y coordinate falls outside the area within the mesh magnification specifying frame range (step S709). If it is in the area, the process returns to the above step S704.

In this way, the number of pixels which recognized that a sample image exists in an individual mesh enlargement specification frame is counted. If the Y coordinate is outside the area within the area range of the mesh enlargement specifying frame (step S709), the number of times to recognize as a sample image in all the X and Y coordinates in the mesh enlargement specifying frame is acquired. The number of pixels recognized as a sample image enters as data in the Chk Counter (step S710).

Returning to the description of FIG. 22 again.

The number of pixels recognized as a sample image is stored in the Chk Counter (I). Here, I represents an arbitrary mesh enlargement specification frame number (corresponds to (0) to (3) in FIG. 21). When the sample image existence check of all the mesh magnification designating frames is finished, the values of the sample image existence check counters are listed in order from ascending order of values. This listed order is stored as a separate variable (step S604).

Next, it is judged whether or not the specimen exists near the center in order from the value of the large number of the sample image presence check counters (step S605). If it is determined that there is no sample image near the center (step S606), the process returns to step S605 again, and a mesh magnification specifying frame having a large number of sample image check counter values is taken out, and the sample image is located near the center. Determine if it exists.

Here, if a sample image exists near the center, it is stored as a position for performing AF first (step S607).

In the manner as described above, it is possible to automatically recognize the position where the sample image has the number of pixels that can be recognized as the most sample image among the plurality of mesh enlargement designation frames and where the sample image exists near the center for the first time.

As mentioned above, although the Example of this invention was described, this invention is not limited to this, Of course, various improvement and change are possible in the range which does not deviate from the main point. For example, in the above embodiment, although the start point of the mesh division is set to the upper left of the still image and the end point is set to the lower right, it is obvious that arbitrary positions in the still picture may be used as the start point and the end point.

As described above, according to the present invention, in the remote observation of a microscope image, a microscope image transmission system which can efficiently and accurately designate the mesh division of the observation region in the initial observation image by simple operation, The recording medium used for a microscope image processing method and a microscope image processing can be provided.

Claims (21)

In the microscope image transmission system which designates the area | region desired to observe with high magnification among a sample image by a predetermined block unit, Imaging means (6, 21, 201) for imaging the specimen to obtain the specimen image; Recording means (1, 5, 26, 31, 202, 206) for recording a still image including a sample image picked up by the imaging means; Display means (2, 4, 35, 219) for displaying a still image recorded in the recording means; Extracting the position of the sample image, and dividing only the region in which the sample image exists based on the predetermined block unit, and superimposing the index indicating the block on the still image to display on the display means; One control means (1, 5, 13,501) Microscope image transmission system comprising a. The method of claim 1, In order to extract the position of the sample image, the control means (1, 5, 13, 501) extracts the luminance information of the still image and sets the maximum luminance level based on the luminance information of the background position of the sample image. A function of setting a minimum luminance level based on luminance information on an unnecessary data position of the sample image, a conversion function of converting the luminance information based on the maximum luminance level and the minimum luminance level, and a converted function. And a function of determining a background position, an unnecessary data position, and a position of the sample image of the sample image based on the luminance information. The method of claim 1, In order to extract the position of the sample image, the control means (1, 5, 13, 501) extracts color information of the sample image and sets the position where the color information is close to the maximum value as the background position of the sample image. And a microscope image including a function of recognizing a position where the color information approaches a minimum value as an unnecessary data position of a sample image, and recognizing a position where the color information becomes above a predetermined threshold value as a sample image position. Transmission system. The method of claim 1, The control means (1, 5, 13, 501) sets an initial block position in the still image, sequentially sets blocks in a first direction with the initial block position as the starting point, and the block position is set to the maximum coordinate. If exceeded, the microscope image transfer further includes a function of shifting by the width of the block in the second direction, sequentially setting additional blocks in the first direction, and causing the display means to display an index relating to the block. system. The method of claim 1, In order to extract the position of the sample image, the control means (1, 5, 13, 501) extracts the color information of the sample image and calculates the difference of the color information at the same position of the sample image; A function of obtaining a position at which the difference of the color information is maximized, a function of setting a predetermined threshold value based on a value at which the difference of the color information is maximized, and distinguishing the difference of the color information into the threshold value, And a function of recognizing a position that is above or below a threshold as the position of the sample image. The method of claim 5, The function of setting the predetermined threshold includes a function of detecting a maximum value of the luminance difference between red and green, a function of detecting a maximum value of the luminance difference between blue and green, and a maximum value of the luminance difference. A microscope image transmission system further comprising the function of dividing by and taking the quotient and setting the value as a threshold. The method of claim 1, Further comprising operation instruction means 37 for instructing addition, deletion, movement of the block set by the control means, And the positional information about blocks added, deleted or moved in accordance with the instruction of the operation instruction means is recorded in the recording means from time to time. The method of claim 1, Detecting the maximum luminance level difference in each block, comparing the maximum luminance level difference for each block, and detecting the position coordinates of the block having the maximum luminance level difference among each block, the focus adjustment reference Microscope image transfer system further comprising focusing means (29, 215) for performing focusing as a position. In the microscope image transmission system which designates the area | region desired to observe with high magnification among a sample image by a predetermined block unit, Imaging means (6, 21, 201) for imaging the specimen to obtain the specimen image; Recording means (15, 26, 31, 202, 206) for recording a still image including a sample image picked up by the imaging means; Display means (2, 4, 35, 219) for displaying at least a still image recorded in the recording means; Detection means for inputting a still image in the absence of the specimen, inputting a still image in the presence of the specimen, and detecting a light quantity shortage portion based on a difference between the luminance information of both; After excluding the light quantity lacking portion detected by the detecting means, the position of the sample image is detected from the still image, and only an area in which the sample image exists is divided into predetermined block units, and the index of the block is displayed. Control means (1, 5, 13, 501) for controlling the display to superimpose on the still image Microscope image transmission system comprising a. In a microscope image transmission system in which at least one of the observation side and the requesting side having control means are communicably connected to each other, and dividing a region of the sample image desired to be observed at a high magnification by a predetermined block unit, The request side, Imaging means 6 for imaging a sample to obtain the sample image; First display means (4) for displaying a still image including a sample image picked up by said imaging means; First transmission / reception means (3, 5) capable of transmitting and receiving information with the observation side, The observation side, Second display means (2) for displaying a still image transmitted from the requesting side; Second transmitting and receiving means (1, 3) capable of transmitting and receiving information with the requesting party, The control means, The position of the sample image is extracted from the still image displayed on the display means on the side using the control means, and based only on that, only an area in which the sample image exists is divided into predetermined block units, and an index indicating the block is obtained. A function of superimposing and displaying on a still image, The still image of the display means on the side not using the control means is also not used from the side using the control means such that the still image is the same image as the still image displayed on the display means on the side using the control means. A function of transmitting at least the positional information of the indicator representing the block, And the same information is shared between the requesting side and the observing side. The method according to any one of claims 1, 9 or 10, The size of said predetermined block unit is a microscope image transmission system determined by the ratio of the visual field size observed with a standard magnification, and the visual field size magnified and observed with high magnification. In the microscope image processing method, Extracting luminance information of a still image including a sample image; Setting a maximum luminance level based on luminance information relating to a background position of the sample image; Setting a minimum luminance level based on luminance information relating to an unnecessary data position of the sample image; Converting the brightness information based on the maximum brightness level and the minimum brightness level; Determining a background position, an unnecessary data position, and a position of the sample image of the sample image based on the converted luminance information. And extracting the position of the sample image. In the microscope image processing method, Extracting color information of the sample image; Recognizing a position at which the color information approaches a maximum value as a background position of a sample image; Recognizing a position at which the color information approaches a minimum value as an unnecessary data position of a sample image; Recognizing a position where the color information becomes equal to or greater than a predetermined threshold value as a position of a sample image; And extracting the position of the sample image. In the microscope image processing method, Extracting different color information at the same position of the sample image, Obtaining a position at which the difference of the color information at the same position of the sample image is maximized; Setting a predetermined threshold value based on a value at which the difference between the color information is maximized; Distinguishing the difference of the color information by the threshold value, and recognizing a position that is above or below a threshold as the position of the sample image. And extracting the position of the sample image. The method of claim 14, The setting of the predetermined threshold may include detecting a maximum value of the luminance difference between red and green, detecting a maximum value of the luminance difference between blue and green, and selecting a maximum value of the luminance difference. And dividing the value by the numerical value and making the value a threshold. The method according to any one of claims 12, 13 or 14, After extracting the position of the sample image, dividing only the region in which the sample image exists by a predetermined block unit to display an index representing the block superimposed on a still image having the sample image; Treatment method. In the recording medium 31 used for microscope image processing, After extracting the luminance information of the still image including the sample image, the maximum luminance level is set based on the luminance information on the background position of the sample image, and based on the luminance information on the unnecessary data position of the sample image. Set a minimum luminance level, convert the luminance information based on the maximum luminance level and the minimum luminance level, and determine the background position, unnecessary data position, and position of the sample image based on the converted luminance information. Thereby recording a program for extracting the position of the sample image. In the recording medium 31 used for microscope image processing, After extracting the color information of the sample image, recognize the position as the background position of the sample image so that the color information is close to the maximum value, and recognize the position where the color information is close to the minimum value as an unnecessary data position of the sample image, A recording medium for recording a program for extracting a position of a sample image by recognizing the position where the color information becomes equal to or larger than a predetermined threshold value as the position of the sample image. In the recording medium 31 used for microscope image processing, After extracting different color information at the same position of the sample image, the position where the difference of the color information at the same position of the sample image is maximized is set, and a predetermined threshold value is set based on the value at which the difference of the color information is maximized. And extracting the position of the sample image by recognizing the difference between the color information as the threshold value and recognizing the position of the sample image as the position of the sample value above or below the threshold value as the position of the sample image. The method of claim 19, The predetermined threshold setting detects the maximum value of red and green, detects the maximum value of the luminance difference between blue and green, divides the maximum value of the luminance difference by an arbitrary value, and takes its share. Recording medium whose value is a threshold. The method according to any one of claims 17, 18 or 19, And after extracting the position of the sample image, only an area in which the sample image exists is divided into predetermined block units, and an index indicating the block is displayed superimposed on a still image including the sample image.