JP2014003990A - Endoscope apparatus and endoscopic observation system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an endoscope apparatus and an endoscopic observation system which can provide a picture quality and operability in accordance with a situation.SOLUTION: By the scope of an endoscope 100, an observation target is imaged, with image data obtained. A network controller 102 transmits the image data picked up by the scope 100 of the endoscope to an image processing server that applies a prescribed image processing to the image data, and controls to receive the image data processed by the image processing server. A simple image processor 103 performs prescribed image processing on the image data picked up by the scope 100 of the endoscope. A display image selection part 104, following the control by a display image switching controller 105, selects either the image data received from the image processing server 106 or the image data processed by the simple image processor and outputs to a monitor 110. The monitor 110 displays an image on the basis of the mage data outputted from an endoscopic image processor 101.

Description

  The present invention relates to an endoscope apparatus and an endoscope observation system.

  In general, an electronic endoscope is inserted into the human body to obtain image information, and digital image data obtained from the endoscope scope is subjected to various image processing to convert it into an image suitable for observation. And an endoscopic image processor.

  On the other hand, the performance improvement of network infrastructure and computer equipment is remarkable, and it has become realistic to realize the functions of the above-described endoscopic image processor by so-called cloud computing. However, since there are a large number of unspecified users in the network, the arrival of data varies depending on the congestion of the line. In the transmission of moving images, variations in the arrival of data lead to a problem of “frame dropping” in which there is no image to be displayed.

  As a countermeasure against this problem, a method of storing an image of a certain amount of frames in a buffer is generally taken, but in this case, the time from image acquisition to display becomes longer. Since this display delay may adversely affect the operability of the endoscope, such a countermeasure is not suitable for the endoscope system.

  Note that there is an image transmission system described in Patent Document 1 as a method for dealing with such fluctuations in delay time. Patent Document 1 describes an invention in which the time required for data transmission is measured and the amount of data transmission is controlled based on that time. The data transmission amount is controlled by a method such as thinning out an image from the image sensor and outputting it.

JP 2008-219479 A

  In the prior art, the emphasis is placed on displaying real-time images rather than image quality. However, an endoscope is a medical device and image quality cannot be sacrificed.

  The present invention has been made in view of the above-described problems, and an object thereof is to provide an endoscope apparatus and an endoscope observation system that can provide image quality and operability according to the situation.

  The present invention has been made to solve the above-described problem, and is an endoscope scope that captures an image of an observation object and obtains image data, and is connected to the endoscope scope and the endoscope scope. An endoscopic image processor connected to an image processing server that performs predetermined image processing on image data captured by the image processing apparatus, and a display unit that displays an image based on the image data output from the endoscopic image processor. The endoscopic image processor transmits image data captured by the endoscope scope to the image processing server, and performs control for receiving image data processed by the image processing server; and An image processing unit that performs predetermined image processing on image data captured by the endoscope scope; first image data that is image data received from the image processing server; and A selection unit that selects any one of the second image data that is image data processed by the processing unit and outputs the second image data to the display unit; a selection control unit that controls an operation of the selection unit to select image data; The endoscope apparatus characterized by having.

  In the endoscope apparatus of the present invention, the selection control unit detects a motion of a subject from image data captured by the endoscope scope, and the selection unit selects image data based on the detected motion. Controlling the operation to be selected.

  In the endoscope apparatus of the present invention, the selection control unit calculates a motion vector from image data captured by the endoscope scope, and detects the absolute value of the calculated motion vector. And

  In the endoscope apparatus of the present invention, the selection control unit calculates a motion vector from image data captured by the endoscope scope, and detects a moving direction of the subject based on the direction of the calculated motion vector. It is characterized by doing.

  In the endoscope apparatus of the present invention, the endoscope scope includes an acceleration sensor that detects movement of the endoscope scope, and the selection control unit is configured to output the acceleration sensor based on an output of the acceleration sensor. The selection unit controls an operation of selecting image data.

  In the endoscope apparatus of the present invention, the selection control unit has a condition for switching the image data output from the selection unit to the display unit between the first image data and the second image data. The switching unit is instructed to switch the image data after being continuously satisfied for a certain period of time.

  In the endoscope apparatus of the present invention, a condition for switching the image data output from the selection unit to the display unit from the first image data to the second image data, and the second image data from the second image data. The condition for switching to the first image data is different.

  The endoscope apparatus according to the present invention further includes a user interface unit for a user to input an instruction to set an observation mode, and the selection control unit is configured to display the image based on the user instruction. It controls the operation of selecting data.

  In addition, the endoscope apparatus of the present invention further includes a light source that generates light that is irradiated onto the subject through the endoscope scope, and the selection control unit is configured to change the light source according to the type of light generated by the light source. The selection unit controls an operation of selecting image data.

  In the endoscope apparatus of the present invention, the image processing server further includes a database, and the selection control unit receives the first image data when the image processing server performs processing using the database. The selection unit is controlled to select.

  The present invention also provides an endoscope scope that captures an image of an observation object and obtains image data, an image processing server that performs predetermined image processing on image data captured by the endoscope scope, and the endoscope scope. And an endoscope image processor connected to the image processing server, and a display unit for displaying an image based on image data output from the endoscope image processor, the endoscope image processor comprising: A communication control unit that performs control to transmit image data captured by the endoscope scope to the image processing server and receive image data processed by the image processing server; and image data captured by the endoscope scope An image processing unit that performs predetermined image processing, first image data that is image data received from the image processing server, and image data processed by the image processing unit A selection unit that selects any one of the second image data and outputs the selected image data to the display unit, and a selection control unit that controls an operation of the selection unit to select image data. Endoscope observation system.

  According to the present invention, the selection control unit controls whether the selection unit selects the first image data from the image processing server or the second image data from the image processing unit according to the situation. It is possible to provide image quality and operability according to the situation.

It is a block diagram which shows the structure of the endoscope observation system by the 1st Embodiment of this invention. It is a flowchart which shows the procedure of operation | movement of the display image switching control part which the endoscope observation system by the 1st Embodiment of this invention has. It is a reference figure which shows the pattern of the motion vector in the 2nd Embodiment of this invention. It is a block diagram which shows the structure of the endoscope observation system by the 3rd Embodiment of this invention. It is a block diagram which shows the structure of the endoscope observation system by the 4th Embodiment of this invention. It is a block diagram which shows the structure of the endoscope observation system by the 5th Embodiment of this invention. It is a block diagram which shows the structure of the endoscope observation system by the 6th Embodiment of this invention.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

(First embodiment)
First, a first embodiment of the present invention will be described. FIG. 1 shows the configuration of the endoscope observation system according to the present embodiment. The endoscope observation system shown in FIG. 1 includes an endoscope apparatus having an endoscope scope 100, an endoscope image processor 101, and a monitor 110, and an image processing server 106.

  The endoscope scope 100 is used by being inserted into the body of a subject. The endoscope scope 100 includes an image sensor at the tip, and acquires an image of an observation object in the body to obtain a digital image signal (image data). The image data output from the endoscope scope 100 is input to the endoscope image processor 101. The endoscope image processor 101 is connected to the endoscope scope 100 and the image processing server 106, and includes a network control unit 102, a simple image processing unit 103, a display image selection unit 104, and a display image switching control unit 105. .

  The network control unit 102 transmits the image data from the endoscope scope 100 to the image processing server 106 via the network. The image processing server 106 is a server that performs software processing using a so-called cloud network. The image processing server 106 performs predetermined image processing on the image data received from the network control unit 102, and returns the processed image data to the network control unit 102. The image processing performed by the image processing server 106 is image processing that has higher functionality than the image processing performed by the simple image processing unit 103 described later. For example, interpolation processing, color and brightness adjustment processing, contour enhancement processing, and enlargement processing are performed. Etc. The network control unit 102 receives the processed image data from the image processing server 106 and outputs the received image data to the display image selection unit 104 as a processing result 107.

  On the other hand, the image data output from the endoscope scope 100 is also input to the simple image processing unit 103. The simple image processing unit 103 performs predetermined image processing on the input image data, and outputs the processed image data to the display image selection unit 104 as a processing result 108. Note that image processing by the simple image processing unit 103 may be simpler than image processing by the image processing server 106, such as demosaicing, white balance, a zooming mechanism that adjusts the number of pixels of the image sensor to the number of pixels of the monitor, etc. Basic processing is sufficient. The image processing by the simple image processing unit 103 is sufficient if a minimum image quality can be ensured, and is not necessarily a process suitable for diagnosis.

  Here, the difference between the processing result 107 and the processing result 108 will be supplemented. The processing result 107 has a larger delay than the processing result 108 due to the reciprocation of the image data through the network, and the operation feeling is not necessarily good, but the image quality is suitable for detailed observation. On the other hand, the processing result 108 is less delayed than the processing result 108 and has a good feeling of operation, but it is not necessarily suitable for detailed observation because only minimal image processing is performed.

  The display image switching control unit 105 selects either the processing result 107 or the processing result 108 based on the result of analyzing the image data from the endoscope scope 100, and displays a selection instruction signal 109 indicating the selection result on the display image. The data is output to the selection unit 104. The display image selection unit 104 selects either the processing result 107 or the processing result 108 according to the selection instruction signal 109 and outputs the selected result to the monitor 110. The monitor 110 displays an image based on the input image data that is the processing result.

  FIG. 2 shows an operation in which the display image switching control unit 105 selects either the processing result 107 or the processing result 108. The display image switching control unit 105 stands by until the frame start timing based on the video synchronization signal is reached. When the frame start timing is reached (step S100), a motion vector is calculated from the image data by block matching or the like. The amount of movement of the subject is detected based on the motion vector (step S105). Although the expression “the amount of movement of the subject” is used, strictly speaking, it is “the amount of movement of the scope” (the movement detected from the image data is equivalent to the movement of the subject). . Subsequently, the display image switching control unit 105 determines whether or not the detected movement amount is greater than or equal to a predetermined value (step S110).

  Here, the fact that the entire screen is moving is emphasized rather than the fact that a specific part has moved greatly. For example, an average of the absolute value of the motion vector over the entire screen may be used as the movement amount. When the movement amount is equal to or greater than the predetermined value and the entire screen is moving greatly, the display image switching control unit 105 determines that the user is in the stage of searching for the observation target part and emphasizes operability. Then, the processing result 108 that minimizes the delay is selected (step S115). On the other hand, when the movement amount is less than the predetermined value and the movement of the entire screen is not large, the display image switching control unit 105 specifies the observation target part (lesion part or the like) and performs detailed observation. It is determined that the process is in progress, and the processing result 107 more suitable for observation is selected so that detailed (high-function) observation is possible (step S120). Thereafter, the above process is repeated.

  As described above, although there is a delay in the processing result 108, in the detailed observation, the user does not move the scope greatly, so the delay caused by passing through the network has little influence on the operability. That is, the user is not stressed even if the operational feeling deteriorates.

  Here, the switching timing of the processing result 107 and the processing result 108 will be described. There is a difference in image quality between the processing result 107 and the processing result 108. Therefore, if both are frequently switched, the screen flickers. Therefore, even if it is determined in step S115 or step S120 that the movement amount satisfies the predetermined condition and the processing result is to be switched in step S115 or step S120, the display image switching control unit 105 does not immediately switch the processing result. After confirming that the condition is satisfied for a predetermined time, the selection instruction signal 109 is output to the display image selection unit 104.

  Furthermore, in the case where the processing result 107 is switched to the processing result 108 and vice versa, the durations in which the movement amount satisfies the predetermined condition may be set to different times. Switching from the search of the observation target to the detailed observation is performed in a state where the scope is stationary, so even if the duration is long, there is no problem. Conversely, in the search for the observation target, the scope does not always move, and it is assumed that the scope temporarily stays at a specific location, but the operation mode switches to the detailed observation mode each time. It can be considered annoying. On the other hand, regarding the transition from the detailed observation to the search for the observation target, since the operation of the scope is related, it is necessary to immediately switch the processing result in order to improve the operability. Therefore, when switching from the processing result 107 to the processing result 108, the duration for which the movement amount satisfies the predetermined condition, and when switching from the processing result 108 to the processing result 107, the movement amount is shorter than the duration for which the predetermined condition is satisfied. That's fine.

  As described above, according to the present embodiment, the display image selection unit 104 controls which of the image data from the image processing server 106 and the image data from the simple image processing unit 103 is selected according to the situation. Accordingly, image quality and operability according to the situation can be provided, and the operational feeling is not impaired even in an image processing system using a network.

  In addition, the user's needs are taken from the movement of the subject in the screen detected from the image data, and the image data is selected adaptively, so that the user is not forced to perform extra work.

  Further, if the image data output to the monitor 110 is frequently switched, the screen flickers. In this embodiment, even if it is determined that the image data needs to be switched, the image data is switched immediately. Without switching, switching is performed after observing that the state continues for a predetermined time, so that unnecessary switching is suppressed and stress applied to the user can be reduced.

  In addition, flicker prevention is maintained by changing the time required for switching the image data output to the monitor 110 from the processing result 107 to the processing result 108 and vice versa. However, it is possible to realize a quick transition to the search for the observation target.

(Second Embodiment)
Next, a second embodiment of the present invention will be described. The components of this embodiment are the same as those of the first embodiment. The present embodiment is different in that the direction of the motion vector is used as a criterion for switching image data.

  Observation with an endoscope is often performed on a hollow organ. FIG. 3 shows a typical pattern of motion vectors detected from image data captured while observing a hollow organ. The motion vector pattern detected from the image P1 is a pattern when the scope is moving forward, that is, when an insertion operation is being performed, and the motion vector pattern detected from the image P2 is That is, this is a pattern when the removal operation is performed.

  The display image switching control unit 105 calculates a motion vector from the image data, and detects the moving direction of the subject based on the calculated direction of the motion vector. The display image switching control unit 105 selects either the processing result 107 or the processing result 108 based on the detected moving direction of the subject. In endoscopic screening, an observation method is used in which observation is performed while the scope is retracted after the scope is inserted to the deepest part. Therefore, the display image switching control unit 105 moves the scope in the insertion direction, that is, when an insertion operation is performed when many motion vectors from the center of the image to the outer periphery are detected. The processing result 108 (image data from the simple image processing unit 103) is selected. Further, the display image switching control unit 105 moves the scope in the direction of extraction when a large number of motion vectors from the outer periphery to the center of the image are detected, that is, the extraction operation (detailed observation). Is selected, the processing result 107 (image data from the image processing server 106) is selected.

  As described above, unnecessary switching of images can be suppressed by determining the observation mode according to the moving direction of the subject, that is, the moving direction of the scope. The movement direction of the scope is also related to the diagnosis contents, and it is possible to determine more appropriate switching by associating the movement direction of the scope with the contents of the image processing. Also in this embodiment, the switching timing control as described in the first embodiment may be used together.

(Third embodiment)
Next, a third embodiment of the present invention will be described. FIG. 4 shows the configuration of the endoscope observation system according to the present embodiment. Components having the same functions as those shown in the first embodiment are given the same symbols, and descriptions thereof are omitted.

  In the present embodiment, the endoscope scope 300 is equipped with an acceleration sensor (not shown). From the endoscope scope 300, image data 301 and movement information 302 from the acceleration sensor are output. The image data 301 is input to the network control unit 102 and the simple image processing unit 103, and the movement information 302 is input to the display image switching control unit 303.

  Based on the movement information 302, the display image switching control unit 303 detects the amount of movement of the subject or the direction of movement of the subject, and controls the display image selection unit 104, as in the first or second embodiment. To do. Although it is necessary to mount a sensor in the endoscope scope, compared to the method of detecting movement by image processing, more accurate determination can be made and the image processing load associated with vector calculation is reduced and the image processing circuit The cost can be reduced.

(Fourth embodiment)
Next, a fourth embodiment of the present invention will be described. FIG. 5 shows a configuration of the endoscope observation system according to the present embodiment. Components having the same functions as those shown in the first embodiment are given the same symbols, and descriptions thereof are omitted.

  In the present embodiment, a user interface unit 400 is provided for a user to input an instruction to set an observation mode. The user interface unit 400 outputs mode information 401 indicating an observation mode based on an instruction input by the user to the display image switching control unit 105. The display image switching control unit 105 selects either the processing result 107 or the processing result 108 according to the mode information 401.

  The fact that the user selects a special observation mode means that the display image switching control unit 105 selects a processing result by using that it can be determined that there is no movement of the subject for a while. For example, when magnifying observation using an endoscope scope having an optical zoom function, the user does not magnify the image from the beginning and uses the magnifying function after specifying the observation target. . Therefore, when magnified observation is performed, the observation target is specified, and the scope does not move greatly. Therefore, the display image switching control unit 105 selects the processing result 107 (image data from the image processing server 106) when the mode for performing the magnified observation is selected, and the processing result when any other mode is selected. 108 (image data from the simple image processing unit 103) is selected.

  In this embodiment, since it is possible to select image data without performing special image processing, the configuration is very simple compared to a method for controlling switching of image data based on the result of image processing. Become. Naturally, the technique of this embodiment may be combined with the technique using the image processing described in the first to third embodiments, and the movement of the scope may be added to the determination condition. Even in situations such as magnified observation, there is no guarantee that the scope will not move at all, so it is effective as a provision in such situations.

(Fifth embodiment)
Next, a fifth embodiment of the present invention will be described. FIG. 6 shows a configuration of the endoscope observation system according to the present embodiment. Components having the same functions as those shown in the fourth embodiment are given the same symbols, and descriptions thereof are omitted.

  In the present embodiment, a light source 500 is provided that can generate light of various wavelengths that are irradiated onto the subject through the endoscope scope 100. An endoscope system has a function of facilitating observation of lesions by irradiating with light of a special spectrum. For example, by observing narrow band light, observation may be made with emphasis on capillaries and fine patterns on the surface of the mucosa (Narrow Band Imaging). When such observation is performed, it is considered that the observation target has already been identified and the detailed observation is performed. Therefore, the display image switching control unit 105 selects the processing result 107 of the image processing server 106 when such a diagnostic mode using observation light is selected, and simplifies when the normal mode using white light is selected. The processing result 108 of the image processing unit 103 is selected.

  In this embodiment, since it is possible to select image data without performing special image processing, the configuration is very simple compared to a method for controlling switching of image data based on the result of image processing. Become. When the normal mode using white light is selected, the display image selection unit 104 uses the method using the image processing described in the first to third embodiments to display the processing result 107 and the processing result 108. The display image switching control unit 105 may perform control so as to select either one.

(Sixth embodiment)
Next, a sixth embodiment of the present invention will be described. FIG. 7 shows a configuration of the endoscope observation system according to the present embodiment. Components having the same functions as those shown in the fourth embodiment are given the same symbols, and descriptions thereof are omitted.

  In the present embodiment, the image processing server 600 includes a database 601. The database 601 stores information such as past cases and sample images that assist diagnosis, or past diagnostic images of the same subject. The following can be considered as uses of the information stored in the database 601.

  For example, the image processing server 600 compares the image acquired from the endoscope image processor 101 with the sample image stored in the database 601, selects a sample image similar to the acquired image, and selects Image processing for assisting the user in overlooking the image acquired from the endoscopic image processor 101 based on the sampled image (for example, processing for superimposing a marker on a portion estimated to be a lesion) )I do. Alternatively, for example, the image processing server 600 performs image processing (for example, acquired from the endoscope image processor 101) for efficiently performing follow-up observation of the same subject on the image acquired from the endoscope image processor 101. Image processing for displaying the images and the past diagnostic images of the same subject side by side).

  In such a situation, the user expects some kind of analysis from the image processing server 600, and it is considered that the operation of the endoscope is paused and the result is awaited. Therefore, the display image switching control unit 105 selects the processing result 107 (image data from the image processing server 600) when the mode for analyzing the currently observed image using the database 601 is selected, When a mode other than is selected, the processing result 108 (image data from the simple image processing unit 103) is selected. Although the database 601 has been described as being in the image processing server 600, the database 601 may be independent of the image processing server 600.

  In this embodiment, since it is possible to select image data without performing special image processing, the configuration is very simple compared to a method for controlling switching of image data based on the result of image processing. Become.

  As described above, the embodiments of the present invention have been described in detail with reference to the drawings. However, the specific configuration is not limited to the above-described embodiments, and includes design changes and the like without departing from the gist of the present invention. .

  100: endoscope scope, 101, 600: endoscope image processor, 102: network control unit (communication control unit), 103: simple image processing unit (image processing unit), 104 ..Display image selection unit (selection unit), 105,303 ... Display image switching control unit (selection control unit), 106 ... Image processing server, 110 ... Monitor (display unit), 400 ... User interface, 500 ... light source, 601 ... database

Claims (11)

An endoscope scope that captures an image of an observation object and obtains image data;
An endoscope image processor connected to the endoscope scope and connected to an image processing server for performing predetermined image processing on image data captured by the endoscope scope;
A display unit that displays an image based on image data output from the endoscopic image processor;
With
The endoscopic image processor includes:
A communication control unit configured to transmit image data captured by the endoscope scope to the image processing server and to receive image data processed by the image processing server;
An image processing unit that performs predetermined image processing on image data captured by the endoscope scope;
Selection for selecting one of first image data that is image data received from the image processing server and second image data that is image data processed by the image processing unit and outputting the selected image data to the display unit And
A selection control unit for controlling an operation in which the selection unit selects image data;
An endoscope apparatus characterized by comprising:   The selection control unit detects movement of a subject from image data captured by the endoscope scope, and controls the operation of the selection unit to select image data based on the detected movement. The endoscope apparatus according to claim 1.   The endoscope according to claim 2, wherein the selection control unit calculates a motion vector from image data captured by the endoscope scope, and detects the magnitude of an absolute value of the calculated motion vector. apparatus.   The selection control unit calculates a motion vector from image data captured by the endoscope scope, and detects a moving direction of the subject based on the direction of the calculated motion vector. Endoscope device. The endoscope scope includes an acceleration sensor that detects movement of the endoscope scope;
The endoscope apparatus according to claim 1, wherein the selection control unit controls an operation in which the selection unit selects image data based on an output of the acceleration sensor.   The selection control unit performs the selection after the condition for switching the image data output from the selection unit to the display unit between the first image data and the second image data is satisfied for a certain period of time. The endoscope apparatus according to claim 2, wherein an instruction to switch image data is given to the unit.   The condition for switching the image data that the selection unit outputs to the display unit from the first image data to the second image data is different from the condition for switching from the second image data to the first image data. The endoscope apparatus according to claim 2. A user interface unit for a user to input an instruction to set an observation mode;
The endoscope apparatus according to claim 1, wherein the selection control unit controls an operation in which the selection unit selects image data based on an instruction from the user. Further comprising a light source for generating light irradiated to the subject through the endoscope scope;
The endoscope apparatus according to claim 8, wherein the selection control unit controls an operation in which the selection unit selects image data according to a type of light generated by the light source. The image processing server further includes a database,
The said selection control part controls the said selection part to select the said 1st image data, when the said image processing server performs the process using the said database. The inside of Claim 1 characterized by the above-mentioned. Endoscopic device. An endoscope scope that captures an image of an observation object and obtains image data;
An image processing server that performs predetermined image processing on image data captured by the endoscope scope;
An endoscope image processor connected to the endoscope scope and the image processing server;
A display unit that displays an image based on image data output from the endoscopic image processor;
With
The endoscopic image processor includes:
A communication control unit configured to transmit image data captured by the endoscope scope to the image processing server and to receive image data processed by the image processing server;
An image processing unit that performs predetermined image processing on image data captured by the endoscope scope;
Selection for selecting one of first image data that is image data received from the image processing server and second image data that is image data processed by the image processing unit and outputting the selected image data to the display unit And
A selection control unit for controlling an operation in which the selection unit selects image data;
An endoscope observation system characterized by comprising: