JP2010158308A - Image processing apparatus, image processing method and image processing program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To appropriately extract an abstractive intracavitary image from a series of intracavitary images. <P>SOLUTION: In this embodiment, a section setting part 16 sets a similar image appearing section in which a similar intracavitary image appears in time-series sections of the series of intracavitary images. A first abstractive image extracting part 17 extracts a first abstractive intracavitary image from the series of intracavitary images based on the change of the scene of the respective intracavitary images constituting the series of intracavitary images. A second abstractive image extracting part 18 extracts a second abstractive intracavitary image based on the similarity among the first abstractive intracavitary images belonging to the same similarity image appearing section. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to an image processing apparatus, an image processing method, and an image processing program for extracting a summary body cavity image from a series of body cavity images obtained by continuously imaging the body cavity.

  In recent years, capsule endoscopes have been developed as medical devices for imaging the body cavity of a subject. After being swallowed from the mouth, the capsule endoscope captures an image of the body cavity at a predetermined imaging rate while moving in the digestive tract by a peristaltic motion and the like, transmits it to the external receiver, and is finally discharged out of the body. . The number of images in the body cavity to be imaged is generally indicated by the imaging rate (2 to 4 frames / sec) × the stay time in the capsule endoscope (8 hours = 8 × 60 × 60 sec). become. An observer such as a doctor spends a lot of time to confirm a large amount of in-vivo images transmitted to these extracorporeal receivers on a diagnostic workstation or the like and to identify a lesioned part. For this reason, there is a strong demand for a technique for improving the efficiency of observing a body cavity image.

  On the other hand, a technique for creating a digest for grasping the content from a series of images arranged in a time series such as a moving image is known. For example, Patent Document 1 discloses a technique for calculating a degree of difference based on a color histogram from the immediately preceding frame and setting a frame whose difference exceeds a threshold as a digest image. By applying this type of technique, it is possible to extract a summary in-vivo image from an in-vivo image captured by a capsule endoscope, so that it is possible to improve the observation efficiency of the in-vivo image.

JP-A-8-294083

  By the way, since the capsule endoscope moves in the body cavity by a peristaltic motion or the like, the moving speed is not constant, and the capsule endoscope may stay in the same place in the digestive tract or repeat forward and backward movements. As a result, there may occur a case where the in-vivo images having the same contents are captured continuously or discontinuously. For this reason, as in the above-described Patent Document 1, in a method of simply obtaining a change between adjacent images (corresponding to “difference” in Patent Document 1), between images existing in positions separated in time series order. In some cases, similarities could not be judged properly.

  In addition, since the capsule endoscope captures an image while moving in the digestive tract, the inside of the esophagus, stomach, small intestine, large intestine, and the like is shown in this order in the series of images in the body cavity obtained. For this reason, in the method of Patent Literature 1, similarity determination is performed between images that clearly belong to different organ sections and based on a change between adjacent images across different organ sections. was there.

  The present invention has been made in view of the above, and provides an image processing apparatus, an image processing method, and an image processing program capable of appropriately extracting a summary body cavity image from a series of body cavity images. With the goal.

  In order to solve the above-described problems and achieve the object, an image processing apparatus according to an aspect of the present invention is an image that extracts a summary body cavity image from a series of body cavity images obtained by continuously imaging the body cavity. A processing device, a section setting unit for setting a similar image appearance section in which a similar in-vivo image appears in a time-series section of the series of in-vivo images, and each of the body cavities constituting the in-body-cavity image A first summary image extraction unit that extracts a first summary body cavity image from the series of body cavity images based on a scene change of the inner image; and the first summary image extraction unit that belongs to the same similar image appearance section And a second summary image extraction unit for extracting a second summary body cavity image based on the similarity between the summary body cavity images.

  According to the image processing apparatus according to this aspect, a similar image appearance section in which a similar in-vivo image appears is set in a time-series section of a series of in-vivo images, and a scene change of each in-vivo image is performed. In addition, the first summary body cavity image can be extracted from the series of body cavity images. The second summary body cavity image can be extracted based on the similarity between the first summary body cavity images belonging to the same similar image appearance section. According to this, when extracting the summary body cavity image, the similarity between the summary body cavity images can be taken into account for each similar image appearance section, so that the contents can be grasped from the series of body cavity images. It is possible to appropriately extract the summary body cavity image.

  An image processing method according to another aspect of the present invention is an image processing method for extracting a summary body cavity image from a series of body cavity images obtained by continuously imaging the inside of the body cavity. Based on the section setting step of setting a similar image appearance section in which a similar in-vivo image appears in the time-series section of the image, and the scene change of each in-body cavity image constituting the series of in-vivo images, A first summary image extraction step of extracting a first summary body cavity image from a series of body cavity images, and a similarity between the first summary body cavity images belonging to the same similar image appearance section And a second summary image extraction step of extracting a second summary body cavity image.

  An image processing program according to another aspect of the present invention is an image processing program for causing a computer to extract a summary body cavity image from a series of body cavity images obtained by continuously imaging the body cavity. A section setting step for setting a similar image appearance section in which a similar in-vivo image appears in a time-series section of the series of in-vivo images, and a scene change of each in-body cavity image forming the series of in-vivo images. Based on the first summary image extraction step of extracting a first summary body cavity image from the series of body cavity images, and between the first summary body cavity images belonging to the same similar image appearance section A second summary image extraction step of extracting a second summary body cavity image based on the degree of similarity is executed by the computer.

  According to the present invention, it is possible to appropriately extract a summary body cavity image from a series of body cavity images.

  DESCRIPTION OF EMBODIMENTS Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the drawings. In this embodiment, a capsule endoscope that moves in the digestive tract is used as an example of an imaging device, and a series of images of body cavities that are continuously captured while the capsule endoscope moves in the digestive tract of a subject. An image processing apparatus to be processed will be described. Note that the present invention is not limited to the embodiments. Moreover, in description of each drawing, the same code | symbol is attached | subjected and shown to the same part.

(Embodiment 1)
FIG. 1 is a schematic diagram showing an overall configuration of an image processing system including an image processing apparatus 10 according to the first embodiment. As shown in FIG. 1, the image processing system receives a capsule endoscope 3 that captures an image (intrabody cavity image) inside the subject 1, and a body cavity image that is wirelessly transmitted from the capsule endoscope 3. The apparatus 5 includes an image processing apparatus 10 that processes and displays the in-vivo image captured by the capsule endoscope 3 based on the in-vivo image received by the receiving apparatus 5. For example, a portable recording medium (portable recording medium) 7 is used for transferring image data between the receiving device 5 and the image processing device 10.

  The capsule endoscope 3 has an imaging function, a wireless function, and the like. The capsule endoscope 3 is swallowed from the mouth of the subject 1 and introduced into the subject 1 and sequentially moves through the digestive tract. Image. The captured in-vivo image is wirelessly transmitted outside the body. Here, the in-vivo image captured by the capsule endoscope 3 has pixel values (RGB) corresponding to, for example, R (red), G (green), and B (blue) color components at each pixel position in the image. Value).

  The receiving device 5 includes receiving antennas A <b> 1 to An distributed at positions on the body surface corresponding to the passage route of the capsule endoscope 3 in the subject 1. The receiving device 5 receives the image data wirelessly transmitted from the capsule endoscope 3 via the receiving antennas A1 to An. The receiving device 5 is configured so that the portable recording medium 7 can be freely attached and detached, and sequentially stores the received image data in the portable recording medium 7. In this way, a series of in-vivo images captured by the capsule endoscope 3 inside the subject 1 is accumulated and stored in the portable recording medium 7 in time series by the receiving device 5.

  The image processing apparatus 10 is for a doctor or the like to observe and diagnose a series of in-vivo images captured by the capsule endoscope 3, and is realized by a general-purpose computer such as a workstation or a personal computer. The image processing apparatus 10 is configured so that the portable recording medium 7 can be freely attached and detached. The image processing apparatus 10 processes a series of in-vivo images stored in the portable recording medium 7, and displays the image on a display such as an LCD or an EL display. Display sequentially in sequence order.

  FIG. 2 is a block diagram illustrating a functional configuration of the image processing apparatus 10 according to the first embodiment. In the present embodiment, the image processing apparatus 10 includes an image acquisition unit 11, an input unit 12, a display unit 13, a recording unit 14, a calculation unit 15, and a control unit 20 that controls each unit of the apparatus.

  The image acquisition unit 11 acquires a series of in-vivo images captured by the capsule endoscope 3 and stored in the portable recording medium 7 by the receiving device 5. For example, the portable recording medium 7 is detachable. The image data of the in-vivo image stored in the mounted portable recording medium 7 is read out and acquired. The image acquisition unit 11 is realized by, for example, a read / write device corresponding to the type of the portable recording medium 7. Note that acquisition of a series of in-vivo images captured by the capsule endoscope 3 is not limited to the configuration using the portable recording medium 7, and includes, for example, a hard disk instead of the image acquisition unit 11. A series of in-vivo images captured by the capsule endoscope 3 may be stored in advance in the hard disk. Or it is good also as a structure which installs a server separately instead of the portable recording medium 7, and preserve | saves a series of body cavity images in this server beforehand. In this case, the image acquisition unit 11 is configured by a communication device or the like for connecting to the server, and is connected to the server via the image acquisition unit 11 to acquire the in-vivo image from the server.

  The input unit 12 is realized by, for example, a keyboard, a mouse, a touch panel, various switches, and the like, and outputs an operation signal corresponding to the operation input to the control unit 20. The display unit 13 is realized by a display device such as an LCD or an EL display, and displays various screens including a display screen of an in-vivo image under the control of the control unit 20.

  The recording unit 14 is realized by various IC memories such as ROM and RAM such as flash memory that can be updated and stored, a hard disk connected by a built-in or data communication terminal, a recording medium such as a CD-ROM, and a reading device thereof. is there. The recording unit 14 records a program for operating the image processing apparatus 10 and realizing various functions provided in the image processing apparatus 10, data used during execution of the program, and the like. Further, an image processing program 141 for extracting a summary body cavity image from a series of body cavity images is recorded.

  The calculation unit 15 is realized by hardware such as a CPU, processes a plurality of in-vivo images acquired by the image acquisition unit 11, and performs various calculation processes for extracting a summary in-vivo image from a series of in-vivo images. I do. The calculation unit 15 includes a section setting unit 16, a first summary image extraction unit 17, and a second summary image extraction unit 18.

  The section setting unit 16 divides a time series section of a series of in-vivo images into sections (organ sections) in which the same organ type is shown, and sets similar image appearance sections. The section setting unit 16 includes an organ type discriminating unit 161 that discriminates the organ type reflected in each body cavity image. The first summary image extraction unit 17 includes an inter-image change amount calculation unit 171 and a scene change detection unit 173, and extracts a first summary body cavity image. The inter-image change amount calculation unit 171 calculates the inter-image change amount of each in-vivo image. The scene change detection unit 173 detects an in-vivo image in which a scene change has occurred based on the amount of change between images. The second summary image extraction unit 18 includes a summary image change amount calculation unit 181 and a similarity calculation unit 183, and extracts a second summary body cavity image from the first summary body cavity image. The inter-summary image change amount calculation unit 181 calculates the inter-summary image change amount of each first summary body cavity image for each similar image appearance section. The similarity calculation unit 183 calculates the similarity of each first summary body cavity image based on the amount of change between summary images.

  The control unit 20 is realized by hardware such as a CPU. The control unit 20 is connected to each unit constituting the image processing apparatus 10 based on image data input from the image acquisition unit 11, operation signals input from the input unit 12, programs and data recorded in the recording unit 14, and the like. Instruction, data transfer, and the like, and overall operation of the image processing apparatus 10 is controlled.

  FIG. 3 is an overall flowchart illustrating a processing procedure performed by the image processing apparatus 10 according to the first embodiment. The processing described here is realized by the calculation unit 15 executing the image processing program 141 recorded in the recording unit 14. 4-6 is explanatory drawing explaining the process sequence of step a3-step a17 shown in FIG.

  As shown in FIG. 3, first, the calculation unit 15 acquires a series of in-vivo images (step a1). Here, the calculation unit 15 acquires image data of a series of in-vivo images read from the portable recording medium 7 by the image acquisition unit 11 via the control unit 20. The acquired image data of each in-vivo image is recorded in the recording unit 14 together with the image numbers indicating the chronological order, and the image data of an arbitrary image number can be read.

  Subsequently, the organ type discriminating unit 161 performs an organ type discriminating process, and discriminates organ types appearing in each body cavity image constituting the series of acquired body cavity images (step a3). In Embodiment 1, for example, four types of esophagus, stomach, small intestine, and large intestine are discriminated as organ types reflected in each body cavity image.

  Subsequently, the section setting unit 16 divides a time series section of a series of in-vivo images into organ sections in which the same organ type is reflected, and a section in which similar in-vivo images appear in each organ section (similar image appearance section ) (Step a5). Specifically, the section setting unit 16 determines the position where the organ type changes in the time series section of the series of in-vivo images based on the organ type determined for each in-vivo image in the organ type determining process in step a3. Divide by. As a result, as shown in FIG. 4 (a), the time-series section of the series of in-vivo images 8 is divided into each organ section including a section where the esophagus is reflected, a section where the stomach is reflected, a section where the small intestine is reflected and a section where the large intestine is reflected. Is done. Then, the section setting unit 16 detects the head and tail of an organ section in which each organ of the esophagus, stomach, small intestine, and large intestine is reflected, and records the detected image numbers of the head and tail of each organ section in the recording unit 14.

  Subsequently, as shown in FIG. 3, the inter-image change amount calculation unit 171 performs an inter-image change amount calculation process to calculate the inter-image change amount of each intra-body cavity image (step a7). Subsequently, the scene change detection unit 173 detects an in-vivo image in which a scene change has occurred based on the inter-image change amount calculated for each in-vivo image in the inter-image change amount calculation processing in Step a7 (Step S7). a9). For example, the scene change detection unit 173 performs threshold processing on the inter-image change amount of each intra-body cavity image, and detects an intra-body image whose inter-image change amount is equal to or greater than a preset threshold value as a scene change. Then, the first summary image extraction unit 17 extracts the body cavity image detected as the scene change in step a9 as the first summary body cavity image (step a11). The extraction result is held in the recording unit 14.

  For example, in FIG. 4B, attention is paid to a stomach organ section set as one of the similar image appearance sections, and nine in-vivo images (first summary body cavity inside) extracted from the similar image appearance section. Image) I11 to I19 are shown together with the inter-image variation amounts D11 to D19. Hereinafter, the first summary body cavity image is referred to as a “primary summary image”.

  Subsequently, as shown in FIG. 3, the inter-summary image change amount calculation unit 181 performs the inter-summary image change amount calculation process to calculate the inter-summary image change amount of each primary summary image for each similar image appearance section. (Step a13). In FIG. 5, attention is paid to one primary summary image I11 extracted from the similar image appearance section which is the organ section of the stomach shown in FIG. 4, and the amount of change between the summary images is shown. In this inter-summary image change amount calculation process, for each primary summary image, the change amount between each primary summary image and another primary summary image belonging to the same similar image appearance section is calculated, and is used as the inter-summary image change amount. . For example, in the case of the primary summary image I11 of FIG. 5, the change amounts D21 to D28 between all the other primary summary images I12 to I19 are calculated, and the calculated change amounts D21 to D28 are calculated as the primary. The amount of change between the summary images of the summary image I11 is set.

  Subsequently, as shown in FIG. 3, the similarity calculation unit 183 performs similarity calculation processing to calculate the similarity of each primary summary image (step a15). As described above, in the summary image change amount calculation process in step a13, the change amount between each primary summary image and all other primary summary images belonging to the same similar image appearance section is the summary image. It is calculated as the amount of change between In the similarity calculation process, each primary summary image is sequentially processed, and the change amount between summary images having the smallest value is selected from the change amounts between summary images of the primary summary image to be processed. And At this time, another primary summary image for which the amount of change between summary images as the similarity is calculated is set as a similar summary image of the primary summary image to be processed. Then, the similarity and the similar summary image are recorded in the recording unit 14 in association with the image number of the primary summary image to be processed. For example, it is assumed that the change amount D22 between the primary summary image I13 and the other primary summary image I13 is the smallest among the change amounts D21 to D28 which are the change amounts between the summary images of the primary summary image I11 shown in FIG. In this case, the amount of change D22 is set as the similarity of the primary summary image I11. Further, another primary summary image I13 for which the change amount D22 as the similarity is calculated is set as a similar summary image of the primary summary image I11.

  Then, as shown in FIG. 3, the second summary image extraction unit 18 performs the summary image re-extraction process to extract the second summary body cavity image from the primary summary image, and the final summary body cavity An inner image is obtained (step a17). FIG. 6 shows how the second summary body cavity image is extracted, focusing on the primary summary images I11 to I19 extracted from the similar image appearance section which is the stomach organ section shown in FIG. . In FIG. 6, the similarities and similar summary images of the primary summary images I11 to I19 are shown. For example, the similarity of the primary summary image I11 is D22, and the similar summary image is I13. In this summary image re-extraction process, first, the level of similarity is determined. In Embodiment 1, since the amount of change between images is set as the similarity, the similarity to the similar summary image is high if the value of the similarity is small. Conversely, if the similarity value is large, the similarity with the similar summary image is low. Here, for example, it is determined that the similarity is high when it is equal to or less than a predetermined threshold, and the similarity is low when it is greater than the threshold.

  Then, the primary summary image having a high similarity and the similar summary image are grouped and assigned to the same similar summary image group. For example, in FIG. 6, it is assumed that the similarity D22 of the primary summary image I11 is equal to or less than a threshold value. In this case, the primary summary image I11 is grouped with the similar summary image I13 and assigned to the same similar summary image group. Further, if the similarity D43 of the primary summary image I13 is equal to or less than the threshold value, the primary summary image I13 is grouped with the similar summary image I14 and assigned to the same similar summary image group. As a result, the primary summary images I11, I12, and I14 are grouped together and assigned to the same similar summary image group G11. Then, from the grouped similar summary image group, for example, a primary summary image with the head in time series order is extracted as a second summary body cavity image. For example, in the illustrated example, the primary summary image I11 is extracted from the similar summary image group G11, and the primary summary image I15 is extracted from the similar summary image group G12 as the second summary body cavity image.

  On the other hand, all the primary summary images whose similarity is larger than the threshold value and low in similarity are extracted as second summary body cavity images. For example, in the illustrated example, primary summary images I12, I16, I18, and I19 are extracted as second summary body cavity images from similar image appearance intervals that are stomach organ sections, respectively.

  Thereafter, the calculation unit 15 outputs the obtained final summary body cavity image (step a19). For example, the calculation unit 15 causes the display unit 13 to sequentially display and output the in-vivo images constituting the second summary in-vivo image via the control unit 20.

  Next, each process of steps a3, a7, a13, a15, and a17 of FIG. 3 will be described sequentially. First, the organ type determination process performed by the organ type determination unit 161 in step a3 of FIG. 3 will be described. FIG. 7 is a flowchart showing a detailed processing procedure of the organ type determination processing.

  As shown in FIG. 7, in the organ type discriminating process, the organ type discriminating unit 161 first sets the target to be processed sequentially from a series of in-vivo images acquired in step a3 of FIG. 3 and recorded in the recording unit 14. The body cavity images are read one by one (step b1). Subsequently, the organ type discriminating unit 161 discriminates the organ type shown in the in-vivo image to be processed (step b3). The determination result is held in the recording unit 14.

  As a specific organ type discrimination method, a known technique can be used as appropriate. For example, using the technique disclosed in Japanese Patent Application Laid-Open No. 2006-288612, the determination is made based on the average R value, G value, and B value of the in-vivo image. Specifically, the numerical ranges of the average R value, G value, and B value for each organ type are set in advance. In Embodiment 1, there are four types of organs to be discriminated: esophagus, stomach, small intestine, and large intestine. Therefore, numerical ranges for the average R value, G value, and B value of the esophagus, stomach, small intestine, and large intestine are set. Keep it. If the average R value, G value, and B value of the in-vivo image to be processed are within the numerical range of the average R value, G value, and B value of the esophagus, the organ type of the observation site reflected in the in-vivo image is determined. Distinguish from the esophagus. If the average R value, G value, and B value of the in-vivo image to be processed are within the numerical range of the average R value, G value, and B value of the stomach, the organ type of the observation site shown in the in-vivo image is defined as the stomach. Determine. If the average R value, G value, and B value of the in-vivo image to be processed are within the numerical range of the average R value, G value, and B value of the small intestine, the organ type of the observation site reflected in the in-vivo image is defined as the small intestine. Determine. If the average R value, G value, and B value of the in-vivo image to be processed are within the range of the average R value, G value, and B value of the large intestine, the organ type of the observation site that appears in the in-vivo image is the large intestine. Determine. It should be noted that any method may be used as long as the organ type appearing in the image can be discriminated.

  Subsequently, the organ type discriminating unit 161 determines whether or not the organ type has been discriminated for all the in-vivo images, and if there is an unprocessed in-vivo image (step b5: No), the process returns to step b1. Then, an unprocessed in-vivo image is read out as a processing target, and the processing in step b3 is performed to determine the organ type. On the other hand, when all the in-vivo images have been processed (step b5: Yes), the process returns to step a3 in FIG. 3 and then proceeds to step a5.

  Next, the inter-image change amount calculation process performed by the inter-image change amount calculation unit 171 in step a7 in FIG. 3 will be described. FIG. 8 is a flowchart showing a detailed processing procedure of the inter-image change amount calculation processing.

  As shown in FIG. 8, in the inter-image change amount calculation processing, the inter-image change amount calculation unit 171 first counts the number of in-vivo images constituting a series of in-vivo images and sets it as a variable max (step c1). ). Subsequently, the inter-image change amount calculation unit 171 sets the inter-image change amount of the in-vivo image having the first time-series order as the maximum value of the inter-image change amount set in advance (step c3). This is because the immediately preceding image does not exist for the first image in time series order.

  Subsequently, the inter-image variation calculation unit 171 initializes the variable i = 2 indicating the time-series order of the images to be processed (step c5). Then, the inter-image change amount calculation unit 171 reads the i-1 th body cavity image I (i-1) and the i th body cavity image I (i) from the recording unit 14 (step c7). Then, the inter-image change amount calculation unit 171 calculates the change amount between the intra-body-cavity image I (i-1) and the intra-body-cavity image I (i), and the inter-image change amount of the intra-body-cavity image I (i). (Step c9). The calculated inter-image change amount is held in the recording unit 14. There are various methods for calculating the amount of change. Here, a method based on a difference in pixel values and a method based on a change in image statistic are shown.

First, a method based on the difference in pixel values will be described. In this method, first, a sum of squares SSD of pixel value differences between the in-vivo image I (i-1) and the in-vivo image I (i) is calculated according to the following equation (1). Here, since each image has a pixel value for each color component of R (red), G (green), and B (blue), an SSD value is also calculated for each color component. Therefore, the total value or average value of the SSDs of the respective color components is obtained, and this value is used as the inter-image change amount of the in-vivo image I (i).

Further, instead of SSD, an absolute value sum SAD of pixel value differences shown in the following equation (2) may be used.

Next, a method based on statistic change will be described. Examples of the image statistic include an average, variance, and histogram of pixel values in the image. For example, these statistics are obtained in the body cavity image I (i-1) and the body cavity image I (i), respectively. Then, a difference D_Euclide of the obtained numerical value Stat is calculated according to the following equation (3), and this value is set as an inter-image variation amount of the in-vivo image I (i).

  Here, when the average or variance of the pixel values in the image is used as a statistic, the numerical value Stat is calculated for each color component of R (red), G (green), and B (blue). There are three statistics. Further, in the case of a histogram, the numerical value Stat is a statistical quantity of 3 × n_edge because the number of histogram divisions n_edge for each color component of R (red), G (green), and B (blue) is calculated. In the above equation (3), the number of dimensions is indicated by d (1 to D).

  It should be noted that a plurality of statistics such as average, variance, and histogram may be combined into a multidimensional feature vector, and the inter-image change amount may be calculated using this multidimensional feature vector. Further, at this time, normalization may be performed so that ranges that each statistic can take, or calculation may be performed by multiplying each statistic by a weighting coefficient.

  As described above, two methods have been shown as a method for calculating the amount of change between images. However, if a value corresponding to a change between the in-vivo image I (i-1) and the in-body-cavity image I (i) is obtained. It may be calculated by a method other than this example. Here, the amount of change between the intra-body-cavity image I (i) and the intra-body-cavity image I (i-1) immediately before in time series is calculated as the inter-image change amount of the intra-body-cavity image I (i). It is not necessary to calculate the amount of change between the images in the body cavity where the sequence order is continuous. For example, it is possible to calculate the amount of change between other images in the body cavity that are not in chronological order but are close to each other.

  Then, as shown in FIG. 8, the inter-image change amount calculation unit 171 increments and updates the variable i indicating the time series order of the image to be processed, and compares it with the variable max. The presence or absence of an image to be determined is determined. If i ≦ max (step c11: No), the process returns to step c7, and the processes in steps c7 and c9 are performed on the next in-vivo image to be processed to calculate the inter-image change amount. On the other hand, if i> max (step c11: Yes), the process returns to step a7 in FIG. 3, and then proceeds to step a9.

  Next, the inter-summary image change amount calculation process performed by the inter-summary image change amount calculation unit 181 in step a13 in FIG. 3 will be described. FIG. 9 is a flowchart illustrating a detailed processing procedure of the inter-summary image variation calculation processing.

  As shown in FIG. 9, in the inter-summary image variation calculation process, first, the inter-summary image variation calculation unit 181 sequentially processes the section where the same organ set as the similar image appearance section in step a5 in FIG. Selected as a target (step d1). Then, the inter-summary image variation calculation unit 181 calculates the primary summary image belonging to the processing target similar image appearance section (hereinafter referred to as inter-summary image variation calculation) from the primary summary images extracted in step a11 of FIG. In the processing, “in-section primary summary image” is read from the recording unit 14 (step d3).

  Subsequently, the inter-summary image change amount calculation unit 181 uses, as the inter-summary image change amount, the change amount between each of the other primary summary images in each section for the primary summary image in each section read in step d1. Calculate (step d5). For example, as illustrated in FIG. 5, the similar image appearance section to be processed is a stomach section, and the primary summary images in the stomach section are configured by nine primary summary images I11 to I19. If this is the case, each of the primary summary images I11 to I19 is sequentially processed, and the amount of change between the other eight primary summary images in the section is calculated as the amount of change between summary images. . The calculated inter-summary image change amount is held in the recording unit 14. Note that, as a method for calculating the amount of change between the first summary images in other sections, the method for calculating the amount of change between images described in step c9 of FIG. 8 can be used.

  Subsequently, as shown in FIG. 9, the inter-summary image change amount calculation unit 181 determines whether or not the inter-summary image change amount has been calculated for all similar image appearance sections, and an unprocessed similar image appearance section. If there is (step d7: No), the process returns to step d1. Then, the processing from step d1 to step d5 is performed on an unprocessed similar image appearance section as a processing target, and a change amount between summary images is calculated. On the other hand, when all the similar image appearance sections have been processed (step d7: Yes), the process returns to step a13 in FIG. 3 and then proceeds to step a15.

  Next, the similarity calculation process performed by the similarity calculation unit 183 in step a15 in FIG. 3 will be described. FIG. 10 is a flowchart showing a detailed processing procedure of the similarity calculation processing.

  As shown in FIG. 10, in the similarity calculation process, the similarity calculation unit 183 first selects a primary summary image as a processing target sequentially (step e1). Subsequently, the similarity calculation unit 183 reads the amount of change between summary images of the primary summary image to be processed from the recording unit 14 (step e3). Then, the similarity calculation unit 183 calculates the similarity of the primary summary image to be processed based on the read change amount between summary images (step e5). That is, as described above, the change amount having the smallest value among the change amounts between summary images of the primary summary image to be processed is selected as the similarity value, and the other primary summaries for which the change amount is calculated. The image is a similar summary image of the primary summary image to be processed.

  Subsequently, the similarity calculation unit 183 determines whether or not the similarity is calculated for all the primary summary images. If there is an unprocessed primary summary image (step e7: No), the similarity calculation unit 183 determines in step e1. Return. Then, the processing of step e1 to step e5 is performed on the unprocessed primary summary image as the processing target, and the similarity is calculated. On the other hand, when all the primary summary images have been processed (step e7: Yes), the process returns to step a15 in FIG. 3 and then proceeds to step a17.

  Next, the summary image re-extraction process performed by the second summary image extraction unit 18 in step a17 of FIG. 3 will be described. FIG. 11 is a flowchart showing a detailed processing procedure of the summary image re-extraction process.

  As shown in FIG. 11, in the summary image re-extraction process, the second summary image extraction unit 18 first selects a primary summary image as a processing target sequentially (step f1). Subsequently, the second summary image extraction unit 18 reads the similarity of the primary summary image to be processed from the recording unit 14 (step f3). Subsequently, the second summary image extraction unit 18 performs threshold processing on the read similarity, and if the similarity is equal to or lower than a predetermined threshold set in advance and the similarity is high (step f5: Yes), processing is performed. The target primary summary image and a similar summary image similar to the image are grouped and assigned to the same similar summary image group (step f7). On the other hand, when the similarity is larger than the threshold and the similarity is low (step f5: No), the process proceeds to step f9.

  In step f9, the second summary image extraction unit 18 determines whether or not the similarity is determined for all the primary summary images, and if there is an unprocessed primary summary image (step f9: No). ), And returns to step f1. Then, the processing of step f1 to step f7 is performed on the unprocessed primary summary image as a processing target, and the primary summary image having a high similarity is assigned to the same summary image group as the similar summary image.

  On the other hand, when all the primary summary images have been processed (step f9: Yes), the second summary image extraction unit 18 selects the first primary image in time series order from among the grouped similar summary image groups. A summary image is selected (step f11). If there are a plurality of similar summary image groups, the first primary summary image in the time-series order is selected for each group. Then, the second summary image extraction unit 18 converts the selected primary summary image and all the primary summary images whose similarity is determined to be greater than the threshold and low in step f9 to the second summary. Extracted as a body cavity image (step f13). The extraction result is held in the recording unit 14. Thereafter, the process returns to step a17 in FIG. 3, and then the process proceeds to step a19.

  Here, the primary summary image with the head in time series order is extracted from the group of grouped similar summary images as the second summary body cavity image. In contrast, a primary summary image suitable for observation may be selected from a group of similar summary images.

  For example, the primary summary image with the least noise may be selected as the primary summary image suitable for observation from the primary summary images constituting the similar summary image group. In this case, for example, values (determination values) such as G / R and B / G are set in advance as color information of a color that does not appear in the body cavity image obtained by imaging the inside of the body. Then, each primary summary image of the similar summary image group is sequentially processed, and the G / R and B / G values of each pixel constituting the primary summary image to be processed are calculated. Then, the smaller the difference between the calculated value and the preset determination value, the more the amount of noise is determined. Then, the primary summary image with the least amount of noise is selected as the primary summary image suitable for observation.

  Moreover, it is good also as selecting the primary summary image in which the halation does not generate | occur | produce as a primary summary image suitable for observation among primary summary images which comprise a similar summary image group. In this case, for example, each primary summary image of the similar summary image group is sequentially processed, and an average value of G values or an average value of Y values of all the pixels constituting the primary summary image to be processed is calculated. calculate. Then, the calculated average value is subjected to threshold processing, and is determined to be an image in which halation occurs when the average value is equal to or greater than a predetermined threshold value. Then, an image in which no halation has occurred is selected as a primary summary image suitable for observation from primary summary images constituting the similar summary image group.

  Alternatively, a brightness threshold range suitable for observation is set in advance, and a primary summary image whose brightness is within the threshold range among primary summary images constituting a similar summary image group is suitable for observation. It is good also as selecting as a primary summary image. According to this, a primary summary image that is neither too bright nor too dark can be selected as a primary summary image suitable for observation.

  Further, the number of images to be extracted as the second summary body cavity image from the group of similar summary images grouped is not limited to one, and a plurality of images may be extracted.

  As described above, according to the first embodiment, a time-series section of a series of in-vivo images is divided into organ sections based on organ types reflected in each in-vivo image, and each organ section is similar to a similar intra-body cavity. It can be set as a similar image appearance section where an image is likely to appear. Then, for each similar image appearance section, by calculating the amount of change between the first summary body cavity images (primary summary images) belonging to each similar image appearance section, the similarity of each first summary body cavity image The degree can be calculated. Based on the calculated similarity, a final second summary body cavity image can be extracted from the first summary body cavity image. Therefore, it is possible to appropriately extract a summary in-vivo image for grasping the contents from a series of in-vivo images.

(Embodiment 2)
Next, a second embodiment will be described. FIG. 12 is a block diagram illustrating a functional configuration of the image processing apparatus 10b according to the second embodiment. In addition, the same code | symbol is attached | subjected about the structure same as the image processing apparatus 10 demonstrated in Embodiment 1. FIG. As shown in FIG. 12, the image processing apparatus 10b controls the operation of the image acquisition unit 11, the input unit 12, the display unit 13, the recording unit 14b, the calculation unit 15b, and the entire image processing apparatus 10b. Part 20b.

  In the recording unit 14b, in the time series section of the series of body cavity images, an organ section in which the same organ type is reflected is set according to the user operation, and a summary body cavity image is obtained from the series of body cavity images based on the organ section. An image processing program 141b for extraction is recorded.

  The calculation unit 15b includes a section setting unit 16b, a first summary image extraction unit 17, and a second summary image extraction unit 18. On the other hand, the control unit 20b includes an organ section designation request unit 201b that requests designation of an organ section and receives an organ section designation operation by the user via the input unit 12. The section setting unit 16b of the calculation unit 15b sets a similar image appearance section according to the organ section specified here.

  FIG. 13 is an overall flowchart illustrating a processing procedure performed by the image processing apparatus 10b according to the second embodiment. Note that the processing described here is realized by the operation of each unit of the image processing apparatus 10b according to the image processing program 141b stored in the recording unit 14b. Moreover, in FIG. 13, the same code | symbol is attached | subjected about the process process similar to Embodiment 1. FIG.

  As shown in FIG. 13, in the second embodiment, after the calculation unit 15b acquires a series of in-vivo images in step a1, the organ segment designation request unit 201b of the control unit 20b subsequently displays the organ segment designation screen. A process of displaying on the display unit 13 and notifying an organ segment designation request is performed (step g3). The organ section designation request unit 201b outputs the organ section designated in response to the designation request notification to the section setting unit 16b of the calculation unit 15b.

  FIG. 14 is a diagram illustrating an example of an organ section designation screen. As shown in FIG. 14, the organ section designation screen includes an image display unit 91 that displays each in-vivo image constituting a series of in-vivo images at the center of the screen. Below the image display unit 91, a reverse button 92, a forward button 93, and a slider bar 94 are arranged for the observer to switch the in-vivo image displayed on the image display unit 91. The user can switch the in-vivo image of the image display unit 91 by pressing the reverse button 92 or the forward button 93 or by sliding the slider bar 94.

  The organ section designation screen includes an organ selection list box 95 for designating an organ section, an OK button 96 for confirming an organ section designation operation, and an end button 97 for ending the designation operation. Yes. The organ selection list box 95 presents a list of options for selecting the head and tail of the organ section. 14 shows options for selecting the head and tail of the esophagus, stomach, small intestine, and large intestine, the options presented in the organ selection list box 95 are organ sections set as similar image appearance sections. It can be set appropriately according to.

  When the user designates an organ section on the organ section designation screen, first, the backward button 92, the forward button 93, and the slider bar 94 are operated to display a series of body cavity images in sequence on the image display unit 91. While browsing, search for an in-vivo image that is the head or tail of an organ section. Then, in a state where these in-vivo images are displayed on the image display unit 91, the corresponding option is selected from the organ selection list box 95, and the OK button 96 is pressed. Further, the user presses an end button 97 when finishing the organ segment designation operation. As internal processing, the organ section designation requesting unit 201b includes the image number of the in-vivo image displayed on the image display unit 91 when the OK button 96 is pressed and the option selected in the organ selection list box 95. Are associated and held in the recording unit 14b. Then, in response to the pressing operation of the end button 97, the organ section designation requesting unit 201b associates the body cavity image held in the recording unit 14b with each option (that is, the body cavity at the head and tail of each organ section). The image number of the inner image) is output to the section setting unit 16b.

  Subsequently, as shown in FIG. 13, the section setting unit 16b divides the time series section of the series of in-vivo images into the organ sections input from the organ section designation requesting section 201b, and each organ section is similar. It is set as a section where a body cavity image appears (similar image appearance section) (step g5). Thereafter, the process proceeds to step a7.

  As described above, according to the second embodiment, the same effect as that of the first embodiment can be obtained, and the similar image appearance section can be set in the time-series section according to the organ section specified by the user.

  In each of the above-described embodiments, the case where the summary body cavity image is extracted from the series of body cavity images captured by the capsule endoscope has been described. However, the image to be processed is the capsule endoscope 3. It is not limited to the captured image. Further, the present invention can be similarly applied to the case where a summary body cavity image is extracted from a continuous image such as a moving image or a sequence of still images captured continuously.

1 is a schematic diagram illustrating an overall configuration of an image processing system including an image processing apparatus. 2 is a block diagram illustrating a functional configuration of the image processing apparatus according to Embodiment 1. FIG. 3 is an overall flowchart illustrating a processing procedure performed by the image processing apparatus according to the first embodiment. It is explanatory drawing explaining the process sequence shown in FIG. It is explanatory drawing explaining the process sequence shown in FIG. It is explanatory drawing explaining the process sequence shown in FIG. It is a flowchart which shows the detailed process sequence of an organ kind discrimination | determination process. It is a flowchart which shows the detailed process sequence of the variation | change_quantity calculation process between images. It is a flowchart which shows the detailed process sequence of the variation | change_quantity calculation process between summary images. It is a flowchart which shows the detailed process sequence of a similarity calculation process. It is a flowchart which shows the detailed process sequence of the summary image re-extraction process. 6 is a block diagram illustrating a functional configuration of an image processing apparatus according to Embodiment 2. FIG. 10 is an overall flowchart illustrating a processing procedure performed by the image processing apparatus according to the second embodiment. It is a figure which shows an example of an organ area designation | designated screen.

DESCRIPTION OF SYMBOLS 3 Capsule endoscope 5 Receiving device A1-An Receiving antenna 7 Portable recording medium 10 Image processing device 11 Image acquisition part 12 Input part 13 Display part 14, 14b Recording part 141, 141b Image processing program 15, 15b Calculation part 16 , 16b Interval setting unit 161 Organ type discrimination unit 17 First summary image extraction unit 171 Inter-image change amount calculation unit 173 Scene change detection unit 18 Second summary image extraction unit 181 Inter-summary image change amount calculation unit 183 Similarity calculation Unit 20, 20b Control unit 201b Organ section designation request unit 1 Subject

Claims (17)

An image processing apparatus for extracting a summary in-vivo image from a series of in-vivo images obtained by continuously imaging the inside of the body cavity,
A section setting unit that sets a similar image appearance section in which a similar in-vivo image appears in the time-series section of the series of in-vivo images;
A first summary image extraction unit for extracting a first summary body cavity image from the series of body cavity images based on a scene change of each body cavity image constituting the series of body cavity images;
A second summary image extraction unit that extracts a second summary body cavity image based on the similarity between the first summary body cavity images belonging to the same similar image appearance section;
An image processing apparatus comprising:   The section setting unit includes an organ type determination unit that determines an organ type reflected in the series of in-vivo images, and sets a section determined as the same organ type by the organ type determination unit as the similar image appearance section. The image processing apparatus according to claim 1, wherein: A section designation requesting unit for requesting designation of an organ section in which the same organ is reflected in the time series section of the series of body cavity images,
The image processing apparatus according to claim 1, wherein the section setting unit sets an organ section specified in response to a request from the section specifying request unit as the similar image appearance section. The first summary image extraction unit includes:
An inter-image change amount calculation unit that calculates an inter-image change amount between other intra-body-cavity images that are close in time series order for each in-vivo image;
A scene change detection unit that detects a body cavity image in which a scene change has occurred from the series of body cavity images based on the amount of change between images;
The image processing apparatus according to claim 1, further comprising: extracting an in-vivo image detected by the scene change detection unit as the first summary in-vivo image.   The scene change detection unit detects an in-vivo image in which a value of the inter-image change amount is within a predetermined threshold range set in advance as an in-vivo image in which the scene change has occurred. 5. The image processing apparatus according to 4. The second summary image extraction unit includes:
An inter-summary image change amount calculation unit that calculates an inter-summary image change amount with respect to each of the other first summary body cavity images belonging to the same similar image appearance section for each of the first summary body cavity images; ,
Based on the amount of change between the summary images, a similarity calculating unit that calculates the similarity of the first summary body cavity image;
The image processing apparatus according to claim 1, further comprising: extracting the second summary body cavity image from the first summary body cavity image based on the similarity.   The similarity calculation unit sets the smallest value among the inter-summary image variation calculated for the first summary body cavity image as the similarity of the first summary body cavity image. Item 7. The image processing apparatus according to Item 6.   The second summary image extraction unit determines the level of similarity of the first summary body cavity image, and uses the first summary body cavity image determined to have high similarity as the similarity. Assigned to the same similar summary image group as the other first summary body cavity images for which the amount of change between summary images is calculated, and extracts at least one image included in the similar summary image group as the second summary body cavity image The image processing apparatus according to claim 6.   9. The image processing apparatus according to claim 8, wherein the second summary image extraction unit extracts a first summary body cavity image that is first in time series order from the similar summary image group. .   The image processing apparatus according to claim 8, wherein the second summary image extraction unit extracts a first summary body cavity image suitable for observation from the group of similar summary images.   The second summary image extraction unit extracts a first summary body cavity image suitable for the observation based on the presence or absence of noise in each of the first summary body cavity images constituting the similar summary image group. The image processing apparatus according to claim 10.   The second summary image extraction unit extracts a first summary body cavity image suitable for the observation based on the presence or absence of halation in each of the first summary body cavity images constituting the similar summary image group. The image processing apparatus according to claim 10.   The second summary image extraction unit extracts a first summary body cavity image suitable for the observation based on the brightness in each first summary body cavity image constituting the similar summary image group. The image processing apparatus according to claim 10.   The second summary image extraction unit determines the level of similarity of the first summary body cavity image, and all the first summary body cavity images determined to have low similarity are the second summary. The image processing apparatus according to claim 6, wherein the image processing apparatus is extracted as an in-vivo image.   The second summary image extracting unit determines that the similarity is high when the value of the similarity is within a predetermined threshold range set in advance and is low when the value is outside the threshold range. The image processing apparatus according to claim 8 or 14. An image processing method for extracting a summary body cavity image from a series of body cavity images obtained by continuously imaging the body cavity,
A section setting step for setting a similar image appearance section in which a similar in-vivo image appears in the time-series section of the series of in-vivo images;
A first summary image extracting step of extracting a first summary body cavity image from the series of body cavity images based on a scene change of each body cavity image constituting the series of body cavity images;
A second summary image extraction step of extracting a second summary body cavity image based on the similarity between the first summary body cavity images belonging to the same similar image appearance section;
An image processing method comprising: An image processing program for causing a computer to extract a summary body cavity image from a series of body cavity images obtained by continuously imaging the body cavity,
A section setting step for setting a similar image appearance section in which a similar in-vivo image appears in the time-series section of the series of in-vivo images;
A first summary image extraction step for extracting a first summary body cavity image from the series of body cavity images based on a scene change of each body cavity image constituting the series of body cavity images;
A second summary image extraction step of extracting a second summary body cavity image based on the similarity between the first summary body cavity images belonging to the same similar image appearance section;
An image processing program for causing the computer to execute.

Images