JP6120762B2 - Image processing device - Google Patents

Description

  The present invention relates to an image processing apparatus that processes an image acquired by an examination using a medical image acquisition apparatus such as a capsule endoscope.

  In recent years, in the field of endoscopes, examination using a capsule endoscope that is introduced into a subject such as a patient and images the inside of the subject is known. A capsule endoscope is a device that incorporates an imaging function, a wireless communication function, etc. in a capsule-shaped casing that is sized to be introduced into the digestive tract of a subject. Image data is generated and wirelessly transmitted sequentially outside the subject. Image data wirelessly transmitted from the capsule endoscope is temporarily stored in a receiving apparatus provided outside the subject, and transferred from the receiving apparatus to an image processing apparatus such as a workstation. In the image processing apparatus, various image processing is performed on the acquired image data, thereby generating a series of images showing an organ or the like in the subject.

  By the way, since the capsule endoscope is passively moved by the peristaltic movement of the digestive tract, the moving speed is very unstable. That is, the gastrointestinal tract may pass smoothly through the gastrointestinal tract, or may stay in the same place for a while because the movement is hindered by the structure of the gastrointestinal tract itself and the residues in the gastrointestinal tract. In addition, the capsule endoscope that has stayed may suddenly move quickly.

  On the other hand, since the structure in the digestive tract is complicated, if the position and orientation of the capsule endoscope changes even a little, the imaging result of the capsule endoscope is often quite different.

  As a technique related to the display of a plurality of images, Patent Document 1 discloses a technique for outputting a notification sound informing that a feature image is displayed when a feature image in an image group is displayed. Patent Document 2 discloses a technique for notifying a user of a change in image between frames that is equal to or greater than a set value.

JP 2006-129950 A JP 2005-176096 A

  For these reasons, when the images acquired by the capsule endoscope are continuously reproduced and displayed, the appearance in the image changes suddenly due to a sudden change in the field of view, or there are notable areas such as lesions. A sudden display occurs repeatedly. Therefore, in order to prepare for a sudden change in information obtained from an image, a user (an image observer) needs to keep an eye on the screen at all times, and the burden is very large.

  Regarding the above problem, in Patent Document 2, the user is notified by displaying a text message on the screen. However, since there is not one type of information change that can occur in an image acquired by a capsule endoscope, it is preferable that notification is made according to the information change. In this regard, a text message can be notified in accordance with changes in information. In this case, however, the user must read the message while changing the viewpoint each time the text message is displayed. Attention may be reduced.

  The present invention has been made in view of the above, and provides an image processing apparatus that enables a user to prepare for various sudden changes in information obtained from an image without feeling an excessive burden. The purpose is to do.

  In order to solve the above-described problems and achieve the object, an image processing apparatus according to the present invention includes a storage unit that stores image data corresponding to an image group arranged in time series, and a predetermined amount from the image group. An image specifying unit that specifies an image having features as a specific image, a display unit that displays each image included in the image group, and image data corresponding to each image from the storage unit in time series or time series A display control unit that acquires the image data in reverse order and displays the images on the display unit in the order in which the image data was acquired, and a display order for the display target image from the display target image from which the display control unit has acquired the image data Includes a determination unit that determines whether the specific image specified by the image specifying unit is included in a range up to an image after a predetermined number of images or a display time is after a predetermined time, and the specific image is included in the range Determined A display data generation unit that generates display data representing information indicating that the specific image exists, and the display control unit creates a screen including the display target image and adds the display data to the display data. Based on this, information indicating that the specific image exists is arranged on the screen in a manner corresponding to the characteristics of the specific image included in the range.

  In the image processing apparatus, the characteristics of the specific image included in the range include the number of images or the display time from the display target image to the specific image, the type of a predetermined feature of the specific image, or the It is a feature amount of a predetermined feature of the specific image.

  In the image processing apparatus, the information indicating that the specific image exists is a display frame provided around the display target image.

  In the image processing apparatus, the display control unit changes at least one of a color tone and a blinking cycle of the display frame according to characteristics of the specific image included in the range.

  In the image processing device, when the predetermined feature included in the specific image is a predetermined feature existing in the specific image, the image specifying unit further includes an existence region of the characteristic portion in the specific image. The display control unit further determines the arrangement of the information indicating that the specific image exists on the screen according to the existence area of the characteristic part in the specific image included in the range. Features.

  In the image processing apparatus, the information indicating that the specific image exists is a partial display frame provided in a part of the periphery of the display target image.

  In the image processing apparatus, the information indicating that the specific image exists is an icon arranged in the vicinity of the image to be displayed, and the display control unit responds to characteristics of the specific image included in the range. Then, at least one of the size of the icon, the display position of the icon, and the blinking cycle of the icon is changed.

  In the image processing apparatus, the information indicating that the specific image exists is an input icon arranged on the screen, an input unit that inputs a signal according to an operation from the outside, and the input on the screen And a skip unit for skipping an image to be displayed on the display unit to the specific image when a signal corresponding to an operation for selecting an icon is input from the input unit.

  In the image processing apparatus, the image group includes a series of images acquired by a capsule endoscope introduced into the subject sequentially capturing an image of the subject, and the specific image An image in which a predetermined feature amount is within a predetermined range, an image in which a predetermined feature portion is detected, an image having a similarity with an image having an image number adjacent to a predetermined value, or a specific position or region It is an image or an image in which the speed change of the capsule endoscope when the image is captured is a predetermined value or more.

  According to the present invention, when it is determined that the specific image is included in the range from the display target image to the image after the predetermined number of display orders or the display time after the predetermined time with respect to the display target image. Since the display indicating that the specific image exists is arranged on the screen in a manner according to the characteristics of the specific image, the user can avoid various excessive changes in information obtained from the image without feeling excessive burden. Can be prepared.

FIG. 1 is a schematic diagram showing a capsule endoscope system including an image processing apparatus according to Embodiment 1 of the present invention. FIG. 2 is a block diagram showing a configuration of the image processing apparatus according to Embodiment 1 of the present invention. FIG. 3 is a schematic diagram showing an image group acquired by the capsule endoscope shown in FIG. FIG. 4 is a flowchart showing the operation of the image processing apparatus shown in FIG. FIG. 5 is a schematic diagram illustrating a display example of a screen including an image to be displayed. FIG. 6 is a schematic diagram illustrating a display example of a screen on which information indicating that a specific image exists is arranged on a screen including an image to be displayed. FIG. 7 is a schematic diagram illustrating an example of a display mode of information indicating that a specific image exists in Modification 1-1. FIG. 8 is a schematic diagram illustrating an example of a display mode of information indicating that a specific image exists in Modification 1-5 of the present invention. FIG. 9 is a block diagram showing a configuration of an image processing apparatus according to Embodiment 2 of the present invention. FIG. 10 is a flowchart showing the operation of the image processing apparatus shown in FIG. FIG. 11 is a schematic diagram showing a display example of a screen on which information indicating that a specific image exists in the second embodiment is arranged. FIG. 12 is a schematic diagram showing a screen displayed after a pointer operation is performed on the skip button shown in FIG. FIG. 13 is a schematic diagram illustrating a display example of a screen on which information indicating that a specific image exists in Embodiment 3 of the present invention is arranged. FIG. 14 is a schematic diagram illustrating an example of a display mode of information indicating that a specific image exists in Modification 3-5 of Embodiment 3 of the present invention. FIG. 15 is a schematic diagram illustrating another example of a display mode of information indicating that a specific image exists in Modification 3-5 of Embodiment 3 of the present invention. FIG. 16 is a schematic diagram illustrating an example of a display mode of information indicating that a specific image exists in Modification 3-6 of Embodiment 3 of the present invention. FIG. 17 is a schematic diagram showing a display example of a screen on which information indicating that a specific image exists in Embodiment 4 of the present invention is arranged. FIG. 18 is a schematic diagram showing another display example of a screen on which information indicating that a specific image exists in Embodiment 4 of the present invention is arranged. FIG. 19 is a schematic diagram showing an image group acquired by the capsule endoscope shown in FIG. FIG. 20 is a block diagram showing a configuration of an image processing apparatus according to Embodiment 5 of the present invention.

  Hereinafter, an image processing apparatus according to an embodiment of the present invention will be described with reference to the drawings. Note that the present invention is not limited to these embodiments. In the following embodiments, a case where an image group acquired by an examination using a capsule endoscope is managed will be described. However, the image processing apparatus according to the present invention is acquired by various medical image acquisition apparatuses. This can be applied when managing images. Moreover, in description of each drawing, the same code | symbol is attached | subjected and shown to the same part.

(Embodiment 1)
FIG. 1 is a schematic diagram showing a capsule endoscope system for performing an examination using a capsule endoscope. The capsule endoscope system shown in FIG. 1 generates image data by imaging the inside of the subject 5 with the image processing apparatus 1 according to Embodiment 1 of the present invention. A capsule endoscope 2 for wireless transmission, and a receiving device 3 for receiving image data transmitted from the capsule endoscope 2 via a receiving antenna unit 4 attached to the subject 5.

  The image processing apparatus 1 is an apparatus that manages a group of images acquired in chronological order by inspection, and is configured by a general-purpose computer such as a workstation or a personal computer. In the first embodiment, a case where an image acquired by an examination using the capsule endoscope 2 is managed will be described. However, images acquired by various image acquisition devices other than the capsule endoscope 2 are described. The image processing apparatus 1 can also be applied when managing the image. The detailed configuration and operation of the image processing apparatus 1 will be described later.

  The capsule endoscope 2 includes a capsule-shaped casing that can be swallowed by a subject 5, an imaging element such as a CCD, an illumination element such as an LED, a memory, a signal processing unit, a wireless communication unit, and other various parts. It is a device with built-in. The imaging element is provided on one end side of the casing, and images a predetermined range outside the casing illuminated by the illumination element and outputs an imaging signal. The capsule endoscope 2 generates image data by performing predetermined signal processing on the imaging signal output from the imaging device, and transmits the image data and related information superimposed on a radio signal.

  The receiving device 3 receives a radio signal transmitted from the capsule endoscope 2 via the receiving antenna unit 4 having a plurality (eight in FIG. 1) of receiving antennas 4a to 4h. Each of the receiving antennas 4a to 4h is configured using, for example, a loop antenna, and corresponds to a predetermined position on the external surface of the subject 5 (for example, each organ in the subject 5 that is a passage path of the capsule endoscope 2). Arranged).

  The receiving device 3 acquires image data and related information by demodulating the received radio signal, and stores the image data and related information in a built-in memory. The related information includes, for example, information such as the serial number of the capsule endoscope 2 and the reception strength when each of the receiving antennas 4a to 4h receives a radio signal. The receiving device 3 includes a data transmission unit including an interface connectable to a communication line such as a USB, a wired LAN, or a wireless LAN, and transfers image data and related information to an external device via these interfaces.

  FIG. 2 is a block diagram illustrating a configuration of the image processing apparatus 1. As shown in FIG. 2, the image processing apparatus 1 includes an input unit 11, an image data acquisition unit 12, a storage unit 13, a calculation unit 14, a display unit 15, and a control unit 16.

  The input unit 11 includes an input device such as a keyboard, various buttons, and various switches, and a pointing device such as a mouse and a touch panel, and inputs a signal corresponding to an operation performed by the user from the outside to the control unit 16.

  The image data acquisition unit 12 is an interface that can be connected to a communication line such as a USB, wired LAN, or wireless LAN, and includes a USB port, a LAN port, and the like. The image data acquisition unit 12 acquires image data and related information via an external device or various lines connected to the USB port, and stores them in the storage unit 13. As shown in FIG. 1, in the first embodiment, the receiving apparatus 3 is connected to the image processing apparatus 1 by setting the receiving apparatus 3 in the cradle 3 a connected to the USB port of the image processing apparatus 1. As a result, the image data stored in the memory of the receiving device 3 is sequentially taken into the image processing device 1.

  The storage unit 13 includes a semiconductor memory such as a flash memory, a RAM, and a ROM, a recording medium such as an HDD, an MO, a CD-R, and a DVD-R, and a writing / reading device that writes and reads information on the recording medium Composed of etc. The storage unit 13 operates a program and various setting information for operating the image processing apparatus 1 to execute various functions, image data captured by the image data acquisition unit 12, and predetermined calculation processing by the calculation unit 14. The applied image data is stored.

  The calculation unit 14 is configured by hardware such as a CPU, for example, and reads an image stored in the storage unit 13 to read an image corresponding to the image data captured by the image data acquisition unit 12 and stored in the storage unit 13. Predetermined arithmetic processing is executed for each group. More specifically, the calculation unit 14 includes an image processing unit 141 that performs predetermined image processing on each image, a position calculation unit 142 that calculates the position of the capsule endoscope 2 when each image is captured, An image specifying unit 143 that extracts and specifies an image having a predetermined feature from the image group.

  The image processing unit 141 performs image processing such as white balance processing, demosaicing, color conversion, density conversion (gamma conversion, etc.), smoothing (noise removal, etc.), sharpening (edge enhancement, etc.) on each image. Display image data is generated, and further, a feature amount calculation process (average color calculation process) for calculating a feature amount such as an average color of each image, and a predetermined feature portion such as a lesion or a lesion candidate is detected from each image Image processing such as lesion detection processing and similarity calculation processing for calculating similarity with adjacent images is performed.

  Based on the reception intensity information when the reception antennas 4a to 4h receive the radio signal among the related information of the image data of each image, the position calculation unit 142 is configured so that the position of the capsule endoscope 2 at that time, That is, the imaging position of the image is calculated. Note that the method for calculating the position of the capsule endoscope 2 is not limited to the method based on the reception intensity of the radio signal, and various known methods can be used (see, for example, Japanese Patent Application Laid-Open No. 2009-222080).

  The image specifying unit 143 extracts an image having a predetermined feature from the image group based on the result of image processing by the image processing unit 141 and the result of position calculation by the position calculation unit 142. As an image having a predetermined feature, for example, an image in which a feature amount such as an average color in the image is within a predetermined range, an image in which a predetermined feature such as a lesion or a lesion candidate is detected, and an image number are adjacent to each other An image whose similarity to an image is equal to or less than a predetermined value, an image showing a specific position or region, and a change in the speed (movement amount per unit time) of the capsule endoscope 2 when the image is captured (adjacent For example, an image having a speed difference with respect to an image is equal to or greater than a predetermined value. The features and threshold values of the image extracted by the image specifying unit 143 are not particularly limited, and user-desired features and threshold values may be set.

  Note that the speed of the capsule endoscope 2 and the change thereof can be obtained for each image from the position information calculated by the position calculation unit 142. In addition, a triaxial acceleration sensor may be provided in the capsule endoscope 2 in advance, and the speed change of the capsule endoscope 2 may be acquired for each image based on the detection value by the triaxial acceleration sensor. Alternatively, based on the similarity calculated by the image processing unit 141, an image having a large change in similarity with an adjacent image may be extracted as an image having a large speed change.

  For an image with a large change in speed, for example, when the capsule endoscope 2 that has stayed suddenly starts moving, the image of the capsule endoscope 2 that has moved in addition to the image whose speed has suddenly increased is displayed. An image in which the speed is suddenly reduced as in the case of sudden stop is also included.

  Hereinafter, the image specified by the image specifying unit 143 is referred to as a specific image. The image specifying unit 143 adds a flag indicating that the image is a specific image to the image data of the specific image and stores the flag in the storage unit 13.

  The display unit 15 is a display device such as a CRT display or a liquid crystal display, for example, and displays a predetermined format screen including images and other information under the control of the control unit 16.

  The control unit 16 is configured by hardware such as a CPU, for example, and by reading a program stored in the storage unit 13, to the units configuring the image processing apparatus 1 based on a signal input from the input unit 11. Instruction, data transfer, etc., and overall operation of the image processing apparatus 1 is controlled.

  The control unit 16 also includes a display control unit 161 that causes the display unit 15 to display an image in a predetermined format based on the image data that has been subjected to predetermined image processing by the image processing unit 141, and a predetermined target image from the display target image. A determination unit 162 that determines whether or not a specific image is included in the range of images, and generates display data to be displayed on a screen on which a display target image is displayed according to a determination result by the determination unit 162 And a display data generation unit 163.

  The display control unit 161 acquires the image data subjected to the image processing by the image processing unit 141 from the storage unit 13 and causes the display unit 15 to display the image. The acquired image data may be all image data, or may be image data selected based on the result of image processing by the image processing unit 141. Further, image data to be acquired may be selected based on other criteria.

  The image display order may be a time-series order along the order in which the capsule endoscope 2 has taken an image, or may be a reverse order of the time series. Further, the display order may be set according to criteria other than the above, such as the result of image processing by the image processing unit 141.

FIG. 3 is a schematic diagram for explaining the operation of the determination unit 162 and shows an image group acquired by the capsule endoscope 2. The determination unit 162 starts from the display target image _Im from which the display control unit 161 has acquired the image data, after a predetermined number (d) of display order with respect to the image _Im , or the display time is a predetermined time ( _Td time). ) after ranging image (hereinafter, also referred to) a range to be determined, in certain image (FIG. 3 determines whether includes an image I _n). Specifically, it is determined whether or not there is an image to which a flag indicating that the image is a specific image exists within the determination target range.

Here, the subscripts m and n of the symbol I are image numbers in the image group, and 1 ≦ m ≦ N, 1 ≦ n ≦ N, and N are the number of images included in the image group. If the display order of the image is chronological order determination unit 162, a range image number is large and _{(I m + 1 ~I m +} d) to determine the target for the image I _m. On the other hand, if the display order of the image is reverse chronological, determination unit 162, a range image number is small and (I _m-1 ~I _md) and determination target for the image I _m. The number of images d or the time T _d that defines the determination target range may be set in advance for the image processing apparatus 1, or the user may be able to set a desired value.

When the determination unit 162 determines that the specific image is included in the determination target range, the display data generation unit 163 generates display data representing information indicating that the specific image exists. Further, the display data generating unit 163 to the display data may include information and time, etc. (the number of images from the image I _m to the image I _n) number of images up to the specific image is displayed. The time until the specific image is displayed, is calculated by integrating the display frame rate from the image I _m the number of images to the image I _n.

Next, the operation of the image processing apparatus 1 will be described. FIG. 4 is a flowchart showing the operation of the image processing apparatus 1.
First, in step S <b> 10, the image data acquisition unit 12 acquires image data of an image group acquired by an examination using the capsule endoscope 2 and stores the image data in the storage unit 13. At this time, the image data acquisition unit 12 may acquire image data from the receiving device 3 illustrated in FIG. 1 via the cradle 3a, or may acquire image data via various communication lines.

  In subsequent step S11, the calculation unit 14 performs predetermined image processing on each image included in the image group corresponding to the image data acquired in step S10, and calculates the imaging position of each image. More specifically, the image processing unit 141 generates image data for display by performing image processing such as white balance processing, demosaicing, color conversion, density conversion, smoothing, and sharpening, and further performs averaging. Image processing such as color calculation processing, lesion detection processing, and similarity calculation processing is executed. Further, the position calculation unit 142 calculates the imaging position of each image based on the reception intensity information of the wireless signal included in the related information of the image data of each image. The calculation unit 14 stores the image data that has undergone image processing and the position information that represents the imaging position in the storage unit 13 in association with each image.

  In step S12, the image specifying unit 143 extracts a specific image from the image group based on the result of the image processing by the image processing unit 141 and the result of the position calculation by the position calculating unit 142, and the image data of the extracted image A flag indicating that the image is a specific image is added. Thereby, for example, an image having a feature amount such as an average color in the image within a predetermined range, an image in which a predetermined feature such as a lesion or a lesion candidate is detected, or an image having an adjacent image number is similar. It is a specific image for an image whose degree is less than or equal to a predetermined value, an image showing a specific position or region, an image whose change in speed of the capsule endoscope 2 at the time of imaging is a predetermined value or more, etc. Is added.

  In step S13, the image processing apparatus 1 starts displaying each image included in the image group. At this time, the display control unit 161 may start displaying an image using a predetermined instruction signal input from the input unit 11 in accordance with a user operation as a trigger. At this time, the display control unit 161 also displays the image display order (time-series order or time-series reverse order) in accordance with an instruction signal (playback or reverse playback instruction signal) input from the input unit 11 in accordance with a user operation. ) May be determined.

  In step S14, the display control unit 161 sets the parameter j indicating the image number to the image number of the image to be displayed first. Here, when the display order of the images is chronological order, the parameter j is set to the image number 1 of the first image in the image group. On the other hand, when the image display order is the reverse order of the time series, the parameter j is set to the image number N of the last image in the image group. Alternatively, the display control unit 161 determines the value of the parameter j in the range of 1 to N according to a predetermined instruction signal (for example, a fast-forward or rewind instruction signal) input from the input unit 11 in accordance with a user operation. You may do it. FIG. 4 shows a case where j = 1 is set.

In step S _< b> 15, the display control unit 161 acquires the image data of the image I _j to be displayed from the storage unit 13.

In step S16, the determination unit 162 determines whether or not an image to which a flag indicating that the image is a specific image is included in the determination target range for the display target image I _j (j = m in FIG. 3). Determine.

When the specific image is not included in the determination target range (step S16: No), the display control unit 161 creates a screen including the display target image I _j (image I _m ) and causes the display unit 15 to display the screen. (Step S17).

FIG. 5 is a schematic diagram illustrating a display example of a screen including the image _Im to be displayed. Screen M1 shown in FIG. 5, another image display area M10 which an image I _m is displayed, the information display field M11 information about information and testing for patients, such as patient's name and age is the subject 5 is displayed And a reproduction operation button group M12.

  The reproduction operation button group M12 is a set of operation buttons used when a user inputs an instruction to control an image reproduction operation in the image display area M10. For example, a search button, a frame advance button, a pause button, and a fast forward button are used. Button, playback button, reverse playback button, rewind button, stop button, etc. When a predetermined pointer operation using the input unit 11 for the reproduction operation button group M12 (for example, a click operation for any one of the operation buttons) is performed, an operation signal corresponding to the pointer operation is input to the display control unit 161. .

  On the other hand, when the specific image is included in the determination target range (step S16: Yes), the display data generation unit 163 generates display data representing information indicating that the specific image exists (step S18). At this time, in the first embodiment, data representing an icon having a predetermined shape is created as display data.

In this case, in the subsequent step S17, the display control unit 161 creates a screen including the image I _j (image I _m in FIG. 3), and a specific image exists based on the display data generated in step S18. the information to the effect that, arranged on said screen in a manner that depends on the characteristics of the (image I _n in FIG. 3) the specific image included in the scope of the determination target, is displayed on the display unit 15. Here, the characteristics of the specific image included in the determination target range include the number of images from the display target image I _j to the specific image or the display time, the characteristic amount and similarity of the specific image, and the specific image. The type of the detected characteristic part, the speed change of the capsule endoscope 2 at the time of capturing the specific image, and the like are included.

Figure 6 is a schematic diagram showing a display example of a screen information indicating that the specific image exists for a screen including the image I _m is arranged. Screen M2 shown in FIG. 6 further includes an icon M20 as information indicating that a specific image exists, compared to screen M1 shown in FIG. The position of the icon M20 in the screen M2 is not particularly limited as long as it is a peripheral area of the image display area M10. However, the user can change the display / non-display of the icon M20 and the display mode without shifting the line of sight from the image display area M10. The icon M20 may be arranged as close as possible to the image display area M10 within a range that does not hinder image observation so that the image can be recognized.

  In addition, the display control unit 161 also displays information such as the number of images and the time until the specific image is displayed as one of the characteristics of the specific image in the icon M20 or a predetermined position on the screen M1. You may display.

In subsequent step S19, the display control unit 161 determines whether there is a next image to be displayed following the image I _j based on the value of the parameter j. Specifically, it is determined whether or not j = N is reached when the image display order is time series, and whether or not j = 1 is determined when the image display order is reverse. .

  When it is determined that there is a next image (step S19: Yes), the control unit 16 updates the parameter j to the image number of the next image to be displayed (step S20). Thereafter, the operation of the image processing apparatus 1 returns to step S15. On the other hand, when it is determined that there is no next image (step S19: No), the operation of the image processing apparatus 1 ends.

  As described above, according to the first embodiment, the specific image exists when the specific image is included in the range from the display target image to the image after the predetermined number of display orders or the display time is after the predetermined time. Since the information to the effect is displayed on the screen, the user can be prepared for a sudden change in the information obtained from the image without feeling an excessive burden. In addition, when displaying the number of images and time until the specific image is displayed together, the user can specifically predict the timing at which the specific image is displayed, so that more efficient observation is performed. Is possible.

  In the first embodiment, the display control unit 161 may change the display mode of the icon M20 displayed on the screen M2 according to the characteristics of the specific image included in the determination target range. Hereinafter, a modified example in which the display mode of the icon M20 is changed according to the characteristics of the specific image will be described.

(Modification 1-1)
In the first embodiment, the number of images and the time until the specific image is displayed are displayed as text in the icon M20. However, the icon M20 may be changed according to the number of images and the time. .

  As an example, when changing the color tone of the icon M20, the icon M20 is first displayed in, for example, light red, and the icon M20 is gradually changed to dark red as the number or time decreases.

  As another example, when changing the brightness of the icon M20, the brightness of the icon M20 is first displayed with a low brightness, and as the number or time decreases, the brightness of the icon M20 is gradually increased.

  As yet another example, the icon M20 may be displayed blinking. In this case, the icon M20 is first displayed with a longer blinking cycle, and as the number or time decreases, the blinking cycle of the icon M20 is gradually shortened.

  As yet another example, when the size of the icon M20 is changed, the icon M20 is first reduced in size (see FIG. 6), and the size of the icon M20 is gradually increased as the number or time decreases. (See FIG. 7).

  As another example, when the display position of the icon M20 is changed, the icon M20 is first displayed at a position slightly away from the image display area M10, and the position of the icon M20 is displayed as an image as the number or time decreases. Gradually approach the region M10.

  Furthermore, the display modes of the icon M20 described above may be combined with each other. For example, the icon M20 is first reduced in size and displayed at a position slightly away from the image display area M10. As the number or time decreases, the icon M20 is gradually increased in size and gradually displayed in the image display area M10. Move closer to.

(Modification 1-2)
You may change the display mode of icon M20 according to the similarity degree of a specific image. Here, as the specific image, an image whose similarity with an adjacent image is lower than a predetermined value, that is, an image having a large change from the adjacent image can be extracted. Therefore, the display mode of the icon M20 is changed according to the similarity of the image extracted as the specific image.

  As an example, when the color tone of the icon M20 is changed, the icon M20 is displayed in, for example, light red as the similarity is higher (the change from the adjacent image is smaller), and as the similarity is lower (change from the adjacent image). For example, the icon M20 is displayed in dark red.

  As another example, when changing the brightness of the icon M20, the higher the similarity (the smaller the change from the adjacent image), the lower the brightness of the icon M20, and the lower the similarity (the change from the adjacent image). The brightness of the icon M20 is increased.

  As another example, when the icon M20 is blinked, the blinking cycle of the icon M20 is lengthened as the similarity is high, and the blinking cycle of the icon M20 is shortened as the similarity is low.

  As another example, when the size of the icon M20 is changed, the size of the icon M20 is decreased as the similarity is higher, and the size of the icon M20 is increased as the similarity is lower.

As another example, when the display position of the icon M20 is changed, the icon M20 is moved away from the image display area M10 as the similarity is higher, and the icon M20 is moved closer to the image display area M10 as the similarity is lower.
Furthermore, the display modes of the icon M20 described above may be combined with each other.

(Modification 1-3)
The display mode of the icon M20 may be changed according to the type of lesion detected in the specific image.
As an example, when changing the color tone of the icon M20, for example, the icon M20 is displayed in green for a specific image in which an ulcer is detected, and the icon M20 is displayed in red for a specific image in which bleeding is detected. And the display aspect of displaying the icon M20 in yellow with respect to the specific image from which the tumor was detected is mentioned.

(Modification 1-4)
The display mode of the icon M20 may be changed according to the degree of the lesion detected in the specific image. As the degree of the lesion, for example, the area of the ulcer or bleeding region, the size of the tumor, and the like are determined.

  As an example, when the color tone of the icon M20 is changed, for example, the icon M20 is displayed in a light red color for a specific image having a small area of ulcer or bleeding or the size of the tumor, and The icon M20 is displayed in dark red for a specific image having a large size.

  Alternatively, the brightness of the icon M20, the blinking cycle when the icon M20 is blinked, the size of the icon M20, and the display of the icon M20 according to the area of the bleeding ulcer, the area of the bleeding, the size of the tumor, etc. The position may be changed.

(Modification 1-5)
The display mode of the icon M20 may be changed according to the position of a lesion or the like detected in the specific image. For example, when a lesion is detected from the lower left area obtained by dividing the specific image into four, as shown in FIG. 8, the icon M20 is arranged in the lower left area in the vicinity of the image display area M10.

  As described above, according to the modified examples 1-1 to 1-5 of the first embodiment, information indicating that a specific image exists is displayed according to the characteristics of the specific image included in the determination target range. Therefore, the user can intuitively grasp information such as when the specific image is displayed or what specific image is displayed. Therefore, the user can prepare for the display of the specific image without feeling an excessive burden, and can perform efficient observation.

  When the user is gazing at the image display area M10, the change in the peripheral visual field is generally more sensitive to the change in brightness than the change in color tone. Therefore, when changing the color tone of the icon M20, it is preferable to combine with a display element other than the color tone (for example, brightness, blinking cycle, size).

(Embodiment 2)
Next, a second embodiment of the present invention will be described.
FIG. 9 is a block diagram showing a configuration of an image processing apparatus according to Embodiment 2 of the present invention. As shown in FIG. 9, the image processing apparatus 6 according to the second embodiment includes a control unit 61 instead of the control unit 16 shown in FIG. The configuration and operation of each unit of the image processing apparatus 6 other than the control unit 61 are the same as those in the first embodiment.

  The control unit 61 further includes a skip unit 611 with respect to the control unit 16 illustrated in FIG. When a signal for selecting a predetermined icon displayed on the screen of the display unit 15 is input from the input unit 11, the skip unit 611 controls to skip the display order of images displayed on the display unit 15 to a specific image. I do. The operation of each unit in the control unit 61 other than the skip unit 611 is the same as that in the first embodiment.

FIG. 10 is a flowchart showing the operation of the image processing apparatus 6. In addition, operation | movement of step S10-S17 shown in FIG. 10 is the same as that of Embodiment 1 (refer FIG. 4).
In step S16, when the specific image is included in the determination target range (step S16: Yes), the display data generation unit 163 generates display data representing information indicating that the specific image exists (step S21). In the second embodiment, as the display data, data representing an icon whose display mode is changed according to the characteristics of the specific image included in the determination target range, and an instruction to skip the display target image to the specific image. Data representing an input icon used when inputting is generated.

In the subsequent step S22, the display control unit 161 creates a screen including the image I _j (I _m in FIG. 3), and displays information indicating that the specific image exists based on the display data generated in step S21. They are arranged on the screen and displayed on the display unit 15.

Figure 11 is a schematic diagram showing a display example of a screen information indicating that the specific image exists for a screen including the image I _m is arranged. Screen M3 shown in FIG. 11 further includes an icon M20 and a skip button M30 as an input icon with respect to screen M1 shown in FIG. Among these, the display mode of the icon M20 is as described in the first embodiment (see FIG. 6).

  On the other hand, the skip button M30 is an icon used by the user when the specific image on which information indicating that the specific image exists is displayed immediately. When a predetermined pointer operation (for example, a click operation) is performed on the skip button M30 using the input unit 11, an instruction signal for skipping an image to be displayed is input to the control unit 61.

  When the instruction signal for skipping the display target image is input (step S23: Yes), the skip unit 611 sets the image number of the display target image to the image number of the specific image (in the case of FIG. 3, j = n). (Step S24). Thereafter, the operation of the image processing apparatus 6 returns to step S15.

FIG. 12 is a schematic diagram showing a screen displayed on the display unit 15 after the pointer operation is performed on the skip button M30. As shown in FIG. 12, in the image display area M10 screen M4, subsequent to the image I _m, which had been previously displayed, an image I _n extracted as the specific image is displayed.

  On the other hand, when the signal for skipping the image to be displayed is not input (step S23: No), the operation of the image processing device 6 proceeds to step S19. The operations in steps S19 and S20 are the same as those in the first embodiment.

  As described above, according to the second embodiment, when the specific image is included in the range until the display order is after the predetermined number of images or the display time is after the predetermined time, the specific display is skipped. Images can be displayed immediately. Therefore, when the user is concerned about the specific image, the user can immediately display and confirm the specific image, so that efficient observation can be performed.

(Embodiment 3)
Next, a third embodiment of the present invention will be described.
In the third embodiment, in the first embodiment, the information indicating that the specific image exists is displayed by the icon, but the display mode of the information indicating that the specific image exists is not limited to the icon. For example, a display frame may be provided around the image to indicate that the specific image exists. Hereinafter, a display mode of information indicating that a specific image exists in Embodiment 3 will be described. Note that the configuration and operation of the image processing apparatus according to the third embodiment are the same as those of the first embodiment as a whole (see FIGS. 2 and 4).

  In the third embodiment, when the specific image is included in the range until the display order is after the predetermined number of images or the display time is after the predetermined time with respect to the display target image, the display data generation unit 163 includes the specific image. Display data representing a display frame surrounding the image display area M10 (see FIG. 5) is created as display data representing the information indicating that it is to be performed (see step S18 in FIG. 4).

  In this case, the display control unit 161 further displays, on the screen including the display target image, information indicating that the specific image exists based on the display data representing the display frame generated in step S18. The image is displayed in a mode corresponding to the characteristics of the specific image included in the range (see step S17 in FIG. 4).

FIG. 13 is a schematic diagram illustrating a display example of a screen on which information indicating that a specific image exists is arranged on a screen including an image to be displayed. Screen M5 shown in FIG. 13 further includes a display frame M50 as information indicating that a specific image exists, compared to screen M1 shown in FIG. The image I _m of the display target, when the display order or display time until after the predetermined number of sheets contain specific image range up after a predetermined time, by displaying such display frame M50, user, It is possible to prepare for a sudden change in information obtained from an image without feeling an excessive burden.

  In the third embodiment, the display frame M50 is provided only when the specific image exists. However, the display frame M50 may be provided even when the specific image does not exist. In this case, the user may be made aware that the specific image exists by changing the color of the display frame M50 depending on whether the specific image exists or not.

  Further, the display mode of the display frame M50 displayed as information indicating that the specific image exists in the third embodiment may be changed according to the characteristics of the specific image included in the determination target range. Hereinafter, a modification in which the display mode of the display frame M50 is changed according to the characteristics of the specific image will be described.

(Modification 3-1)
The display mode of the display frame M50 may be changed according to the number of images and the time until the specific image is displayed.
As an example, when the color tone of the display frame M50 is changed, when the number of sheets is large or the time is long, the display frame M50 is displayed in, for example, light red, and the display frame M50 is displayed as the number of sheets or time decreases. Gradually change to dark red.

  As another example, when changing the brightness of the display frame M50, when the number of sheets is large or when the time is long, the brightness of the display frame M50 is lowered and the brightness of the display frame M50 is decreased as the number of sheets or time decreases. Increase gradually.

  As another example, when the display frame M50 is displayed in a blinking manner, when the number of sheets is large or the time is long, the blinking cycle of the display frame M50 is lengthened, and as the number or time of the display frame M50 decreases, Gradually shorten the blinking cycle.

As another example, when the width of the display frame M50 is changed, when the number of sheets is large or the time is long, the width of the display frame M50 is narrowed, and as the number of sheets or time decreases, as shown in FIG. In this way, the width of the display frame M50 is gradually increased.
Furthermore, the display modes of the display frame M50 described above may be combined with each other.

(Modification 3-2)
The display mode of the display frame M50 may be changed according to the similarity of the specific image.
As an example, when the color tone of the display frame M50 is changed, the higher the similarity is (the smaller the change from the adjacent image is), for example, the display frame M50 is displayed in light red, and the lower the similarity is (the adjacent image is The display frame M50 is displayed in, for example, dark red.

  As another example, when the brightness of the display frame M50 is changed, the brightness of the display frame M50 is decreased as the similarity is higher, and the brightness of the display frame M50 is increased as the similarity is lower.

  As yet another example, when blinking the display frame M50, the higher the similarity is, the longer the blinking cycle of the display frame M50 is, and the lower the similarity is, the shorter the blinking cycle of the display frame M50 is.

As another example, when the width of the display frame M50 is changed, the width of the display frame M50 is narrowed as the similarity is high, and the width of the display frame M50 is widened as the similarity is low.
Furthermore, the display modes of the display frame M50 described above may be combined with each other.

(Modification 3-3)
The display mode of the display frame M50 may be changed according to the type of lesion detected in the specific image.
As an example, when changing the color tone of the display frame M50, for example, the display frame M50 is displayed in green for a specific image in which an ulcer is detected, and the display frame M50 is displayed for a specific image in which bleeding is detected. A display mode in which the display frame M50 is displayed in yellow for a specific image that is displayed in red and in which a tumor is detected can be mentioned.

(Modification 3-4)
The display mode of the display frame M50 may be changed according to the degree of the lesion detected in the specific image. As the degree of the lesion, for example, the area of the ulcer or bleeding region, the size of the tumor, and the like are determined.

  As an example, when the color tone of the display frame M50 is changed, for example, the display frame M50 is displayed in a light red color for a specific image with a small area of ulcer or bleeding or tumor size, and the area or tumor of the ulcer or bleeding is displayed. For a specific image having a large size, the display frame M50 is displayed in dark red.

  Alternatively, the brightness of the display frame M50, the blinking cycle when the display frame M50 is blinked, the size of the display frame M50, the display, etc., according to the area of the bleeding ulcer, bleeding area, tumor size, etc. The display position of the frame M50 may be changed.

(Modification 3-5)
The display mode of the display frame as information indicating that the specific image exists may be changed according to the position of a lesion or the like detected in the specific image. 14 and 15 are schematic diagrams illustrating examples of display modes of display frames in Modification 3-5.

  In this case, the image specifying unit 143 specifies an existing area such as a lesion in the specific image, and adds a flag representing the existing area to the image data of the specific image. The display control unit 161 determines a position where the display frame is displayed on the screen based on a flag indicating the presence area added to the image data. Note that the presence area of a lesion or the like may be roughly defined as an upper half area of the image and an upper right quarter area.

  For example, when a lesion is detected from the upper right area obtained by dividing the specific image into four parts, the partial display frame M51 is changed to the upper right 1/4 area around the image display area M10 as shown in a screen M6 in FIG. Deploy. When a lesion is detected from the upper half area of the specific image, a partial display frame M52 is arranged in the upper half area around the image display area M10 as shown in a screen M7 in FIG.

(Modification 3-6)
FIG. 16 is a schematic diagram illustrating an example of a display mode of information indicating that a specific image exists in Modification 3-6. As in the screen M8 shown in FIG. 16, the skip button M30 described in the second embodiment may be displayed together with the display frame M50 as information indicating that the specific image exists.

  As described above, according to the modified examples 3-1 to 3-6 of the third embodiment, the information indicating that the specific image exists is displayed according to the characteristics of the specific image included in the determination target range. Therefore, the user can intuitively grasp information such as when the specific image is displayed or what specific image is displayed. Therefore, the user can prepare for the display of the specific image without feeling an excessive burden, and can perform efficient observation.

  In the third embodiment and its modifications 3-1 to 3-6, the display frames M50, M51, and M52 are displayed in contact with the outer periphery of the image display area M10. However, the image display area M10 and the display frame are displayed. You may provide a clearance gap between 50, 51, and 52. Even in this case, the user can recognize the display / non-display of the display frames M50, 51, and 52 and the change in the display mode while gazing at the image display area M10, so long as the user does not interfere with the image observation. The display frames 50, 51, and 52 are preferably arranged as close to the image display area M10 as possible.

  Further, as described above, when the user is gazing at the image display region M10, the change in the peripheral visual field is generally more sensitive to the change in brightness than the change in color, so the user changes the color tone of the display frame M50. In this case, it is preferable to combine with display elements other than the color tone (for example, brightness, blinking cycle, size).

(Embodiment 4)
Next, a fourth embodiment of the present invention will be described.
In the fourth embodiment, the specific image itself is displayed instead of the icon as information indicating that the specific image exists. The configuration and operation of the image processing apparatus according to the fourth embodiment are the same as those of the first embodiment as a whole.

  In the fourth embodiment, when the specific image is included in the determination target range with respect to the display target image, the display data generation unit 163 displays the image of the specific image as display data indicating information indicating that the specific image exists. Data is acquired from the storage unit 13 (see step S18 in FIG. 4).

  In this case, the display control unit 161 further applies the specific image itself included in the determination target range to the screen including the display target image based on the image data acquired in step S18. Is displayed in a manner corresponding to (see step S17 in FIG. 4).

FIG. 17 is a schematic diagram illustrating a display example of a screen on which information indicating that a specific image exists is arranged on a screen including an image to be displayed. A screen M9 shown in FIG. 17 includes a specific image display area M90 as information indicating that a specific image exists, compared to the screen M1 shown in FIG. The Specific image display area M90, the image I _n, which is determined that the flag indicating that a specific image is added in step S16 is disposed.

The position of the specific image display area M90 may be changed according to the number of images and the time until the specific image is displayed. For example, as該枚number or time decreases, as shown in FIG. 18, gradually while enlarging the specific image display area M90, by approximating the image display area M10 gradually, specific image (image I _n) is far You may display so that it may approach from. Thereby, the user can intuitively grasp the timing at which the specific image is displayed.

(Modification 4)
Next, a fourth modification of the first to fourth embodiments will be described.
In the first to fourth embodiments, the display target image I _m (see FIG. 3) is specified in a range (determination target range) from the display order after d images or the display time after T _d time. The case where only one image (image I _n ) is included has been described. However, a plurality of specific images may be included in the determination target range. In such a case, the determination unit 162 may process a plurality of specific images as one specific image group.

For example, as shown in FIG. 19, when a plurality of specific image which is continuous from the scope of the determination target (image I _n1 ~I _n4) is detected, the first image determined as the specific image from the image I _m to be displayed Depending on the number of images up to In ₁ or the time, an indication that a specific image exists (icon M20, display frame M50, etc.) is performed.

  As described above, according to the fourth modification of the first to fourth embodiments, the user can check a plurality of display data at once. Therefore, the user can easily grasp the information that the specific image exists, and thus can perform efficient observation.

  Furthermore, you may combine suitably the said Embodiment 1-4 and the modification of each embodiment.

For example, when changing the display mode of the information indicating that the specific image exists according to the type and degree of the lesion detected from the specific image, the type of lesion detected in the first image In ₁ determined as the specific image it may be changed the display mode such as icons M20 and display frame M50 based on the degree and, changing these display modes on the basis of the average of the detected lesion type and extent of the image I _n1 ~I _n4 May be.

  In addition, when a plurality of image groups including a plurality of specific images that are consecutive from the determination target range are detected, only one icon corresponding to the first image group may be displayed as an indication that the specific image exists. Alternatively, a plurality of icons corresponding to a plurality of image groups may be displayed.

(Embodiment 5)
Next, a fifth embodiment of the present invention will be described.
FIG. 20 is a block diagram showing a configuration of an image processing apparatus according to Embodiment 5 of the present invention. As shown in FIG. 20, the image processing apparatus 7 according to the fifth embodiment includes a control unit 71 instead of the control unit 16 illustrated in FIG. 2, and further includes a notification unit 72. The configuration and operation of each part of the image processing apparatus 7 other than the control unit 71 and the notification unit 72 are the same as those in the first embodiment.

The control unit 71 further includes a notification control unit 711 with respect to the control unit 16 illustrated in FIG. When the determination unit 162 determines that the specific image is included in the determination target range, the notification control unit 711 indicates that the specific image exists by means that can be perceived by hearing such as voice or notification sound. Control to notify.
The notification unit 72 is a sound generating unit that generates, for example, a voice or a notification sound.

The notification that the specific image exists is performed in the following manner as a specific example.
(1) When a specific image exists, a notification sound is generated.
(2) A notification sound is generated when a specific image exists, and the notification sound is generated as the number of images until the specific image is displayed decreases or as the time until the specific image is displayed decreases. Increase the volume gradually or gradually increase the pitch. Alternatively, the notification sound may be generated intermittently and the generation cycle may be gradually shortened.
(3) A notification sound is generated when a specific image exists, and the volume, pitch, or generation cycle of the notification sound is changed according to the similarity between the specific image and an adjacent image.
(4) A notification sound is generated when a specific image exists, and the type of notification sound is changed according to the type of lesion detected in the specific image.
(5) A notification sound is generated when a specific image exists, and the volume or pitch of the notification sound is changed according to the degree of a lesion detected in the specific image.
(6) When a specific image exists, a message indicating that the specific image exists and the number of images or time until the specific image is displayed are read out by voice.
(7) When a specific image exists, a message to the effect that the specific image exists and characteristics of the specific image (similarity and detected lesion type and degree) are read out by voice.

  Furthermore, these modes (1) to (7) may be appropriately combined. Further, these modes (1) to (7) may be combined with the visual (screen display) notification described in the first to fourth embodiments.

  According to the fifth embodiment, the user can grasp that the specific image exists by voice or notification sound while gazing at the image displayed on the screen.

(Modification 5)
Next, Modification 5 of Embodiment 5 of the present invention will be described.
When the determination unit 162 determines that the specific image is included in the determination target range, the notification control unit 711 and the notification unit 72 illustrated in FIG. The user may be notified. Examples of means that can be perceived by tactile sensation include vibration and temperature change. In this case, the notification unit 72 includes, for example, a vibrator that generates vibration, a heater that changes temperature, and the like.

The notification that the specific image exists is performed in the following manner as a specific example.
(1) When a specific image exists, a device such as a mouse used as the input unit 11 of the image processing apparatus 7 or a member such as a chair on which a user sits is vibrated.
(2) When a specific image exists, the members such as the mouse and the chair are vibrated, and the vibration amount (strength) and the period are changed according to the number of images and time until the specific image is displayed.
(3) When the specific image exists, the temperature of the member such as the mouse or the chair is changed.
(4) When a specific image exists, the shape of the member such as the mouse or the chair is changed. Specifically, when a specific image exists, there is a mode in which a protruding member is protruded from the mouse, or the inclination or hardness of the seating surface or backrest of the chair is changed.

  Furthermore, you may combine these aspects (1)-(5) suitably. These modes (1) to (5) may be combined with the visual (screen display) notification described in the first to fourth embodiments or the auditory notification described in the fifth embodiment.

  The present invention described above is not limited to Embodiments 1 to 5 and modifications thereof, and can be variously modified according to specifications and the like. For example, Embodiments 1 to 5 and these can be modified. Alternatively, some components may be excluded from all the components shown in the modified example. It is apparent from the above description that various other embodiments are possible within the scope of the present invention.

DESCRIPTION OF SYMBOLS 1, 6, 7 Image processing apparatus 2 Capsule endoscope 3 Receiving apparatus 3a Cradle 4 Receiving antenna unit 4a-4h Receiving antenna 5 Subject 11 Input part 12 Image data acquisition part 13 Storage part 14 Calculation part 15 Display part 16, 61, 71 Control unit 72 Notification unit 141 Image processing unit 142 Position calculation unit 143 Image identification unit 161 Display control unit 162 Determination unit 163 Display data generation unit 611 Skip unit 711 Notification control unit

Claims (5)

A storage unit for storing the image group;
From the images, and to identify the specific image having a predetermined feature, further image specifying unit for specifying a region in which the predetermined characteristic of the specific image exists,
A display control unit when acquired in reverse sequence order or chronological, and displays the image on the obtained sequence on the screen of Table radical 113 an image included in the image group from the storage unit,
By the image specifying unit, a range from a display target image displayed on the display unit by the display control unit to an image after a predetermined number of displays or a display time after a predetermined time with respect to the display target image. a determination section for determining whether or not include the specific image specified,
A display data generation unit that generates display data representing information indicating that the specific image exists when it is determined that the specific image is included in the range;
With
The image processing apparatus, wherein the display control unit arranges the display data at a position of the screen corresponding to a region where the predetermined feature exists in the specific image included in the range.   The characteristics of the specific image included in the range include the number of images from the display target image to the specific image or the display time, the type of a predetermined feature included in the specific image, or a predetermined characteristic included in the specific image. The image processing apparatus according to claim 1, wherein the image processing apparatus has a feature amount. The image processing apparatus according to claim 1 , wherein the display data is a partial display frame provided in a part of the periphery of the image to be displayed. The display data is an input icon arranged on the screen,
An input unit for inputting an external operation;
A skip unit that causes an image to be displayed on the display unit to be skipped to the specific image when an operation for selecting the icon for input on the screen is input from the input unit;
The image processing apparatus according to claim 1 , further comprising: The image group includes images acquired by sequentially imaging the inside of the subject by a capsule endoscope introduced into the subject,
The specific image is an image in which a predetermined feature amount in the image is within a predetermined range, an image in which a predetermined feature is detected, an image in which the similarity with an adjacent image is a predetermined value or less, a specific position, The image processing apparatus according to claim 1 , wherein the image processing apparatus is an image in which an area is captured, or an image in which a change in movement of the capsule endoscope when the image is captured is a predetermined value or more.

Images