JP5973770B2 - Image processing apparatus, program, and image processing method - Google Patents

Description

  The present invention relates to an image processing apparatus, a program, an image processing method, and the like.

  When still image capturing is continued in time series at a given time interval, when a subject having a spatial spread is covered by a large number of images, or after each moving image is captured, each moving image is configured. When an image is captured as a still image, a very large amount of images (hereinafter also referred to as image sequences) that are temporally or spatially continuous are acquired. In such a case, there is a high possibility that images that are close in the image sequence (that is, close in time or space) are similar images. The need to check everything is not high. In the first place, it is not uncommon for the number of images to be tens of thousands or more, and checking everything with the user's hand itself is a heavy burden.

  Therefore, there is a demand for deleting a part of the image from the image sequence and summarizing it into an image sequence having a smaller number than the original image sequence (hereinafter, this process is referred to as an image summarization process). For example, Patent Document 1 discloses an image summarization processing method that leaves an image that makes it easy to grasp the contents of an image sequence by extracting an image of a boundary where the scene in the image sequence changes or an image representative of the image sequence. Has been.

JP 2009-5020 A JP 2011-24763 A

  For example, when an image summarization technique is applied to the medical field, it is necessary to suppress the occurrence of an area that cannot be observed by deleting an image from the viewpoint of avoiding oversight of a disease. In particular, an important area such as a lesion area or an abnormal area should not be included in an area where observation is impossible.

  However, if only the image of the boundary where the scene changes is left as in the method of Patent Document 1 or if image summarization is performed from the viewpoint of whether the image sequence after summarization is intuitively easy to see, the image is deleted. This may cause an area that cannot be observed, which is not preferable. Further, since the degree of occurrence of the region that cannot be observed depends on the contents of the image, it is difficult to control the degree of oversight of the disease by the conventional image summarization processing method.

  According to some aspects of the present invention, when image summarization processing is performed by deleting some images, an image processing device, a program, and an image that suppress the occurrence of a region of interest that cannot be observed due to image deletion A processing method or the like can be provided.

  According to an aspect of the present invention, an image sequence acquisition unit that acquires an image sequence having a plurality of images, and a summary image sequence by deleting a part of the plurality of images of the image sequence acquired by the image sequence acquisition unit. A processing unit that performs image summarization processing to be acquired, wherein the processing unit selects a reference image and a determination target image from the plurality of images, and uses deformation information between the reference image and the determination target image. And an image processing apparatus that determines whether or not the determination target image can be deleted based on the result of the processing performed and the result of the processing using the structural element corresponding to the attention area.

  In one aspect of the present invention, whether or not to delete a determination target image is determined based on processing using deformation information and processing using a structural element. Therefore, by associating the structural element with the attention area, which is an area in which the possibility of oversight is to be suppressed, image summarization processing according to the imaging state of the attention area in the reference image and the determination target image can be performed.

  In the aspect of the invention, the process using the deformation information is a process of deforming at least a part of one of the reference image and the determination target image using the deformation information, and the structural element is The process used may be a contraction process using the structural element or a process for determining whether or not the structural element is included in an uncovered area that is an area where the determination target image is not covered by the reference image. .

  As a result, the deformation process can be performed on at least a part of the reference image or the determination target image as a process using the deformation information, and the contraction process or the structure element can be applied to the uncovered area as the process using the structural element. Can be determined.

  In the aspect of the invention, the processing unit may perform processing using the deformation information by deforming the reference image using the deformation information, and in an area where the determination target image is covered by the reference image. As a process using the structural element, a contraction process is performed on the non-covered area as a process using the structural element. May be performed.

  As a result, it is possible to determine the non-covering area and determine whether or not to delete based on the shrinkage process for the determined non-covering area.

  In the aspect of the invention, the processing unit may determine that the determination target image cannot be deleted when there is a remaining region as a result of the contraction processing.

  As a result, it is possible to determine whether or not deletion is possible according to the presence or absence of a residual area by the shrinking process.

  Moreover, in one aspect of the present invention, the processing unit may perform a process of obtaining a covering area by performing a contraction process using the structural element on the determination target image as a process using the structural element. .

  As a result, it is possible to determine whether or not deletion is possible based on the required coverage area obtained from the contraction process on the determination target image.

  Moreover, in one aspect of the present invention, the processing unit performs a process of deforming the covering area based on the deformation information between the reference image and the determination target image as the process using the deformation information. The processing unit may determine that the determination target image can be deleted when the cover area after deformation is included in the reference image.

  Thereby, it becomes possible to determine the inclusion relationship between the reference image and the required coverage area by deforming the required coverage area.

  In the aspect of the invention, the processing unit performs a process of deforming the reference image based on the deformation information between the reference image and the determination target image as the process using the deformation information. The processing unit may determine that the determination target image can be deleted when the required coverage area is included in the deformed reference image.

  As a result, by deforming the reference image, it is possible to determine the inclusion relationship between the reference image and the area requiring coverage.

  In the aspect of the invention, the processing unit may perform processing using the deformation information by deforming the reference image using the deformation information, and in an area where the determination target image is covered by the reference image. A certain covering area is obtained, and at each point of the plurality of points set on the obtained boundary of the covering area, a shortest distance to the outer periphery of the determination target image is obtained, and is obtained at each of the plurality of points. In addition, a process for obtaining a maximum value among the shortest distances is performed, and as a process using the structural element, a minimum diameter of the structural element is acquired, and the acquired minimum diameter and the maximum value among the shortest distances are acquired. Based on the comparison process, a process for determining whether or not the structural element is included in a non-covered area that is an area other than the covered area in the determination target image may be performed.

  As a result, it is possible to easily determine the possibility of oversight of a region of interest without shrinking processing.

  In the aspect of the invention, the processing unit may perform processing using the deformation information by deforming the reference image using the deformation information, and in an area where the determination target image is covered by the reference image. While obtaining a certain covering region, at each point of a plurality of points set on the outer periphery of the determination target image, the shortest distance to the boundary of the covering region is obtained, and the above-mentioned obtained at each point of the plurality of points A process of obtaining a maximum value of the shortest distance is performed, and as a process using the structural element, a minimum diameter of the structural element is acquired, and the acquired minimum diameter and the maximum value of the shortest distance Based on the comparison process, a process of determining whether or not the structural element is included in an uncovered area that is an area other than the covered area in the determination target image may be performed.

  As a result, it is possible to easily determine the possibility of oversight of a region of interest without shrinking processing.

  In one aspect of the present invention, the processing unit calculates a coverage of the determination target image by the reference image based on the deformation information between the reference image and the determination target image, and calculates the calculated Based on the results of the first deleteability determination process for determining whether or not the determination target image can be deleted based on the coverage, the process using the deformation information, and the process using the structural element, the determination target A second deleteability determination process for determining whether or not an image can be deleted may be performed.

  As a result, in addition to the determination as to whether or not deletion is possible, it is possible to make a determination based on the coverage ratio, so that it is possible to improve the accuracy of the determination as to whether or not deletion is possible.

  In the aspect of the invention, the processing unit may be arranged adjacent to the reference image, the determination target image, and the image between the reference image and the determination target image in the image sequence among the plurality of images. The deformation information between the matching images may be obtained, and the deformation information between the reference image and the determination target image may be obtained based on the obtained deformation information between the obtained adjacent images.

  Thereby, the deformation information between the reference image and the determination target image can be calculated based on the deformation information between adjacent images.

  In the aspect of the invention, the processing unit may set the structural element having a size proportional to the size of the region of interest.

  Thereby, the size of the structural element can be associated with the size of the attention area.

  In one aspect of the present invention, when the first to Nth images (N is an integer of 2 or more) are input as an input image sequence, the processing unit is configured to output the pth (p is 1 ≦ p ≦ N). Image) is selected as the first reference image, the qth image (q is an integer equal to or greater than p + 2) is selected as the second reference image, and the rth image (where r is p + 1 ≦ r ≦ q−). (An integer satisfying 1) is selected as the determination target image, and processing using the deformation information between the first reference image and the determination target image, the second reference image and the determination target image Whether or not the determination target image is to be deleted may be determined based on the results of the process using the deformation information and the process using the structural element.

  This makes it possible to set a reference image in front of and behind the determination target image.

  In one aspect of the present invention, the processing unit selects the second reference image from a second reference image selection section in which a start point and an end point corresponding to the p + 2 to Nth images are set, It is determined whether or not the determination target image can be deleted based on the first reference image and the second reference image, and when it is determined that the p + 1 to q−1th images can be deleted, the first reference image and the second reference image are deleted. The xth image (x is an integer satisfying x> q) included in the second reference image selection section is selected as a new second reference image, and the start point of the second reference image selection section is selected as the second reference image selection section. You may update to the qth image.

  As a result, it is possible to set the second reference image selection section, update the second reference image to the back image, update the start point of the selection section, and the like based on the determination result of the deletion possibility. Become.

  In one aspect of the present invention, the processing unit is included in the second reference image selection section when it is determined that at least one of the p + 1 to q-1th images cannot be deleted. The y-th image (y is an integer satisfying y <q) is selected as the new second reference image, and the end point of the second reference image selection section is updated to the q-th image. Good.

  As a result, the second reference image can be updated to the front image, and the end point of the selected section can be updated.

  In the aspect of the invention, the processing unit may update the first reference image when the start point and the end point are adjacent to each other as a result of updating the start point or the end point of the second reference image selection section. One of the plurality of images selected as the image is included in the summary image sequence, and one of the plurality of images corresponding to the start point and one of the plurality of images corresponds to the start point. A partial image sequence consisting of a rear one in the input image sequence rather than one of the plurality of images is set as the input image sequence, and the value of p is set to 1 for the set input image sequence Then, the process may be performed again.

  Accordingly, when the start point and the end point of the second reference image selection section are adjacent to each other, it is possible to end the processing for the input image sequence and set the partial image sequence.

  In the aspect of the invention, the processing unit may select the q + 1th image as the second reference image when it is determined that the p + 1st to q−1th images can be deleted. Good.

  As a result, it is possible to update the second reference image to an image one backward.

  In one aspect of the present invention, the processing unit is selected as the first reference image when it is determined that at least one of the p + 1 to q−1 images is not erasable. A process of including one of a plurality of images in the summary image sequence, a partial image sequence composed of the q-1st to Nth images is set as the input image sequence, and the set input image sequence On the other hand, the value of p may be set to 1 and the process may be performed again.

  As a result, the first reference image can be left in the summary image sequence, an appropriate partial image sequence can be set as a new input image sequence, and the first reference image and the second reference image can be updated. Therefore, more efficient image summarization processing becomes possible.

  According to another aspect of the present invention, an image sequence acquisition unit that acquires an image sequence having a plurality of images, and a part of the plurality of images in the image sequence acquired by the image sequence acquisition unit are deleted and summarized. A processing unit that performs an image summarization process for obtaining an image sequence, wherein the processing unit selects a reference image and a determination target image from the plurality of images, and deforms between the reference image and the determination target image Based on the information, obtaining a covered area that is the area covered with the determination target image by the reference image, and determining, based on the covered area, the possibility of missing the attention area that is the area to be noted, The present invention relates to an image processing apparatus that determines whether or not a determination target image can be deleted.

  Another aspect of the present invention relates to a program that causes a computer to function as each of the above-described units.

  In another aspect of the present invention, an image sequence having a plurality of images is acquired, a reference image and a determination target image are selected from the plurality of images in the image sequence, and the reference image and the determination target image are selected. Based on the result of the process using the deformation information between, and the result of the process using the structural element corresponding to the region of interest, it is determined whether the determination target image can be deleted, and based on the determination result of the deletion permission, The present invention relates to an image processing method for performing image summarization processing for obtaining a summary image sequence by deleting a part of the plurality of images in the image sequence.

  In another aspect of the present invention, an image sequence having a plurality of images is acquired, a reference image and a determination target image are selected from the plurality of images in the image sequence, and the reference image and the determination target image are selected. Based on the deformation information in between, a cover area that is an area where the determination target image is covered by the reference image is obtained, and based on the cover area, a possibility of missing an attention area that is an attention area is determined. The image to be subjected to image summarization processing for determining whether or not the determination target image can be deleted, and acquiring a summary image sequence by deleting a part of the plurality of images in the image sequence based on the determination result of the deletion availability Related to processing method.

The example of calculation of the covering area based on deformation information. FIGS. 2A to 2C are views for explaining the relationship between the shape of the uncovered area and the possibility of overlooking the attention area. 1 is a system configuration example of an image processing apparatus according to the present embodiment. 1 is a system configuration example of an image processing apparatus according to a first embodiment. The flowchart for demonstrating the process of 1st Embodiment. FIG. 6A to FIG. 6E are diagrams for explaining a contraction process by a structural element for an uncovered region. FIG. 7A to FIG. 7D are diagrams for describing the reference image and determination target image selection processing according to the first embodiment. 9 is a system configuration example of an image processing apparatus according to a second embodiment. The flowchart for demonstrating the process of 2nd Embodiment. The figure explaining the process which calculates | requires a covering area | region based on a some reference | standard image. FIG. 11A and FIG. 11B are diagrams for explaining the reference image and determination target image selection processing according to the second embodiment. FIGS. 12A to 12D are diagrams for explaining an example of a method for selecting a first reference image in the second embodiment. FIG. 13A to FIG. 13G are diagrams illustrating a modification of the second embodiment. The flowchart for demonstrating the process of the modification of 2nd Embodiment. 9 is a system configuration example of an image processing apparatus according to a third embodiment. FIG. 16A to FIG. 16C are diagrams for explaining contraction processing by a structural element for a determination target image. FIG. 17A and FIG. 17B are examples of determination of inclusion of the reference image and the area requiring coverage. The flowchart for demonstrating the process of 3rd Embodiment. FIG. 19A to FIG. 19C are views for explaining processing using the structural elements of the fourth embodiment. 10 is a system configuration example of an image processing apparatus according to a fifth embodiment. The other example of the method of calculating | requiring a coverage.

  Hereinafter, this embodiment will be described. In addition, this embodiment demonstrated below does not unduly limit the content of this invention described in the claim. In addition, all the configurations described in the present embodiment are not necessarily essential configuration requirements of the present invention.

1. First, the method of this embodiment will be described. When an image sequence composed of a large number of images that are temporally or spatially continuous is acquired, the user uses the image sequence to perform some processing (for example, medical practice such as diagnosis if an endoscopic image sequence). When performing, it is desirable to perform image summarization processing. This is because the number of images included in the image sequence is very large, and it is necessary for the user to make a determination after viewing all of the images. In addition, there is a high possibility that there are images that are similar to each other in the image sequence, and even if all such similar images are checked, the amount of information that can be obtained is limited and is not commensurate with the effort. .

  As a specific example, an image sequence captured using a capsule endoscope can be considered. The capsule endoscope is a capsule-shaped endoscope with a built-in small camera, and takes an image at a given time interval (for example, twice per second). Since the capsule endoscope requires several hours (in some cases, several tens of hours) from taking the clothes to discharging, tens of thousands of captured images are acquired in one examination by one user. In addition, the capsule endoscope stays in the same place or returns in the opposite direction due to the influence of the movement of the living body when moving in the living body. For this reason, in a large number of images, the same subject as that of other images is imaged, and there are many images that are not highly useful in finding lesions.

  In the conventional image summarization processing, an image at a boundary where the scene changes and an image representing an image sequence are extracted. However, with such a method, when deleting an image, the relationship between the subject captured in the image to be deleted and the subject captured in the remaining image is not particularly considered. For this reason, it is possible that the subject captured on the image included in the pre-summarization image sequence is not captured on any image included in the post-summary image sequence.

  This is not preferable particularly in image summarization processing in the medical field. In the medical field, for the purpose, oversight of a region of interest (for example, a lesion) that is a region of interest must be suppressed as much as possible. For this purpose, it is desirable to image the widest possible range in the living body, and in the image summarization process, it should be prevented that a subject range that cannot be observed by deleting a given image is generated.

  On the other hand, a reference image (an image to be left, an image to be a candidate to be left depending on the setting method of the reference image) and a determination target image (an image to be deleted or not) are selected from the image sequence, A technique for performing image summarization processing based on the coverage of the determination target image is effective. Specifically, as shown in FIG. 1, the coverage area is calculated on the determination target image by deforming the reference image. The subject imaged with the reference image corresponds to the subject imaged on the covering area of the determination target image. That is, the range outside the covered area (hereinafter referred to as a non-covered area) in the determination target image is an area that cannot be covered even if the reference image is left when the determination target image is deleted.

  Therefore, the ratio of the coverage area in the determination target image is calculated as the coverage ratio, and by determining whether to delete the determination target image based on the calculated coverage ratio, the degree of occurrence of the subject range that cannot be observed is determined. Control. For example, if the image to be judged is deleted when the coverage is equal to or higher than the threshold, and the image to be judged is not deleted when the coverage is less than the threshold, the degree of occurrence of an area that cannot be covered is controlled according to the threshold setting. it can.

  However, the coverage represented by the ratio or the like of the covered area in the determination target image does not consider the shape of the covered area (or non-covered area). For example, FIGS. 2A and 2B show the determination target image, and the area of the covering region is 75% of the total area of the determination target image (that is, the coverage is a value such as 0.75 or 75). become). However, even if the coverage is the same, if the size and shape of the region of interest is FIG. 2C, FIG. 2A and FIG. This is because the non-covered region in FIG. 2A does not include the entire region of interest in FIG. Therefore, regardless of the position of the attention area on the determination target image in FIG. 2A (however, the entire attention area is captured on the determination target image), at least the attention area on the reference image is displayed. A part of the image is captured. On the other hand, the uncovered region in FIG. 2B can include the entire region of interest in FIG. Therefore, when there is a region of interest at the position indicated by the dotted line in the determination target image in FIG. 2B, the region of interest is not captured at all on the reference image.

  In other words, from the viewpoint that at least a part of the attention area must be captured in the reference image regardless of the position of the attention area in the determination target image, the determination target image in FIG. In contrast, the determination target image in FIG. 2B cannot be deleted. However, as described above, since there is no difference in coverage, it is not possible to distinguish between FIG. 2A and FIG. 2B in the determination based on the coverage.

  Accordingly, the present applicant proposes a method of selecting a reference image and a determination target image and determining whether or not the determination target image can be deleted based on a processing result using a structural element corresponding to the region of interest. Specifically, as will be described later with reference to FIGS. 6A to 6E, the shrinkage process using the structural element is performed on the uncovered area to determine whether or not there is a remaining area. That's fine. When there is no remaining area, the entire area having a size corresponding to the attention area is not included in the non-covering area (for example, as shown in FIG. 2A), and the determination target image can be deleted. On the contrary, when there is a remaining area, the entire image having a size corresponding to the attention area may be included in the non-covering area as shown in FIG.

  In this way, even when the determination target image is deleted, it is guaranteed that at least a part of the attention area is imaged on the reference image (in some embodiments, 100% is not guaranteed). Image summarization processing becomes possible. In the method of this embodiment, it is necessary to set the shape and size of the structural element, but a shape corresponding to the region of interest may be used. The shape or the like of the region of interest varies depending on the imaging target and the situation of the image. For example, if the imaging is performed for medical purposes using a capsule endoscope, a minimum lesion size that should not be overlooked may be set. In this case, even when the determination target image is deleted, at least a part of a lesion larger than the lesion size set as the attention area is captured on the reference image, and the size is large ( In other words, it is possible to suppress the oversight of the lesion (which is assumed to be highly serious).

  Note that the structural element in the present embodiment is assumed to be information representing an image region used for the contraction process as described above, but is not limited to this. Like a structural element in the fourth embodiment to be described later, information that is set based on a region of interest and is used for determining whether or not a determination target image can be deleted is included widely. As in the first embodiment and the fourth embodiment, the attention area and the structural element may have the same shape and size, but even in such a case, the same shape and size as the attention area. It is considered that the setting process of the structural element is performed in the form of setting the area.

  As an embodiment of the image processing apparatus here, an apparatus including a processing unit 100 and an image sequence acquisition unit 200 as shown in FIG. 3 can be considered. The image sequence acquisition unit 200 acquires an image sequence having a plurality of images. Then, the processing unit 100 performs image summarization processing that acquires a summary image sequence by deleting some of the plurality of images included in the image sequence acquired by the image sequence acquisition unit 200. Specifically, the processing unit 100 selects a reference image and a determination target image from a plurality of images, performs processing using deformation information between the reference image and the determination target image, and a structural element corresponding to the attention area. Based on the result of the processing used, it is determined whether or not the determination target image can be deleted.

  Hereinafter, first, a basic method in the first embodiment will be described. However, in the method of the present embodiment, two processes are performed: a reference image and determination target image selection process, and a determination target image deletion permission determination process based on the selected reference image and determination target image. Various modifications can be considered for the processing. Therefore, in the second embodiment, another method for selecting the reference image and the determination target image and a modification example thereof will be described. Similarly, the deletion permission / prohibition determination process may be different from the first embodiment, and will be described in detail in the third to fifth embodiments.

  Note that the selection process of the reference image and the determination target image may use any of the methods of the first embodiment and the second embodiment (and modifications thereof), and the deletion feasibility determination process is also the first embodiment. Any method of the form and the third to fifth embodiments may be used. In addition, since there is no restriction on the combination of the two processes, modifications in various combinations such as combining the selection method of the second embodiment and the deletion possibility determination method of the third embodiment are possible.

2. First Embodiment A basic method of this embodiment will be described. Specifically, a system configuration example of the image processing apparatus will be described, and a processing flow will be described using a flowchart.

  FIG. 4 shows a system configuration example of the image processing apparatus in the present embodiment. The image processing apparatus includes a processing unit 100, an image sequence acquisition unit 200, and a storage unit 300.

  The processing unit 100 performs image summarization processing on the image sequence acquired by the image sequence acquisition unit 200 by deleting some of the plurality of images included in the image sequence. The function of the processing unit 100 can be realized by hardware such as various processors (CPU and the like), ASIC (gate array and the like), a program, and the like.

  The image sequence acquisition unit 200 acquires an image sequence that is an object of image summarization processing. In addition to storing the image sequence acquired by the image sequence acquisition unit 200, the storage unit 300 serves as a work area for the processing unit 100 and the like, and its function can be realized by a memory such as a RAM or an HDD (hard disk drive).

  Further, as illustrated in FIG. 1, the processing unit 100 includes a reference image selection unit 1001, a determination target image selection unit 1002, a coverage area calculation unit 1003, a deletion possibility determination unit 1005, and a partial image sequence setting unit 1008. The summary image sequence determination unit 1009, the attention area missing possibility determination unit 1013, the structural element generation unit 1014, and the deformation information acquisition unit 1015 may be included. Note that the processing unit 100 is not limited to the configuration of FIG. 4, and various modifications such as omitting some of these components or adding other components are possible. Each of the above-described units is set to explain each subroutine when the image summarization process executed by the processing unit 100 is divided into a plurality of subroutines, and the processing unit 100 does not necessarily include the above-described units. Does not have as.

  The reference image selection unit 1001 selects a reference image from a plurality of images in the image sequence. The determination target image selection unit 1002 selects an image different from the reference image from among the plurality of images in the image sequence as the determination target image.

  The covering area calculation unit 1003 uses the deformation information (deformation parameters) between the reference image and the determination target image to project the reference image onto the determination target image to obtain a covering area. In the determination target image, an area that is not a covered area may be set as a non-covered area.

  The deletion possibility determination unit 1005 determines whether or not the determination target image can be deleted based on the determination result in the attention area miss possibility determination unit 1013 described later.

  When the deletion permission determination unit 1005 determines that the determination target image cannot be deleted, the partial image sequence setting unit 1008 determines a part of the image sequence based on the position in the image sequence of the determination target image at that time. Thus, an image sequence composed of one or more images is set as a partial image sequence.

  The summary image sequence determination unit 1009 determines a summary image sequence that is an image sequence after the summary processing. In the present embodiment, the reference image selected by the reference image selection unit 1001 is included in the summary image sequence. Further, it is assumed that an image determined to be deleteable among the determination target images is deleted and not included in the summary image sequence.

  When the determination target image is deleted, the attention area missing possibility determination unit 1013 may be in a situation where the attention area captured on the determination target image is not captured in the reference image (that is, the attention area is missed). The determination process about sex is performed. Details will be described later.

  The structural element generation unit 1014 generates a structural element used for processing in the attention area missing possibility determination section 1013 based on the attention area. Here, an area having the same shape and size as the attention area is set, but the present invention is not limited to this.

  The deformation information acquisition unit 1015 acquires deformation information between two images. Here, the deformation information represents what shape (range) the range captured in one image is captured in the other image. For example, the deformation parameter disclosed in Patent Document 2 Etc. The deformation information acquisition unit 1015 acquires deformation information between the reference image selected by the reference image selection unit 1001 and the determination target image selected by the determination target image selection unit 1002.

  FIG. 5 shows a flowchart for explaining the image summarization processing of this embodiment. When this processing is started, first, an image sequence to be subjected to image summarization processing is acquired (S101). The image sequence is acquired by the image sequence acquisition unit 200, and an RGB 3-channel image arranged in time series can be considered. Alternatively, it may be an image sequence that is spatially continuous, such as an image sequence that is spatially arranged and photographed by imaging devices arranged in a horizontal row. In addition, the image which comprises an image sequence is not limited to RGB 3 channel image, You may use other color spaces, such as a Gray1 channel image.

  Further, based on the region of interest, a structural element used for the image summarization process (specifically, whether or not the determination target image can be deleted) is generated (S102). In this embodiment, a region having the same shape and size as the region of interest may be used as a structural element, but other methods may be used. For example, for a circular attention area, a rectangular area circumscribing the attention area may be used as the structural element. In particular, when the shape of the attention area is complicated, it is possible to reduce the amount of calculation by setting a simplified shape as the structural element.

  The reference image selection unit 1001 selects, as a reference image, the first image of the input image sequence (the image sequence acquired in S101 in the first process, and then the partial image sequence set in S109 described later) (S103). ). The reference image selected here is left in the summary image sequence. Note that when the reference image cannot be selected from the input image sequence due to an error or the like (for example, when there is no image in the image sequence), the process ends.

  Then, the determination target image selection unit 1002 selects a determination target image from images included in the input image sequence (S104). If the determination target image is not set, the next image of the reference image (second image in the input image sequence) is selected as the determination target image. If the kth image in the input image sequence has already been selected as the determination target image, the selection position is shifted by 1 and the k + 1th image in the input image sequence is selected as the new determination target image. If the determination target image cannot be selected (for example, if the number of images included in the input image sequence is less than 2 or k + 1), the process ends.

  When the reference image and the determination target image are selected, the deformation information acquisition unit 1015 acquires deformation information between the reference image and the determination target image (S105). The coverage area calculation unit 1003 uses the acquired deformation information to project the reference image onto the determination target image to obtain a coverage area (S106). Here, the deformation information may be a non-rigid deformation parameter or the like estimated by the method described in Patent Document 2. An example of the covering region is shown in FIG. Here, the deformation parameter represents how the subject imaged in the reference image is deformed on the determination target image. In other words, the subject imaged in the reference image and the subject imaged in the covering area on the determination target image are corresponding (same in the narrow sense).

  When the covering area is calculated, the attention area missing possibility determination unit 1013 determines the possibility of missing the attention area (S107). Specifically, a contraction process using a structural element is performed on an uncovered area that is an area other than the covered area in the determination target image to determine whether there is a remaining area.

  A specific example of the contraction process will be described with reference to FIGS. 6 (A) to 6 (E). As shown in FIG. 6A, the non-covering region is always a closed region, and its boundary can be set. For example, in FIG. 6A, BO1 that is the outer boundary and BO2 that is the inner boundary are set.

  At this time, the contraction process by the structural element is a process of cutting the overlapping area between the non-covering area and the structural element when the reference point of the structural element is set on the boundary of the non-covering area. For example, when a circular area is set as a structural element as shown in FIG. 2C and the reference point is the center of the circle, a circle having a center is drawn on the boundary of the non-covered area. And a process of excluding a portion where the non-covering region overlaps from the non-covering region. Specifically, as shown in FIG. 6A, a circle centered on a point on the outer boundary BO1 of the non-covering region is drawn, and an overlapping region with the non-covering region (in this case, a half-line indicated by diagonal lines) (Circular area) is excluded.

  Considering that the outer boundary BO1 is discretely processed, the outer boundary BO1 is composed of a plurality of points. Therefore, the above-described processing may be performed for each point of the plurality of points. As an example, as shown in FIG. 6A, starting from one point on the boundary, a circle centering on the point on the boundary BO1 is sequentially drawn in a given direction, and the overlapping area with the non-covering area is not covered. What is necessary is just to exclude from a covering area | region.

  As shown in FIG. 2B, there may be one boundary depending on the shape of the uncovered region. In this case, the above-described processing may be performed for the one boundary. In addition, as shown in FIG. 6A, when two boundaries of BO1 and BO2 are considered as boundaries of the uncovered area, the above-described processing is performed for each. Specifically, as shown in FIG. 6B, for the inner boundary BO2, a process is performed in which a circle having a center is drawn on BO2 and an overlapping area with the non-covering area is excluded. What is necessary is just to repeat about each point which comprises.

  By performing such shrinkage processing, the area of the non-covering region is reduced. For example, when attention is paid to the left part of the non-covered region in FIG. 6A, the non-covering process is performed by BO1 shown in FIG. 6A and BO2 shown in FIG. 6B. The covered area is completely deleted and there is no remaining area. On the other hand, when attention is paid to the lower right portion of the non-covered region, as shown in FIG. 6C, a residual region RE remains that is not excluded in both the shrinking process in BO1 and the shrinking process in BO2. . Therefore, the result of performing the contraction process with the structural element on the entire uncovered region here is as shown in FIG. 6D, and a residual region RE is generated.

  Here, the meaning of contraction processing when a circle having a radius r is used as a structural element will be considered. A non-covered area that is a closed area is an area inside a boundary (may be a different boundary such as BO1 and BO2 or a single boundary as shown in FIG. 2B). Can think. By performing the above-described shrinkage processing on this boundary, among the points included in the uncovered area, points within the distance r from the points on the boundary are to be deleted. That is, when a point included in the remaining area that has not been deleted is considered, the distance from the point to an arbitrary point on the boundary is greater than r. Therefore, when a circle having a radius r centering on an arbitrary point on the residual region is drawn, the circumference of the circle does not intersect any boundary. In other words, this represents a situation in which the region of interest represented by a circle with a radius R (= r) is completely contained in the non-covered region with the point in the remaining region as its center. Even when a shape other than a circle (such as a square) is used as a structural element, the basic concept is the same.

  That is, the case where the residual region exists is a case where the region corresponding to the structural element is included in the uncovered region as shown in the lower right of FIG. If there is a region of interest and the determination target image is deleted, there is a possibility that the region of interest cannot be observed even if the reference image is left. Conversely, when there is no remaining area, as shown in the upper left of FIG. 6E, at least a part of the attention area is included in the covering area, and even if the determination target image is deleted, At least a part of the attention area can be left in the reference image. From the above, the attention area missing possibility determination unit 1013 performs the contraction process by the structural element on the non-covered area and outputs whether or not the remaining area exists as a result. It becomes possible to determine whether or not to delete.

  Then, the deletion permission / inhibition determination unit 1005 determines whether or not the determination target image can be deleted based on whether or not there is a residual area as a result of the contraction process (S108). If there is a remaining area, the entire area of interest may be included in the uncovered area, that is, the area of interest may be missed. The process proceeds to image row setting processing. If there is no remaining area, the entire area of interest is not included in the non-covered area, and at least a part of the area of interest is included in the covered area. Therefore, in this case, since it is unlikely that the attention area will be missed, it is determined that the determination target image can be deleted, and the process returns to S104 to select the determination target image again.

  If it is determined in S108 that the determination target image cannot be deleted, the partial image sequence setting unit 1008 sets a partial image sequence (S109). Specifically, an image sequence composed of a determination target image determined to be unremovable and subsequent images may be set as a partial image sequence. When the partial image sequence is set, the process returns to S103, and the above-described processing is executed using the partial image sequence as the input image sequence.

  FIGS. 7A to 7D illustrate the above image summarization processing. As shown in FIG. 7A, when an image sequence having N images is acquired by the image sequence acquisition unit 200, first, the first image is selected as the reference image, and the second image is selected. Selected as a determination target image. Then, the possibility of missing the attention area is determined between the reference image and the determination target image, and it is determined whether the determination target image can be deleted.

  If it is determined that the determination target image can be deleted, a new determination target image is selected. Specifically, the position of the determination target image is shifted backward, and the third image is selected as the determination target image as shown in FIG. 7B. Then, whether or not the determination target image can be deleted is determined between the reference image and the new determination target image, and the image selected as the determination target image is updated until a determination target image that is determined to be undeleteable is found.

  As shown in FIG. 7C, when it is determined that the 2nd to k-1st images can be deleted and the kth image is determined to be undeleteable, the 2nd to k-1th images. The image means that the region of interest is covered by the reference image so as not to miss the attention area, and is deleted and not included in the summary image sequence. On the other hand, the k-th image cannot be sufficiently covered by the reference image, so it is necessary to leave it in the summary image sequence. For this purpose, the k-th image and the subsequent images (k to N-th images) are set as partial image sequences.

  Then, the processes in FIGS. 7A to 7C may be repeated for the partial image sequence. Specifically, as shown in FIG. 7D, a partial image sequence composed of N−x + 1 images is set as an input image sequence, and the first image (k-th in FIG. 7C) is set as a reference image, The second (k + 1th in FIG. 7C) image is processed as a determination target image. The subsequent processing is the same. When it is determined that the determination target image can be deleted, the next image is selected as a new determination target image. Further, if it is determined that the determination target image cannot be deleted, the reference image is left in the summary image sequence, the image determined to be deleteable is deleted, and the image after the determination target image at that time is replaced with a new partial image sequence. Set. Eventually, if it is determined that all the images in the input image sequence can be deleted, or if only one image is included in the input image sequence and the determination target image cannot be set, the process ends. It will be.

  In the above embodiment, as shown in FIG. 4, the image processing apparatus has an image sequence acquisition unit 200 that acquires an image sequence having a plurality of images, and a plurality of images in the image sequence acquired by the image sequence acquisition unit 200 A processing unit 100 that performs image summarization processing for deleting a part of the image and acquiring a summary image sequence. Then, the processing unit 100 selects a reference image and a determination target image from a plurality of images, performs processing using deformation information between the reference image and the determination target image, and processing using a structural element corresponding to the region of interest. Based on the result of this, it is determined whether or not the determination target image can be deleted.

  Here, the process using the deformation information may be a process of deforming at least a part of one of the reference image and the determination target image using the deformation information. Further, the process using the structural element corresponding to the attention area determines whether or not the structural element is included in the non-covered area that is the area where the determination target image is not covered by the reference image or the contraction process by the structural element. It may be a process.

  In addition, the attention area is an area where the priority of observation for the user is higher than other areas. For example, when the user is a doctor and desires treatment, an area in which a mucous membrane portion or a lesion is copied is displayed. Point to. As another example, if the object that the doctor desires to observe is a bubble or stool, the region of interest is a region in which the bubble or stool portion is copied. In other words, the object that the user should pay attention to depends on the purpose of observation, but in any case, in the observation, an area in which the priority of observation for the user is relatively higher than other areas is the attention area.

  As a result, processing using the deformation information becomes possible, so that image summarization processing can be performed in consideration of the correspondence between the subject imaged in the reference image and the subject imaged in the determination target image. At that time, by performing processing using a structural element corresponding to the attention area, processing according to the imaging state of the attention area becomes possible. Specifically, since it is possible to determine whether or not at least a part of the region of interest captured on the determination target image is captured in the reference image by performing contraction processing or the like by the structural element, Even if the determination target image is deleted, it is always possible to leave an image in which at least a part of the attention area is captured in the summary image sequence. This makes it possible to suppress the possibility that the user misses the attention area.

  In addition, as processing using deformation information, the processing unit 100 deforms the reference image using the deformation information, obtains a covered area that is an area where the determination target image is covered by the reference image, and covers the determination target image. You may perform the process which calculates | requires areas other than an area | region as an uncovered area | region. Further, as the process using the structural element, the shrinkage process using the structural element may be performed on the uncovered region. Then, the processing unit 100 may determine that the determination target image cannot be deleted when there is a remaining region as a result of the contraction processing.

  This makes it possible to accurately determine whether or not the structural element (here, the same size as the region of interest) fits in the uncovered region. Since the uncovered area is an area that is not covered by the reference image in the determination target image, as described above with reference to FIGS. 6 (A) to 6 (E), the presence / absence of the remaining area is determined by the uncovered area. This corresponds to whether or not the structural element is completely contained. Since the determination here is strict, the accuracy of the deletion permission / inhibition determination can be increased as compared with the methods of the third and fourth embodiments described later.

  Further, the processing unit 100 obtains deformation information between adjacent images for an image between the reference image and the determination target image in the image sequence among the reference image, the determination target image, and the plurality of images, and the acquired adjacent images. The deformation information between the reference image and the determination target image may be obtained based on the deformation information.

  Thereby, when the reference image and the determination target image are not adjacent to each other, it is possible to calculate by accumulating the deformation information obtained between the adjacent images instead of directly obtaining the deformation information between the two images. Become. The deformation information can be calculated by the method shown in Patent Document 2 or the like, but generally, the process of combining a plurality of pieces of deformation information is much lighter than the process of calculating the deformation information from scratch. For example, if the deformation information is a matrix or the like, the process of obtaining the matrix from two pieces of image information is heavy, but combining a plurality of already obtained matrices (for example, it is only necessary to take the matrix product) ) It will be very easy.

  This method is particularly effective in a process where the number of times of using deformation information is used, for example, as in a second embodiment described later. For example, in the second embodiment, a reference image (second reference image) is set not only in front of but also behind the determination target image, and the second reference image is updated according to conditions. Specifically, when the first image is the first reference image and the k-th image is the second reference image, it is determined whether or not deletion is possible between the second to (k-1) -th images and each reference image. Depending on the conditions, the second reference image is updated to the (k + 1) th image without changing the first reference image. In this case, deformation information between each of the second to kth images and the (k + 1) th image that is the second reference image is necessary, and it is necessary to obtain the deformation information k−1 times. And since it is different from the deformation information between the 2nd to (k-1) th images used in the immediately preceding process and the kth image (the immediately preceding second reference image), it must be newly determined. . For example, the image sequence acquisition unit 200 acquires N images as an image sequence, and the second reference image is sequentially updated from the third to the Nth image without changing the first reference image. If a case is assumed, it is necessary to obtain deformation information 1 + 2 + 3 +... + N−2 = (N−2) (N−1) / 2 times. In other words, the number of times of calculating deformation information with a large load is large and inefficient.

  In that respect, if adjacent deformation information is used, if the image sequence acquisition unit 200 acquires N images as an image sequence, it is sufficient to obtain the deformation information N-1 times between adjacent images. In this case, when the reference image and the determination target image are selected from the N images, a process of combining necessary ones of the N-1 pieces of deformation information is necessary. The processing is lighter than the deformation information calculation processing.

  The processing unit 100 may set a structural element having a size proportional to the size of the region of interest.

  This makes it possible to associate the size of the attention area with the size of the structural element. In this embodiment, the size of the attention area and the size of the structural element may be equal (same in the narrow sense), but in the third embodiment described later, the size of the structural element is set to the size of the attention area. It will be set to 2 times. Here, since the determination target image deletion determination is performed based on whether or not a region having the same size as the region of interest completely fits in the non-covered region (or an equivalent region), the size of the region of interest The ratio of the size of the structural elements is determined according to the processing content.

  When first to Nth images (N is an integer equal to or greater than 2) are input as an input image sequence, the processing unit 100 selects the first image as a reference image, and the kth (k is An integer satisfying 2 ≦ k ≦ N−1) may be selected as the determination target image. Then, based on the deformation information between the reference image and the determination target image and the process using the structural element corresponding to the attention area, it is determined whether or not the determination target image can be deleted. Further, when it is determined that the kth image can be deleted, the (k + 1) th image is selected as the determination target image.

  Here, the input image sequence is an image sequence that is a target of this processing (selection of a reference image / determination target image, determination of whether or not to delete, and update of a determination target image when deletion is possible). The image sequence acquired by the unit 200 may be used, or an image sequence including a part of the image sequence may be used.

  As a result, when an input image sequence is input, processing as shown in FIGS. 7A to 7B and the like can be performed. Here, the determination target image is selected from the rear of the reference image (for example, the image adjacent to the reference image at first), and when the selected determination target image can be deleted, the determination target image is further updated to the rear one. . That is, it is determined whether it can be deleted in order from a place close to the reference image (whether it can be sufficiently covered by the reference image), and a place where it cannot be deleted is searched. Note that the determination target image determined to be deleteable here is basically deleted and is not included in the summary image sequence, but is not limited to this, and the determination target image determined to be deleteable May be included in the summary image sequence.

  In addition, the processing unit 100 may perform processing for including the image selected as the reference image in the summary image sequence. At the same time, when it is determined that the kth image selected as the determination target image cannot be deleted, the partial image sequence including the kth to Nth images is set as a new input image sequence and set. Processing may be performed again on the input image sequence.

  As a result, the processing shown in FIG. 7C becomes possible. Since the deletion permission / inhibition determination process is performed based on the contraction process by the structural element as described above, even if the determination target image determined to be deleteable is deleted, by leaving the reference image, at least a part of the attention area Can be observed. That is, in this embodiment, the reference image is included in the summary image sequence. If the k-th image is determined not to be deleted, it means that the k-th image cannot be sufficiently covered by the reference image, and therefore the k-th image should remain in the summary image sequence. That is, the kth image may be set as the next reference image. Specifically, for example, a partial image sequence including the kth to Nth images may be set as a new input image sequence. In this way, as shown in FIG. 7D, the first image of the input image sequence, that is, the k-th image in the image sequence acquired by the image sequence acquisition unit 200 is selected as the reference image, and the (k + 1) -th image is selected. Images after the image are sequentially selected as determination target images. Note that a partial image sequence including a part of the images in the image sequence of FIG. 7D may be newly set as an input image sequence, and this process is repeated (or recursively).

  In addition, the image processing apparatus according to the present embodiment includes an image sequence acquisition unit 200 that acquires an image sequence having a plurality of images, and a summary obtained by deleting some of the plurality of images in the image sequence acquired by the image sequence acquisition unit. A processing unit 100 that performs an image summarization process for acquiring an image sequence. The processing unit 100 selects a reference image and a determination target image from a plurality of images, and is based on deformation information between the reference image and the determination target image. Then, a covering area that is an area where the determination target image is covered by the reference image is obtained, and based on the covering area, the possibility of overlooking the attention area that is the attention area is determined, and based on the determination result of the overlooking possibility Alternatively, it may be determined whether or not the determination target image can be deleted.

  Here, the possibility of missing the attention area means that the attention area may not be observed by the user. For example, in a situation where a region of interest captured on a given image included in the image sequence before image summarization processing is not captured on any image included in the summary image sequence after image summarization processing, A user who sees only the summary image sequence cannot observe the region of interest, and the possibility of missing is 100%. On the contrary, when the attention area is imaged on the image included in the summary image sequence, the possibility of missing may be set to 0%. In other words, the missed possibility here does not consider the situation where the attention area is imaged on the image included in the summary image sequence but is missed by the user's carelessness, and is simply summarized. Whether or not the image is captured on the image is a problem. However, the possibility of missing is not limited to two values of 0% and 100%, but may be a value between them. For example, the ratio of the area of the attention area on the summary image to the area of the entire attention area may be used as a possibility of overlooking.

  This enables image summarization processing that takes into consideration the possibility of missing the attention area. As described above, since the attention area is a lesion in the medical field, it is very useful to prevent the attention area from being overlooked.

  Note that the image processing apparatus or the like according to the present embodiment may realize part or most of the processing by a program. In this case, a processor such as a CPU executes the program, thereby realizing the image processing apparatus according to the present embodiment. Specifically, a program stored in the information storage medium is read, and a processor such as a CPU executes the read program. Here, the information storage medium (computer-readable medium) stores programs, data, and the like, and functions as an optical disk (DVD, CD, etc.), HDD (hard disk drive), or memory (card type). It can be realized by memory, ROM, etc. A processor such as a CPU performs various processes of the present embodiment based on a program (data) stored in the information storage medium. That is, in the information storage medium, a program for causing a computer (an apparatus including an operation unit, a processing unit, a storage unit, and an output unit) to function as each unit of the present embodiment (a program for causing the computer to execute processing of each unit) Is memorized.

3. Second Embodiment Here, another method for selecting a reference image and a determination target image will be described. In the present embodiment, two images of the first reference image and the second reference image are selected as the reference images. First, a basic method will be described, and then two modifications will be described. Note that the difference between the present embodiment and the other embodiments lies in the selection method of the reference image and the determination target image, and the deletion permission determination processing based on the selected reference image and determination target image will be described in another embodiment. Since the method is used, detailed description is omitted.

3.1 Basic Method of Second Embodiment FIG. 8 shows a system configuration example of an image processing apparatus according to this embodiment. The second reference image selection unit 1007 is added to the processing unit 100 of FIG.

  The reference image selection unit 1001 selects the first reference image. The second reference image selection unit 1007 selects two or more images behind the first reference image as the second reference image. Further, the determination target image selection unit 1002 selects an image that is behind the reference image and ahead of the second reference image as the determination target image. In FIG. 8, the reference image selection unit 1001 and the second reference image selection unit 1007 are separated. However, the present invention is not limited to this, and the reference image selection unit 1001 uses the first reference image and the second reference image. You may select both.

  FIG. 9 shows a flowchart for explaining the image summarization processing of this embodiment. S201 to S203 are the same as S101 to S103. After S203, two or more images behind the first reference image selected in S203 are selected as second reference images (S210). Then, a determination target image is set (S204). When the determination target image is not set, the image next to the first reference image (second image in the input image sequence) is selected as the determination target image. If the kth image in the input image sequence has already been selected as the determination target image, the selection position is shifted by 1 and the k + 1th image in the input image sequence is selected as the new determination target image. However, the selection range of the determination target image is not up to the last image in the input image sequence but until the second reference image matches.

  If the determination target image does not match the second reference image, deformation information is acquired (S205), and a covering area is calculated based on the acquired deformation information (S206). Here, the first candidate region is calculated based on the deformation parameter between the first reference image and the determination target image, and the first candidate region is calculated based on the deformation parameter between the second reference image and the determination target image. 2 candidate regions are calculated. Then, as shown in FIG. 10, an area corresponding to the union of the first candidate area and the second candidate area may be set as the covering area. This is because the method according to the present embodiment determines whether the determination target image is sufficiently covered when both the first reference image and the second reference image are left. This is because an area covered by at least one of the candidate area and the second candidate area can be a covered area. That is, the non-covering region in the present embodiment is a region that is not included in either the first candidate region or the second candidate region (a region that is not a covering region in the determination target image as in the first embodiment). ) Will be set.

  S207 to S209 are the same as S107 to S109. If it is determined in S208 that the image can be deleted, the process returns to S204 and the determination target image is updated to the next image. If the determination target image matches the second reference image as a result of the update, the process returns to S210, and the second reference image is updated to the next image. In addition, when the second reference image is updated, the selection state of the determination target image is reset. If the determination target image does not match the second reference image, the processing from S205 is performed.

  If it is determined in S208 that the determination target image cannot be deleted, the reference image and the current second reference image cannot cover all the images sandwiched between them. Therefore, it is necessary to leave an image one previous to the current second reference image in the summary image sequence. Therefore, an image sequence including an image immediately before the current second reference image and the subsequent image is set as the partial image sequence (S209), and the process returns to S203.

  FIGS. 11A and 11B illustrate the above image summarization processing. It is assumed that the entire image sequence kth image has been selected as the first reference image (note that the processing has been completed for the 1st to (k-1) th images, and the kth to Nth images are the partial image sequences. Corresponds to the case where it is set as). Then, the k + 2nd image is selected as the second reference image.

  Further, a determination target image is selected from the front of the image between the first reference image and the second reference image, and whether or not the determination target image can be deleted is determined.

  As shown in FIG. 11A, when it is determined that all the images between the first reference image and the second reference image can be deleted, the first reference image is set as the first reference image. Since there is a possibility that an image further away from the reference image may be selected, the second reference image is reselected as shown in FIG. Specifically, the second reference image that was k + 2 may be updated to k + 3.

  Then, it is determined again whether or not the image between the first reference image and the second reference image can be deleted. As shown in FIG. 11B, when there is a determination target image that is determined not to be deleted, all of the first reference image and the current second reference image are included between them. This means that it is impossible to cover the image (specifically, the determination target image that is determined not to be deleted cannot be covered). It is thought.

  Therefore, an image sequence including the image immediately before the current second reference image (corresponding to the second reference image in FIG. 11A) and subsequent images is set as a partial image sequence. Accordingly, the second reference image at the time of FIG. 11A is selected as the reference image in the next process, and the image to be deleted is covered by the image remaining in the summary image sequence. Is guaranteed.

  In the above description, the head image of the input image sequence is selected as the reference image in S203. However, this is not the case when the process of S203 is performed for the first time. In the present embodiment, the determination target image can be deleted if it is covered by the second reference image behind the determination target image. That is, as shown in FIG. 12A, for example, if the first and second images are covered by the third image, it is necessary to leave the first and second images in the image after the summary processing. There is no. Therefore, it is not always necessary to leave the top image, and there is a possibility that the number of images included in the summary image sequence will be unnecessarily increased in the method using the top image as a reference image.

  Therefore, in the present embodiment, the first reference image does not have to be the top image of the image sequence acquired by the image sequence acquisition unit 200. An example of a specific method is shown. As shown in FIG. 12B, the 0th image that does not actually exist is selected as the first reference image (the selection process here is for convenience, and the 0th image is actually selected). Preparation or other processing is not necessary). If it does so, the 2nd image will be selected by the 2nd standard picture selection processing in S210, and the image in the meantime (only the 1st picture here) will be selected one by one. The processing of S206 to S208 is performed between the determination target image and the second reference image as long as the first reference image does not actually exist. If the first image is covered by the second image, the second reference image is updated as shown in FIG. 12C in accordance with the processing of FIG. 9, and the third image is transferred. It is determined whether the first and second images can be covered by the second image. Hereinafter, if the process is repeated, as shown in FIG. 12D, if the k-1st image is set as the second reference image, all of the 1st to k-2th images can be covered. Even if the 1st image is used as the second reference image, k that cannot cover all the 1st to (k-1) th images can be found. In that case, it is determined in S208 that the image cannot be deleted. In S209, the image sequence including the k-1 to Nth images is set as the partial image sequence, and the process returns to S203. In the second and subsequent processings in S203, as described above, the first image of the input image sequence is selected as the reference image, and therefore the (k-1) th image remains in the summary image sequence as the reference image. As described above, since the 1st to 2nd images can be covered by the (k-1) th image, the 1st to 2nd images can be deleted, and the images included in the summary image sequence The number of sheets can be reduced.

  In the present embodiment described above, when the first to Nth images (N is an integer of 2 or more) are input as the input image sequence, the processing unit 100 selects the pth image as the first reference image. The qth image (q is an integer satisfying p + 2 ≦ q ≦ N−1) is selected as the second reference image, and the rth image (r is an integer satisfying p + 1 ≦ r ≦ q−1) is selected. Select as a determination target image. Then, the deformation element between the first reference image and the determination target image, the process based on the deformation information between the second reference image and the determination target image, and the structural element corresponding to the attention area are used. It is determined whether the determination target image can be deleted based on the processing result. Further, when it is determined that the p + 1 to q−1th images can be deleted, the q + 1th image is newly selected as the second reference image.

  As a result, as shown in FIGS. 11A and 11B, the image summarization processing based on the coverage can be performed after setting the reference images in front of and behind the determination target image. In this case, since two reference images are used, there is a high possibility that it is possible to determine that the determination target image can be deleted, and the number of images after the summarization process can be reduced. When it is determined that the determination target image set between the first reference image and the second reference image can be deleted (in a narrow sense, all determination target images can be deleted. (It is not necessary to limit the second reference image.) In this case, even if the space between the first reference image and the second reference image is further widened, there is a possibility that the image between them may be covered. Is updated to an image later than the current second reference image.

  Note that, as described above, in the present embodiment, the reference image may be set behind the determination target image, and therefore it is not necessary to set the first reference image as the first image in the first process. This is because if all the previous images are covered by the second and subsequent given images, the previous images can be deleted by using the given image as the reference image.

  In addition, the processing unit 100 may perform a process of including the image selected as the first reference image in the summary image sequence. If it is determined that at least one of the p + 1 to q-1th images cannot be deleted, a partial image sequence composed of the q-1st to Nth images is set as an input image sequence and set. The input image sequence is processed again after setting the value of p to 1.

  As a result, the first reference image can be included in the summary image sequence in this embodiment as well as the reference image is included in the summary image sequence in the first embodiment. The case where at least one of the determination target images between the first reference image and the second reference image cannot be deleted is a case where the space between the first reference image and the second reference image is excessively widened. Therefore, the image that is in front of the second reference image at that time (in the narrow sense, in front and closest) should remain in the summary image sequence. Therefore, the partial image sequence including the q-1st to Nth images is set as an input image sequence, and the first and second reference images and the determination target image are selected from the set input image sequence. Depending on the conditions for determining whether or not to delete, the second image update process or the like is performed again. Note that, for the set partial image sequence, the head image should remain in the summary image sequence.

3.2 Modification (Other updating method of second reference image)
Next, a modification of the second embodiment will be described. In this modified example, the same notation is repeated for the determination of whether or not deletion is possible because the second reference image selection method is described. Therefore, as a result of selecting the qth image as the second reference image and determining whether or not to delete in order to simplify the sentence, all images between the first reference image and the second reference image can be deleted. Is described as “the qth image is OK”, and at least one image between the first reference image and the second reference image cannot be deleted. “The image is NG”.

  In the above method, when the q-th image is OK, the second reference image is selected again. However, the new second reference image to be selected is limited to the q + 1-th image. It was.

When the first to Nth images are input as the input image sequence and the first image is selected as the first reference image, the determination target image candidate is selected when the qth image is selected as the second reference image Q-2 images (second to q-1th images) can be considered, and the determination process is performed q-2 times. Assuming that the image summarization process is completed without determining that the determination target image has been deleted once, q is selected from 3 to N (may include N + 1 if a virtual image is considered). Therefore, at least 1 + 2 + 3 +... + N−2 = (N−2) (N−1) / 2 processes are required, and the order of calculation amount is N ² . That is, in the above-described method, if N becomes very large, the amount of calculation increases dramatically, which is not preferable.

  Therefore, here, when the second reference image is selected again, the amount of calculation is reduced by expanding the selection range without limiting the target to adjacent images. Specifically, when the q-th image is OK, the new second reference image is not limited to the q + 1-th image, but is allowed to be selected from the q + 2 image and the image behind it. In this case, even if the qth image is NG, it is not known whether or not the q-1th image is OK (the q-1st image may not be selected as the second reference image). Because there is). Therefore, unlike the basic method of the second embodiment, the fact that the q-th image is NG does not immediately leave the q-1-th image as a summary image, but basically the q-th image. By selecting an image ahead of the first image as a new second reference image, the front image is determined.

  That is, in this modification, until the end condition is satisfied, the second reference image is updated backward in the case of OK, and forward in the case of NG, thereby searching for the next summary image of the first reference image. It will be. By appropriately setting the position of the new second reference image, the number of images selected as the second reference image until the next summary image is found can be reduced, and the amount of calculation can also be reduced. Note that the calculation amount reduction here is only to reduce the expected value of the calculation amount, and depending on the position of the summary image next to the first reference image, the above-described method requires less calculation amount. The possibility cannot be denied. Hereinafter, the method of the modification will be described in detail.

  The system configuration example of the image processing apparatus is the same as that in FIG. 8, and the second reference image selection process (update process) in the second reference image selection unit 1007 is different. Therefore, detailed description of similar parts is omitted, and different points will be described.

  When the input image sequence is input, the reference image selection unit 1001 selects the first reference image. Here, as described above, the top image (first image) of the input image sequence is selected. When the input image sequence is an image sequence acquired by the image sequence acquisition unit 200 (when the first first reference image selection process is performed), an image other than the head (for example, a virtual 0th image) However, in the following description, the first reference image is assumed to be the first image unless otherwise specified.

  Thereafter, the second reference image is selected. Here, a second reference image selection section (actually equivalent to a range for searching for a summary image next to the first reference image) corresponding to an image to be selected for the second reference image is set. The half-open interval [i, j) corresponding to the i-th image to the j-th image is set as the second reference image selection interval, and i is made to correspond to the next image of the first reference image (i = 2 in a narrow sense). ), J = N + 2. Note that j = N + 2 is set even if a virtual N + 1th image is set as the second reference image, so that a virtual 0th image may be set as the first reference image. Because it is good. The case where the second reference image is the (N + 1) th image corresponds to the case where it is possible to cover all the images behind the first reference image alone and determine whether the second reference image is unnecessary. To do.

  Then, the second reference image is selected from the set second reference image selection section. Here, in order to perform processing efficiently, the second reference image is determined based on a given condition. First, when the second reference image is selected for the first time after setting the first reference image, the i + 1th image (third image in a narrow sense) is selected as the second reference image. This is the same as the basic method of the second embodiment.

  FIG. 13A illustrates the processing so far. Here, N = 12 image sequences are considered, the first reference image is the first image, the second reference image selection section is the second image to the 14th image (i = 2, j = 14). The second reference image is the third image.

  After the second reference image is selected, the determination target image selection process, the covered area (non-covered area) calculation process, the attention area missing possibility determination process, the deletion possibility determination process, and the repetition of these processes Since it is the same as the above-mentioned method, detailed description is abbreviate | omitted.

  If a given image (initially the third image) is selected as the second reference image and the image is OK, the position of the second reference image can be further separated from the first reference image. Since it is good, an image behind the current one is selected as a new second reference image. This idea itself is the same as the basic method of the second embodiment. However, although the second reference image is moved backward one by one in the above-described method, it may be shifted to two or more backward images here.

  As an example, if the current second reference image is the a-th image counted from the first reference image, the 2 × a-th image counted from the first reference image is replaced with the new second reference image. It may be an image. Specifically, as shown in FIG. 13B, when the third image (second from the first reference image) is selected as the second reference image, the third image is If OK, the next second reference image is selected as the fifth (fourth counted from the first reference image) image.

  By the way, if the q-th image is OK, it is not necessary to select the q−1-th and previous images as the summary images left in the summary image sequence. Therefore, since there is no merit to select an image ahead of the current position (qth) as the second reference image, the second reference image selection section may be updated. Specifically, the starting point i of the selected section may be i = q. In this modification, since the second reference image is selected from the second reference image selection section, this prevents the image ahead of the current position from being selected. For example, as shown in FIG. 13B, when the third image is OK, the second image is not a summary image, so it may be excluded from the selected section, and the start point of the selected section is the third point. Update to an image.

  Similarly, if the fifth image is OK, the ninth image is selected as a new second reference image as shown in FIG. 13C, and the start point of the second reference image selection section is set to 5. Update to the second image.

  However, as can be understood by considering the case where the ninth image is OK in FIG. 13C, when the qth image is the second reference image and the qth image is OK, q If the value increases, the new second reference image may become extremely rearward. For example, the image behind N + 1 becomes a candidate and the second reference image cannot be selected, or the interval between the second reference images before and after the update becomes too wide, The search for the summary image becomes inefficient.

  Therefore, when selecting an image behind the current position as a new second reference image, another method may be used in combination. As an example, a new second reference image is determined based on the value of (q + j) / 2. For example, when the 9th image is OK, the start point of the second reference image selection section is updated to the 9th image, and thus the [9, 14) half-open section. That is, by setting the image near the center as a new second reference image, the center of the search range is set as a processing target. The method of halving the search range by determining the center of the search range is not limited to the widely known binary search, and it is also widely known that the binary search has an advantage in terms of computational complexity. Yes. The second reference image selection section of the present embodiment is that if a given image is OK, all images ahead of it may be considered OK, and if a given image is NG, the image behind it is All have the property of being considered NG, and a binary search technique can be applied. In other words, efficient processing can be expected by selecting a new second reference image from the vicinity of the middle between the second reference image before update and the end point of the second reference image selection section.

Here, the method of doubling the distance of the first reference image starting point and the method corresponding to the binary search are used together. For example, when the q-th image is the second reference image before update, the k-th image that satisfies the following expression (1) may be used as the next second reference image. Here, min (a, b) represents the smaller one of a and b.

  On the other hand, when the q-th image is NG as described above, a new second reference image is selected from the front of the current position, contrary to the case of OK. How much forward image is selected can be determined by various methods. For example, a method corresponding to the binary search may be used here. In this case, since the starting point of the second reference image selection section is the i-th image, a new second reference image is determined based on the value of (i + q) / 2. Further, as long as the qth image is NG, the qth image and the image behind it are not selected as the summary image. Therefore, the end point of the second reference image selection section may be updated, and j = q may be set. FIG. 13D shows an example where the ninth image is NG. The seventh image is selected as a new second reference image, and the end point j of the second reference image selection section is updated to j = 9.

  Note that the reason why the second reference image selection section is a half-open section is for convenience of explanation here. That is, when the q-th image is OK, there remains a possibility that the q-th image is selected as a summary image. Therefore, when the starting point i of the second reference image selection section is i = q. , I may be included in the second reference image selection section. On the other hand, when the q-th image is NG, the q-th image is not selected as the summary image. Therefore, when the end point j of the second reference image selection section is j = q, j is the second reference image. It is better not to include it in the image selection section. From the above, the second reference image selection section is merely [i, j), and depending on the notation of the sign and the expression, there is nothing to represent the second reference image selection section by an open section or a closed section. No problem.

  With the above processing, the second reference image selection section (in the narrow sense, the search range of the next summary image) is narrowed. The next summary image is a k-th image when the k-th image is OK and the k + 1-th image is NG. Therefore, when a location where the OK image and the NG image are adjacent to each other is found. Processing will be terminated. In the above example, it is assumed that the process is performed in a binary search immediately before the end, for example, as shown in FIG. The i-th image is OK, the j-th image next to it is NG, and the q-th image between them is the second reference image. In this case, if the q-th image is OK, it will be as shown in FIG. 13 (F), and if it is NG, it will be as shown in FIG. 13 (G). The image corresponding to the start point is OK, and the image corresponding to the end point is NG. Therefore, since the image corresponding to the start point may be selected as the next summary image, the search process for the input image sequence ends.

  If the next summary image is found, it is the same as the basic method of the second embodiment in that a partial image sequence including the image and subsequent images may be set as an input image sequence. Therefore, the partial image sequence setting unit 1008 sets the start point of the second reference image selection section and the subsequent images as the partial image sequence, and sets the partial image sequence as a new input image sequence. When a new input image sequence is given, the subsequent processing is the same, and detailed description thereof is omitted.

  FIG. 14 shows a flowchart for explaining this process. S301 to S303 are the same as S201 to S203. After the first reference image is selected in S303, a second reference image selection section is set (S311). As the processing of S311 performed immediately after S303, for example, as described above, a half-open section of [i, j) that satisfies i = 2 and j = N + 2 may be set. As will be described later, when the processing of S311 is performed after S304 or S308, the second reference image selection section that has already been set is updated.

  When setting (or updating) of the second reference image selection section is performed in S311, it is determined whether the start point and the end point are adjacent to each other (j = i + 1 is satisfied) (S312). In the case of Yes in S312, since the i-th image has been found to be the next summary image of the first image as shown in FIG. 13F, the i-th image and the subsequent images Is set as a partial image sequence (S309), and the process returns to S303.

  In the case of No in S312, since the next summary image has not yet been found, the second reference image is selected from the second reference image selection section set in S311 (S310). When the process of S310 is performed for the first time after the setting of the first reference image in S303, for example, an i + 1th image (two images behind the first reference image) may be selected. In other cases, a process of selecting a new second reference image is performed according to the position of the immediately preceding second reference image.

  When the second reference image is selected in S310, a determination target image is selected (S304). The deformation information acquisition process (S305), the cover area calculation process (S306), the attention area miss possibility determination process (S307), and the image deletion possibility determination process (S308) after selection of the determination target image are the same as S205 to S208. . If it is determined in S308 that the image can be deleted, the process returns to S304, the determination target image is updated to the next image, and the same processing is performed. By repeating the processes of S304 to S308, it is determined whether all the images between the first reference image and the second reference image can be deleted, or at least one of them cannot be deleted. If all can be deleted, it is determined in S304 that the determination target image is equal to the second reference image, and the process returns to S311. If at least one image cannot be deleted, the determination in S308 indicates that the image cannot be deleted, and the process returns to S311. Although not shown in FIG. 14, it is necessary to retain information indicating whether the process returns from S304 to S311 or whether the process returns from S308 to S311 and the process in the next S311 or the like needs to be changed accordingly. is there.

  When the process returns from S304 to S311, since all images can be deleted, a process of updating the start point of the second reference image selection section is performed. As a result, in S310, the previous second reference An image behind the image is selected as a new second reference image. On the other hand, when returning from S308 to S311, since at least one image cannot be deleted, a process of updating the end point of the second reference image selection section is performed. An image ahead of the second reference image is selected as a new second reference image.

  In the above modification, when the pth image is selected as the first reference image and the qth image is selected as the second reference image from the input image sequence including the first to Nth images, The processing unit 100 selects the second reference image from the second reference image selection section in which the start point and the end point corresponding to the p + 2 to Nth images are set. Then, processing using deformation information between the first reference image and the determination target image, processing using deformation information between the second reference image and the determination target image, and a structural element corresponding to the attention area are used. It is determined whether or not the determination target image can be deleted based on the result of the processing performed. When it is determined that the (p + 1) -th to (q-1) -th images can be deleted, the x-th (x is an integer satisfying x> q) image included in the second reference image selection section is added to the new second image. Select as reference image. At the same time, the starting point of the second reference image selection section may be updated to the qth image.

  Here, the second reference image selection section includes the (p + 2) th to Nth images in view of the property of being an image that is a candidate for the second reference image. However, since a virtual image such as the (N + 1) th image may be selected as the second reference image, the end point of the second reference image selection section may be larger than N. In addition, since the second reference image selection section also has an aspect of a search range for the next summary image, an image that can be selected as a summary image is included in the selection section even if the image is not selected as the second reference image. It may be included. In that case, an image (p + 1-th image) one behind the first reference image may be set as the starting point of the second reference image selection section.

  Thereby, when the second reference image is updated, the position of a new second reference image can be determined flexibly. In other words, the basic method of the second embodiment is a method of checking the search range one by one from the top and reducing the search range, so that the amount of calculation becomes very large depending on the position of the correct answer. End up. In that respect, by making it possible to select images that are not adjacent to each other as a new second reference image, the search range can be greatly reduced by one unit of determination (determination of whether the qth image is OK or NG). Therefore, an effect of reducing the amount of calculation can be expected, and the load on the system can be reduced and the processing time can be shortened.

  In addition, when it is determined that at least one of the p + 1 to q−1th images cannot be deleted, the processing unit 100 determines that the yth (y is y <q) included in the second reference image selection section. An integer that satisfies (integer integer) may be selected as a new second reference image. At the same time, the end point of the second reference image selection section is updated to the qth image.

  As a result, when the second reference image is updated, an image ahead of the current second reference image can be selected as a new second reference image. As described above, as long as the backward search is not limited to selecting adjacent images, an unsearched range may remain ahead of the current second reference image. Can be considered that there is a correct answer in the unsearched range. In that case, it is possible to perform appropriate processing by performing a forward search. Further, as in the backward search, selection of a new second reference image is not limited to adjacent images.

  Further, the processing unit 100 may set the value of x based on the value of (q + j) / 2 when the j-th image (j is an integer) corresponds to the end point of the second reference image selection section. Good. Alternatively, when the i-th (i is an integer) image corresponds to the start point of the second reference image selection section, the value of y may be set based on the value of (i + q) / 2.

This makes it possible to use a binary search method in selecting a new second reference image. In the case of a backward search, an image that is intermediate between the current second reference image and the end point is selected, and in the case of a forward search, an intermediate between the current second reference image and the start point is selected. Will be selected. Therefore, the search range (corresponding to the length of the second reference image selection section) can be halved. If logN images are selected as the second reference image, the entire search range can be searched. Expected to end. Therefore, the order of the calculation amount can be suppressed to N × logN, and when N is very large, the calculation amount is reduced as compared with the basic method of the second embodiment (the order of calculation amount is N ² ). Great effect. Since (q + j) / 2 and (i + q) / 2 are not necessarily integers, there may be cases where there is no image corresponding to each value. In that case, for example, a maximum integer not exceeding (q + j) / 2, or an integer larger by 1 may be considered.

  In addition, the processing unit 100 performs processing for including the image selected as the first reference image in the summary image sequence when the start point and the end point are adjacent as a result of updating the start point or the end point of the second reference image selection section. May be performed. Also, an image corresponding to the start point and a partial image sequence consisting of images behind the image corresponding to the start point are set as the input image sequence, and the value of p is set for the set input image sequence. The processing may be performed again by setting it to 1.

  Here, that the start point and the end point are adjacent indicates that the image corresponding to the start point and the image corresponding to the end point are adjacent in the input image sequence. When N images are given as an input image sequence, it is assumed that the input image sequence is a set of images that are continuous in time series or spatially. You can define forward and backward. For example, an image acquired at an earlier time in time series is an image ahead of an image acquired at a later time. Specifically, each image in the input image sequence is represented as first to Nth images, and an image with a smaller shaken numerical value is assumed to be in front. Therefore, the i-th image and the j-th (> i) image in the image sequence are adjacent to each other in a situation where j = i + 1 is satisfied.

  This makes it possible to set a condition based on the start point and end point (or length) of the second reference image selection section as the process end condition for the input image sequence. By setting such an end condition, an image that is predicted to be the farthest from the first reference image in the image group determined to be OK when selected as the second reference image It can be selected as the first image (corresponding to the next summary image) of the image sequence. This is because, as shown in FIG. 13F and the like, this end condition is equivalent to searching for a position where an OK image and an NG image are adjacent to each other. Therefore, the number of summary images included in the finally output summary image sequence can be reduced, and the burden on the user can be reduced.

3.3 Modification (Initial setting of second reference image)
In the above second embodiment and its modifications, an input image sequence (which may be an image sequence acquired by the image sequence acquisition unit 200 or a partial image sequence consisting of a part of the images) is input. In this case, the second reference image that is initially set is limited to an image that is two images behind the first reference image.

  However, the initial position of the second reference image may be different. For example, in an actual image sequence, a section in which similar images continue for a long time and a section in which there are not many similar images are not often adjacent. In other words, the length of the next summary section (representing how far the adjacent summary images are separated) is predicted to be close to the length of the previous summary section. Therefore, if a plurality of summary images have already been obtained and information corresponding to “the length of the previous summary section” has been acquired, the initial position of the second reference image is determined from the first reference image. By setting it at a position separated by “the length of the previous summary section”, it is expected that the correct answer will be reached sooner, and a reduction in the amount of calculation can also be expected.

  Specifically, the length g of the summary section is acquired from the immediately preceding summary image and the previous summary image. Then, when the second reference image selection section is [i, j), the second reference image may be set to the i + g image instead of the i + 1th image. Note that g cannot be acquired when the number of summary images already determined is zero or one. In this case, the initial position of the second reference image is set without using g. For example, if there are no summary images, the second reference image may be the (i + 1) th image as described above. If there is one summary image, the length from the first image in the image sequence acquired by the image sequence acquisition unit 200 to the summary image is g ′, and the first second reference image is i + g. It may be '

  Note that when the second reference image needs to be updated, various updating methods can be considered. For example, the next second reference image may be selected using a binary search method as in the above-described modification.

  However, in view of the assumption that there is a high possibility that the next summary image is likely to be in the vicinity of the i + g image, the fact that the updated second reference image is far away from the i + g image is until the next summary image is found. It may increase the number of searches and may be considered undesirable. In that case, as described as the basic method of the second embodiment, an image adjacent to the previous second reference image may be selected as a new second reference image. However, as long as the determination is not made for the (i + 1) -th to i + g-1 images, there is a possibility that the next summary image exists in the range. Therefore, the update of the second reference image is not limited to moving backward one by one, and may be shifted forward one by one depending on the determination result of deletion availability.

4). Third Embodiment Next, a modification of determining whether or not a determination target image can be deleted, which is performed after the reference image and the determination target image are selected, will be described. In the present embodiment, the covering region and the non-covering region are not necessarily obtained, but based on whether or not the attention region completely fits in the region not covered by the reference image (corresponding to the non-covering region), It is the same in that the determination of whether or not deletion is possible.

  FIG. 15 shows a system configuration example of the image processing apparatus in the present embodiment. The processing area 100 of FIG. 4 has a configuration in which a required coverage area generation section 1011 and a required coverage area inclusion determination section 1012 are added.

  The required coverage area generating unit 1011 generates a required coverage area based on the determination target image and the structural element. The required coverage area inclusion determination unit 1012 determines whether the generated required coverage area is included in the reference image. Details of the processing in the covering area generation unit 1011 and the covering area inclusion determination unit 1012 will be described later. In addition, the attention area missing possibility determination unit 1013 according to the present embodiment determines the possibility of missing the attention area based on the inclusion relationship between the coverage area and the reference image.

  Details of the processing will be described below. In the present embodiment, the deletion permission / inhibition determination is performed by performing a contraction process using a structural element on the entire determination target image. Specifically, as shown in FIG. 16A and FIG. 16B, contraction processing is performed on the determination target image using a structural element corresponding to the attention area, and the remaining area is set as a covering area. If the structural element is circular, a circle centered on the boundary (periphery) of the determination target image is drawn, an overlapping area between the circle and the determination target image is deleted, and the remaining area becomes a required covering area.

  Here, in the present embodiment, the region to be cut by the contraction process is set so as to satisfy the condition that the region of interest is not completely included. Under this condition, the attention area does not completely fit into the trimmed area, and therefore it is not necessary to cover the trimmed area with the reference image. In other words, the area that needs to be covered without being cut is an area that is required to be covered with the reference image.

  In other words, based on the deformation information between the reference image and the determination target image, the reference image is compared with the coverage area, and if the coverage area is covered by the reference image, the determination target image is deleted. If it is determined that the area to be covered is not covered by the reference image, it may be determined that the determination target image cannot be deleted.

  However, it is desirable to set the maximum area to be cut by the shrinking process among those satisfying the above conditions. As an extreme example, let us consider a case in which a region to be cut by the shrinking process does not occur (corresponding to the case where the shrinking process is not performed or an ideal point having no size as a structural element is set). In this way, as long as there is no area to be cut, the condition that the attention area does not completely fall within the cut area is satisfied. However, since the entire determination target image is a covering area, the determination target image can be deleted only when the entire determination target image is covered by the reference image (in the case of 100% coverage). . For this reason, the effect of reducing the number of images by the image summarization process is remarkably reduced, and it is difficult to achieve the original purpose of the image summarization process. That is, in order to effectively reduce the number of images, the possibility of being determined to be deleteable should be increased. In this embodiment, even when the determination target image is deleted, the reference image is displayed. In a situation where at least a part of the region of interest is imaged, it is better to determine that the determination target image can be deleted with a high probability (in the narrow sense, always). For this purpose, as described above, the area to be cut by the shrinking process may be set to the maximum area among the conditions satisfying the condition, and is basically realized by increasing the size of the structural element as much as possible.

  In the first embodiment, the target of shrinkage processing by the structural element is an uncovered region, and it is determined whether or not the region corresponding to the attention region is completely contained in the uncovered region based on the presence or absence of the remaining region. Was. However, in the present embodiment, the target of the shrinkage process is the entire determination target image, and the fact that the entire region of interest fits in the determination target image is natural during normal observation (depending on the setting of the imaging magnification, etc.) Are not considered here). Therefore, in this embodiment, it is natural that there is a remaining area, and there is no meaning in the determination based on the presence or absence of the remaining area. This difference in processing is different from that in the first embodiment in which the contraction process is performed for the region to be determined whether or not the region of interest completely fits. This is due to the difference that the region of interest does not completely fit into the region cut by the shrinking process.

  Due to this difference, the structure element setting method based on the region of interest in this embodiment is different from that in the first embodiment. In the first embodiment, there can be a shape change process such as setting a rectangular structural element from a circular attention area, but basically the same size as the attention area (in the narrow sense, the same size and shape). ) Was set. However, as shown in FIG. 16B, when a circle having a radius r is a structural element and the reference point is the center of the circle, the area to be cut by the shrinking process is the boundary (the main area of the target area of the shrinking process). In the embodiment, the region is a distance r or less from the boundary of the determination target image. That is, when a circle with a radius R satisfying R = r is considered as the attention area, the condition that the attention area does not completely fit in the area to be cut by the contraction process in FIG. Is satisfied, but the area is smaller than the largest region (in other words, the above condition is satisfied even if a larger region is cut). For this reason, when the same structural elements as those in the first embodiment are set, the effect of reducing the number of images in the image summarization process is reduced as described above.

  Therefore, in the present embodiment, unlike the first embodiment, a structural element having a size larger than the size of the region of interest may be set. For example, when a circle with a radius r is considered as the structural element and the reference point is the center of the circle, the range within the distance r from the boundary of the determination target image is deleted in the contraction process. Therefore, if considered simply, the radius that is completely within the deleted range is r / 2. That is, if the region of interest is a circle with a radius R, the condition that the region of interest does not completely fit in the region to be cut will satisfy R ≧ r / 2. From this, the radius r of the structural element is r ≦ 2R, and for example, a circle of r = 2R where the equal sign holds may be used as the structural element. Similarly, if the region of interest is a square with a side L, the structural element may be a square with a side 2L. That is, basically, a structural element twice as large as the region of interest (comparison with the diameter and the maximum diameter, not the area) may be used.

  Depending on the determination target image and the shape of the structural element, the above-described setting may reduce the accuracy of the determination as to whether or not deletion is possible (determined that deletion is possible although there is a possibility of oversight). In that case, if the accuracy is to be increased, the size of the structural element needs to be set according to the shape of the determination target image or the like, which may increase the load of the setting process. Therefore, if the method of the present embodiment is used as a simpler and lighter processing load than the method of the first embodiment (that is, a certain degree of accuracy is allowed to be reduced), the determination target image, etc. Regardless of the shape, a structural element that is twice the area of interest may be set.

However, an appropriate size of the structural element may be determined according to the shape of the determination target image or the like. A specific example will be described with reference to FIGS. When a contraction process is performed on a rectangular determination target image with a circular structural element having a radius r as shown in FIGS. 16A and 16B, the peripheral portion of the determination target image is cut away. Become. In this case, when the radius of the maximum circle that can be accommodated in the region to be cut is strictly considered, it becomes maximum at the four corners of the determination target image as shown in FIG. 16C, and the following equation (2) needs to be satisfied.

In other words, if the region of interest is a circle with a radius R, if a circular shape is used as the structural element, the radius r must satisfy the following formula (3), and in the narrow sense, r holds an equal sign. What is necessary is just to use as a radius.

  In addition, as for the comparison process between the reference image and the required coverage area, a deformation process based on the deformation information may be performed in the same manner as the process for obtaining the coverage area in the first embodiment. For example, as shown in FIG. 17A, the covering area may be deformed and projected onto the reference image. In this case, if the covered area after deformation is included in the reference image, the covered area is covered by the reference image, and the determination target image in this case can be deleted. Conversely, when at least a part of the required coverage area after deformation is not included in the reference image, the required coverage area is not covered by the reference image, and the determination target image in this case cannot be deleted. Become.

  In addition, as shown in FIG. 17B, the reference image side may be modified in the comparison process between the reference image and the area requiring coverage. That is, the reference image is deformed based on the deformation information and projected onto the determination target image. The deformed reference image corresponds to the covering area in the first embodiment. If the covering area is included in the covering area, the determination target image can be deleted, and at least one of the covering areas is required. If the portion is not included in the covering area, the determination target image cannot be deleted.

  FIG. 18 shows a flowchart for explaining the processing of this embodiment. Since S401 to S405 of this process are the same as S101 to S105 of FIG. 5, detailed description thereof is omitted. After S405, a covering area is generated based on the structural element generated in S402 (S410). Specifically, as shown in FIGS. 16A and 16B, a contraction process using a structural element may be performed on the determination target image, and the remaining area may be set as a required covering area. Then, it is determined whether the covering area is included in the reference image (S411). Specifically, as shown in FIG. 17A, the required coverage area may be deformed based on the deformation information, and the deformable coverage area may be compared with the reference image, as shown in FIG. As described above, the reference image may be deformed based on the deformation information, and the covering required area may be compared with the deformed reference image.

  After that, as attention area missing possibility determination processing (S407), the presence / absence of possibility of missing is determined based on whether it is determined in S411 that the required coverage area is included in the reference image. S408 and S409 are the same as S108 and S109 in FIG.

  In the present embodiment described above, the processing unit 100 may perform a process of obtaining a necessary area by performing a contraction process using a structural element on a determination target image as a process using a structural element. Then, it is determined whether or not the determination target image can be deleted based on the required coverage area and the deformation information.

  This makes it possible to easily determine the possibility of missing the attention area. The multiple images included in the image sequence are unlikely to change in size or shape (in the narrow sense, all images have the same size and shape). Many of them have the same shape. When a shrinking process using the same structural element is performed on a plurality of images (areas) having the same size and shape, the areas after the shrinking process are the same. In other words, if the sizes and shapes of the plurality of determination target images are the same, the required coverage area obtained from each determination target image should be the same, and the contraction process using the structural element is performed on all the determination target images. The need is low. Therefore, when the process for obtaining the required coverage area is performed once, the determination target images having the same size and shape can be used as they are, so that the processing load can be reduced. Furthermore, since the determination target image is assumed to have a simple shape (for example, a square shape or a circular shape), the processing is easier than the contraction processing for the uncovered region in the first embodiment. It is assumed that the size of the structural element is set to be twice the size of the region of interest, but there is a possibility that the region of interest may be overlooked when r = 2R as shown in FIG. Caution must be taken. Therefore, it is better to change the process according to the situation, and in a situation where a certain degree of oversight is allowed and reduction of processing load is important, the size of the structural element is the size of the attention area regardless of the judgment target image or the shape of the attention area. Further reduction of the processing load can be achieved by assuming that it is twice as large. Also, if importance is placed on suppression of the possibility of overlooking, the ratio of the size of the structural element to the size of the attention area may be determined according to the determination target image and the shape of the attention area.

  Moreover, the process part 100 may perform the process which deform | transforms a covering required area | region based on the deformation | transformation information between a reference | standard image and a determination target image as a process using deformation | transformation information. Then, the processing unit 100 may determine that the determination target image can be deleted when the cover area after deformation is included in the reference image.

  Or the process part 100 may perform the process which deform | transforms a reference | standard image based on the deformation | transformation information between a reference | standard image and a determination target image as a process using deformation | transformation information. Then, the processing unit 100 may determine that the determination target image can be deleted when the required coverage area is included in the deformed reference image.

  Thereby, the inclusion determination shown in FIG. 17A or FIG. 17B becomes possible. The required coverage area here is an area that is required to be covered with the reference image, and the determination target image (that is, the required coverage area) is generally deformed with respect to the reference image. For this reason, at least one of the comparison processing with the reference image and the area requiring coverage needs to be deformed based on the deformation information.

  When an intermediate image exists between the reference image and the determination target image in the input image sequence, the reference image is deformed based on deformation information between the reference image and the intermediate image, and the determination target image and the intermediate image It is also possible to determine whether or not deletion is possible by deforming the required coverage area based on the deformation information between the two and determining whether or not to include the deformed reference image and the required coverage area after deformation. However, in consideration of the processing load and the like, it is desirable to use a method of deforming either the reference image or the area requiring coverage.

5. Fourth Embodiment In addition, a deletion determination process for a determination target image (that is, a determination as to whether or not an attention area completely fits in an uncovered area) may be performed without performing a contraction process using a structural element. . Here, as in the first embodiment, the reference image is deformed based on the deformation information, and the covering area is obtained.

  The system configuration example of the image processing apparatus according to this embodiment is the same as that of FIG. 4 used in the first embodiment, and thus detailed description thereof is omitted. Further, details of the processing are the same as those in the flowchart shown in FIG. The difference from the first embodiment is the attention area missing possibility determination unit 1013 in FIG. 4 and the processing contents of S107 in FIG.

  In the present embodiment, a plurality of points are set on the boundary of the covering region, and the minimum value of the distance to the boundary of the determination target image is obtained for each point. For example, when the covering region and the determination target image as illustrated in FIG. 19A are acquired, the minimum value for the point p1 is k1, and the minimum value is k2 to k4 for p2 to p4. In FIG. 19A, four points p1 to p4 are shown. However, in actual processing, a sufficient number of N points to ensure accuracy (in a narrow sense, all pixels corresponding to the boundary of the covering region). )) Find the minimum value.

  When N minimum values k1 to kN are acquired for the points p1 to pN, the largest one of k1 to kN is obtained. Since the largest of these k1 to kN can be regarded as information corresponding to the maximum diameter of the non-covered region, the value and the structural element (here, the same as the region of interest as in the first embodiment) By performing the comparison process with the minimum diameter), it can be easily determined whether or not the attention area completely fits in the non-covering area. Specifically, when the maximum value among k1 to kN is larger than the minimum diameter of the structural element, it is determined that the attention area may be completely contained in the non-covering area, and the determination target image is Cannot be deleted. On the contrary, when the maximum value of k1 to kN is equal to or smaller than the minimum diameter of the structural element, it is determined that the attention area is not completely contained in the non-covering area, and the determination target image can be deleted. .

  When the determination target image is a quadrangle, wide uncovered areas may appear at the four corners. For example, in FIG. 19A, the minimum value at p4 is k4, but it is appropriate to consider k ′ indicated by a dotted line as the maximum diameter of the uncovered region. Therefore, it is possible to set a plurality of points on the boundary of the determination target image, obtain the minimum value of the distance to the boundary of the covering region for each point, and use the maximum value among the obtained minimum values together Good. In this way, k ′ can also be used for the processing, and information corresponding to the maximum diameter of the uncovered area can be obtained appropriately.

  As shown in FIG. 19B, if the determination target image is circular, the minimum value at each point (for example, k5, k6 at p5, p6, etc.) appropriately represents the minimum diameter of the non-covered area. It is thought that. That is, when using a circular determination target image, unlike the case of using a quadrangle determination target image as shown in FIG. 19A, a method of setting a plurality of points on the boundary of the covering region, and the determination target image It is sufficient to use one of the methods for setting a plurality of points on the boundary of. Therefore, the method of the present embodiment reduces the processing load when processing a circular determination target image as compared with a case where processing of the same degree of accuracy is performed on a quadrangle determination target image. It is thought that the effect to do. In addition, although the accuracy may be lower than that of a circular shape, a method of setting a plurality of points on the boundary of the covering region even when a polygon having a large number of vertices (such as an octagon or a hexagon) is used. Thus, it is considered that a certain degree of accuracy can be ensured by any one of the methods of setting a plurality of points at the boundary of the determination target image. However, if importance is attached to the determination accuracy, the above two methods may be used in combination.

  However, in the method of the present embodiment, there is a case where the attention area does not completely fit in the uncovered area even when the maximum value of k1 to kN is larger than the minimum diameter of the structural element. For example, as shown in FIG. 19C, even in a situation where it is clear that the region of interest does not completely fit in the uncovered region, the method of this embodiment uses the minimum value k7 at p7. However, since it is used for the comparison process with the minimum diameter of the structural element, it is determined that the determination target image in FIG. Therefore, even if a determination target image such as a circular shape is used, it is not possible to ensure the same determination accuracy as when the contraction process of the first embodiment is used (in FIG. 19C, what can be originally deleted is not deleted). Therefore, it should be noted that there is a possibility that the effect of reducing the number of images will be reduced).

  In the above-described embodiment, the processing unit 100 may deform the reference image using the deformation information as a process using the deformation information, and obtain a covered area that is an area where the determination target image is covered by the reference image. Good. Then, at each point of the plurality of points set on the boundary of the obtained covering region, the shortest distance to the outer periphery of the determination target image is obtained, and the maximum value among the shortest distances obtained at each point of the plurality of points The process which calculates | requires is performed. In addition, the processing unit 100 acquires the minimum diameter of the structural element as the process using the structural element, and based on the comparison process between the acquired minimum diameter and the maximum value of the shortest distances, You may perform the process which determines whether a structural element is contained in the non-covering area | regions other than a covering area | region.

  Here, the minimum diameter of the structural element is a value representing the size of the structural element. For example, if the structural element is a circle, the diameter is a diameter, and if the structural element is a square, the length is a short side. In addition, when the shape of the attention area is complex and it is difficult to obtain the minimum diameter, the structural element is a shape obtained by simplifying the shape of the attention area (for example, a figure having a reduced number of vertices compared to the attention area). And a shape circumscribing the region of interest), the minimum diameter of the structural element can be easily obtained.

  This makes it possible to easily determine the possibility of missing the attention area. The shortest distance from the point on the boundary of the covering region to the boundary (outer periphery) of the determination target image is easy as shown in FIGS. 19A and 19B, although it depends on the shape of the determination target image. Can be calculated. In addition, the comparison process with the minimum diameter of the structural element (here, the same shape and size as the region of interest) is easy, and the method of this embodiment has a lighter processing load than the first embodiment.

  Further, the processing unit 100 obtains the shortest distance to the boundary of the covering region at each of the plurality of points set on the outer periphery of the determination target image, and among the shortest distances obtained at each of the plurality of points. You may perform the process which calculates | requires the maximum value of. And the process part 100 acquires the minimum diameter of a structural element as a process using a structural element, Based on the comparison process of the acquired minimum diameter and the maximum value of the shortest distance, among determination object images Processing is performed to determine whether or not a structural element is included in a non-covering region that is a region other than the covering region.

  Accordingly, it is possible to perform a similar process by providing a plurality of points on the outer peripheral side of the determination target image. In the present embodiment, since the information corresponding to the maximum diameter of the non-covering region is obtained from the distance between the outer periphery of the determination target image and the boundary of the covering region, the plurality of points serving as the starting points of the distance calculation are as follows: It may be provided on the determination target side or on the covering region side. The method of setting a plurality of points on the boundary of the covering region and the method of setting a plurality of points on the boundary of the determination target image may be used in combination, or one of them may be used. For example, two methods may be used together if importance is attached to the determination accuracy, and one method may be used if reduction of processing load is important. Alternatively, as described above, it may be determined whether or not the two methods are used in consideration of the shape of the determination target image (whether it is circular or square).

6). Fifth Embodiment In the first embodiment, the third embodiment, and the fourth embodiment, different processes have been described as determinations on whether or not deletion is possible using a structural element. In the present embodiment, any one of them has been described. A method of using both the above process and the deletion determination process based on the coverage will be described.

  FIG. 20 shows a configuration example of the image processing apparatus of this embodiment. The image processing apparatus according to the present embodiment has a configuration in which a coverage rate calculation unit 1004 is added to the processing unit 100 with respect to the image processing apparatus of FIG. The coverage calculation unit 1004 calculates the coverage based on the coverage area calculated by the coverage area calculation unit 1003. Further, the deletion possibility determination unit 1005 of the present embodiment is based on the coverage calculated by the coverage calculation unit 1004 and the possibility of missing the attention area determined by the attention area missing possibility determination unit 1013. Determining whether images can be deleted.

  Details of the processing will be described below. The process for deforming the reference image to obtain the coverage area is the same as in the first embodiment. Here, the ratio of the area of the covering region to the area of the determination target image may be set as the covering rate. However, the coverage may be information indicating the degree of coverage of the determination target image by the reference image, and is not limited to the ratio / ratio. For example, as shown in FIG. 21, a plurality of points set in the determination target image shape are projected on the reference image based on the deformation information, and the number of points included in the reference image among the plurality of points. May be used as the coverage.

  Deletability determination based on the coverage rate may be performed by comparing with a given threshold value. If the coverage rate is equal to or greater than the threshold value, the determination target image can be deleted. If the coverage rate is smaller than the threshold value, the determination target image Can be deleted. Even if the determination target image is deleted based on the determination as to whether or not deletion is possible based on the coverage ratio, it is guaranteed that a region of a certain proportion (a level corresponding to the threshold value) captured in the determination target image can be covered by the reference image.

  Various methods are conceivable for integrating the deletion permission determination based on the structural element and the deletion permission determination based on the coverage ratio. For example, two determinations may be made and the AND condition may be taken. In other words, when it is determined that the image can be deleted based on the structural element and it is determined that the image can also be deleted based on the coverage, it is determined that the image can be deleted as a final determination result for the determination target image. From the viewpoint of reducing the possibility of overlooking the attention area, it is desirable to delete it when it can be deleted by both of the two determinations instead of the OR condition.

  Alternatively, one process may be performed first, and then the other process may be performed. For example, if it is determined that deletion is possible based on the coverage ratio first and the deletion is not possible, the final determination result is also determined as non-deletable, and the determination based on the structural element is not performed. On the other hand, for those that are determined to be deletable based on the coverage rate, a determination is made based on the structural element, and those that can be deleted there can also delete the final result, and those that cannot be deleted cannot be deleted. do it. By doing so, it is possible to reduce the number of images to be processed later in time series, so that the processing load can be reduced. In this case, the processing load can be efficiently reduced by performing the processing that is assumed to have a large number of images that are determined not to be deleted among the two processings first.

  In the above embodiment, the processing unit 100 calculates the coverage of the determination target image by the reference image based on the deformation information between the reference image and the determination target image, and determines the determination target based on the calculated coverage. You may perform the 1st deletion permission determination process which performs the deletion permission determination of an image. At the same time, the processing unit 100 performs a second deleteability determination process for determining whether or not the determination target image can be deleted based on the results of the process using deformation information and the process using the structural element.

  As a result, it is possible to perform the deletion permission determination process based on both the first deletion permission determination process and the second deletion permission determination process, so that the determination accuracy can be improved. At that time, both the first and second determinations may be performed on all the images. However, if one of the determinations is performed first and the image to be subjected to the other determination is limited, the processing is performed. The load can be reduced.

  As mentioned above, although 5 Embodiment 1-5 and its modification which applied this invention were demonstrated, this invention is not limited to each Embodiment 1-5 and its modification as it is, Implementation stage The constituent elements can be modified and embodied without departing from the spirit of the invention. Various inventions can be formed by appropriately combining a plurality of constituent elements disclosed in the above-described first to fifth embodiments and modifications. For example, some constituent elements may be deleted from all the constituent elements described in the first to fifth embodiments and the modified examples. Furthermore, you may combine suitably the component demonstrated in different embodiment and modification. In addition, a term described together with a different term having a broader meaning or the same meaning at least once in the specification or the drawings can be replaced with the different term anywhere in the specification or the drawings. Thus, various modifications and applications are possible without departing from the spirit of the invention.

BO1 outer boundary, BO2 inner boundary, RE residual area, 100 processing section,
200 image sequence acquisition unit, 300 storage unit, 1001 reference image selection unit,
1002 determination target image selection unit, 1003 covering area calculation unit,
1004 Coverage rate calculation unit, 1005 Deletability determination unit, 1007 Reference image selection unit,
1008 Partial image sequence setting unit, 1009 Summary image sequence determination unit,
1011 required coverage area generation unit, 1012 required coverage area inclusion determination unit,
1013 attention area miss possibility determination unit, 1014 structural element generation unit,
1015 Deformation information acquisition unit

Claims (23)

An image sequence acquisition unit for acquiring an image sequence having a plurality of images;
A processing unit that performs image summarization processing for acquiring a summary image sequence by deleting a part of the plurality of images of the image sequence acquired by the image sequence acquisition unit;
Including
The processor is
A reference image and a determination target image are selected from the plurality of images,
Determining whether or not the determination target image can be deleted based on a result of processing using deformation information between the reference image and the determination target image and processing using a structural element corresponding to a region of interest. An image processing apparatus. In claim 1,
The process using the deformation information is as follows:
A process of deforming at least a part of one of the reference image and the determination target image using the deformation information;
The process using the structural element is as follows:
An image processing apparatus comprising: a shrinking process using the structural element; or a process for determining whether or not the structural element is included in an uncovered area that is an area where the determination target image is not covered by the reference image. . In claim 1,
The processor is
As a process using the deformation information,
The reference image is deformed using the deformation information to obtain a covered area that is an area where the determination target image is covered by the reference image, and an area other than the covered area of the determination target image is set as an uncovered area Perform the requested process,
As a process using the structural element,
An image processing apparatus that performs shrinkage processing by the structural element on the uncovered region. In claim 3,
The processor is
An image processing apparatus that determines that the determination target image cannot be deleted when there is a remaining area as a result of the contraction process. In claim 1,
The processor is
As a process using the structural element,
An image processing apparatus that performs a process of obtaining a covering area by performing a contraction process using the structural element on the determination target image. In claim 5,
The processor is
As a process using the deformation information,
Based on the deformation information between the reference image and the determination target image, performing a process of deforming the coverage area,
The processor is
An image processing apparatus that determines that the determination target image is erasable when the required coverage area after deformation is included in the reference image. In claim 5,
The processor is
As a process using the deformation information,
Based on the deformation information between the reference image and the determination target image, a process of deforming the reference image,
The processor is
An image processing apparatus that determines that the determination target image is erasable when the required coverage area is included in the deformed reference image. In claim 1,
The processor is
As a process using the deformation information,
The reference image is deformed using the deformation information to obtain a covering region that is an area where the determination target image is covered by the reference image, and a plurality of points set on the obtained boundary of the covering region are obtained. At each point, the shortest distance to the outer periphery of the determination target image is obtained, and a process for obtaining a maximum value among the shortest distances obtained at each point of the plurality of points is performed.
As a process using the structural element,
The minimum diameter of the structural element is acquired, and based on a comparison process between the acquired minimum diameter and the maximum value of the shortest distances, non-covering that is an area other than the covering area of the determination target image An image processing apparatus that performs a process of determining whether or not the structural element is included in an area. In claim 1,
The processor is
As a process using the deformation information,
The reference image is deformed using the deformation information to obtain a covered region that is a region where the determination target image is covered by the reference image, and each of a plurality of points set on the outer periphery of the determination target image At the point, the shortest distance to the boundary of the covering region is obtained, and the maximum value of the shortest distances obtained at each point of the plurality of points is determined.
As a process using the structural element,
The minimum diameter of the structural element is acquired, and based on a comparison process between the acquired minimum diameter and the maximum value of the shortest distances, non-covering that is an area other than the covering area of the determination target image An image processing apparatus that performs a process of determining whether or not the structural element is included in an area. In any one of Claims 1 thru | or 9,
The processor is
Based on the deformation information between the reference image and the determination target image, a coverage of the determination target image by the reference image is calculated, and based on the calculated coverage, whether the determination target image can be deleted is determined. A first deletion permission determination process for performing
A second deleteability determination process for determining whether or not the determination target image can be deleted based on a result of the process using the deformation information and the process using the structural element;
An image processing apparatus characterized by In any one of Claims 1 thru | or 10.
The processor is
For the image between the reference image and the determination target image in the image sequence among the reference image, the determination target image, and the plurality of images, the deformation information between the adjacent images is obtained, and the obtained neighbor An image processing apparatus, wherein the deformation information between the reference image and the determination target image is obtained based on the deformation information between the matching images. In any one of Claims 1 thru | or 11,
The processor is
An image processing apparatus, wherein the structural element having a size proportional to the size of the region of interest is set. In any one of Claims 1 to 12,
When the first to Nth images (N is an integer of 2 or more) are input as the input image sequence,
The processor is
The pth (p is an integer satisfying 1 ≦ p ≦ N) image is selected as the first reference image, the qth image (q is an integer equal to or greater than p + 2) is selected as the second reference image, and the first an image of r (r is an integer satisfying p + 1 ≦ r ≦ q−1) is selected as the determination target image;
The process using the deformation information between the first reference image and the determination target image, the process using the deformation information between the second reference image and the determination target image, and the structural element are used. An image processing apparatus that determines whether or not the determination target image can be deleted based on a result of the processing performed. In claim 13,
The processor is
The second reference image is selected from the second reference image selection section in which the start point and the end point corresponding to the p + 2 to Nth images are set, and the first reference image and the second reference image are selected. Based on the determination whether the determination target image can be deleted,
When it is determined that the (p + 1) -th to (q-1) -th images can be deleted, an x-th image (x is an integer satisfying x> q) included in the second reference image selection section is newly added to the first image. The image processing apparatus is selected as a second reference image and updates the start point of the second reference image selection section to the qth image. In claim 14,
The processor is
If it is determined that at least one of the p + 1 to q−1 images cannot be deleted, the y th image (y is an integer satisfying y <q) included in the second reference image selection section. Is selected as a new second reference image, and the end point of the second reference image selection section is updated to the qth image. In claim 14 or 15,
The processor is
As a result of updating the start point or the end point of the second reference image selection section, when the start point and the end point are adjacent,
A process of including one of the plurality of images selected as the first reference image in the summary image sequence;
One of the plurality of images corresponding to the start point, and among the plurality of images, a partial image sequence composed of a rear one in the input image sequence than one of the plurality of images corresponding to the start point An image processing apparatus that is set as an input image sequence and performs the process again with the p value set to 1 for the set input image sequence. In claim 13,
The processor is
An image processing apparatus, wherein when it is determined that the (p + 1) -th to (q-1) -th images can be deleted, the (q + 1) -th image is selected as the second reference image. In claim 17,
The processor is
If it is determined that at least one of the p + 1 to q−1 images cannot be deleted, one of the plurality of images selected as the first reference image is included in the summary image sequence. And a partial image sequence composed of the q-1st to Nth images is set as the input image sequence, and the value of p is set to 1 for the set input image sequence. An image processing apparatus that performs processing again. An image sequence acquisition unit for acquiring an image sequence having a plurality of images;
A processing unit that performs image summarization processing for acquiring a summary image sequence by deleting a part of the plurality of images of the image sequence acquired by the image sequence acquisition unit;
Including
The processor is
A reference image and a determination target image are selected from the plurality of images,
Based on deformation information between the reference image and the determination target image, obtain a covered area that is an area where the determination target image is covered by the reference image;
An image processing apparatus that determines whether or not the determination target image can be deleted by determining a possibility of missing an attention area that is an area of interest based on the covering area. An image sequence acquisition unit for acquiring an image sequence having a plurality of images;
As a processing unit that performs image summarization processing for acquiring a summary image sequence by deleting a part of the plurality of images of the image sequence acquired by the image sequence acquisition unit,
Make the computer work,
The processor is
A reference image and a determination target image are selected from the plurality of images,
Determining whether or not the determination target image can be deleted based on a result of processing using deformation information between the reference image and the determination target image and processing using a structural element corresponding to a region of interest. Program to do. An image sequence acquisition unit for acquiring an image sequence having a plurality of images;
As a processing unit that performs image summarization processing for acquiring a summary image sequence by deleting a part of the plurality of images of the image sequence acquired by the image sequence acquisition unit,
Make the computer work,
The processor is
A reference image and a determination target image are selected from the plurality of images,
Based on deformation information between the reference image and the determination target image, obtain a covered area that is an area where the determination target image is covered by the reference image;
A program for determining whether or not the determination target image can be deleted by determining a possibility of missing an attention area that is an area of interest based on the covering area. Obtain an image sequence with multiple images,
A reference image and a determination target image are selected from the plurality of images in the image sequence,
Based on the result of processing using deformation information between the reference image and the determination target image and processing using a structural element corresponding to the region of interest, it is determined whether or not the determination target image can be deleted,
An image processing method for performing image summarization processing for acquiring a summary image sequence by deleting a part of the plurality of images of the image sequence based on the determination result of the deletion possibility. Obtain an image sequence with multiple images,
A reference image and a determination target image are selected from the plurality of images in the image sequence,
Based on deformation information between the reference image and the determination target image, obtain a covered area that is an area where the determination target image is covered by the reference image;
Based on the covering area, by determining the possibility of missing the attention area that is the area to be noticed, it is possible to delete the determination target image,
An image processing method for performing image summarization processing for acquiring a summary image sequence by deleting a part of the plurality of images of the image sequence based on the determination result of the deletion possibility.

Images