JP2008217493A - Animation by recording image changing over time, image data for creating animation and its creation method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an animation creation method in which, based on a plurality of photographs with change over time of an observation object recorded, difference generated due to a cause other than an observation purpose, existing between each of the photographs, can be appropriately modified and the smoothed animation can be provided. <P>SOLUTION: The method of this invention is equipped with a difference modifying process between images P1, a continuity granting process between the images P2, and an animated process P3. The previous two processes among these are constituted with each of the processing processes by a proper image processing program, to be executed using a computer, and the last animated process P3 is constituted with each of the processing processes by the proper animation creation program to be executed using the computer. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to a moving image by a time-varying recording image, image data for moving image creation, and a method for creating the moving image, and in particular, based on a plurality of photographs in which changes with time of an observation target are recorded, The present invention relates to a method for creating a moving image using a time-varying recording image, in which a difference caused by a cause is corrected and a smooth moving image can be provided.

  In the fundus photograph, the optic nerve head, the retina, the blood vessels of the retina, etc., which are the exit of the optic nerve that conveys information captured by the retina to the cerebrum, are observed and recorded. In ophthalmology, fundus photographs are widely used as basic data for diagnosis, treatment, and surgery of various ophthalmic diseases. In addition to being able to obtain detailed information about the current state as a photograph, that is, a still image, the progression of the disease state, especially by comparing and observing multiple fundus photographs that have been observed, photographed and stored over time from the past to the present Since the healing process can be easily observed and compared, fundus photography is indispensable in ophthalmology.

  Fundus photographs are also widely used in health examinations because they are the only ones in the human body that can directly observe and record arteries and veins from outside the body. In fundus examinations, blood vessels can be directly observed to obtain information on the color and shape of the blood vessels, or the shape of running, which can be used for the diagnosis and treatment of diseases such as eye disease, hypertension, and diabetes. it can. In addition, since the blood vessels in the fundus are a part of the brain blood vessels, it is possible to estimate the state of the blood vessels in the brain.

  For such important fundus photography technology, various technical proposals such as fundus camera, fundus image processing device and computer program, fundus photo diagnosis system, fundus photo panorama system, fundus stereoscopic display / coordinate measurement device, etc. And improvements have been made. The following patent documents are a part thereof.

JP 2002-269539 A "Image processing apparatus, image processing apparatus method, computer-readable recording medium recording an image processing program, and diagnosis support system using the same" Japanese Patent Laid-Open No. 2002-34925, “Method and apparatus for restoring fundus three-dimensional pattern by overlapping fundus images taken from multiple directions” JP 2006-246991 A "Information processing apparatus and method, and program" Japanese Laid-Open Patent Publication No. 2000-166874 “Fundamental Image Brightness Correction Method, Apparatus, and Recording Medium Recording the Program”

  In the diagnosis and treatment of various ophthalmic diseases in ophthalmology, the ability to compare and observe a plurality of fundus photographs that have been observed, photographed and stored over time in time series is very useful as described above. Rather than looking at each one, if you make these videos time-sequentially edited and switched at high speed, you can observe and grasp the changes in the fundus in a time-sequential manner more easily, clearly, and dynamically Can do.

  However, there is a problem that fundus photographs are different depending on the shooting conditions even for the same person and the same date and time. That is, even in the fundus of the same subject, the images are completely different depending on the shooting conditions of each time, and a shift between the images occurs. Specifically, the photographer's skill, the condition of the subject, the difference in camera angles, and the like. In addition, the fundus image changes due to changes in the refractive state, that is, myopia, hyperopia, astigmatism, corneal shape, and even the axial length. Therefore, even if a series of fundus photographs taken in time series is simply edited into a moving image using moving image creation software or the like, it becomes a confused image, and it is difficult to grasp the expected fundus change.

  Although the situation and problems of the related art in fundus photography have been briefly described above, an example in which such problems exist when moving a photograph image observed and photographed in time series is not limited to fundus photographs.

  Therefore, the problem to be solved by the present invention is based on a plurality of photographs in which changes in observation objects over time are recorded based on the above-mentioned state of the art, and there are differences caused by causes other than the observation purpose existing between the photographs. As a new technology capable of providing a smooth moving image that is appropriately modified, it is to provide a moving image using time-lapse recorded images, image data for moving image generation, and a method for generating the same. In particular, the present invention is to provide a moving image by using a time-change recording image, image data for moving image creation, and a method for producing the same, which can be used in fundus photography and can be applied to other fields without being limited thereto.

  As a result of studying the above problems, the inventor of the present application has set a reference point in a specific photo using appropriate image processing software before using the video creation software, and matches each photo with each other. The present inventors have found that the above-described problems can be solved by correcting a difference caused by a cause other than the observation purpose existing in the image and applying continuity between the photographs. That is, the invention claimed in the present application, or at least the disclosed invention, as means for solving the above-described problems is as follows.

[1] Using an image processing program and a moving image creation program executed using an electronic computer, the temporal change of the target is animated by a plurality of time-change recorded images obtained by continuously recording the temporal change of the target. A “moving image creation method using time-varying recorded images”, which comprises (I) an inter-image difference correction process, (II) an inter-image continuity providing process, and (III) an animation process,
The inter-image difference correction process (I) is performed by an image processing program.
(I-1) An adjustment reference point is set in one specified image (I-2) A corresponding point in another image is overlaid on the set reference point (I-3) As a center, differences such as rotation / distortion / enlargement / reduction between both images are corrected. (I-4) By going through each procedure in which (I-2) and (I-3) are repeated as many times as necessary, This is a process in which each image is obtained by correcting the differences between multiple images for animation.
The inter-image continuity imparting step (II) is performed by an image processing program.
(II-1) Between two adjacent images in time series, the opacity of the previous image is set to 100%, or preparation is made in such a state (II-2) opacity of the image after Is changed from 0% to less than 100% (II-3) An integrated image formed by integrating the previous image and the subsequent image is separately stored (II-4) as necessary When the above (II-1) to (II-3) are repeated, and a plurality of integrated images with different opacity of the subsequent images are formed and stored, and then converted into a moving image. Each image is processed to obtain an integrated image so that smooth video changes can be obtained.
The animation process (III) is performed by a movie creation program.
(III-1) Each integrated image, that is, the playback time of one frame (hereinafter referred to as “image playback time”) is set. (III-2) Each integrated image created for moving images in (II) above. Are read in the order in which the opacity is changed (III-3) Each read integrated image is added to the timeline or edited (III-4) If necessary, for a predetermined integrated image (III-5) A method of creating a moving image using a time-varying recording image, comprising: (III-5) each step of saving the edited and created moving image.

[2] The “identified one image” in (I-1) is the earliest among a plurality of time-varying recorded images obtained by continuously recording a subject's time-varying still image. The moving image creating method using the time-change recorded image according to [1].
[3] In changing the opacity in the subsequent images in (II-2) to (II-3) and storing the integrated image, a plurality of images in which the opacity in the subsequent images is changed at equal intervals. The moving image creating method using the time-varying recorded image according to [1] or [2], wherein the moving image is created and stored based on the above.
[4] The method of creating a moving image using a time-varying recording image according to any one of [1] to [3], wherein the time-varying recording image is a fundus photograph.

[5] The method for creating a moving image using a time-change recorded image according to [4], wherein any one of a macular fovea, a retinal blood vessel, or an optic disc blood vessel is used as the reference point.
[6] A moving image based on a time-change recorded image created by any one of the methods [1] to [5].
[7] Using an image processing program and a moving image creation program executed using an electronic computer, the temporal change of the target is animated by a plurality of time-change recorded images in which the temporal change of the target is continuously recorded as a still image. “Image data for making moving image of time-varying recording image” that can be used in “moving image creation method with time-varying recording image”, the image data being set in one specified image by an image processing program Differences such as rotation, distortion, and enlargement / reduction from the image are corrected based on the correction reference point, and the opacity of the previous image between two images adjacent in all time series Is set to 100%, the opacity of the subsequent image is set to 0%, and the further image is integrated with the image after the change in which the opacity is set to more than 0% and less than 100%. One or a plurality of integrated images are separately formed, and image data for animation of a time-varying recording image is provided.

  The moving image by the time-varying recording image of the present invention, the image data for moving image creation and the creation method thereof are configured as described above, and according to this, based on a plurality of photographs that record the temporal change of the observation target, Differences caused by causes other than the observation purpose existing between the photographs are appropriately corrected, and a smooth moving image can be created.

  In addition, as for the case where the present invention is applied to the conversion of fundus photographs, it is possible to correct differences due to photographic conditions that are different from changes in the fundus, and changes in fundus images due to changes in the refraction state, using image processing software. Furthermore, by changing the opacity of each image, it is possible to make the original fundus change more easily understandable, and by making it into a moving image using moving image creation software, it is possible to grasp the change over time more dynamically.

Hereinafter, the present invention will be described in more detail with reference to the drawings.
FIG. 1 is a flowchart showing a moving image creation method using a time-varying recording image of the present invention. As shown in the figure, the method of the present invention comprises an inter-image difference correction process P1, an inter-image continuity imparting process P2, and an animation process P3. Of these, the previous two steps are constituted by respective processing steps by an appropriate image processing program executed using an electronic computer. Further, the final animation process P3 is configured by each processing process by an appropriate animation creation program executed using an electronic computer.

  First, the inter-image difference correction process P1 is set as a process P1-1 (reference point setting process) in which a correction reference point is set in one specified image among a plurality of images for creating a moving image. A process P1-2 (corresponding point superimposition process) in which corresponding points in other images are superimposed on the reference point, and differences such as rotation, distortion, and enlargement / reduction between the two images around the reference point are corrected. This process includes processes P1-3 (difference modification / deformation process). Here, the processing steps P1-2 and P1-3 are repeated for the required number of sheets. Note that this repeated processing process may be grasped as a processing process P1-4 (repeated processing process).

  As described above, in the inter-image difference correction process P1 including the three (or four) process steps P1-1 and the like, the difference between a plurality of images for animation is corrected through these process steps. Each image is obtained. That is, in the processing step P1-1, the retouching reference point is set using the specified one image, and then in the processing step P1-2, the corresponding point in the other image matches the set reference point. In the process P1-3, the differences such as rotation, distortion, and enlargement / reduction between the two images are corrected with the superimposed reference point as the center.

  The “difference” corrected here is not a difference related to the “time-dependent change” of the target object of the moving image finally created by this method, but is an error in the observation / measurement method such as the shooting conditions. Differences or noise differences are the main.

  As the “specified one image” in the process P1-1 in which the retouching reference point is set, the earliest one of the plurality of time-change recorded images obtained by continuously recording the time-dependent change of the target still image. It can be devoted to it. Moreover, it is natural as a flow for smooth implementation of moving image creation, which is an object of the present invention, and it is also desirable from the viewpoint of workability and work efficiency.

  Next, the inter-image continuity imparting process P2 is a process P2-1 (pre-image opacity setting process in which the opacity of the previous image is set to 100% between two adjacent images in time series. Includes a wide range of cases where preparation is made in such a state without passing through special processing.) A process P2 in which the opacity of a subsequent image is changed from 0% to less than 100% is added. -2 (opacity change process), and a process P2-3 (image composition process) in which an integrated image formed by integrating the previous image and the subsequent image is separately stored. Is done.

  Here, the processing steps P2-1 to P2-3 are repeated as necessary, and a plurality of integrated images having different opacity of the subsequent images are formed and stored. Note that this repetitive processing process may be grasped as a processing process P2-4 (repetitive processing process). As will be described later in the embodiment, rather, the process P2-4 is repeated to some extent, so that a predetermined number of integrated images in which the opacity of the subsequent images is gradually different can be created and stored. It is desirable to enhance the effect of providing continuity between images, which is the purpose of this process P2.

  Particularly, in the change of opacity in the subsequent images in (II-2) to (II-3) and the storage of the integrated image, based on a plurality of images in which the opacity in the subsequent images is changed at equal intervals. It is desirable to create and save.

  For example, the combination “PS” of the opacity of the previous image (hereinafter “first image” in the present paragraph and the next paragraph) and the subsequent image (similarly “rear image”) is “100-20”. , “100-40”, “100-60”, “100-80”, newly creating each integrated image “100-0”, which displays only the front image, and only the rear image In addition, the integrated image group having a five-step gradation effect is formed between both images together with “(100) -100”.

  More preferably, as will be described later in the embodiment, the opacity combination “PS” is “100-10”, “100-20”, “100-30”,. If each integrated image of “100-90” is newly created, an integrated image group having a gradation effect of 10 steps is formed between both images, and the inter-image continuity providing effect is Furthermore, the smoothness at the time of moving image reproduction further increases. Of course, there is no limitation on further increasing the number of integrated image formations as long as it is allowed, or regardless of the existence of such circumstances, in consideration of workability, cost, cost effectiveness, and the like.

  That is, it can be said that the inter-image continuity providing process P2 is typically a process of forming an integrated image group having a fade-in effect by gradually increasing the opacity of the subsequent image.

  As described above, in the inter-image continuity imparting process P2 including the three (or four) process processes P2-1 and the like, each process is performed so that a smooth video change can be obtained when moving to a moving image. The images are processed to create the required number of integrated images. That is, in the process P2-1, the opacity of the previous image is set or prepared as 100% between two adjacent images in time series, and is used as a reference for providing continuity between images. In P2-2, a change is made so that the opacity of the image after being superimposed on the previous image becomes an image having a smaller opacity than the previous image.

  Next, in the processing step P2-3, the previous image and the image whose opacity has been changed to be smaller than this are integrated to form an integrated image, which is stored separately. As described above, it is desirable to repeat the process in the process P2-4 to some extent. As a result, a predetermined number of integrated images in which the opacity of subsequent images is gradually different can be created and stored, and smoothness during reproduction of a finally created moving image can be improved.

  The inter-image continuity providing process P2 described above is a process for preparing different images by changing each of the still images that will eventually form a moving image little by little in time series. It is. For this purpose, an appropriate image processing computer program (such as PHOTOSHOP (registered trademark)) is used to set the opacity of one original image, for example, (0%) 10%, 20%,. By changing 10%, such as 80%, 90%, and 100%, 10 images (integrated images) are newly synthesized to prepare a total of 11 images.

  Finally, in the animation process P3, a process P3-1 (image reproduction time setting process) in which the image reproduction time of each integrated image, that is, one frame is set by the animation creation program, and the inter-image continuity provision process P2 is set. Process P3-2 (image reading process) in which each integrated image created in step 1 is read in the order in which the opacity is changed, and each read integrated image is added to the timeline or edited Process P3-3 (image editing process), process P3-4 (reproduction time setting changing process) in which the setting of the image reproduction time is changed for a predetermined integrated image, if necessary, edited and created moving image Is stored in each step of process P3-5 (moving image storage process). As a result, the “moving image based on the time-change recorded image” of the present invention is finally created.

  As an image processing program used for the inter-image difference correction process P1 and the inter-image continuity imparting process P2, for example, Adobe (registered trademark) Photoshop (registered trademark, manufactured by Adobe Systems Incorporated, USA) is used, and animation is made. As a moving image creation program used in the process P3, Windows (registered trademark) movie maker (manufactured by Microsoft Corporation, USA) can be suitably used. However, other software can be used as a matter of course, and the present invention is not limited by the computer program used.

  Hereinafter, taking a fundus photograph as an example, the present invention “moving image creation method using time-varying recorded images” will be described. However, this is an example, and it goes without saying that the present invention is not limited to the fundus photograph moving image. .

  FIGS. 2 to 49 << 005 to 160 >> are explanatory diagrams showing the process of creating a moving image actually performed using the method of the present invention, and how the fundus changes based on the fundus photographs recorded over time. It shows the process of creating a “fundus change movie” to be expressed. This is a copy of the display screen of image processing software or video creation software. The characters of Adobe (registered trademark), Photoshop (registered trademark), and Windows (registered trademark) displayed in each figure are registered trademarks. A description of each drawing showing details of the moving image production method will be described later. First, a specific outline of moving image creation in this example will be described. (Note that << >> is the applicant's management drawing number. The same applies below.)

  Even if the fundus photograph is the fundus of a specific person, the obtained image has a large difference or image shift depending on the photographing conditions. This is not a manifestation of the change that should be tracked, which is the original purpose, but should be excluded as a disturbance factor. For example, the lack of skill of the photographer, wrinkles, the condition of the subject, the camera angle, etc. There is an element with uncertainty. Image processing software such as PHOTOSHOP (registered trademark) solves the problem of moving image creation by removing such disturbance factors. In other words, after the retouching reference point is determined as described above, image deformation processing is performed on software for removing a disturbance factor between two consecutive images, and the images are matched.

In this example, the above-mentioned Adobe (registered trademark) Photoshop (registered trademark, manufactured by Adobe Systems Incorporated, USA) is used as the image processing software. Made).
In the following, the points for creating this example movie are indicated by bullets. And B. The image processing software is C.I. Let's use video creation software.

A. Points of the fundus photo difference correction work in the inter-image difference correction process (P1 in FIG. 1) a. Set a reference point for the first fundus photo. As the reference point, a macular central fovea, a retinal blood vessel, an optic disc blood vessel, or the like can be used, but in the illustrated example, the macular central fovea was used as a reference point.
b. Based on the set reference point, the next time-series fundus photo is superimposed. Here, only positions corresponding to the reference points are matched.
c. The difference between the two images is corrected around the reference point. Usually, the image that is overlaid later is modified. As a correction method, a “rotation” process for correcting a positional shift, a “distortion” correction process for correcting a positional relationship or a ratio, an “enlargement / reduction” process, and the like are appropriately used.
d. b. c. Repeat as necessary to correct differences in all fundus photos used to create the video.

B. Points of image creation work for moving images in inter-image continuity provision process (P2 in FIG. 1) a. After correcting the difference, the opacity of the later fundus photograph in the time series is superimposed and synthesized on the previous fundus photograph image.
b. The opacity is changed sequentially from 0 to 100% (10% interval or 5% interval), a. Each of the synthesis described above is performed, and the synthesized image (integrated image) is stored.
c. Note that the composite image (integrated image) when the opacity is 0% is the same as the original image, and the composite image (integrated image) when the opacity is 100% is the superimposed image, that is, the back image. It becomes the same as the image of.

C. Points of animation creation work in the animation process (P3 in FIG. 1) a. The image playback time, that is, the playback time for one frame is set to 0.250 seconds.
b. Read images created for moving images in the order of changing opacity.
c. Add the scanned image to the timeline.
d. Change the playback time of the first and last images to 2 seconds.
e. For other images, the playback time is changed to 1.5 seconds for each cycle of 0 to 100% opacity. However, when using FAG fundus photographs (fundus photographs taken with fluorescein fluorescence fundus angiography. Fluorescence dyes injected into veins / fundus photograph taken when fluorescein circulates in the blood vessels of the fundus), playback time should not be changed. And
f. Save the edited movie (movie) for playback.

2 to 49 << 005 to 160 >> are explanatory diagrams showing a series of creation processes for the “fundus change video” according to the present embodiment described above, and the explanation of each figure is as follows.
<A-0 Difference adjustment work preparation process>
FIG. 2 << 005 >>: An image processing layer (working layer) is created on the activated image processing software.
FIG. 3 << 007 >>: Image data (file) to be read is selected.
FIG. 4 << 008 >>: The selected image data is read.
FIG. 5 << 014 >>: The read image data is copied to the work layer. The copied image is treated as “Layer 1” and displayed on the screen.
FIG. 6 << 015 >>: Similarly, the selected second image data is read.
FIG. 7 << 017 >>: The read second image data is copied to the work layer, treated as “layer 2”, and displayed on the screen.

<A-1 Reference point setting process>
FIG. 8 << 021 >>: Layer 1 (the previous image) is appropriately changed (50% in the figure) while layer 1 (the previous image) is changed as a reference for setting a reference point for correcting differences between images. The situation of the specific difference between the images) is confirmed.
FIG. 9 << 025 >>: After the validity of the reference point setting position for difference correction is confirmed, layer 2 is switched to the “non-display” state on the screen, and layer 1 is displayed on the screen.
FIG. 10 << 030 >>: A predetermined region of layer 1 is enlarged and displayed, and a tool for marking a reference point is selected.
FIG. 11 << 031 >>: A mark is added to the position on the image determined as the reference point.

<A-2 Corresponding point overlapping process>
FIG. 12 << 034 >>: The “non-display” state of layer 2 is switched to the “display” state, and it is confirmed whether or not there is a deviation between the mark of the reference point set in layer 1 and the corresponding position of layer 2 Is done.
FIG. 13 << 035 >>: The image is enlarged and displayed with the reference point as the center, and the state of deviation between the reference point mark and the position of the layer 2 corresponding thereto is confirmed more accurately.
FIG. 14 << 036 >>: The movement process of layer 2 is performed so that the corresponding position of layer 2 overlaps the mark of the reference point.
FIG. 15 << 040 >>: In a state where the layers 1 and 2 are aligned with the reference point mark, the difference in the image between the layers is present in which position and area of the image, in what form, and to what extent. ,It is confirmed. This operation is performed while adjusting the opacity of the layer 2 and switching between “display” and “non-display” as appropriate.

<A-3 Difference Modification / Deformation Process (Part 1 Rotation Process)>
FIG. 16 << 044a >>: The correction process for the difference between layers 1 and 2 is started. First, the angular position around the reference point in the image of layer 2 is corrected. “Deformation” processing and “rotation” processing are selected from the “Edit” menu of the software, and the processing target area is set using a frame displayed on the layer 2 image.
FIG. 17 << 045 >>: The rotation center of the “rotation” process is set as the reference point.
FIG. 18 << 047 >>: The “rotation” process is executed so that the difference between layers 1 and 2 is eliminated.
FIG. 19 << 049 >>: As a result of the “rotation” process, the difference in angular position between layers 1 and 2 is corrected.

<A-3 Difference Modification Processing Process (Part 2 Distortion Modification Processing)>
FIG. 20 << 051 >>: Next, distortion that becomes prominent on the image of layer 2, particularly in the peripheral region, is corrected. The “deformation” process and the “distortion” process are selected from the “edit” menu of the software, and the processing target area is set using a frame displayed on the layer 2 image.
FIG. 21 << 052 >>: The center of the “distortion” process is set as the reference point.
FIG. 22 << 054 >>: The distortion in the lower right region of the layer 2 image is corrected by the expansion / contraction operation in the appropriate direction of the frame.
FIG. 23 << 057 >>: Similarly, the distortion in the upper left region of the layer 2 image is corrected by the expansion / contraction operation in the appropriate direction of the frame.
FIG. 24 << 059 >>: The distortion correction is completed.
FIG. 25 << 067 >>: Whether each deformation process executed for correcting the difference between layers 1 and 2 is appropriate, fine adjustment of opacity of layer 2 (10% interval, etc.), “display”, “non-display” It is confirmed and evaluated while switching the “display” as appropriate.
<A-4 Repeat process>
The difference correction processing performed as described above is repeated according to the required number of images.

<Preparation process of B-0 inter-image continuity provision processing>
The procedure shown in FIGS. 26 to 30 << 084 to 098 >> is an operation for erasing a part that is not necessary for a moving image and adjusting the appearance. Normally, an area is set and selected so as to be the greatest common divisor of each image area based on the earliest image, the selected area is inverted, and all images are similarly deleted. That is, all images are cut into the same shape. However, it is also possible to animate without performing these series of operations. Further, for example, as in a panoramic image, the erase area may be different for each image. In short, it is the work of adjusting the shape so that it is finally formed as a moving image that satisfies the observation purpose.
FIG. 26 << 084 >>: An image area necessary for the earliest image is set. This is usually set at the maximum.
FIG. 27 << 088 >>: Based on the area set in the process of FIG. 27, the area is changed for setting the area of the greatest common divisor with the subsequent image. This is usually smaller than the area set in the first image. In this way, a repeated area is set for all images.
FIG. 28 << 092 >>: The area of the greatest common divisor previously set is inverted.
FIG. 29 << 094 >>: Layer 2 is hidden, Layer 1 is displayed and selected, and the selected area is erased.
(The selection area is applied to the selected layer. As shown in the drawing, layer 1 is highlighted. Note that the highlighted display is displayed in blue on the actual screen.
FIG. 29-2 << 096 >>: Similarly, layer 2 is selected, and layer 2 is highlighted. On the actual screen, the reverse display is displayed in blue.
FIG. 30 << 098 >>: the selected area has been erased.

Next, in the procedure shown in FIGS. P31-1 to FIG. 31, all layers other than the background / reference point are linked and all layers are moved together. The link is necessary here because all layers need to be moved together.
Fig. P31-1 << 103 >>: shows a state where a link is set. A chain mark is displayed on layer 1 of the layer tab (the icon displayed in the second column from the left of the layer tab).
FIG. P31-2 << 104 >>: A state in which the moving tool (arrow mark) is selected is shown.
Fig. P31-3 << 105 >>: shows a state in which each image is moved while being linked.
FIG. 31 << 106 >>: This is a state in which the link icon is released.
After each image is prepared by the above processing procedure, the position of the image is appropriately adjusted.

<B-1 First Image (First Image) Creation Process>
32 and 33 show the procedure for creating the earliest image. Only the background and layer 1 are selected, the layer 1 has 100% opacity, that is, the layer 1 itself, and the background and layer 1 are integrated to create the earliest image.
FIG. 32 << 108 >>: a state in which image integration processing of layer 1 and the background is selected,
FIG. 33 << 111 >>: shows a state in which image integration processing of layer 1 and the background is performed and image data is stored.

<B-2 Opacity change process> and
<B-3 Image composition process>
FIG. 34 << 118 >>: The opacity of layer 2 is set to 10% and displayed.
FIG. 35 << 119 >>: Image integration processing of layer 2 (the subsequent image, opacity of 10%) and layer 1 (the original image) is performed.
FIG. 36 << 121 >>: Integrated image data is stored. The image stored here is a layer 1, that is, a previous image (original image) and a layer 2 (later image) having an opacity of 10%.

FIG. 37 << 125 >>: Similarly, layer 2 (the subsequent image) with an opacity of 20% is subjected to the image integration process and stored in layer 1 (the original image).
FIG. 38 «126»: Similarly, layer 1 (original image) and layer 2 (subsequent image) processed to a predetermined opacity are subjected to image integration processing and stored. The figure displays layer 2 with an opacity of 90%.
FIG. 39 << 127 >>: Finally, layer 2 (the subsequent image) with an opacity of 100% is subjected to image integration processing on layer 1 (the original image) and stored. The image stored here is Layer 2, that is, the subsequent image itself, and the trace of the original image is not displayed at all.

  FIG. 40 << 129 >>: As described above, layer 2 (the subsequent image) of each opacity is subjected to the image integration process on layer 1 (the original image) and stored separately. In this way, by preparing image data obtained by combining the original image with the gradually changing opacity of the subsequent image, and displaying them in the order of opacity in the animation process described later, The effect of fading out the original image and fading in the subsequent image can be obtained.

<B-4 Repeat process>
Through the above-described procedure, the series of opacity changing images between two images is repeated as necessary between necessary images, and image data necessary for creating a moving image is synthesized and stored.

<C-1 image playback time setting process>
FIG. 41 << 131 >>: The moving image creation software is activated and an initial screen is displayed.
FIG. 42 << 134 >>: The reproduction time of the image is set.
<C-2 Image reading process>
FIG. 43 << 137 >>: Each image data for moving image creation previously stored is read.
<C-3 Image Editing Process>
FIG. 44 << 141 >>: Each image is sequentially added to the timeline. That is, assembly editing (insert editing as necessary) is performed to create a moving image using a plurality of still images.
FIG. 45 << 145 >>: The moving image initially configured by editing each image is confirmed to be reproduced.

<C-4 Playback time setting change process>
FIG. 46 << 149 >>: Trimming of a moving image, that is, an appropriate change in playback time is set. In the figure, the first image reproduction time is changed longer than the initial setting.
FIG. 47 << 151 >>: Similarly, the last image playback time is changed longer than the initial setting.
FIG. 48 «155»: The finish of the moving image for which the processing for changing the image playback time is completed is played back and confirmed.
<C-5 Movie storage process>
FIG. 49 << 160 >>: Completed moving image data is stored.
Thus, the whole process of creating a moving image is completed.

  When the following fundus change videos were created by the movie creation method described in this example, all of them were animated well, and the temporal change of the fundus in each case was compared with the conventional still image method. It was possible to observe as much as possible and to express it clearly.

<1> Young glaucoma case 1 (female, both eyes) Animation based on fundus photographic records of 13, 15, and 18 years (late stage).
Timing: 9 seconds of use Fundus photo number: 3 (taken at the age of 13, 15 and 18, respectively)
FIG. 50A shows the created moving image start image (at the first visit). Also,
FIG. 51Z shows an end image (at the time of the last visit) of the created moving image.

<2) Glaucoma case 2 (female, both eyes) for young people Animation based on fundus photo records of 15, 17, 18, and 20 years (late stage).
Timing: 16-second fundus photograph number: 4 (shooted at the age of 15, 17, 18, and 20, respectively)
FIG. 52A shows the created moving image start image (at the first visit). Also,
FIG. 53Z shows an end image (at the time of the last visit) of the created moving image.

  In each of the above cases, information on the onset process of glaucoma was obtained in an easy-to-understand manner and in detail. The importance of following time-series changes in the diagnosis of glaucoma and the high degree of usefulness in the diagnosis and treatment process were shown. Note that all of the above cases are young glaucoma, but it goes without saying that the method of the present invention can also be applied to cases of adult glaucoma.

  Each fundus hemorrhagic disease case below is an example in which animation was performed in addition to glaucoma cases. In these cases, the animation was well formed, and the temporal changes of the fundus in each case could be observed dynamically and clearly expressed compared to the conventional method using still images.

<10> Case of central retinal vein occlusion (male, right eye) 62 years old at first visit 128 days after final visit. Animation based on the fundus photo recording of the treatment process.
Timing: 47 seconds Number of fundus photographs used: 13 days Elapsed days from the first visit of fundus photographs used: 0, 2, 4, 7, 14, 23, 30, 37, 47, 82, 96, 110, 128
FIG. 54 shows a start image (at the first visit) and an end image (at the last visit) of the created moving image.

<11> Central retinal vein occlusion case (male, right eye) 62 years old at first visit Same as the case of <10> above. Animation based on FAG fundus photo recording of the course of treatment.
Timing: 1 minute 1 second

<12> Case of ruptured retinal arteriole aneurysm (male, right eye) at the first visit, 64 years old, 821 days after the last visit. Animation based on the fundus photo recording of the treatment process.
Timing: 1 minute 28 seconds Number of fundus photographs used: 24 sheets Number of days elapsed since first visit of used fundus photographs: 0, 2, 9, 25, 39, 46, 53, 61, 71, 78, 88, 128, 142, 172, 204, 233, 278, 313, 340, 365, 456, 548, 730, 821
FIG. 55 shows a start image (at the first visit) and an end image (at the last visit) of the created moving image.

<13> Central retinal vein occlusion case (female, left eye) 45 years old at first visit 469 days after last visit. Animation based on the fundus photo recording of the treatment process.
Timing: 1 minute 18 seconds Number of fundus photographs used: 24 days Number of days elapsed since the first visit of fundus photographs used: 0, 2, 10, 14, 20, 28, 34, 41, 49, 55, 63, 70, 76, 107, 126, 150, 206, 237, 245, 293, 307, 371, 398, 469
FIG. 56 shows a start image (at the first visit) and an end image (at the last visit) of the created moving image.

  In addition to the above cases, the myopia change was also converted into a moving image based on the photographed image, and good results were obtained.

  In addition, although the animation example in the above-mentioned Example and each case is an example of animation creation in a normal fundus photograph or FAG fundus photograph, in the field of ophthalmology, for example, an HRT image (HRT: Heidelberg Retina Tomograph glaucoma Images used for diagnosis), OCT images (images obtained by optical coherence tomography, images used for diagnosis of macular edema, etc.), images of the anterior segment of the eye (anterior segment: generic term for eyelid, conjunctiva, sclera, cornea, iris, etc.) Needless to say, this can also be applied.

  According to the moving image, the moving image creating image data and the creating method thereof according to the present invention, the non-observation causes existing between the respective photos Thus, the difference caused by the above can be appropriately corrected, and a smooth moving image can be created. As an embodiment, the case where the present invention is applied to moving a fundus photograph has been described, but it is of course an invention that can be applied to other medical fields and other fields and has high industrial utility value.

It is a flowchart which shows the moving image production method by a time-change recording image of this invention. It is a figure which shows a part of process of "A-0" in explanatory drawing which shows the preparation process of the "fundus change animation" concerning the example of the present invention. It is a figure which shows a part of process of "A-0" in explanatory drawing which shows the preparation process of the "fundus change animation" concerning the example of the present invention. It is a figure which shows a part of process of "A-0" in explanatory drawing which shows the preparation process of the "fundus change animation" concerning the example of the present invention. It is a figure which shows a part of process of "A-0" in explanatory drawing which shows the preparation process of the "fundus change animation" concerning the example of the present invention. It is a figure which shows a part of process of "A-0" in explanatory drawing which shows the preparation process of the "fundus change animation" concerning the example of the present invention. It is a figure which shows a part of process of "A-0" in explanatory drawing which shows the preparation process of the "fundus change animation" concerning the example of the present invention. It is a figure which shows a part of process of "A-1" in explanatory drawing which shows the preparation process of the "fundus change animation" concerning the present invention example. It is a figure which shows a part of process of "A-1" in explanatory drawing which shows the preparation process of the "fundus change animation" concerning the present invention example. It is a figure which shows a part of process of "A-1" in explanatory drawing which shows the preparation process of the "fundus change animation" concerning the present invention example. It is a figure which shows a part of process of "A-1" in explanatory drawing which shows the preparation process of the "fundus change animation" concerning the present invention example. It is a figure which shows a part of process of "A-2" in explanatory drawing which shows the preparation process of the "fundus change animation" concerning an example of the present invention. It is a figure which shows a part of process of "A-2" in explanatory drawing which shows the preparation process of the "fundus change animation" concerning an example of the present invention. It is a figure which shows a part of process of "A-2" in explanatory drawing which shows the preparation process of the "fundus change animation" concerning an example of the present invention. It is a figure which shows a part of process of "A-2" in explanatory drawing which shows the preparation process of the "fundus change animation" concerning an example of the present invention. It is a figure which shows a part of process of "A-3 (the 1)" in explanatory drawing which shows the preparation process of the "fundus change animation" concerning an example of the present invention. It is a figure which shows a part of process of "A-3 (the 1)" in explanatory drawing which shows the preparation process of the "fundus change animation" concerning an example of the present invention. It is a figure which shows a part of process of "A-3 (the 1)" in explanatory drawing which shows the preparation process of the "fundus change animation" concerning an example of the present invention. It is a figure which shows a part of process of "A-3 (the 1)" in explanatory drawing which shows the preparation process of the "fundus change animation" concerning an example of the present invention. It is a figure which shows a part of process of "A-3 (the 2)" in explanatory drawing which shows the preparation process of the "fundus change animation" concerning an example of the present invention. It is a figure which shows a part of process of "A-3 (the 2)" in explanatory drawing which shows the preparation process of the "fundus change animation" concerning an example of the present invention. It is a figure which shows a part of process of "A-3 (the 2)" in explanatory drawing which shows the preparation process of the "fundus change animation" concerning an example of the present invention. It is a figure which shows a part of process of "A-3 (the 2)" in explanatory drawing which shows the preparation process of the "fundus change animation" concerning an example of the present invention. It is a figure which shows a part of process of "A-3 (the 2)" in explanatory drawing which shows the preparation process of the "fundus change animation" concerning an example of the present invention. It is a figure which shows a part of process of "A-3 (the 2)" in explanatory drawing which shows the preparation process of the "fundus change animation" concerning an example of the present invention. It is a figure which shows a part of process of "B-0" in explanatory drawing which shows the preparation process of the "fundus change animation" concerning the example of the present invention. It is a figure which shows a part of process of "B-0" in explanatory drawing which shows the preparation process of the "fundus change animation" concerning the example of the present invention. It is a figure which shows a part of process of "B-0" in explanatory drawing which shows the preparation process of the "fundus change animation" concerning the example of the present invention. It is a figure which shows a part of process of "B-0" in explanatory drawing which shows the preparation process of the "fundus change animation" concerning the example of the present invention. It is a figure which shows a part of process of "B-0" in explanatory drawing which shows the preparation process of the "fundus change animation" concerning the example of the present invention. It is a figure which shows a part of process of "B-0" in explanatory drawing which shows the preparation process of the "fundus change animation" concerning the example of the present invention. It is a figure which shows a part of process of "B-0" in explanatory drawing which shows the preparation process of the "fundus change animation" concerning the example of the present invention. It is a figure which shows a part of process of "B-0" in explanatory drawing which shows the preparation process of the "fundus change animation" concerning the example of the present invention. It is a figure which shows a part of process of "B-0" in explanatory drawing which shows the preparation process of the "fundus change animation" concerning the example of the present invention. It is a figure which shows a part of process of "B-0" in explanatory drawing which shows the preparation process of the "fundus change animation" concerning the example of the present invention. It is a figure which shows a part of process of "B-1" in explanatory drawing which shows the preparation process of the "fundus change animation" concerning an example of the present invention. It is a figure which shows a part of process of "B-1" in explanatory drawing which shows the preparation process of the "fundus change animation" concerning an example of the present invention. It is a figure which shows a part of process of "B-2" and "B-3" in explanatory drawing which shows the preparation process of the "fundus change animation" concerning the present invention example. It is a figure which shows a part of process of "B-2" and "B-3" in explanatory drawing which shows the preparation process of the "fundus change animation" concerning the present invention example. It is a figure which shows a part of process of "B-2" and "B-3" in explanatory drawing which shows the preparation process of the "fundus change animation" concerning the present invention example. It is a figure which shows a part of process of "B-2" and "B-3" in explanatory drawing which shows the preparation process of the "fundus change animation" concerning the present invention example. It is a figure which shows a part of process of "B-2" and "B-3" in explanatory drawing which shows the preparation process of the "fundus change animation" concerning the present invention example. It is a figure which shows a part of process of "B-2" and "B-3" in explanatory drawing which shows the preparation process of the "fundus change animation" concerning the present invention example. It is a figure which shows a part of process of "B-2" and "B-3" in explanatory drawing which shows the preparation process of the "fundus change animation" concerning the present invention example. It is a figure which shows a part of process of "C-1" in explanatory drawing which shows the preparation process of the "fundus change animation" concerning an example of the present invention. It is a figure which shows a part of process of "C-1" in explanatory drawing which shows the preparation process of the "fundus change animation" concerning an example of the present invention. It is a figure which shows a part of process of "C-2" in explanatory drawing which shows the preparation process of the "fundus change animation" concerning the present invention example. It is a figure which shows a part of process of "C-3" in explanatory drawing which shows the creation process of the "fundus change animation" concerning the example of the present invention. It is a figure which shows a part of process of "C-3" in explanatory drawing which shows the creation process of the "fundus change animation" concerning the example of the present invention. It is a figure which shows a part of process of "C-4" in the explanatory view which shows the preparation process of the "fundus change animation" concerning the example of the present invention. It is a figure which shows a part of process of "C-4" in the explanatory view which shows the preparation process of the "fundus change animation" concerning the example of the present invention. It is a figure which shows a part of process of "C-4" in the explanatory view which shows the preparation process of the "fundus change animation" concerning the example of the present invention. It is a figure which shows a part of process of "C-5" in explanatory drawing which shows the preparation process of the "fundus change animation" concerning an example of the present invention. It shows the start image (at the first visit) of the created video. The end image (at the time of the last visit) of the created moving image is shown. It shows the start image (at the first visit) of the created video. The end image (at the time of the last visit) of the created moving image is shown. It shows the start image (at the first visit) and the end image (at the last visit) of the created moving image. It shows the start image (at the first visit) and the end image (at the last visit) of the created moving image. It shows the start image (at the first visit) and the end image (at the last visit) of the created moving image.

Explanation of symbols

P1... Inter-image difference correction process P1-1... Reference point setting process P1-2. Corresponding point overlapping process P1-3. Difference correction deformation process P1-4. Repeat process P2. -1 ... Previous image opacity setting process P2-2 ... Opacity changing process P2-3 ... Image composition process P2-4 ... Repetitive process P3 ... Animation process P3-1 ... Image reproduction time setting process P3- 2 ... Image reading process P3-3 ... Image editing process P3-4 ... Playback time setting change process P3-5 ... Movie storage process

Claims (7)

Using an image processing program and a moving image creation program executed using an electronic computer, the temporal change of the target is animated by a plurality of temporally recorded images obtained by continuously recording the temporal change of the target. A moving image creation method using recorded images ”, which includes (I) an inter-image difference correction process, (II) an inter-image continuity imparting process, and (III) an animation process.
The inter-image difference correction process (I) is performed by an image processing program.
(I-1) An adjustment reference point is set in one specified image (I-2) A corresponding point in another image is overlaid on the set reference point (I-3) As a center, differences such as rotation / distortion / enlargement / reduction between both images are corrected. (I-4) By going through each procedure in which (I-2) and (I-3) are repeated as many times as necessary, This is a process in which each image is obtained by correcting the differences between multiple images for animation.
The inter-image continuity imparting step (II) is performed by an image processing program.
(II-1) Between two adjacent images in time series, the opacity of the previous image is set to 100%, or preparation is made in such a state (II-2) opacity of the image after Is changed from 0% to less than 100% (II-3) An integrated image formed by integrating the previous image and the subsequent image is separately stored (II-4) as necessary When the above (II-1) to (II-3) are repeated, and a plurality of integrated images with different opacity of the subsequent images are formed and stored, and then converted into a moving image. Each image is processed to obtain an integrated image so that smooth video changes can be obtained.
The animation process (III) is performed by a movie creation program.
(III-1) Each integrated image, that is, the playback time of one frame (hereinafter referred to as “image playback time”) is set. (III-2) Each integrated image created for moving images in (II) above. Are read in the order in which the opacity is changed (III-3) Each read integrated image is added to the timeline or edited (III-4) If necessary, for a predetermined integrated image (III-5) A method of creating a moving image using a time-varying recording image, comprising: (III-5) each step of saving the edited and created moving image. The “identified one image” in (I-1) is the first of a plurality of time-varying recorded images obtained by continuously recording a subject's time-varying still image, A method for creating a moving image using a time-varying recorded image according to claim 1. In changing the opacity in the subsequent images in (II-2) to (II-3) and storing the integrated image, based on a plurality of images in which the opacity in the subsequent images is changed at equal intervals. 3. The method for creating a moving image using a time-varying recording image according to claim 1, wherein the moving image is created and stored. 4. The moving image creation method using a time-varying recording image according to claim 1, wherein the time-varying recording image is a fundus photograph. 5. The method of creating a moving image using a time-varying recorded image according to claim 4, wherein any one of a macular fovea, a retinal blood vessel, or an optic disc blood vessel is used as the reference point. A moving image by a time-varying recording image created by the method according to claim 1. Using an image processing program and a moving image creation program executed using an electronic computer, the temporal change of the target is animated by a plurality of temporally recorded images obtained by continuously recording the temporal change of the target. "Image data for making moving image of time-varying recording image" that can be used for "moving image creation method by recorded image", and the image data is a modification standard set in one specified image by an image processing program Differences such as rotation, distortion, and enlargement / reduction from the image are corrected based on the point, and the opacity of the previous image is 100% between two adjacent images in all time series. The opacity of the subsequent image is set to 0%, and the previous image is integrated with the image after the change in which the opacity is changed from more than 0% to less than 100%. And a plurality of integrated image is formed by separately formed, the image data for a moving image of the temporal change recording images.