KR20230066526A - Image Processing Method, Apparatus, Computing Device and Storage Medium - Google Patents

Abstract

According to the present disclosure, provided are a method, a device, and a computing device for processing an image and a storage medium thereof. The method of the present invention comprises the steps of: obtaining a first image sequence corresponding to a first body region and a second image sequence corresponding to the first body region, wherein at least one of a collection position, a collection depth, and a collection angle is different in images of the first image sequence, and images of the second image sequence have basically the same collection angle, and are sequentially collected at different time points; and generating a third image sequence on the basis of the first and second image sequences, wherein the third image sequence includes at least one update image, wherein the updated image corresponds to at least one image of the second image sequence and has a different observation angle from the image.

Description

Image processing method, apparatus, computing device and storage medium {Image Processing Method, Apparatus, Computing Device and Storage Medium}

This disclosure relates to the field of image processing, and more particularly to image processing methods, apparatus, computing devices, and storage media.

Currently, doctors use medical image sequences obtained by scanning with medical scanning devices (e.g., nuclear magnetic resonance imaging (NMRI) scanners, magnetic resonance imaging (MRI) scanners, computed tomography (CT) scanners, etc.) for medical diagnosis, or Image sets are often used. After obtaining one or more original images with the medical scanning device, the one or more original images are processed to facilitate the doctor's confirmation, so that the doctor can refer to the processing result to make a diagnosis more easily.

According to one aspect of the present disclosure, an image processing method is provided, comprising: acquiring a first image sequence corresponding to a first body region and a second image sequence corresponding to the first body region - the first image sequence corresponding to the first body region; The images in one image sequence differ from each other in at least one of the collection position, collection depth, or collection angle, the images in the second image sequence basically have the same collection angle, and the images in the second image sequence are at different points in time. Collected sequentially -; and generating a third image sequence based on the first image sequence and the second image sequence, wherein the third image sequence includes at least one update image, wherein the at least one update image corresponds to the second image sequence. corresponds to at least one image in the sequence, and has a different viewing angle than the at least one image; includes

According to another aspect of the present disclosure, an image processing device is provided, comprising: obtaining an image sequence for acquiring a first image sequence corresponding to a first body region and a second image sequence corresponding to the first body region; unit - the images in the first image sequence differ from each other in at least one of the collection position, collection depth, or collection angle, the images in the second image sequence have essentially the same collection angle, and the images in the second image sequence are sequentially collected at different time points; and an image sequence generating unit for generating a third image sequence based on the first image sequence and the second image sequence, the third image sequence comprising at least one updated image, the at least one updated image comprising: corresponding to at least one image of the second image sequence, and having a different viewing angle than the at least one image; includes

According to another aspect of the present disclosure, there is provided a computing device including a memory, a processor, and a computer program stored in the memory, wherein the processor implements the image processing method of one or more embodiments by executing the computer program. It is configured to run a computer program.

According to another aspect of the present disclosure, a non-transitory computer readable storage medium in which a computer program is stored is provided, and when the computer program is executed by a processor, an image processing method according to one or more embodiments is implemented.

According to another aspect of the present disclosure, a computer program stored in a computer readable storage medium including instructions and implementing an image processing method according to one or more embodiments when the instructions are executed by at least one processor is provided. .

According to the embodiments described below, these and other aspects of the present disclosure will become clear and will be explained with reference to the embodiments described below.

Additional details, features and advantages of the present disclosure are set forth in the following description of exemplary embodiments in conjunction with the drawings.
1 is an exemplary diagram illustrating an exemplary system capable of implementing various methods described herein according to exemplary embodiments.
Fig. 2 is a flowchart illustrating an image processing method according to an exemplary embodiment.
3A-3D show some example images according to example embodiments of the present disclosure.
Fig. 4 is an exemplary block diagram illustrating an image processing device according to an exemplary embodiment.
Fig. 5 is a block diagram illustrating an exemplary computer device that may be applied to an exemplary embodiment.

Unless otherwise specified in the present disclosure, in describing various elements using terms such as “first” and “second”, there is no intention to limit the positional relationship, time series relationship, or importance relationship of these elements, and these terms is only used to distinguish one component from another. In some instances, a first element and a second element may refer to the same instance of an element, but in some cases may refer to different instances depending on the context.

Terms used in various exemplary descriptions in this disclosure are for the purpose of describing specific examples and are not intended to be limiting. Unless the context clearly indicates otherwise, unless the number of elements is specifically limited, one or a plurality of corresponding elements may exist. As used herein, the term "plurality" means two or more, and the term "... "Based on" means "at least in part... should be interpreted as “based on”. Also, the terms "and/or" and "at least one of..." include any one of the listed items and all possible combinations.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 is an exemplary diagram illustrating an exemplary system 100 capable of implementing various methods described herein according to exemplary embodiments.

Referring to FIG. 1 , the system 100 includes a client device 110 , a server 120 and a network 130 communicatively coupling the client device 110 and the server 120 .

The client device 110 includes a display 114 and a client application (APP) 112 that can be displayed on the display 114 . The client application 112 may be an application program that needs to be downloaded and installed before execution or a light-weight application, such as an applet (liteapp). If the client application 112 is an application that needs to be downloaded and installed before execution, the client application 112 may be pre-installed on the client device 110 and activated. If the client application 112 is an applet, the user 102 searches for the client application 112 (eg, the name of the client application 112, etc.) from the host application, without installing the client application 112, or The client application 112 may be directly executed on the client device 110 through a method such as scanning a graphic code (eg, barcode, QR code, etc.) of the client application 112 . In some embodiments, client device 110 is any type of mobile device, including a mobile computer, cell phone, wearable computer device (eg, smart watch, head-mounted device encompassing smart glasses, etc.) or other type of mobile device. It may be a computer device. In some embodiments, client device 110 may alternatively be a fixed computer device, such as a desktop, server computer, or other type of fixed computer device. In some optional embodiments, the client device 110 may be or include a medical image printing device.

Server 120 is a server typically built by an Internet Service Provider (ISP) or Internet Content Provider (ICP). Server 120 may represent a single server, a multi-server cluster, a distributed system, or a cloud server providing basic cloud services (eg, cloud database, cloud computing, cloud storage, cloud communication). Although FIG. 1 shows server 120 as communicating with only one client device 110 , it will be appreciated that server 120 may provide background services to multiple client devices simultaneously.

Examples of networks 130 include combinations of communication networks such as LANs, WANs, Personal Area Networks (PANs), and/or the Internet. Network 130 may be a wired or wireless network. In some embodiments, data exchanged over network 130 is in a format and/or technology including Hyper Text Markup Language (HTML), Extensible Markup Language (XML), and the like. processed using In addition, it uses encryption technologies such as Secure Sockets Layer (SSL), Transport Layer Security (TLS), Virtual Private Network (VPN), and Internet Protocol SECurity (IPsec). You can also encrypt all or some links. In some embodiments, custom and/or proprietary data communication technologies may be used to replace or supplement the data communication technologies noted above.

System 100 may also include image collection device 140 . In some embodiments, the image collection device 140 shown in FIG. 1 is a Positron Emission Tomography (PET) system, a Positron Emission Tomography with Computerized Tomography (PET/CT) system, or a single photon emission tomography (PET/CT) system. Single Photon emission Computed Tomography with Computerized Tomography (SPECT/CT), Computed Tomography (CT), Medical Ultrasonograph, Nuclear Magnetic Resonance Imaging, Medical scans, including but not limited to scanning or imaging devices used in NMRI), Magnetic Resonance Imaging (MRI), Cardiac Angiograph (CA), Digital Radiograph (DR), etc. can be a device. For example, the image acquisition device 140 may include a digital subtractive angiography scanner, a magnetic resonance angiography scanner, a tomographic blood vessel scanner, a positron emission tomography scanner, a positron emission tomography scanner, a single photon emission computed tomography scanner, a computed tomography scanner, and medical ultrasound. It may include a diagnostic device, a nuclear magnetic resonance imaging scanner, a magnetic resonance imaging scanner, a digital radiography scanner, and the like. The image collection device 140 may be configured to perform image data processing including but not limited to scan data conversion (eg, conversion to a medical image sequence), compression, pixel correction, 3D reconstruction, etc. For example, it may be connected to the server 120 in FIG. 1 or a separate server of an imaging system (not shown).

The image collection device 140 may be connected to the client device 110 via, for example, the network 130 or otherwise directly connected to the client device to communicate with the client device.

Optionally, the system may also include a smart computing device or computing card 150 . The image collection device 140 may include or be connected (eg, detachably connected) to the computing card 150 or the like. As one example, computing card 150 may implement processing of image data including but not limited to transformation, compression, pixel correction, reconstruction, and the like. As another example, the computing card 150 may implement an image processing method according to an embodiment of the present disclosure.

The system may also include other parts not shown, such as a data storage unit. The data storage unit may be a database, a data store, or one or more devices of another form for storing data, and may be a conventional database, or may be a cloud database, a distributed database, and the like. For example, direct image data formed by the image collection device 140 or medical image sequences or 3D image data acquired through image processing are searched by the subsequent server 120 and the client device 110 from the data storage unit. It can be stored in the data storage unit so that it can be used. In addition, the image collection device 140 may directly provide image data or medical image sequences or 3D image data obtained through image processing to the server 120 or the client device 110 .

The user uses the client device 110 to check collected images or videos (including preliminary image data or images that have undergone analysis processing), check analysis results, and collect images or analysis results. You can interact with it, enter collected commands and configuration data, and so on. The client device 110 may transmit configuration data, commands, or other information to the image collection device 140 to control collection and data processing of the image collection device.

In order to implement the objectives of the embodiments of the present disclosure, in the example of FIG. 1 , the client application 112 may perform various functions, such as, for example, storage management, indexing, sorting, classification, etc. on the collected image sequences. It may be an image sequence management application program that can provide. Correspondingly, the server 120 may be a server used together with an image sequence management application program. The server 120 may provide an image sequence management service to the client application 112 running on the client device 110 based on a user's request or a command generated according to an embodiment of the present disclosure. For example, it manages storage of image sequences in the cloud, stores specified indexes (sequence type, patient identifier, body part, collection object, collection stage, collection machine, lesion detection status, severity, etc.), and searches according to the specified index to client You can provide a device with an image sequence, and the like. Alternatively, the server 120 may provide or allocate such service capabilities or storage space to the client device 110 , by a client application 112 running on the client device 110 , upon user request or according to the present disclosure. Based on the command generated according to the embodiment of, providing a corresponding image sequence management service, and the like. Only one example has been provided above, and it can be understood that the present disclosure is not limited thereto.

Fig. 2 is a flowchart illustrating an image processing method 200 according to an exemplary embodiment. The method 200 may be executed in a client device (eg, the client device 110 shown in FIG. 1), that is, the execution subject of each step of the method 200 is the client device 110 shown in FIG. ) can be. In some embodiments, method 200 may be executed on a server (eg, server 120 shown in FIG. 1 ). In some embodiments, method 200 may be executed by a mutual combination of a client device (eg, client device 110 ) and a server (eg, server 120 ).

Hereinafter, each step of the method 200 will be described in detail by taking the execution entity as the client device 110 as an example.

Referring to FIG. 2 , in step 210, a first image sequence corresponding to a first body region and a second image sequence corresponding to the first body region are acquired, wherein the images in the first image sequence are mutually related to each other. At least one of the acquisition position, acquisition depth, and acquisition angle is different, the images in the second image sequence basically have the same acquisition angle, and the images in the second image sequence are sequentially acquired at different time points.

In step 220, a third image sequence is generated based on the first image sequence and the second image sequence, wherein the third image sequence includes at least one update image, and the at least one update image comprises the first image sequence. Corresponds to at least one image in the sequence of two images, and has a different viewing angle than the at least one image.

According to this embodiment, it is possible to perform 3D reconstruction of a 2D image sequence on a time series based on a 3D image sequence.

In the field of medical image acquisition, several approaches already exist to create a three-dimensional image representation, such as performing three-dimensional enhancement on, for example, CT images, MR images, and the like. However, although various technical methods for generating 3D images based on 2D images already exist, since the 2D images collected for the human body region change in time series according to human breathing, blood vessel pulsation, etc., these methods are It is difficult to directly implant in a sequence of two-dimensional images collected in time series. In contrast, according to an embodiment of the present disclosure, a reconstruction of at least one different angle of a time-series two-dimensional image may be generated by using a 3D image sequence collected for the same body region as a reference.

The images in the first image sequence differ from each other in at least one of acquisition position, acquisition depth, or acquisition angle, which means that the image sequences are images acquired at different positions (e.g., different probe positions), same position but different imaging It can be understood that it can mean included in images collected at depth and/or images collected at different angles, and/or combinations thereof. As a non-limiting example, a first sequence of images may include one set of images collected from the same angle but from a different location or depth, and may include another set of images collected from essentially similar locations and different angles; This disclosure is not limited to this. It can be understood that this may mean that the first image sequence can perform a 3D representation of the first body region. As a non-limiting example, the first image sequence may be a coronary CT angiography (CTA), CT venography (CTV), magnetic resonance angiography (MRA), or other image capable of three-dimensional representation comprehensible to those skilled in the art. can be a sequence. It is to be understood that the first image sequence is a 3D representation of the first body region, and images in the first image sequence are not required to be 3D images or have specific 3D representation capabilities. In contrast, the first image sequence may include a plurality of 2D images that may include a predetermined depth or 3D space information collected for the first body region. For example, the first image sequence may be implemented in a 3D form through post-processing, rendering, and enhancement, but the present disclosure is not limited thereto.

In addition, the images in the second image sequence basically have the same collection angle, and this description indicates that the collection angle of the images in the second image sequence satisfies a predetermined threshold value (e.g., a probe shake error threshold value). I can understand what it means to do. In other words, the second image sequence may be a sequence of time-series two-dimensional images of the first body region, and may be used to show changes over time of some features of the corresponding body region to the user as an example. Prior to applying the method according to the embodiment of the present disclosure, it does not have 3D expression capability. Illustratively, the images in the second image sequence may also have essentially the same collection location and/or collection depth (eg, same collection cross-section). As a non-limiting example, the second image sequence may be a digital subtraction angiography (DSA) image, but the present disclosure is not limited thereto.

Throughout the text, the descriptions “The first image sequence is for a first body region…” and “The second image sequence is for a first body region…” indicate that the first image sequence and the second image sequence are It can be understood that it does not mean having exactly the same imaging or collection range. The descriptions "The first image sequence is for a first human body region..." and "The second image sequence is for a first human body region..." even indicate that the first image sequence and the second image sequence are essentially the same. It is not required to have an imaging or collection range, provided that the first image sequence contains sufficient information about the topology, human tissue, human organ, lesion, etc. in the corresponding human body region, and the specific topology, human tissue in the second image sequence , 3D data for data such as human organs, lesions, and contrast agents need only be created as a reference. As another example, a first image sequence may be collected for a spatial extent of a first body region, whereas an image in a second image sequence may be located in only a cross-section of one of the first body regions.

In addition, it can be understood that the “viewing angle” may mean the angle of view of the image, and may be expressed as the direction of the two-dimensional plane on which the image is located, the direction of the normal vector, and the like. When the image is an image obtained by direct collection or an image that has undergone only flat processing, the observation angle may be the same as the collection angle. In other cases, the viewing angle may correspond to any particular collection angle in space, and the viewing angle may indicate that the image may not have been collected at that angle, but the object collected at that angle (e.g., a first body part or its It corresponds to confirming a part).

In some examples, the third image sequence may include a plurality of update images. As a non-limiting example, the third image sequence may include one set of update images, and the number of images in the update image set is equal to the number of images in the second image sequence. In other words, in this example, the third image sequence may include a time-series image sequence with all images of the second image sequence updated. In another example, the number of updated images in one set is equal to the number of images satisfying a predetermined condition in the second image sequence, and the predetermined condition is an image at a predetermined time interval, an image satisfying a quality requirement, a number of An example of one image frame selected for each image frame... is not limited thereto. As another non-limiting example, the third image sequence can include a plurality of sets of update images, where each set of update images corresponds to one viewing angle, thereby plural time series viewed from different angles. sequence can be obtained. In some examples, the third image sequence may further include the second image sequence itself or the processed second image sequence, in addition to the updated one or more images or one or more set of images. It can be understood that the foregoing is only an example, and the present disclosure is not limited thereto.

According to some embodiments, the image in the second image sequence is a contrast auxiliary imaging image.

The method according to the present disclosure is particularly advantageous for contrast-assisted imaging image sequences. On the one hand, since the contrast agent imaging image sequence often requires a plurality of images collected in time sequence to observe the movement of the contrast agent in the human body, images on a time series including the increase of the contrast agent and the pulsation of blood vessels at different time points. change is particularly evident. On the other hand, it is difficult to collect enough 2D images from enough different angles and positions to perform reconstruction from 2D images to 3D images according to existing methods due to the limitations of existing contrast imaging devices. There are many cases. For at least the above reasons, there is currently no method or product for three-dimensionalizing a contrast agent imaging image sequence. In contrast, according to an embodiment of the present disclosure, a 3D auxiliary image of a contrast agent imaging image may be generated.

According to some embodiments, generating a third image sequence based on the first image sequence and the second image sequence may include the first image sequence based on the first image sequence and the second image sequence. at least one body organ, region or tissue in the first body region appearing in at least one image of and at least one body organ in the first body region appearing in at least one image of the second image sequence; determining a correspondence between parts or tissues; and generating the third image sequence based on the correspondence, the first image sequence, and the second image sequence.

According to this embodiment, 3D visual assistance or 3D reconstruction can be implemented by easily generating images of different observation angles based on the second image sequence based on the 3D first image sequence based on the correspondence relationship. there is.

It can be understood that throughout the text, the use of terms such as “organ”, “tissue”, “body part”, etc. is not limited to the strict medical meaning, and for example, a set of one or more organs, one or more It means a set of tissues, a part of an organ, or a part of a tissue, and is intended to include the broadest meaning that can be understood as a human body part or a human body part. For example, internal organs or parts thereof, lesion tissues or parts thereof, small segment blood vessels, topologies composed of multiple blood vessels, and the like may all be referred to as human body parts, but the present disclosure is not limited thereto.

As an example, by mapping the blood vessel segment region, shape, contrast agent level, and other parameters in the second image sequence to the corresponding blood vessel segment region in the 3D topology structure of the first image sequence, the correspondence relationship, the first image At least one image of the third image sequence may be generated based on the sequence and the second image sequence. For example, based on an image in the first image sequence having a different viewing angle than the corresponding image in the second image sequence, or based on a three-dimensional representation of the first image sequence different from the corresponding image in the second image sequence. A corresponding image of the third image sequence may be generated based on the topology of the corresponding 2D view by mapping to an image having an observation angle. As another example, the generated third image sequence may have a simpler topology than the first image sequence. For example, two blood vessels intersecting in a 2D plane exist in the second image sequence, and the method of the present disclosure uses the first image sequence to view the blood vessels in the 2D plane of the corresponding image of the second image sequence. Separating the two blood vessels may be included, thus achieving the purpose of 3-dimensionalization. In such an example, it may not be necessary to create a vessel topology that completely matches the first image sequence. Other image generation or 3D reconstruction schemes understood by those skilled in the art may also be used, and the present disclosure is not limited thereto. As an example, at least one updated image such that the topological structure of the corresponding image in the second image sequence is updated to compensate for intersections, overlaps, occlusions, etc. present in the images, as described further below. An observation angle of can be selected.

As another example, the angle, dimension (window width), position (window position), etc. of the updated image in the third image sequence may be selected according to the angle, position, and CT value information of the lesion or abnormal region existing in the corresponding image. there is.

In such examples, methods of the present disclosure may include processing at least one of the first image sequence and the second image sequence to identify a lesion or abnormal region therein. For example, a method of the present disclosure may include processing a first sequence of images to identify a lesion or abnormal region therein, and further, based on, for example, a correspondence established by the two images, the identification The affected lesion or abnormal region is projected onto a corresponding position in at least one image of the second image sequence. For another example, a method of the present disclosure may include processing a second image sequence to identify a lesion or abnormal region therein. The method of the present disclosure may include combining and processing a first image sequence and a second image sequence, and exemplarily performs mutual verification, combination, supplementation, etc. on lesions or abnormal data extracted from both. can do. In another example, at least one of the acquired first image sequence and the second image sequence may have pre-processed or pre-identified lesion or abnormal information, and the method of the present disclosure includes generating an updated image based on such information. , and no additional processing steps are required.

According to one or more examples, the method of the present disclosure may include generating at least one updated image to better represent the identified or pre-identified lesion or abnormal area. For example, an updated image can be created by selecting an observation angle that maximizes the cross-section of the lesion or abnormal region, and the dimension, center position of the updated image can be adjusted so that the lesion or abnormal region is located in the middle of the image or elsewhere visible to the observer. location, etc. As another example, the number, resolution, etc. of generated updated images may be adjusted based on the shape, size, complexity, and severity of lesion or abnormal information. For example, if the identified lesion has a high level of importance (eg, associated with more severe lesions), complex shape, and/or small dimensions, images of two or more viewing angles are generated for the lesion area, or more Higher resolution, more precise images can be created.

According to one or more embodiments, the angle, number, resolution, etc. of the third images are intelligently selected based on the identified lesions and other information so that the generated third images better represent information of the lesions or abnormal regions. can make it advantageous.

A more specific, non-limiting example according to the present disclosure will be described with reference to FIGS. 3A to 3B. As shown in FIG. 3A, an image 310 of a second image sequence (not shown) of a plurality of images is shown, and blood vessels 311, 312, 313, and 314 are shown, and blood vessels 313 The blood vessels 314 intersect in the 2D plane of the image 310 .

3B is a corresponding image 320 of a third image sequence generated based on image 310 . It is to be understood that when the third image sequence uses a 3D data format, the image 320 may be a 2D projection of a corresponding one of the third image sequence, and the present disclosure is not limited thereto. As shown in FIG. 3B, an image 320 of a second image sequence that may include a plurality of images is shown, and blood vessels (311, 312, 313, and 314 respectively corresponding to the blood vessels 311, 312, 313, and 314 of FIG. 3A) are shown. 321, 322, 323, 324) are shown. In the diagram of FIG. 3B , blood vessels 323 and blood vessels 324 no longer intersect in the two-dimensional plane of the image 320 .

As an example, a method according to an embodiment of the present disclosure may, based on blood vessels at which intersections exist in the image 310, the angle at which corresponding blood vessels among them no longer intersect (eg, the angle of the image 320). angle), and creates and presents an image 320 for the user to view. Exemplarily, based on the determined angle, an image sequence (not shown) on a time series including the image 320 is generated to completely represent changes in the blood vessels of the human body by the contrast agent at an angle at which the corresponding blood vessels do not cross each other. . For example, the generated image sequence may have the same time range, number, and the like as the second image. In this case, the third image sequence may refer to a new 2D image sequence that has been generated, and the form of the 2D image sequence may be used, but information of a different view of the second image sequence may be provided. In some cases, the third image sequence may provide time-series 3-dimensional information, and thus may be referred to as a 3-dimensional representation of the second image sequence, but the present disclosure is not limited thereto. As another example, the third image sequence may include a combination of the second image sequence and the generated image sequence of the new angle, wherein the third image sequence includes at least two images of the human body region at at least one collection point in time. It has an angular image. In another example, one or more angles may be determined, one or more images (or one or more image sequences) may be generated, and a third image sequence may include the generated image or image sequence, and the present disclosure relates to this Not limited.

As another example, a method according to an embodiment of the present disclosure generates a 3D image for at least one viewpoint of a second image sequence based on the first image sequence and the second image sequence (ie, the second image sequence). generating a 3D image for a plurality of images or each image in the sequence). Accordingly, the user observes images of at least one viewpoint or even all viewpoints in the second image sequence from different angles by dragging, rotating, or the like. In these examples, images 310 and 320 may be views from specific angles selected by a user, and it is to be understood that the present disclosure is not limited thereto.

In some examples, the contrast agent concentration value may be visually displayed to the user. Referring to FIG. 3C , a local image 330 of one image of the acquired second image sequence is shown, where blood vessels 331 and blood vessels 332 have intersections. In image 330, the concentration of contrast agent in regions 3311, 3312, 3321, 3322, and 333 at different depths are shown. Understandably, in this two-dimensional image, the contrast agent concentration of the collected intersection area 333 is a superposition of the contrast agent concentration values of both blood vessels 331 and blood vessels 332 . In addition, although the density of each area is indicated by shading in the drawing, the drawing is only an example, and the density of different areas can be displayed in various forms such as color, contrast, depth, gradation, numerical value, numerical range, level, etc. Concentrations between regions may be equal to each other, or concentrations at different locations in the same region may be different, and it is to be understood that the present disclosure is not limited thereto.

Continuing to refer to FIG. 3D , according to one or more embodiments of the present disclosure, an image or image sequence including a local image 340 is generated by generating images of different viewing angles based on the second image sequence. can do. Blood vessels 341 and blood vessels 342 in the image 340 may correspond to blood vessels 331 and blood vessels 332 in the image 330, respectively, and the angles of the image 340 are the blood vessels 341 and blood vessels. Let (342) no longer intersect. In this example, the contrast value density of regions 3411, 3412, 3413, 3421, 3422, and 3423 are calculated, respectively, and displayed in any desired manner, including but not limited to pattern, shade, contrast, color, numerical, and the like. It can be. Areas 3411, 3412, 3421, and 3422 correspond to areas 3311, 3312, 3321, and 3322 in the second image sequence, respectively, and density values may be obtained from them, for example, directly obtained, or Adjust based on changes in angle and vessel shape. The concentrations of the regions 3413 and 3423 may be obtained through the concentration of the region 333, for example, determined by comprehensively subtracting the concentrations of adjacent regions of the non-intersecting region from the concentrations of the intersecting regions, or may be simulated. obtained through other methods including, but not limited to, computation, neural network analysis, and the like. It can be understood that the present disclosure is not limited thereto.

3A to 3D have described some specific examples of the embodiments of the present disclosure, focusing on the presence or absence of cross blood vessels, but it can be understood that the present disclosure is not limited thereto. The present disclosure can achieve the effect of visual assistance or 3D reconstruction by generating at least one image of a different angle for a 2D image time series sequence based on a 3D image sequence as a reference, and this technical object is the first. It is not required that blood vessel intersections exist in the images of the two-image sequence, and it is not required that blood vessel intersections not necessarily exist in the generated updated image, and the user (eg, doctor, diagnostician) determines the blood vessel shape of the human body region. It is only necessary to create images of different viewing angles so that the contrast agent concentration, etc. can be checked more conveniently.

According to some embodiments, at least one image of the first image sequence includes one or a plurality of identified blood vessel segment regions, the identified blood vessel segment regions having a corresponding name; The correspondence is a naming mapping from the identified blood vessel segment region in the at least one image of the first image sequence to at least one blood vessel segment region in the at least one image of the second image sequence.

According to this embodiment, since the information identified in the first image sequence can be effectively used, the converted visual expression form of the second image sequence can be conveniently obtained. For example, in the related art, there is already a presentation scheme of CTA images capable of providing the naming of each blood vessel. Accordingly, a 3D spatial extent of the blood vessel corresponding to the nomenclature may be further obtained, and the first image sequence is time-sequentially collected (eg, but not limited to, contrast imaged) using the nomenclature. 2 Map to image sequence. As some non-limiting examples, images collected via CTA or MRA may include identified blood vessel names RCA, LM, LCX, LAD, etc., through these naming, the corresponding second image sequence (e.g., DSA images (but not limited thereto) corresponding to one or a plurality of blood vessels may be identified, thereby completing mapping between different types of image sequences.

According to some embodiments, at least one image of the first image sequence comprises an identified lesion area, wherein the third image sequence based on the correspondence, the first image sequence and the second image sequence. The generating may include generating an image portion corresponding to the identified lesion area in at least one image of the third image sequence, based on the correspondence.

In this embodiment, the generated third image sequence facilitates the user to view the identified lesion of the first image sequence together with the time series situation of the second image sequence. In particular, since the user can check the second image sequence that changes over time in a time series and in three dimensions (eg, from at least two angles), it is easy for the user to check the collection result. For example, there is no need to repeatedly jump through different image sequences to check and compare. For example, this type of presentation can help make a more accurate diagnosis. As an example, the identified lesion may be an identified CTA plaque, but the present disclosure is not limited thereto. For example, when the second image sequence is a contrast agent auxiliary imaging sequence, the generated third image sequence can facilitate the user to observe the lesion and the contrast agent situation together, and in particular, the user can view at least two It is possible to check the change over time of the contrast medium from the dog's angle, and the change over time of the contrast medium in combination with the shape of the lesion can be checked, but the present disclosure is not limited thereto.

According to some embodiments, the determining of the correspondence may include determining a reference image in the second image sequence; obtaining a reference two-dimensional view of the first image sequence based on the reference image; and determining the corresponding relationship based on the reference 2D view and the reference image.

According to this embodiment, by determining one image from the time-series image sequence as a reference, determining one two-dimensional image as a reference view based on the first image sequence, and comparing the reference image and the reference view, the two images Correspondence (eg, naming correspondence, etc.) corresponding to blood vessels in the sequence may be obtained. Thus, it is possible to simply and accurately establish a mapping relationship between two image sequences of different data formats. For example, after establishing a mapping relationship between a reference image and a reference view, a mapping relationship or correspondence between the reference image and other images in the second image sequence may be established.

Exemplarily, in the reference 2D view of the first image sequence obtained based on the reference image, the reference 2D view has the same angle as the key image frame (eg, exactly the same or within a threshold range). Deviation angle at ) can be made. The reference 2D view may be any image among images included in the first image sequence. Alternatively, a corresponding reference 2D view may be obtained by projecting 3D spatial data generated based on the first image sequence onto a certain 2D plane.

The corresponding relationship determined based on the reference 2D view and the reference image may be obtained based on a shape of a blood vessel in the reference 2D view and the reference image. As an example, a reference image may also be referred to as a key image frame. The reference image may be determined based on one or more criteria, some examples of which are described below.

According to some embodiments, the obtaining of the reference 2D view of the first image sequence based on the reference image includes obtaining the reference 2D view based on a collection angle of the second image sequence. do.

According to this embodiment, the reference 2D view of the first image sequence may be obtained by directly acquiring or indirectly calculating the collection angle of the second image sequence. For example, for some collection devices, or under some collection parameter settings, an angle in the real world corresponding to an angle in three-dimensional space of a first image sequence is known, and/or a collection of a second image sequence. angle is known. Alternatively, the collection angle may be determined based on the image data, for example, based on the location and orientation of a specific human body organ or tissue in the image, the location of a light source, etc., the collection angle is calculated from the image data. .

It can be understood that the obtaining of the reference 2D view may include selecting an image corresponding to, close to, or within a certain angular range of the collection angle of the second image sequence from the first image sequence, The description of “obtaining the reference 2D view” does not require that the reference 2D view be an image of one of the initially collected first image sequences. As already mentioned above, in some examples, full-angle 3D spatial data may be generated based on the first image sequence, and the reference 2D view obtained thereby is such that the 3D spatial data is viewed from any one angle. It may be a two-dimensional projection.

According to some embodiments, obtaining a plurality of observation views having a plurality of different viewing angles based on the first image sequence; and determining the reference 2D view from the plurality of observation views by comparing the reference image with the reference image.

According to this embodiment, the acquisition angle of the first image sequence or the second image sequence is obtained by comparing the plurality of observation views of the first image sequence with the second image sequence (specifically, the reference view of the second image sequence). If this is unnecessary or impossible, the required reference two-dimensional view can still be determined. For example, an image having the closest shape may be determined as a reference 2D view by comparing a blood vessel shape or a shape of another human tissue organ in each observation view among a plurality of observation views with a blood vessel shape in a reference image.

As already mentioned above, acquiring a plurality of observation views having a plurality of different observation angles based on the first image sequence includes acquiring images having a plurality of different collection angles from the first image sequence. and generating a plurality of projection results by performing projection on the 3D representation of the first image sequence according to a plurality of projection angles, but the present disclosure is not limited thereto.

According to some embodiments, determining a first vessel segment region satisfying a first criterion from among the second image sequence, wherein the first criterion includes at least one of a vessel segment length, a development degree of a contrast agent, and collection sharpness. -; and determining an image frame in which the first blood vessel segment region is located as the reference image.

In this embodiment, a more accurate correspondence can be obtained by selecting, as a reference image, an image having the longest blood vessel, the best phenomenon, and/or the most distinct blood vessel from the second image sequence.

In some examples, a specific vessel in the image, eg, a specific vessel related to the current collection target, an aorta, the thickest vessel in the image, a vessel in which the developer arrived first, a vessel in which the developer concentration is the highest or higher than a certain threshold value, etc., first. It can be identified, and an image conforming to the standard is selected as a reference image by using the specific blood vessel as the “first blood vessel segment region”. In some other examples, one may choose from all images and all vessels, and select the vessel segment region that best meets the requirements of the first criterion (eg, one vessel that is the longest in all images, one vessel that has the best sharpness). The first blood vessel segment region is determined, and a reference image is selected based on the first blood vessel segment region. It can be understood that the above is an example and the present disclosure is not limited thereto.

In some other examples, the first criterion may additionally or alternatively include the greatest number of blood vessels and/or the most distinct blood vessel bifurcations, and the like. According to some other examples, the first standard may include a case where the total concentration of the developer exceeds a specific threshold or a specific time elapses after the developer is ingested, and the vascular path of the developer in these images. Select the longest or longest image as a reference, and the present disclosure is not limited thereto.

According to some embodiments, the determining of the reference image in the second image sequence may include: obtaining a first shape point in the second image sequence, wherein the first shape point is at least one of a bifurcation point, a connection point, and a turning point of a blood vessel; corresponds to one -; and determining an image frame in which the first shape point is located as the reference image.

According to this embodiment, the selected reference image may have some desired symbolic features such as divergence and turning points. In the 3D reconstruction process, these symbol points are very important, because shape points including relationships between blood vessels, such as branching, connection, and transition, often have a certain angle and special shape, at least compared to straight sections. Therefore, these reference points Based on the points, corresponding angles in the first image sequence are more easily identified, and corresponding reference two-dimensional views are determined, thereby accurately establishing the correspondence between the first image sequence and the second image sequence.

In some instances, these feature points may be identified by analyzing the topology in the identified image. In some examples, the identification of the shape point may focus on identifying a specific bifurcation point corresponding to a coronary artery based on parameters such as the purpose of collection and the collected body region (eg, based on the current collection target being a coronary artery). can In some other examples, shape points may be identified through a pre-trained network model. In another embodiment, if the accuracy requirements are not satisfied in identifying the shape points by the existing algorithm, the user (e.g., the doctor) is prompted to manually mark the shape points to assist in determining the most important shape points. can also be entered. It can be understood that the present disclosure is not limited thereto.

According to some embodiments, the method of the present disclosure may further include acquiring a second shape point corresponding to the first shape point in the reference 2D view, wherein the reference 2D view and the The determining of the correspondence based on the reference image may include one or a plurality of blood vessel segment regions related to the second shape point in the reference 2D view and one or more blood vessel segment regions related to the first shape point in the reference image. and determining a correspondence between the plurality of blood vessel segment regions.

According to this embodiment, it is possible to more simply and accurately establish a correspondence between the first image sequence and the second image sequence by identifying shape points similarly having a constant angle or direction in the reference 2D view.

According to some embodiments, if the second image includes contrast value information, the method 200 may further include generating an enhanced first image sequence, wherein the enhanced first image sequence comprises the second image sequence. Include contrast values based on the sequence. In such an embodiment, reconstruction of different angles may be performed on a sequence of time-series images based on a sequence of three-dimensional images (eg, CTA, MRA, etc.), and the first image based on the contrast values in the second image sequence. Enhancements and rendering can be performed on sequences, thereby reducing the need for users (e.g., diagnosing physicians) to repeatedly compare different sequences, making it easier for users to obtain more comprehensive information on the image sequence they are currently viewing. It is more helpful for the user's diagnosis and the like.

It should be understood that while each action is shown in a specific order in the figures, it should not be understood that these actions must be performed in the specific order shown or in sequential order, rather than all actions shown to achieve a desired result. should not be understood as having to

Throughout this disclosure, an image sequence is image data that can be directly collected and stored or otherwise transmitted to a terminal device for use by a user. The image sequence may be processed image data after various image processing. The image sequence may also be subjected to other analysis processes (eg, the presence or absence of lesion features or the process of analyzing lesions) and may further include analysis results (eg, circle the region of interest, tissue segmentation results, etc.). can It can be understood that the present disclosure is not limited thereto.

Fig. 4 is a schematic block diagram of an image processing device 400 according to an exemplary embodiment. The image processing device 400 may include an image sequence acquisition unit 410 and an image sequence generation unit 420 . The image sequence acquiring unit 410 may be configured to acquire a first image sequence corresponding to a first body region and a second image sequence corresponding to the first body region, wherein an image in the first image sequence is At least one of a collection position, a collection depth, or a collection angle is different from each other, the images in the second image sequence basically have the same collection angle, and the images in the second image sequence are sequentially acquired at different time points. The image sequence generating unit 420 may be for generating a third image sequence based on the first image sequence and the second image sequence, the third image sequence including at least one updated image, The at least one updated image corresponds to at least one image of the second image sequence and has a different viewing angle than the at least one image.

It should be understood that each module of the apparatus 400 shown in FIG. 4 corresponds to each step of the method 200 described in FIG. 2 . Accordingly, the operations, features, and advantages described for method 200 described above apply equally to device 400 and the modules it includes. For brevity, some operations, features, and benefits are not duplicated herein.

According to an embodiment of the present disclosure, the computer further includes a computing device including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to perform an image processing method according to an embodiment of the present disclosure and variations thereof. It is configured to implement the steps.

According to an embodiment of the present disclosure, a non-transitory computer readable storage medium in which a computer program is stored is further disclosed, wherein, when the computer program is executed by a processor, steps of an image processing method according to an embodiment of the present disclosure and variations thereof implement

According to an embodiment of the present disclosure, a computer program product including a computer program is further disclosed, wherein the computer program implements steps of an image processing method according to an embodiment of the present disclosure and variations thereof when executed by a processor.

Although specific functions have been discussed above with reference to specific modules, it should be noted that the functions of each module discussed herein may be divided into multiple modules, and/or at least some functions of multiple modules may be combined into a single module. Be careful. An operation of executing a specific module as discussed herein refers to an operation in which the specific module itself executes an operation or, alternatively, the particular module invokes or otherwise accesses another component or module that executes the operation (see above). and other components or modules that execute the above actions) together with the specific module. Thus, a particular module executing an operation may include the particular module executing the operation and/or other modules executing the operation that are called by or otherwise accessed by the particular module. For example, multiple modules described above may be combined into a single module in some embodiments, and vice versa. As used herein, the phrase “entity A initiates action B” may mean that entity A instructs action B to be executed, but entity A itself does not necessarily execute that action B.

It should also be understood that various techniques may be described herein in the general context of software hardware elements or program modules. Each of the modules described above with respect to FIG. 4 may be implemented in hardware, or in hardware combined with software and/or firmware. For example, these modules may be implemented as computer program codes/instructions configured to be executed on one or more processors and stored in a computer-readable storage medium. Alternatively, these modules may be implemented with hardware logic/circuits. For example, in some embodiments, one or multiple of the modules described herein may be implemented together in a System on Chip (SoC). SoC is an integrated circuit chip (processor (eg, central processing unit (CPU), microcontroller, microprocessor, digital signal processor (DSP), etc.), memory, one or more communication interfaces) , and/or one or more components of other circuitry), and may include embedded firmware for selectively executing received program code and/or performing functions.

According to one aspect of the present disclosure, a computing device comprising a memory, a processor and a computer program stored in the memory is provided. The processor is configured to execute a computer program for implementing the steps of any method embodiment described above.

According to one aspect of the present disclosure, there is provided a non-transitory computer-readable storage medium storing a computer program, and when the computer program is executed by a processor, the steps of any of the foregoing method embodiments are implemented.

According to one aspect of the present disclosure, a computer program product comprising a computer program is further disclosed, wherein the computer program when executed by a processor implements the steps of any of the foregoing method embodiments.

Illustrative examples of such a computer device, a non-transitory computer-readable storage medium and a computer program product are described below in conjunction with FIG. 5 .

5 shows an exemplary configuration of a computer device 500 for practicing the methods described herein. For example, server 120 and/or client device 110 shown in FIG. 1 may include an architecture similar to computer device 500 . The image processing device/device may be fully or at least partially implemented by the computer device 500 or a similar device or system.

Computer device 500 may be a variety of different types of devices, such as a server of a service provider, a device associated with a client (eg, a client device), a system on a chip, and/or any other suitable computer device or computing system. . Examples of computer device 500 include a desktop computer, server computer, notebook computer or netbook computer, mobile device (e.g. tablet computer, cellular or other wireless phone (e.g. smart phone), notepad computer) , mobile stations), wearable devices (eg glasses, wristwatches), entertainment devices (eg entertainment devices, set-top boxes communicatively coupled to display devices, game consoles), TVs or other displays devices, automotive computers, and the like, but are not limited thereto. Accordingly, the computer device 500 ranges from fully resourced devices with large amounts of memory and processor resources (eg, personal computers, game consoles) to low-resource devices with limited memory and/or processing resources (eg, personal computers, game consoles). , existing set-top boxes, handheld game consoles).

Computer device 500 includes at least one processor 502, memory 504, (plural) communication interfaces 506, display device 508 capable of communicating with each other via a system bus 514 or other suitable connection. , another input/output (I/O) device 510 and one or more mass storage devices 512.

Processor 502 may be a single processing unit or multiple processing units, and all processing units may include single or multiple computing units or multiple cores. Processor 502 may be implemented as one or more microprocessors, microcomputers, microcontrollers, digital signal processors, central processing units, state machines, logic circuits, and/or any device that manipulates signals based on operational instructions. there is. Processor 502, in addition to other functions, may include computer readable instructions (e.g., program code of operating system 516, application programs 518) stored in memory 504, mass storage device 512, or other computer readable medium. program code of, program code of other programs 520, etc.) may be configured to obtain and execute.

The memory 504 and the mass storage device 512 are examples of computer readable storage media for storing instructions executed by the processor 502 to execute each function described above. For example, memory 504 may typically include both volatile and non-volatile memory (eg, RAM, ROM, etc.). In addition, mass storage device 512 typically includes hard disk drives, solid state drives, removable media, external and removable drives, memory cards, flash memory, floppy disks, optical disks (eg CDs, DVDs), storage may include arrays, network attached storage, storage area networks, and the like. Memory 504 and mass storage device 512 may both be collectively referred to herein as a memory or computer-readable storage medium, which is a non-transitory medium capable of storing computer-readable instructions and processor-executable program instructions in the form of computer program code. , and the computer program code may be executed by the processor 502, which is a specific machine configured to implement the operations and functions described in the examples herein.

A plurality of program modules may be stored in the mass storage device 512 . These programs include an operating system 516, one or a plurality of application programs 518, other programs 520, and program data 522, which can be loaded into memory 504 and executed. Examples of such application programs or program modules include, for example, program logic for implementing elements/functions of method 200 (including any suitable steps of method 200) and/or other embodiments described herein. (eg, computer program code or instructions).

Although shown as stored in memory 504 of computer device 500 in FIG. 5 , modules 516 , 518 , 520 and 522 , or portions thereof, are in any form of computer readable medium accessible by computer device 500 . can be implemented using As used herein, “computer readable media” includes at least two types of computer readable media: computer storage media and communication media.

Computer storage media includes volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Computer storage media may include RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital universal disk (DVD) or other optical storage device, magnetic box, magnetic tape, magnetic disk storage device or other magnetic storage device, or It includes, but is not limited to, other non-transport media that may be used to store information for access by a computing device.

In contrast, communication media may embody computer readable instructions, data structures, program modules, or other data in a modulated data signal, such as, for example, a carrier wave or other transport mechanism. Computer storage media, as defined herein, does not include communication media.

Computer device 500 may further include one or more communication interfaces 506 for exchanging data with other devices via a network, direct connection, or the like, as described above. Such communication interface may be any type of network interface (e.g. NIC), wired or wireless (e.g. IEEE 802.11 wireless LAN (WLAN)) interface, Wi-MAX interface, Ethernet interface, Universal Serial Bus (USB) interface, a cellular network interface, a Bluetooth ^™ interface, a near field communication (NFC) interface, or the like. Communications interface 506 may facilitate communication within a variety of network and protocol types, including wired networks (eg, LAN, cable, etc.) and wireless networks (eg, WLAN, cellular, satellite, etc.), the Internet, and the like. there is. Communications interface 506 may also provide communication with external storage devices (not shown) such as storage arrays, network attached storage, storage area networks, and the like.

In some examples, it may include a display device 508 such as a monitor for displaying information and images to a user. The other I/O device 510 may be a device that receives various inputs from the user and provides various outputs to the user, and may include a touch input device, a gesture input device, a camera, a keyboard, a remote controller, a mouse, a printer, an audio input/output device, and the like. can include

Although the present disclosure has been shown and described in detail in the drawings and foregoing description, such description is to be regarded as explanatory and schematic rather than limiting, and the present disclosure is not limited to the disclosed embodiments. Variations to the disclosed embodiments can be understood and implemented by those skilled in the art in practicing the claimed subject matter by studying the drawings, the disclosure and the appended claims. In the claims, the word “comprising” does not exclude other unlisted elements or steps, and the words “an” or “an” do not exclude a plurality. The mere fact that certain measures are recited in different dependent claims does not indicate that a combination of these measures cannot be used to advantage.

Claims (16)

In the image processing method,
acquiring a first image sequence corresponding to a first human body region and a second image sequence corresponding to the first human body region, images in the first image sequence having at least one of a collection position, a collection depth, and a collection angle with respect to each other; is different, the images in the second image sequence basically have the same acquisition angle, and the images in the second image sequence are sequentially collected at different time points; and
generating a third image sequence based on the first image sequence and the second image sequence, the third image sequence including at least one update image, the at least one update image comprising the second image sequence corresponds to at least one image of and has a different viewing angle than the at least one image; Image processing method comprising a.
According to claim 1,
The image of the second image sequence is a contrast agent auxiliary imaging image.
According to claim 1 or 2,
Generating a third image sequence based on the first image sequence and the second image sequence,
at least one human body organ, part or tissue in the first body region appearing in at least one image of the first image sequence, based on the first image sequence and the second image sequence; and the second image sequence determining a correspondence between at least one human body organ, part, or tissue in the first body region appearing in at least one image of the body; and
generating the third image sequence based on the correspondence, the first image sequence, and the second image sequence; Image processing method comprising a.
According to claim 3,
at least one image of the first image sequence includes one or a plurality of identified blood vessel segment regions, and the identified blood vessel segment regions have corresponding names;
The correspondence relationship is a naming mapping from the identified blood vessel segment region in the at least one image of the first image sequence to at least one blood vessel segment region in the at least one image of the second image sequence.
According to claim 3 or 4,
At least one image of the first image sequence includes the identified lesion area, wherein generating the third image sequence based on the correspondence, the first image sequence, and the second image sequence comprises: and generating an image portion corresponding to the identified lesion area in at least one image of the third image sequence based on the correspondence relationship.
According to any one of claims 3 to 5,
The step of determining the correspondence relationship is,
determining a reference image of the second image sequence;
obtaining a reference two-dimensional view of the first image sequence based on the reference image; and
determining the correspondence based on the reference 2D view and the reference image; Image processing method comprising a.
According to claim 6,
Obtaining a reference 2D view of the first image sequence based on the reference image,
and obtaining the reference 2D view based on a collection angle of the second image sequence.
According to claim 6,
Obtaining a reference 2D view of the first image sequence based on the reference image,
obtaining a plurality of observation views having a plurality of different observation angles based on the first image sequence; and
comparing with the reference image, determining the reference 2D view from the plurality of observation views; Image processing method comprising a.
According to any one of claims 6 to 8,
Determining a reference image in the second image sequence,
determining a first blood vessel segment region from the second image sequence that satisfies a first criterion, wherein the first criterion includes at least one of a blood vessel segment length, a development degree of a contrast agent, and collection sharpness; and
determining an image frame in which the first blood vessel segment region is located as the reference image; Image processing method comprising a.
According to any one of claims 6 to 8,
Determining a reference image in the second image sequence,
acquiring a first shape point in the second image sequence, wherein the first shape point corresponds to at least one of a branch point, a connection point, and a turning point of blood vessels; and
and determining an image frame in which the first shape point is located as the reference image.
According to claim 10,
The method further includes obtaining a second shape point corresponding to the first shape point in the reference 2D view, wherein determining the correspondence relationship based on the reference 2D view and the reference image determining a correspondence between one or more blood vessel segment regions associated with the second shape point in the reference 2D view and one or more blood vessel segment regions associated with the first shape point in the reference image; Image processing method comprising a.
According to any one of claims 2 to 10,
The image processing method further comprising generating an enhanced first image sequence, wherein the enhanced first image sequence includes an contrast value based on the second image sequence.
In the image processing device,
an image sequence acquiring unit for acquiring a first image sequence corresponding to a first body region and a second image sequence corresponding to the first human body region, the images in the first image sequence having a collection position, a collection depth or a collection point relative to each other; at least one of the angles is different, the images in the second image sequence basically have the same acquisition angle, and the images in the second image sequence are sequentially collected at different time points; and
image sequence generation unit for generating a third image sequence based on the first image sequence and the second image sequence, the third image sequence comprising at least one updated image, the at least one updated image comprising the corresponding to at least one image of the second image sequence, and having a different viewing angle than the at least one image; Image processing device comprising a.
a memory, a processor and a computer program stored in the memory;
A computing device, wherein the processor is configured to execute the computer program to implement the method according to any one of claims 1 to 12.
A non-transitory computer-readable storage medium in which a computer program is stored,
A non-transitory computer-readable storage medium embodying the steps of the method according to any one of claims 1 to 12 when the computer program is executed by a processor.
A computer program stored in a computer readable storage medium,
13. A computer program comprising instructions, which when executed by at least one processor implement the steps of the method according to any one of claims 1 to 12.

Images