JP6202960B2 - Image processing apparatus, image processing method, and program - Google Patents

Description

  The present invention relates to an image processing apparatus, a control method for the image processing apparatus, and a program, and in particular, a nuclear magnetic resonance imaging apparatus (MRI), an X-ray computed tomography apparatus (X-ray CT), an ultrasonic diagnostic imaging apparatus (US), and the like. The present invention relates to a technique for processing medical images taken by various medical image collection devices (modalities).

  In the medical field, when a site of interest is present on an image of a certain modality, the corresponding site may be identified on an image of another modality, and diagnosis may be performed by comparing the identified site. When the imaging posture differs between modalities, the shape of the subject at the time of imaging is different, so that identification becomes difficult. Therefore, an attempt has been made to estimate the deformation of the subject between the two (that is, to perform alignment between images accompanying the deformation). Thereby, it is possible to estimate the position of the corresponding part based on the position information of the target part, or to generate an image in which one image is deformed to have the same shape as the other.

  Non-Patent Document 1 discloses a technique for displaying an MRI image photographed in a prone position and an ultrasonic image photographed in a supine position with the positions thereof aligned. Specifically, a finite element model (FEM) is generated based on the MRI image photographed in the prone position, and a deformation simulation from the prone position to the supine position is performed using the model. Based on the simulation result, the MRI image photographed in the prone position and the lesion area drawn in the MRI image are superimposed on the ultrasound image in the supine position. Thereby, the position difference due to the deformation between the prone position and the supine position is corrected.

T. Carter, C. Tanner, N. Beechey-Newman, D. Barratt and D. Hawkes, "MR navigated breast surgery: Method and initial clinical experience," MICCAI2008.

  According to the method described in Non-Patent Document 1, it can be expected that highly accurate deformation estimation is performed between the prone position and the supine position. On the other hand, a change in the body position from the prone position to the supine position causes a deformation that compresses the breast of the subject. Therefore, it is necessary to generate a deformed image in which the MRI image taken in the prone position is compressed. At that time, there is a problem that the quality of the generated deformed image is deteriorated as compared with the quality of the original MRI image taken in the prone position.

  The present invention has been made in view of the above problems, and provides a technique capable of generating a deformed image without degrading the quality of the original image as much as possible when the deformed image is generated by deforming the original image. With the goal.

  In order to achieve the above object, an image processing apparatus according to an aspect of the present invention has the following arrangement.

That is, image acquisition means for acquiring a first image of a subject;
Deformation information acquisition means for acquiring deformation information of the subject;
Derivation means for deriving the resolution of the second image obtained by deforming the first image based on the resolution of the first image and the deformation information;
Generating means for generating the second image at the derived resolution.

  According to the present invention, when a deformed image is generated by deforming the original image, the deformed image can be generated without degrading the quality of the original image as much as possible.

1 is a diagram illustrating a functional configuration of an image processing system according to a first embodiment. 1 is a diagram showing a hardware configuration of an image processing system according to a first embodiment. 3 is a flowchart showing a processing procedure of the image processing apparatus according to the first embodiment. FIG. 5 is a diagram for explaining in detail processing in step S310 of the image processing apparatus according to the first embodiment. 9 is a flowchart showing a processing procedure of the image processing apparatus according to the second embodiment. 10 is a flowchart showing a processing procedure of the image processing apparatus according to the third embodiment. 10 is a flowchart showing a processing procedure of the image processing apparatus according to the fourth embodiment. FIG. 10 is a diagram for explaining in detail processing in step S730 of the image processing apparatus according to the fourth embodiment.

  Hereinafter, preferred embodiments of an image processing apparatus and method according to the present invention will be described in detail with reference to the accompanying drawings. However, the scope of the invention is not limited to the illustrated examples.

(First embodiment)
<Outline of the first embodiment>
The image processing apparatus according to the present embodiment changes the posture from the prone position to the supine position based on the MRI image (target image, that is, the first image) obtained by photographing the breast of the target case in the prone position. MRI image (deformed image, that is, a second image obtained by deforming the first image) is virtually generated as if the breast deformation was estimated and the image was taken in the supine position. In the following description, a case where an MRI image of a breast imaged in the prone position is a processing target will be described. However, the scope of the present invention is limited to the body position and modality type of the subject. is not.

<Functional configuration of image processing system>
FIG. 1 is a diagram illustrating a functional configuration of the image processing system according to the present embodiment. As shown in FIG. 1, the image processing system according to the present embodiment includes an image processing apparatus 100, an MRI image photographing apparatus 110, a data server 120, a monitor 130, and an operation input unit 140.

  The image processing apparatus 100 includes a target image acquisition unit 1000, a deformation information acquisition unit 1010, a resolution derivation unit 1020, a deformation image generation unit 1030, and a display control unit 1040. Then, it is connected to the data server 120 and the monitor 130. Note that the image processing apparatus 100 is connected to the MRI image capturing apparatus 110 via the data server 120 in the example of FIG. 1, but may be directly connected to the MRI image capturing apparatus 110.

  Each processing unit included in the image processing apparatus 100 will be described. The target image acquisition unit 1000 acquires from the data server 120 an MRI image (target image, that is, the first image) of the prone breast related to the subject (target case) targeted by the image processing apparatus 100 in the present embodiment. The image processing apparatus 100 takes it in.

  Based on the target image (first image) acquired by the target image acquisition unit 1000, the deformation information acquisition unit 1010 derives information related to the deformation from prone to supine on the target case. The resolution deriving unit 1020 is generated by a deformed image generating unit 1030 described later based on information on the resolution of the target image acquired by the target image acquiring unit 1000 and information on the deformation of the target case derived by the deformation information acquiring unit 1010. The resolution of the deformed image (second image) is derived.

  The deformed image generation unit 1030 uses the prone breast MRI image (first image) acquired by the target image acquisition unit 1000 based on the information about the deformation of the target case acquired by the deformation information acquisition unit 1010. A deformed image (second image) deformed to the back is generated with the resolution derived by the resolution deriving unit 1020. The display control unit 1040 outputs the deformed image generated by the deformed image generating unit 1030 to the monitor 130.

  The MRI imaging apparatus 110 is an apparatus that acquires an image obtained by nuclear magnetic resonance, that is, an MRI image, of information related to a three-dimensional region inside a subject that is a human body. The MRI image capturing apparatus 110 is connected to the data server 120 and transmits the acquired MRI image to the data server 120. The data server 120 is an apparatus that holds the MRI image captured by the MRI image capturing apparatus 110. The monitor 130 displays the deformed image output from the display control unit 1040. The operation input unit 140 accepts pointing input and input of characters, commands, and the like by the user.

<Hardware configuration of image processing system>
FIG. 2 is a diagram illustrating an example of a hardware configuration of the image processing system according to the present embodiment. The image processing system according to the present embodiment includes an image processing apparatus 100, an MRI image capturing apparatus 110, a data server 120, a monitor 130, a mouse 1401, and a keyboard 1402.

  The image processing apparatus 100 can be realized by a personal computer (PC), for example. The image processing apparatus 100 includes a central processing image processing apparatus (CPU) 201, a main memory 202, a magnetic disk 203, and a display memory 204.

  The CPU 201 mainly controls the operation of each component of the image processing apparatus 100. The main memory 202 stores a control program executed by the CPU 201 and provides a work area when the CPU 201 executes the program. The magnetic disk 203 stores an operating system (OS), device drives of peripheral devices, various application software including programs for performing processing described later, and the like.

  The display memory 204 temporarily stores display data for the monitor 130. The monitor 130 is a CRT monitor or a liquid crystal monitor, for example, and displays an image based on the data acquired from the display memory 204. A mouse 1401 and a keyboard 1402 are an example of the operation input unit 140 described with reference to FIG. 1, and perform a pointing input and input of characters, commands, and the like by a user. Each component is communicably connected to each other via a common bus 205.

<Processing flow>
Next, with reference to a flowchart of FIG. 3, a procedure of processing performed by the image processing apparatus 100 according to the present embodiment will be described. The processing according to the present embodiment is realized by the CPU 201 reading and executing a program that realizes the function of each unit stored in the main memory 202. The results of each process performed by the image processing apparatus 100 described below are recorded by being stored in the main memory 202.

[Step S300: Acquisition of target image]
In step S300, the target image acquisition unit 1000 acquires the MRI image of the case targeted for processing according to the present embodiment from the data server 120, and executes a process of developing and holding it on the main memory 202 of the image processing apparatus 100. To do.

  In this embodiment, a case where an MRI image of a breast in a prone position of a target case is acquired will be described as an example. In the present embodiment, this MRI image is referred to as a target image. Here, the target image is a three-dimensional image obtained by photographing the inside of the target case with the MRI image photographing device 110, and specifically, is configured by pixel values (luminance values) in pixels that are three-dimensionally arranged in the image region. Is done. Further, in order to define the image area, the size (pixel size) of each pixel in the real space is also attached to the target image as supplementary information. The target image in the present embodiment is configured by a plurality of pixels that can be identified by orthogonal three-axis (X, Y, Z) direction components, and the pixel size as supplementary information is for each of the three-axis directions. Defined. In the present embodiment, a specific example will be described in which the pixel sizes in the directions of the three axes are v_size_x = 1.0 mm, v_size_y = 1.0 mm, and v_size_z = 1.0 mm.

  Note that the luminance value of the target image can also be regarded as a function derived with reference to pixel addresses in a three-dimensional array of pixels. In the present embodiment, the target image is represented as a function I (x, y, z). The function I (x, y, z) is a function that returns the pixel value at the position, using the three-dimensional real space coordinate value (x, y, z) of the shooting area of the target image as an argument. In addition, the image processing apparatus 100 holds the target image as a set of pixels constituting the target image. At this time, the pixels are recorded in a three-dimensional memory array. In the present embodiment, the memory array that holds the target image is represented as I_mem (dx, dy, dz). Here, dx, dy, dz are integers representing addresses of the memory array. The position of the pixel indicated by this address in the real space is (dx × size_x, dy × size_y, dz × size_x). The ranges of the addresses dx, dy, dz are 1 ≦ dx ≦ Nx, 1 ≦ dy ≦ Ny, and 1 ≦ dz ≦ Nz, respectively. Here, the address ranges Nx, Ny, Nz are directly related to the shooting range of the target image in the real space, and (Nx × size_x, Ny × size_y, Nz × size_z) is the shooting range.

[Step S310: Derivation of deformation information]
In step S310, the deformation information acquisition unit 1010 executes processing for deriving information related to deformation from the prone position to the supine position in the breast of the target case. Here, the information regarding deformation includes, for example, a deformation field from the prone position to the supine position.

  As a specific example of this processing, a case will be described as an example in which a deformation occurring in the breast due to a difference in the gravity direction with respect to the breast between the prone position and the supine position is targeted. In this case, the breast is deformed in the compressing direction due to a change in the load due to the weight of the breast. This deformation can be simulated by using, for example, a finite element method. A specific procedure will be described with reference to FIG. As shown in FIG. 4 (a), the target image 400 includes a region obtained by photographing the inside of the breast region 410 of the target case. In FIG. 4 (a), the breast region 410 is a region surrounded by the body surface 420 and the pectoral muscle surface 430. The body surface 420 and the pectoral muscle surface 430 can be extracted automatically or manually by subjecting the target image 400 to image processing by a known method. Next, a mesh is generated to apply the finite element method to the breast region 410.

  Specifically, as shown in FIG. 4B, a mesh that divides the breast region 410 by mesh elements 440 is generated. Here, it is assumed that the total number of mesh elements is N_elem. The mesh element can take the form of a tetrahedron or a hexahedron. The position coordinates of the vertex of the generated mesh are expressed as Pi = (pix, piy, piz). Here, i is a subscript indicating the index of the vertex of the mesh, and in this embodiment, it is an integer of 1 ≦ i ≦ N_node. Here, N_node is the total number of vertices of the mesh. A deformation simulation by the finite element method can be executed by setting the mechanical characteristics, external force, boundary conditions, and the like of each mesh element for the mesh structure thus generated. In the present embodiment, as shown in FIG. 4 (c), the displacement amount of the vertex position of the mesh from the prone position to the supine position is derived by deformation simulation. This amount of displacement is a difference value between the position Pi of the mesh and the position after deformation, and is expressed as Di = (dix, diy, diz) here. That is, the position of the vertex of the i-th mesh in the supine position is Pi + Di.

  In step S310, a deformation field from the prone position to the supine position of the target case is derived based on the information regarding the connection of Pi, Di, and the vertexes of the mesh. Specifically, the displacement inside the mesh element is obtained by linear interpolation or the like. This method can be executed by a well-known method related to deformation expression using a mesh model. As a result, a three-dimensional function f_deform (x, y, z) representing the deformation related to the target case is obtained. Here, the function f_deform (x, y, z) is a function that takes the image coordinates (3D) of the target image obtained by photographing the target case in the prone position as an argument and returns the transformed position (3D) of the coordinate position It is.

  Information on the mesh derived by the above method is recorded and held as information on deformation. Here, the information on the mesh includes information on the vertex position Pi, the displacement amount Di, and the mesh vertex connection. In addition, an area that is the breast area 410 in the target image 400 is recorded and held. Here, a mask image f_mask (x, y, z) representing the breast region 410 is generated, recorded and held. However, f_mask (x, y, z) takes the three-dimensional coordinate value (x, y, z) in the real space of the shooting area of the target image as an argument, 1 if the position is inside the breast area 410, otherwise In some cases it is a scalar function that returns a value of 0. The function f_deform (x, y, z) is recorded as an actual value (a pair of an argument and a return value) inside the breast region 410.

[Step S320: Derivation of local compression ratio]
In step S320, the resolution deriving unit 1020 executes a process for deriving a local compression rate for each local region constituting the target image associated with the deformation based on the information regarding the deformation of the target case derived in step S310. .

  The derivation of the local compression rate is executed based on the information regarding the mesh generated in step S310. First, for each of the N_elem mesh elements generated in step S310, a change in the size of the mesh element is derived. Specifically, for example, when attention is paid to the j-th mesh element, a bounding box surrounding the positions of a plurality of vertices constituting the mesh element is derived. If the form of the mesh element of interest here is a hexahedron, it becomes a bounding box for the six vertex positions that make up the hexahedron. Let bbXj, bbYj, and bbZj be the sizes of the derived bounding box in the X-axis, Y-axis, and Z-axis directions. Next, the size after deformation of the mesh element of interest (the size of the bounding box) is similarly derived. Here, the sizes are assumed to be bbXj ′, bbYj ′, and bbZj ′. Next, the change rate of the size accompanying the deformation of the derived bounding box is derived. Specifically, for each coordinate axis, the value is derived by equation (1). In this embodiment, the compression rate is derived, but the rate of change may be defined as the expansion rate. The subsequent processing can be similarly applied to the expansion rate.

  The above processing is performed for all mesh elements (local regions), and the derived compXj, compYj, and compZj (1 ≦ j ≦ N_elem) are recorded and retained.

[Step S330: Derivation of resolution of deformed image]
In step S330, the resolution deriving unit 1020 executes a process of deriving the resolution of the deformed image generated by the deformed image generating unit 1030 in step S340 described later based on the local compression rate related to the target case derived in step S320. . Specific processing will be described below. First, the maximum value of the compression rate of all mesh elements is derived based on the compression rate of each mesh element derived in step S320. This process is executed by equation (2).

  Then, based on the derived maximum value of compression rate (maxCompX, maxCompY, maxCompZ) and the resolution (size_x, size_y, size_z) of the target image, the resolution (size_x ', size_y', size_z ') of the deformed image is expressed by Derived by (3).

  Accordingly, the resolution of the deformed image can be determined based on the compression ratio of the local region that is most compressed by the deformation of the target case derived in step S310. In accordance with the above processing, the resolution deriving unit 1020 executes processing for deriving the resolution of the deformed image generated in step S340.

[Step S340: Generation of deformation image]
In step S340, the deformed image generation unit 1030 executes processing for generating a deformed image obtained by deforming the target image based on the information related to the deformation derived in step S310. Here, the deformed image is an image spatially aligned with the target case in the supine position, and is an image similar to an image obtained when the target case is photographed in the supine position. In the present embodiment, the deformed image is expressed as I _deformed (x, y, z). The deformed image is a three-dimensional image having the resolution derived in step S330. Hereinafter, specific processing for generating a deformed image will be described.

  First, the deformed image generation unit 1030 generates a three-dimensional array of memory areas for storing deformed images. The shooting range in the real space where the target image is shot is (Nx × size_x, Ny × size_y, Nz × size_z), and the deformed image is also an image in the same range. However, the value derived in step S330 is used as the resolution of the deformed image. That is, the memory area for storing the deformed image is (Nx × size_x ÷ size_x ′, Ny × size_y ÷ size_y ′, Nz × size_z ÷ size_z ′). The memory area generated by the above processing is expressed as I_deform_mem (dx, dy, dz). The meaning of the notation is the same as in the case of the memory area I_mem (dx, dy, dz) of the target case described in step S300, and the description is omitted here.

  Next, based on the deformation f_deform (x, y, z) of the target case derived in step S310, a deformed image I_deform (x, y, z) is generated by deforming the target image I (x, y, z). To do. However, the actual processing in the image processing apparatus 100 is based on the modified f_deform (x, y, z) for the luminance value of the memory area I_mem (dx, dy, dz) in the three-dimensional array that holds the luminance value of the target image. Thus, a process of mapping to the memory area I_deform_mem (dx, dy, dz) of the deformed image is executed. Specifically, the deformed image can be generated by executing the calculation shown in the following equation with respect to a plurality of pixels constituting the image.

However, “*” is an operator that means a product operation between vector elements. F ^-1 _deform is an inverse function of the function f_deform representing the deformation field acquired in step S310, and takes a position in the supine position of the target case as an argument, and returns a position of the prone position corresponding to that position. It is. Since f ⁻¹ _deform can be obtained by a well-known method based on f_deform, detailed description is omitted here. A deformed image, I _deformed (x, y, z) is generated as a three-dimensional array memory area I_deform_mem (dx, dy, dz) in the image processing apparatus 100 by the processing in step S340 described above.

[Step S350: Display deformed image]
In step S350, the display control unit 1040 transmits information related to the deformed image generated in step S340 to the monitor 130 connected to the image processing apparatus 100, and controls to display the information. At this time, the display control unit 1040 can execute processing for generating a two-dimensional cross-sectional image obtained by cutting the three-dimensional deformed image along a predetermined surface. For example, the predetermined surface can be interactively operated by the user using the mouse 1401, the keyboard 1402, and the like. Accordingly, a cross-sectional image obtained by cutting out an arbitrary region desired by the user at an arbitrary angle from the three-dimensional deformed image can be displayed at an arbitrary enlargement / reduction ratio. The image display method is not limited to the above method. For example, a cross-sectional image obtained by cutting out the corresponding cross-sections of the target image and the deformed image from both sides may be generated, and the images may be displayed side by side or superimposed. good. Further, a three-dimensional deformed image may be displayed by volume rendering.

  As described above, each process of the flowchart of FIG. 3 is performed. This provides a high-quality and efficient mechanism for generating a deformed image of a prone position of the target case (target image) transformed into the supine position without missing as much luminance information as possible from the target image. it can.

<Modification 1-1: Manual deformation and application to alignment between two images>
In the description of the first embodiment, the case where the derivation of the deformation from the prone position to the supine position is performed by simulation using the finite element method in step S310 has been described as an example. Not limited to. For example, input information from the user may be acquired via the mouse 1401 or the keyboard 1402, and a deformation applied to the target image may be acquired based on the acquired information. In addition to this, for example, a low-resolution MRI image obtained by photographing the breast of the target case in a supine position is acquired, and the target image is deformed and aligned with the image by a well-known method, and the deformation is determined from the result of the deformed positioning. You may make it acquire. That is, the derivation or acquisition of the deformation of the target case in step S310 may be executed by any method. In either case, the same effects as those described in the first embodiment can be obtained.

<Modification 1-2: It may be based on an average value of compression ratio, a histogram, etc.>
In the description of the first embodiment, the case in which the resolution of the deformed image is derived based on the maximum value of the local compression rate derived in step S320 in step S330 has been described as an example. Not limited to this. For example, the resolution of the deformed image may be derived based on the average value or median value of the local compression rates derived in step S320. Alternatively, a local compression ratio histogram derived in step S320 may be generated, and the resolution of the deformed image may be derived based on the compression ratio value corresponding to a predetermined displacement value from the top. In addition, it derives a plurality of resolution candidates based on a plurality of indices such as the maximum value, average value, and median value of the local compression rate as described above, and allows the user to select a more suitable resolution from them, You may make it have a mechanism to select automatically. According to these methods, it is possible to generate a deformed image with a resolution that balances the quality of the deformed image and the storage capacity even in the case where high compression occurs in a very small local region in the entire breast region. Is obtained.

<Modification 1-3: Further may be based on luminance distribution>
In the description of the first embodiment, the case where the resolution of the deformed image is derived based on the local compression rate derived in step S320 in step S330 has been described as an example. However, the embodiment of the present invention is not limited thereto. Absent.

  For example, the resolution of the deformed image may be derived based on the luminance distribution of the target image in addition to the local compression rate derived in step S320. For example, the local evaluation value is derived based on the local compression rate and the complexity value of the luminance value of the target image at the local position. Then, the resolution of the deformed image is derived based on the local maximum evaluation value. Here, the complexity of the luminance value is a value calculated based on, for example, the dispersion of pixel values in the local region of the target image and the spatial frequency. This value may be, for example, the sum or average of luminance values that vary when the luminance values of the local area are smoothed. The local evaluation value may be a value obtained by an operation such as a sum or product of a local compression rate and a luminance value and a complexity value. This value is related to the resolution required when generating the deformation image of the local region. For example, if the local compression rate is high and the complexity of the brightness value is high, if the deformed image of the area is generated at a low resolution, the information on the brightness value of the original target image is greatly lost. become. In this case, the evaluation value is a high value, and is an evaluation value indicating the necessity of generating a deformed image of the area with high resolution.

  By deriving the resolution of the deformed image based on the above-described local evaluation value, for example, in the case where there is an area where the distribution of luminance values is faint although there is local compression, the first embodiment Compared to the above, there is an effect that a high-quality deformed image can be generated with high recording efficiency.

  As described above, in the present embodiment, the resolution (pixel size) of the deformed image (second image) is determined based on the deformation information when generating the deformed image (second image). Thereby, when generating a deformed image obtained by deforming the original image (first image), it is possible to generate a high-quality deformed image (second image) reflecting the luminance distribution of the original image. .

(Second embodiment; may be based on the compression ratio of the region of interest)
In the first embodiment, the case in which the resolution of the deformed image is derived based on the maximum value of the local compression rate derived in step S320 in step S330 has been described as an example, but the embodiment of the present invention is not limited thereto. Not exclusively. In the second embodiment, a target image is acquired, and a region of interest (region of interest) such as a tumor position depicted in the target image is extracted or acquired, and based on a local compression rate at that position (region of interest) An example of deriving the resolution of the deformed image will be described.

  The functional configuration and hardware configuration of the image processing apparatus 100 according to the present embodiment are the same as those of the first embodiment, but further include a notable region acquisition unit (not shown). Hereinafter, with reference to a flowchart of FIG. 5, a procedure of processing performed by the image processing apparatus 100 according to the present embodiment will be described. Since the process of step S500 is the same as the process of step S300 of the first embodiment, a description thereof will be omitted.

  In step S505, the attention area acquisition unit of the image processing apparatus 100 executes processing for acquiring the attention position in the target image. This process is, for example, a process of acquiring a position of a part of interest of the user such as a tumor depicted in the target image. For example, by directly specifying an input according to a user operation using the operation input unit 140 such as a mouse 1401 or a keyboard 1402 get. In addition to this, it is also possible to automatically detect a part that the user pays attention to or a part that the user should pay attention to by performing image processing on the target image and acquire the position. Since the processes in steps S510 and S520 are the same as the processes in steps S310 and S320 of the first embodiment, the description thereof is omitted.

  In step S530, the resolution deriving unit 1020 executes a process for deriving the resolution of the deformed image generated in step S540, which will be described later, based on the local compression rate related to the deformation of the target case derived in step S520. In the first embodiment, the case where the resolution is derived based on the maximum value of the local compression rate of the target case has been described as an example. However, in the present embodiment, the attention position acquired in step S505 and the local position in the vicinity thereof are described. The resolution is derived based on the compression ratio. For example, the resolution is derived based on the average value or the maximum value of the compression rate in a region within a predetermined distance range from the target position. Since the processes of Step S540 and Step S550 are the same as the processes of Step S340 and Step S350 of the first embodiment, description thereof will be omitted.

  As described above, according to the present embodiment, there is an effect that a deformed image can be generated without degrading the quality of an image in a target region to be observed in particular in the target image as much as possible.

(Third embodiment: Preprocessing such as image rotation is performed based on the main axis of deformation)
In the third embodiment, an example will be described in which the processing described in the first embodiment is more effectively performed by performing preprocessing that substantially matches the main axis of deformation of the target case with the coordinate axis of the image coordinates of the target image. To do.

  The functional configuration and hardware configuration of the image processing apparatus 100 according to the present embodiment are the same as those in the first embodiment, but further include a conversion unit (not shown). Hereinafter, with reference to a flowchart of FIG. 6, a procedure of processing performed by the image processing apparatus 100 according to the present embodiment will be described. Since the processing of step S600 and step S610 is the same as the processing of step S300 and step S310 of the first embodiment, description thereof will be omitted.

  In step S615, the conversion unit of the image processing apparatus 100 derives a deformation main axis, which will be described later, based on the deformation information derived in step S610, and executes a process of performing coordinate conversion of the target image, Pi, and Di based on the main axis. . That is, the third principal component is derived from the first principal component by performing linear principal component analysis on Di (where 1 ≦ i ≦ N_node) derived in step S610. Here, each principal component is a three-dimensional orthonormal vector. These three orthonormal vectors are referred to as deformation main axes. Using this orthogonal vector, orthogonal transformation is performed so that the image coordinate axes (X, Y, Z) of the target image substantially coincide with the main axis of deformation. That is, rotation conversion is performed. At this time, the resolution and the like of the image after rotation are appropriately set so as not to lose as much luminance information as possible of each pixel of the target image before rotation. For example, the resolution may be determined using the same configuration as in the first embodiment. Further, Pi and Di derived in step S610 are similarly subjected to rotation processing. The subsequent processing is performed using the rotated target image, Pi, and Di obtained by the above processing. The process from step S620 to step S640 executes the same process as step S320 to step S340 of the first embodiment.

  The process of step S650 executes the same process as step S350 of the first embodiment. However, when generating and displaying a cross-sectional image from a three-dimensional deformed image based on user input, etc., the display enlargement / reduction ratio can be switched according to the resolution of the deformed image in the region for generating the cross-sectional image, The range of the enlargement / reduction ratio may be limited. For example, an area with a high resolution can be displayed with a high magnification according to a user input, and conversely, an area with a low resolution cannot be displayed with a high resolution. Thereby, since the enlargement / reduction ratio is controlled in accordance with the density of the luminance information included in the display area, the user can display an image at an appropriate enlargement / reduction ratio.

  As described above, in the present embodiment, when the main axis of deformation that occurs in the target case is different from the coordinate axis of the image, preprocessing is performed to substantially match them. Thereby, there is an effect that the setting of the compression rate for each coordinate axis executed in the processing after step S620 can be more effectively performed for the deformation of the target case.

(Fourth embodiment; resolution is determined locally according to local deformation)
In the first embodiment, the case where the resolution derived based on the local deformation of the target case is applied to the entire deformed image has been described as an example, but the embodiment of the present invention is not limited to this. In the fourth embodiment, a case where the resolution derived based on the local deformation of the target case is applied to a local region of the deformed image will be described as an example.

  Since the functional configuration and hardware configuration of the image processing apparatus 100 according to the present embodiment are the same as those in the first embodiment, description thereof will be omitted. Hereinafter, with reference to a flowchart of FIG. 7, a procedure of processing performed by the image processing apparatus 100 according to the present embodiment will be described. Since the processing from step S700 to step S720 is the same as the processing of step S300 and step S320 of the first embodiment, description thereof will be omitted.

  In step S730, the resolution deriving unit 1020 executes a process for deriving the resolution of the deformed image generated in step S740, which will be described later, based on the local compression rate related to the deformation of the target case derived in step S720. In the first embodiment, the case where a single resolution to be applied to the entire deformed image is derived based on the maximum value of the local compression rate of the target case has been described as an example. On the other hand, the present embodiment is different from the first embodiment in that the resolution is derived for each local region and a deformed image in which a plurality of resolutions are mixed is generated. In step S730, the local resolution is derived.

  A specific example of this process will be described in detail with reference to FIG. FIG. 8 (a) is a diagram in which memories in a three-dimensional array that store pixels of the target image are arranged to reflect the size of each pixel in real space. However, although it is originally a three-dimensional array, for convenience of explanation on paper, a cross section of Z = (arbitrary constant) is cut out and described using a two-dimensional diagram. The pixel size (resolution) of the target image of the present embodiment is size_x = 1.0 mm, size_y = 1.0 mm, and size_z = 1.0 mm, and pixels of isotropic sizes are arranged in an orderly manner. Here, a case is considered where the information related to the deformation of the target case derived in step S710 is compressed vertically (in the Y-axis direction) near the center of the target image 400 as described with reference to FIG.

  First, as shown in FIG. 8B, the deformed image 800 is divided into a plurality of regions 801 to 809. Next, using the compression rates compXj, compYj, and compZj (1 ≦ j ≦ N_elem) derived in step S720, the maximum values of the compression rates in the respective regions 801 to 809 are derived. This process will be described in more detail.

  First, one of a plurality of areas 801 to 809 included in the deformed image 800 is selected. Here, the description will be continued assuming that the area 801 is selected. Next, a deformed mesh element included in the region 801 is extracted based on the information regarding the deformation derived in step S710. This is executed based on the determination as to whether or not the vertex position Di of the mesh constituting the deformed mesh element is within the region 801. Then, the same process as step S330 of the first embodiment is performed on the extracted mesh element. That is, the maximum value of the compression rate of the plurality of extracted mesh elements is derived, and the resolution of the image after deformation of the region 801 is derived based on the maximum value. The above processing is performed on all the regions 801 to 809 included in the deformed image 800. Thereby, the resolution regarding each area | region 801-area 809 is each derived | led-out. FIG. 8 (b) shows an example of the resolution derived by the above processing. Here, a result is shown in which the resolution in the Y-axis direction of the region 805 in which compression occurs in the Y-axis direction is higher than the resolution of the target image 400 (that is, the pixel size is reduced). In other areas where compression has not occurred, the image is applied without changing the resolution of the target image.

  Through the processing in step S730 described above, the resolution of the deformed image generated in step S740 is derived based on the local compression rate of the target case derived in step S720. The process of step S740 and step S750 performs the same process as step S340 and step S350 of the first embodiment.

  As described above, in the present embodiment, a deformed image in which the resolution is adaptively set according to the local deformation characteristics of the target case is generated. As a result, even when a large deformation occurs locally in the target case, there is an effect that a high-quality deformed image that does not lose as much luminance information as possible in the target image can be generated with high recording efficiency.

  Note that the description in this embodiment described above is an example of a suitable image processing apparatus according to the present invention, and the present invention is not limited to this.

(Other embodiments)
The present invention can also be realized by executing the following processing. That is, software (program) that realizes the functions of the above-described embodiments is supplied to a system or apparatus via a network or various storage media, and a computer (or CPU, MPU, or the like) of the system or apparatus reads the program. It is a process to be executed.

Claims (15)

Image acquisition means for acquiring a first image of the subject;
Deformation information acquisition means for acquiring deformation information of the subject;
Derivation means for deriving the resolution of the second image obtained by deforming the first image based on the resolution of the first image and the deformation information;
An image processing apparatus comprising: generating means for generating the second image at the derived resolution. It said deriving means derives information on compression or expansion of the object based on the deformation information, in claim 1, characterized in that to derive the resolution of the second image based on the information The image processing apparatus described. The derivation means derives a local compression rate or expansion rate for each local region constituting the first image of the subject based on the deformation information, and based on the compression rate or expansion rate the image processing apparatus according to claim 1 or 2, characterized in that to derive the resolution of the second image. The derivation means derives a local compression rate or expansion rate for each local region constituting the first image of the subject based on the deformation information, and the derived maximum compression rate or the image processing apparatus according to any one of claims 1 to 3, characterized in that to derive the resolution of the second image based on the extended rate. The derivation means derives a local compression rate or expansion rate for each local region constituting the first image of the subject based on the deformation information, and the derived compression rate or expansion rate the image processing apparatus according to any one of claims 1 to 3 mean or on the basis of the median and wherein the deriving the resolution of the second image. The derivation means derives a local compression rate or expansion rate for each local region constituting the first image of the subject based on the deformation information, generates a histogram, and based on the histogram the image processing apparatus according to any one of claims 1 to 3, characterized in that to derive the resolution of the second image Te. Said deriving means, based on the deformation information to derive information about the compression or expansion of the subject, deriving resolution of the second image based on the luminance distribution of the with the information first image The image processing apparatus according to claim 1, wherein the image processing apparatus is an image processing apparatus. A region of interest acquisition means for acquiring a region of interest from the first image of the subject;
Said deriving means, on the basis of the said modified information in the resolution and the region of interest of the first image, the claims 1 to 7, characterized in that to derive the resolution of the second image of the object The image processing apparatus according to any one of the above.   The image processing apparatus according to claim 8, wherein the attention area acquisition unit acquires the attention area by performing image processing on the first image. An operation input means for receiving designation of the attention area in the first image according to a user operation;
The image processing apparatus according to claim 8, wherein the attention area acquisition unit acquires the attention area according to the user operation. Based on the deformation information, provided to the al converting means for rotational transform the first image and the deformation information,
The derivation means derives the resolution of the second image based on the rotation-transformed deformation information,
11. The generation unit according to claim 1, wherein the generation unit generates the second image from the rotation-converted first image at the derived resolution based on the rotation-transformed deformation information. The image processing apparatus according to any one of the above. The derivation means derives a local compression rate or expansion rate for a local region constituting the first image of the subject based on the deformation information, and based on the compression rate or expansion rate, the image processing apparatus according to resolution of the second image to any one of claims 1 to 11, wherein the deriving for each of the local region.   The image processing apparatus according to claim 1, further comprising a display control unit that causes the display device to display the second image generated by the generation unit. A picture Zosho sense how,
Obtaining a first image of a subject ;
A step of acquiring a deformation information before Symbol subject,
And deriving the previous SL on the basis of the resolution and the deformation information of the first image, the resolution of the second image obtained by modifying the first image,
Image processing how, characterized in that in front Symbol derived resolution and a step of generating said second image. Program for executing the respective steps of the image processing how according to the computer to claim 14.

Images