JP2022128882A - Image processing device, image processing program, and image processing method - Google Patents

Abstract

To correct contour lines of blood vessels extracted from captured images.SOLUTION: An image processing device 1 corrects contour lines of blood vessels generated on the basis of captured images in which the blood vessels are captured. The contour lines include a first contour line and a second contour line. The image processing device comprises: a storage unit 12 which stores the captured image in association with position information of contour pixels corresponding to the contour lines of the blood vessels; a contour line selection unit 131 which selects one of the first contour line and the second contour line as a target contour line of correction; an extraction unit 132 which extracts the plurality of contour pixels as extraction pixels on the basis of the position information of the contour pixels; a pixel selection unit 133 which selects two extraction pixels having a prescribed positional relation as selection pixels; a correction pixel determination unit 134 which determines correction pixels corresponding to the target contour line after the correction on the basis of pixel information on a plurality of reference pixels arranged at prescribed positions from the two selection pixels; and a correction contour line identifying unit 136 which identifies the target contour line after the correction on the basis of the position information of the correction pixels.SELECTED DRAWING: Figure 2

Description

The present invention relates to an image processing device, an image processing program, and an image processing method.

Medical equipment for visualizing the inside of the human body, such as CT (Computed Tomography) and MRI (Magnetic Resonance Imaging), is used for early detection of intracerebral vascular disorders. However, these medical devices are expensive. Therefore, the burden of equipment investment is large, and the number of hospitals that have such medical equipment is limited. In addition, since the medical expenses are high, the burden of medical expenses on the patient is also large.

The brain is an embryologically identical organ to the eye. Therefore, the state of blood vessels in the brain can be estimated from the state of retinal blood vessels in the fundus. Therefore, fundus examination, which does not require the above-mentioned medical equipment, has been used for diagnosing the risk of intracerebral vascular disorders. In recent years, due to the revision of the 3rd specific health checkup, fundus examination has been included in internal medicine examination items. However, the experience and skill of the judgment doctor greatly contributes to fundus examination. Therefore, it is difficult for physicians who are not specialists to perform an accurate fundus examination. Therefore, there is a demand for automation of fundus examination. Automated fundus examination requires an image processing technique for accurately extracting blood vessels from captured images of the fundus.

Image processing techniques for extracting blood vessels from captured images have also been proposed (see, for example, Non-Patent Document 1).

The technique disclosed in Non-Patent Document 1 selects solution candidates for the blood vessel wall by searching using an energy function, and extracts the solution candidate with the highest degree of similarity to the blood vessel wall as the blood vessel contour line. However, the visibility of the fundus image is poor, and the extraction accuracy of the blood vessel contour line by the technique disclosed in Non-Patent Document 1 is not necessarily good. Therefore, there is a demand for a technique for correcting the contour lines of blood vessels extracted from captured images.

Yuki Okami, 3 others, “Calculation of the caliber of fundus blood vessels in cataract patients”, Proceedings of the 50th Annual Meeting of the Chugoku-Shikoku Branch of the Japan Ergonomics Society, 2017/12/9, p. 49-50

SUMMARY OF THE INVENTION An object of the present invention is to provide a technique for correcting the contour line of a blood vessel extracted from a captured image.

An image processing apparatus according to the present invention is an image processing apparatus that corrects a contour line of a blood vessel generated based on a captured image of the blood vessel. and a second contour line indicating the other contour in the width direction. a storage unit for storing the following: a contour selection unit for selecting either one of the first contour and the second contour as a correction target contour; an extraction unit for extracting a plurality of contour pixels as extracted pixels from among the contour pixels, and a pixel selection unit for selecting two extracted pixels having a predetermined positional relationship from among the plurality of extracted pixels as selected pixels; For each of the two selected pixels, based on the pixel information of each of a plurality of reference pixels arranged at predetermined positions from the two selected pixels, correction corresponding to the target contour line after correction from among the pixels constituting the captured image It is characterized by comprising a correction pixel determination unit that determines pixels, and a correction contour line identification unit that identifies a target contour line after correction based on position information of the correction pixels.

According to the present invention, it is possible to correct the contour line of a blood vessel extracted from a captured image.

1 is a network connection diagram showing an embodiment of an image processing apparatus according to the present invention; FIG. 1 is a functional block diagram showing an embodiment of an image processing device according to the present invention; FIG. FIG. 3 is a schematic diagram showing an example of a captured image stored in a storage unit included in the image processing apparatus of FIG. 2; 1 is a flow chart showing an embodiment of an image processing method according to the present invention; 5 is a flowchart of primary correction processing included in the image processing method of FIG. 4; 6A and 6B are schematic diagrams showing captured images at each stage of the primary correction processing in FIG. 5 ; FIG. 6 is a schematic diagram showing a captured image in the middle of the primary correction process of FIG. 5; FIG. FIG. 6 is a schematic diagram showing an example of the luminance value of a reference pixel and the amount of change thereof in the primary correction process of FIG. 5; 5 is a flowchart of secondary correction processing included in the image processing method of FIG. 4; 9A and 9B are schematic diagrams showing captured images at each stage of the secondary correction process of FIG. 8; FIG. 9 is a schematic diagram showing a virtual circle generated by the secondary correction processing of FIG. 8; FIG. 3 is a functional block diagram showing another embodiment of an image processing device according to the present invention; Fig. 4 is a flow chart showing another embodiment of the image processing method according to the present invention; 13 is a flowchart of primary correction processing included in the image processing method of FIG. 12; 14A and 14B are schematic diagrams showing captured images at each stage of the primary correction processing of FIG. 13; FIG. 14 is a schematic diagram showing a captured image in the middle of the primary correction process of FIG. 13;

Hereinafter, with reference to the drawings, an image processing apparatus (hereinafter referred to as "this apparatus"), an image processing program (hereinafter referred to as "this program"), and an image processing method (hereinafter referred to as "this method") according to the present invention. and an embodiment will be described.

The present invention corrects the contour line of a blood vessel generated based on a captured image of the blood vessel. Specifically, the present invention selects pixels (hereinafter referred to as “corrected pixels”) corresponding to the contour line after correction from among the pixels forming the captured image based on the pixel information of the pixels forming the captured image. Then, the post-correction outline is specified based on the positional information of the corrected pixels. Also, the present invention corrects the position information of the correction pixel based on the curvature of each correction pixel.

The “captured image” is, for example, a fundus image captured by a fundus camera, an X-ray image captured by an X-ray imaging device, a CT image captured by a CT device, or an image captured by a normal optical camera. Also includes In the present invention, at least a blood vessel is captured as a subject in the captured image. As described above, in the present embodiment, the captured image is a fundus image.

"Vessels" includes not only human blood vessels, but also non-human animal blood vessels. In this embodiment, the blood vessel is a human fundus blood vessel.

A “contour line” is a line that traces the contour of the blood vessel captured in the captured image. In the present embodiment, the contour line is generated, for example, by performing known image processing on the captured image. Here, since the blood vessel is a tubular body having a width, the contour line of one blood vessel captured in the captured image is a contour line indicating the contour on one side of the blood vessel in the width direction (hereinafter referred to as "first contour line ”) and a contour line indicating the contour on the other side in the same width direction (hereinafter referred to as “second contour line”).

"Pixel information" is information for each pixel. In this embodiment, the pixel information includes the brightness value of the pixel and the amount of change in brightness value (brightness gradient) between adjacent pixels.

"Positional information" is information about the position of a pixel, that is, information indicating the coordinates of a pixel.

●Image processing device (1)●
First, an embodiment (first embodiment) of this device will be described.

FIG. 1 is a network connection diagram showing an embodiment of this device.
The figure shows that the device 1 is connected to the information storage device 2 via a network (communication line) N using a wired communication system or a wireless communication system.

The apparatus 1 corrects the contour line of the blood vessel captured in the captured image i1 based on the pixel information of the pixels forming the captured image i1 (see FIG. 3; the same shall apply hereinafter). The specific configuration and operation of this device 1 will be described later.

The information storage device 2 associates and stores the captured image i1 and the position information of the pixels (hereinafter referred to as "contour pixels") corresponding to the contour lines of the blood vessels captured in the captured image i1. The information storage device 2 is, for example, an information storage server, NAS, or personal computer.

The network N is, for example, a communication network such as the Internet, a mobile communication network, a LAN (Local Area Network), a WAN (Wide Area Network), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

●Configuration of Image Processing Apparatus (1) FIG. 2 is a functional block diagram showing an embodiment of the apparatus 1. As shown in FIG.

The device 1 is realized by, for example, a personal computer. In this device 1, this program operates, and this program cooperates with the hardware resources of this device 1 to realize this method.

Here, by causing a computer (not shown) to execute this program, the program can cause the computer to function in the same manner as the apparatus 1 and cause the computer to execute the method.

The device 1 includes a communication section 11 , a storage section 12 and a control section 13 .

The communication unit 11 acquires the captured image i1 from the information storage device 2 via the network N. FIG. The communication unit 11 is composed of, for example, a communication module and an antenna. The captured image i<b>1 is stored in the storage unit 12 .

The storage unit 12 stores information necessary for the apparatus 1 to execute the method described later. The storage unit 12 is, for example, a recording device such as a HDD (Hard Disk Drive) or an SSD (Solid State Drive) included in the apparatus 1, a volatile memory such as a RAM (Random Access Memory), and/or a flash memory. A portable storage medium such as In the present embodiment, the storage unit 12 associates and stores the captured image i1 received in advance from the information storage device 2 and the position information of the contour pixels of the blood vessels captured in the captured image i1.

The control unit 13 controls the operation of the entire device 1 and executes the method described later. The control unit 13 includes, for example, a CPU (Central Processing Unit) provided in the device 1, a volatile memory such as a RAM that functions as a work area for the CPU, and a ROM (Read Only Memory) that stores various information such as the program. ) and a non-volatile memory such as The control unit 13 includes a contour line selection unit 131 , an extraction unit 132 , a pixel selection unit 133 , a correction pixel determination unit 134 , a secondary correction unit 135 , and a correction contour line identification unit 136 .

The contour selection unit 131 selects either one of the first contour and the second contour as a contour to be corrected (hereinafter referred to as a “target contour”). A specific operation of the outline selection unit 131 will be described later.

The extraction unit 132 extracts a plurality of contour pixels as extracted pixels from contour pixels forming the target contour (corresponding to the target contour) based on the position information of the contour pixels. A specific operation of the extraction unit 132 will be described later.

The pixel selection unit 133 selects two extracted pixels having a predetermined positional relationship among the plurality of extracted pixels as selected pixels. A specific operation of the pixel selection unit 133 and a predetermined positional relationship will be described later.

Based on the pixel information of each of the pixels arranged at the same distance from the two selected pixels (hereinafter referred to as "reference pixels"), the correction pixel determination unit 134 determines: A corrected pixel corresponding to the target contour line after correction is determined from the pixels forming the captured image i1. A specific operation of the correction pixel determination unit 134 will be described later.

"A position at the same distance" is an example of a predetermined position in the present invention, and means a position at which the linear distance from two selected pixels is the same in the captured image i1. Here, since the captured image i1 is composed of a plurality of pixels, it is possible that a single pixel does not exist at a position where the linear distances from the two selected pixels are exactly the same. Therefore, pixels arranged at the same distance from two selected pixels are not only pixels arranged at positions with the same linear distance from two selected pixels, but also pixels adjacent to the same position (linear distance from two selected pixels). are “substantially the same”). That is, for example, when there are an even number (for example, 10) of pixels between two selected pixels, a pixel located at a position of 6 pixels from one selected pixel (a pixel located at a position of 4 pixels from another selected pixel). A pixel) can be the pixel that has the same linear distance from the two selected pixels.

A secondary correction unit 135 corrects the position information of the correction pixel. A specific operation of the secondary correction unit 135 will be described later.

The corrected contour specifying unit 136 specifies the target contour after correction based on the position information of the corrected pixels. A specific operation of the correction contour specifying unit 136 will be described later.

●Operation of Image Processing Apparatus (1) Next, the operation of the apparatus 1, that is, the embodiment of the method executed by the apparatus 1 will be described with reference to FIG.

In the following description, the device 1 acquires a captured image i1 from the information storage device 2 via the network N and position information of contour pixels corresponding to the contour lines of the blood vessels captured in the captured image i1. are stored in the storage unit 12 in association with each other. Also, it is assumed that a contour line is superimposed on the captured image i1 used in the following description. Further, in the following description, the contour is denoted by "L", the first contour is denoted by "L1", the second contour is denoted by "L2", and the target contour shall be given the code "Lt". Furthermore, in the following description, each pixel is assigned a reference numeral beginning with "P" in order to distinguish each pixel.

FIG. 3 is a schematic diagram showing an example of the captured image i1.
For convenience of explanation, this figure shows an enlarged part of one blood vessel. The figure shows that contour lines L (first contour line L1, second contour line L2) are superimposed on a part of the imaged blood vessel in the captured image i1.

FIG. 4 is a flow chart illustrating an embodiment of the method.

The method performs a primary correction process (S1) and a secondary correction process (S2).

● Primary Correction Processing First, the apparatus 1 executes the primary correction processing (S1).

FIG. 5 is a flow chart of the primary correction process (S1).
FIG. 6 is a schematic diagram showing a captured image i1 at each stage of the primary correction process (S1).

The "primary correction process (S1)" is a process of correcting the position information of the contour pixels corresponding to the contour line L associated with the captured image i1 based on the pixel information of the pixels forming the captured image i1. .

First, the contour line selection unit 131 selects one of the first contour line L1 and the second contour line L2 as the target contour line Lt (S101). That is, for example, the contour line selection unit 131 determines the positional relationship between the first contour line L1 and the second contour line L2 in the captured image i1 (for example, the origin (0, 0) of the coordinates of the pixels forming the captured image i1). distance), the target contour line Lt is selected.

Note that the contour line selection unit in the present invention may select a contour line selected by the user of the apparatus using a selection means (not shown) as the target contour line, or a relative positional relationship to the blood vessel. The target contour line may be selected based on (eg, up, down, left, or right with respect to the blood vessel).

Next, the correction pixel determining unit 134 performs mask processing on pixels included in a predetermined region including the contour line L (the second contour line L2 in the present embodiment) that is not selected as the target contour line Lt ( S102), the same predetermined area is excluded from correction pixel targets, which will be described later. Here, the correction pixel determination unit 134 performs, for example, a known expansion process on the second contour line L2, and defines an area (predetermined area) covered by the expanded second contour line L2.

Note that the predetermined area excluded from the correction pixel target in the mask processing may be a certain area based on the contour line not selected as the target contour line. Also, the masking process may be executed before calculating the amount of change in luminance value of a reference pixel, which will be described later.

Next, the extraction unit 132 extracts a plurality of contour pixels out of the contour pixels corresponding to the target contour line Lt based on the position information of the contour pixels corresponding to the target contour line Lt among the pixels forming the captured image i1. It is extracted as an extraction pixel P1(n) (S103). That is, for example, the extracting unit 132 extracts a certain number of contour pixels at approximately equal intervals as extracted pixels P1(n) from a plurality of contour pixels. In the following description, it is assumed that the extraction unit 132 has extracted six extraction pixels P1(1) to P1(6).

Note that the extraction unit in the present invention may extract all contour pixels corresponding to the target contour line as extracted pixels.

Next, the pixel selection unit 133 selects adjacent two extracted pixels P1(n) and P1(n+1) among the plurality of extracted pixels P1(1) to P1(6) as selected pixels P2(1) and P2(2). ) (S104). That is, for example, when six extracted pixels P1(1) to P1(6) are arranged in numerical order, the pixel selection unit 133 selects the extracted pixels P1(1) and P1(2) as the selected pixel P2(1). , P2(2).

Here, in the present embodiment, the "adjacent" positional relationship is an example of a predetermined positional relationship in the present invention.

FIG. 7 is a schematic diagram showing a captured image i1 in the middle of the primary correction process (S1).
For convenience of explanation, the drawing shows an enlarged view of the vicinity of two selected pixels P2(1) and P2(2).

Returning to FIGS. 5-7.
Next, the correction pixel determining unit 134 generates a virtual line segment Lm connecting the two selected pixels P2(1) and P2(2), and creates a virtual line segment Lm that is orthogonal to the virtual line segment Lm at the midpoint of the virtual line segment Lm. A perpendicular line Ln is generated (S105). Here, the length of the virtual perpendicular line Ln is set in advance to, for example, a predetermined length (for example, 9 pixels with the midpoint as the central pixel).

Note that the length of the virtual perpendicular line may be set, for example, based on the distance between the first contour line and the second contour line (that is, the width and diameter of the blood vessel). That is, for example, the length of the virtual perpendicular may be set to less than twice the same distance.

Next, the correction pixel determination unit 134 identifies, as reference pixels P3(n), a plurality of pixels corresponding to the virtual perpendicular line Ln among the pixels forming the captured image i1 (S106). That is, for example, when the length of the virtual perpendicular line Ln is 9 pixels, the 9 pixels are specified as the reference pixels P3(1) to P3(9). Here, the pixels corresponding to the imaginary perpendicular line Ln are pixels arranged at substantially the same distance from the two selected pixels P2(1) and P2(2). That is, the pixels corresponding to the imaginary perpendicular line Ln are pixels arranged at predetermined positions in the present invention.

Next, the correction pixel determining unit 134 acquires the luminance values of the reference pixels P3(1) to P3(9) from the storage unit 12, and calculates the luminance values of the plurality of reference pixels P3(1) to P3(9). Based on (S107), the amount of change in luminance value for each of two adjacent reference pixels P3(n) and P3(n+1) is obtained. That is, for example, the correction pixel determination unit 134 calculates the primary luminance gradient and the secondary luminance gradient of each of the reference pixels P3(1) to P3(9).

Note that the correction pixel determination unit in the present invention may calculate only the primary luminance gradient, or may calculate the amount of change in the luminance value by other calculations.

FIG. 8 is a schematic diagram showing an example of the pixel value of each reference pixel P3(n) and its variation (primary gradient). The figure shows that the reference pixel P3(6) has the largest amount of change in luminance value.

Returning to FIGS. 5-8.
Next, the correction pixel determination unit 134 determines the reference pixel P3(n) having the largest luminance value variation among the plurality of reference pixels P3(1) to P3(9) as the correction pixel P4(n) ( S108). Usually, in the captured image i1, the blood vessel, which is the subject, is imaged darker than the background. Therefore, a pixel with a large amount of change in luminance value can be regarded as a pixel corresponding to the edge of a blood vessel.

Here, the virtual perpendicular line Ln can pass over the contour line L (the second contour line L2) that is not selected as the target contour line Lt. Therefore, the reference pixel P3(n) with the largest change in pixel value can also exist near the second contour line L2. As a result, the correction pixel determination unit 134 can determine the reference pixel P3(n) near the second contour line L2 as the correction pixel P4(n). However, in the present embodiment, the reference pixels P3(n) near the second contour line L2 are excluded from the correction pixels P4(n) in the masking process (S102). Therefore, in the process (S108), the single reference pixel P3(n) is the pixel with the largest change in luminance value.

It should be noted that the correction pixel determination unit in the present invention does not need to perform the mask processing described above. In this case, the correction pixel determining unit of the present invention, for example, determines a pixel closer to the target contour line as a correction pixel, out of two reference pixels having a large amount of change in pixel value.

Next, the control unit 13 (for example, the correction pixel determination unit 134) determines whether there is an unselected extracted pixel P1(n) as the selected pixels P2(1) and P2(2) (S109). When there is an unselected extracted pixel P1(n) as the selected pixels P2(1) and P2(2) ("Y" in S109), the primary correction process (S1) returns to the process (S103). That is, for example, the pixel selection unit 133 selects the extracted pixel P1(3) adjacent to the already selected extracted pixel P1(2) among the unselected extracted pixels P1(3) to P1(6), and the already selected extracted pixel P1(3). Among the already extracted pixels P1(1) and P1(2), the extracted pixel P1(2) adjacent to the unselected extracted pixel P1(3) is used as the next selected pixels P2(1) and P2(2). Select as

On the other hand, when there is no unselected extracted pixel P1(n) as the selected pixels P2(1) and P2(2) (“N” in S109), the correction pixel determination unit 134 determines all the correction pixels P4(1) to The position information of P4(5) is stored in the storage unit 12 as the position information of the corrected pixel group in association with the captured image i1 (S110).

Next, the control unit 13 (for example, the correction pixel determination unit 134) determines whether or not there is an unselected contour line L as the target contour line Lt (S111). When there is a contour line L that has not been selected as the target contour line Lt ("Y" in S111), the primary correction process (S1) returns to the process (S101). On the other hand, when there is no contour line L that has not been selected as the target contour line Lt ("N" in S111), the primary correction process (S1) ends.

●Secondary Correction Processing Next, the apparatus 1 executes the secondary correction processing (S2).

FIG. 9 is a flowchart of secondary correction processing (S2).
FIG. 10 is a schematic diagram showing captured images i1 at each stage of the secondary correction process (S2).
FIG. 10 shows a state where a vein crosses an artery and a portion of the vein is obscured by a portion of the artery.

The "secondary correction process (S2)" is a process of correcting the position information of the correction pixel P4(n) determined in the first correction process (S1). As described above, the blood vessels are imaged darker than the background in the captured image i1. corresponds to the contour of However, for example, at a blood vessel intersection where an artery and a vein intersect, there is little difference between the brightness value of the artery and the brightness value of the vein. Thus, for example, the venous correction pixel P4(n) at the intersection may protrude toward the artery or recede into the vein. The secondary correction process (S2) is a process of correcting the positional information of the correction pixel P4(n) determined in the region where the luminance value difference is small based on the element (curvature) different from the luminance value. is.

First, the secondary correction unit 135 reads a correction pixel group to be subjected to secondary correction from the storage unit 12 (S201). In the following description, it is assumed that the correction pixel group is composed of five correction pixels P4(11) to P4(15).

Next, the secondary correction unit 135 selects the correction pixel P4(11) as the start point and the correction pixel P4(15) as the end point among the correction pixels P4(11) to P4(15) forming the correction pixel group. Curvatures corresponding to each of the three correction pixels P4(12) to P4(14) are calculated. Here, each of the start point and the end point is determined in advance by, for example, the order of determination of the correction pixel P4(n).

Specifically, of the corrected pixels P4(12) to P4(14), the secondary correction unit 135 converts the corrected pixel P4(12), which is closest to the starting point and whose curvature has not been calculated, to the calculated pixel P5(1). ) (S202). Next, the secondary correction unit 135 selects two correction pixels P4(11) and P4(13) arranged on both sides of the calculation pixel P5(1) as calculation auxiliary pixels P5(2) and P5(3). (S203).

Next, the secondary correction unit 135 generates a virtual circle C that connects the calculated pixel P5(1) and the two calculated auxiliary pixels P5(2) and P5(3) (S204).

FIG. 11 is a schematic diagram showing the virtual circle C. As shown in FIG.
For convenience of explanation, the figure also shows parameters used to calculate the radius "R" of the virtual circle C. As shown in FIG.

Returning to FIGS. 9-11.
Next, the secondary correction unit 135 calculates the radius "R" of the virtual circle C using the following formula (1) (S205).

ここで、
「||・||」はベクトルのノルムを意味する（以下同じ）。
「ｐ_ｉ－１」は、算出補助画素Ｐ５（２）の座標である。
「ｐ_ｉ＋１」は、算出補助画素Ｐ５（３）の座標である。
「ｑ」は、算出補助画素Ｐ５（２）と仮想円Ｃの中心点「Ｏ」とを結ぶ仮想線Ｌｏと、仮想円Ｃとの交点「Ｑ」の座標である。
「θ」は、算出補助画素Ｐ５（３）と交点「Ｑ」とを結ぶ仮想線Ｌｐと、仮想線Ｌｏと、の間の角度である。

here,
"||·||" means the norm of the vector (same below).
“p _i−1 ” is the coordinates of the calculated auxiliary pixel P5(2).
“p _i+1 ” is the coordinates of the calculated auxiliary pixel P5(3).
"q" is the coordinate of the intersection point "Q" between the virtual line Lo connecting the calculated auxiliary pixel P5(2) and the center point "O" of the virtual circle C and the virtual circle C;
“θ” is the angle between the virtual line Lp connecting the calculation auxiliary pixel P5(3) and the intersection point “Q” and the virtual line Lo.

Next, the secondary correction unit 135 calculates the curvature "k _i " of the virtual circle C using the following equation (2) (S206).

The calculated curvature “k _i ” is stored in the storage unit 12 in association with the corresponding calculated pixel P5(1) (corrected pixel P4(12)).

Next, the secondary correction unit 135 determines whether or not there is a corrected pixel P4(n) for which the curvature "k _i " has not been calculated (S207). When there is a correction pixel P4(n) whose curvature "k _i " has not been calculated ("Y" in S207), the secondary correction process (S2) returns to the process (S202).

On the other hand, when there is no correction pixel P4(n) for which the curvature "k _i " has not yet been calculated ("N" in S207), the secondary correction unit 135 calculates the average value "V _ave. " of all the curvatures "k _i ". and the standard deviation "σ" are calculated (S208).

Next, the secondary correction unit 135 determines whether there is a corrected pixel P4(n) having a relatively large curvature “k _i ” based on the calculated average value “V _ave. ” and standard deviation “σ”. (S209). That is, for example, the secondary correction unit 135 determines whether or not there is a correction pixel P4(n) having a curvature "k _i " that deviates from "±2σ" of the normal distribution centered on the average value "V _ave. ". That is, in the present embodiment, "the curvature 'k _i ' is relatively large" means that the curvature 'k _i ' deviates from '±2σ' of the normal distribution centered on the average value 'V _ave. means.

When there is a corrected pixel P4(n) with a relatively large curvature “k _i ” (“Y” in S209), the secondary correction unit 135 selects the corrected pixel P4(n) closest to the starting point and , the correction pixel P4(n) that has not been selected as the position correction pixel P6(n) is selected as the position correction pixel P6(1) (S210). In the following description, it is assumed that the correction pixel P4(13) is selected as the position correction pixel P6(1).

Next, the secondary correction unit 135 selects two correction pixels P4(12) and P4(14) arranged on both sides of the position correction pixel P6(1) as reference pixels P7(1) and P7(2). (S211).

Next, the secondary correction unit 135 calculates the position information of the pixel (hereinafter referred to as "midpoint pixel") Po corresponding to the midpoint of the reference pixels P7(1) and P7(2) (S212).

Next, the secondary correction unit 135 corrects the position information of the position correction pixel P6(1) to the position information of the midpoint pixel Po (S213). That is, the secondary correction unit 135 moves the position of the correction pixel P4(13) corresponding to the position correction pixel P6(1) to the position of the midpoint pixel Po. That is, the secondary correction unit 135 corrects the position information of the position correction pixel P6(1) based on the position information of the reference pixels P7(1) and P7(2).

Next, the secondary correction unit 135 determines whether or not there is an unselected correction pixel P4(n) as the position correction pixel P6(n) (S214). When there is an unselected correction pixel P4(n) as the position correction pixel P6(n) ("Y" in S214), the secondary correction process (S2) returns to the process (S210).

On the other hand, when there is no correction pixel P4(n) with a relatively large curvature "k _i "("N" in S209), or there is no correction pixel P4(n) unselected as the position correction pixel P6(n) When (“N” in S214), the secondary correction unit 135 combines the position information of the corrected pixel P4(n) whose position information is not corrected and the position information of the corrected pixel P4(n) whose position information is corrected. , are stored in the storage unit 12 as the position information of the pixels corresponding to the contour line Lx after correction (S215).

Next, the corrected contour line identifying unit 136 identifies the corrected contour line Lx based on the position information of the pixels corresponding to the corrected contour line Lx (S216).

In this way, in the primary correction process (S1), the apparatus 1 determines the correction pixel P4(n) based on the pixel information of the extraction pixel P1(n) extracted from the contour line L, and corrects the contour line L. is corrected, and in the secondary correction process (S2), the position information of the corrected pixel P4(n) is further corrected based on the curvature "k _i " of the corrected pixel P4(n). That is, the apparatus 1 corrects the position information of the contour line L in two stages based on two different pieces of information, ie, the pixel information and the curvature "k _i ". As a result, the apparatus 1 can highly accurately correct the contour line L of the blood vessel extracted from the captured image i1.

●Summary (1)
According to the embodiment described above, the apparatus 1 includes the contour line selection unit 131 that selects one of the first contour line L1 and the second contour line L2 as the correction target contour line Lt, and the contour pixel An extraction unit 132 for extracting a plurality of contour pixels as extraction pixels P1(n) from among the contour pixels corresponding to the target contour line Lt based on the position information of the extraction unit 132, and a predetermined A pixel selection unit 133 that selects two extracted pixels P1(n) having a positional relationship as selected pixels P2(1) and P2(2), and for each of the two selected selected pixels P2(1) and P2(2) Then, based on the pixel information of each of the plurality of reference pixels P3(n) arranged at predetermined positions from the two selected pixels P2(1) and P2(2), the pixels forming the captured image i1 after correction are selected. A correction pixel determination unit 134 that determines the correction pixel P4(n) corresponding to the target contour line Lt of the correction contour line that specifies the target contour line Lt after correction based on the position information of the correction pixel P4(n) and a specifying unit 136 . According to this configuration, the apparatus 1 determines one correction pixel P4(n) based on pixel information (amount of change in luminance value) of the reference pixel P3(n) for each two extracted pixels P1(n). can do. Then, the apparatus 1 can specify the corrected target contour line Lt (contour line L) based on the determined positional information of the correction pixel P4(n). That is, the apparatus 1 can correct the contour line L of the blood vessel extracted from the captured image i1.

Moreover, according to the embodiment described above, the pixel information includes the luminance value of the reference pixel P3(n). Further, the correction pixel determination unit 134 acquires the amount of change in luminance value for each of two adjacent reference pixels P3(n) based on the luminance values of each of the plurality of reference pixels P3(n), Among P3(n), the reference pixel P3(n) having the largest amount of change is determined as the corrected pixel P4(n). Usually, in the captured image i1, the blood vessel, which is the subject, is imaged darker than the background. Therefore, a pixel with a large amount of change in pixel value can be regarded as a pixel corresponding to the edge of a blood vessel. According to this configuration, the device 1 can easily identify the pixels corresponding to the edge of the blood vessel (the pixels of the contour line Lx after correction, if related).

Furthermore, according to the embodiment described above, when the first contour line L1 is selected, the correction pixel determining unit 134 selects the pixels included in the predetermined area including the second contour line L2 as the correction pixels P4(n ), and when the second contour line L2 is selected, the pixels included in the predetermined region including the first contour line L1 are excluded from the correction pixel P4(n). According to this configuration, when calculating the amount of change in luminance value on the virtual perpendicular line Ln, the change in luminance value near the contour line L not selected as the target contour line Lt is not affected. As a result, the pixel with the largest change in luminance value is the single reference pixel P3(n), which can be easily identified.

Furthermore, according to the embodiment described above, the correction pixel determination unit 134 demarcates a predetermined area by performing expansion processing on the first contour line L1 or the second contour line L2. According to this configuration, the device 1 can easily determine the predetermined region to be masked.

Furthermore, according to the embodiment described above, the device 1 has the secondary correction section 135 that corrects the position information of the correction pixel P4(n). The secondary correction unit 135 calculates the curvature corresponding to each of the plurality of corrected pixels P4(n), and corrects the position information of the corrected pixel P4(n) based on the curvature of each corrected pixel P4(n). According to this configuration, the apparatus 1 can obtain the position information of the correction pixel P4(n) in the portion where there is no large difference in the amount of change in the luminance value and the position information of the correction pixel P4(n) is likely to be erroneous. can be corrected. That is, the device 1 can correct the contour line L with high accuracy. Further, the apparatus 1 can specify the correction pixel P4(n) whose position information is corrected based on the curvature, that is, the difference in position information relative to other correction pixels P4(n).

Furthermore, according to the embodiment described above, the secondary correction unit 135 selects the correction pixel P4(n) to be corrected as the position correction pixel P6(n) based on the calculated curvature. . According to this configuration, the apparatus 1 identifies the corrected pixel P4(n) whose position information relative to the other corrected pixel P4(n) is greatly deviated based on the curvature, and determines the corrected pixel P4(n). Only the position information of n) can be corrected. Further, the secondary correction unit 135 selects two correction pixels P4(n) arranged on both sides of the position correction pixel P6(n) as reference pixels P7(1) and P7(2), and selects two reference pixels P7(1) and P7(2). The position information of the position correction pixel P6(n) is corrected based on the position information of P7(1) and P7(2). According to this configuration, the apparatus 1 can detect pixels (reference pixels) whose position information is not deviated from the contour of the blood vessel and which are closer to the correct positions of the position-corrected pixels P6(n) than the position-corrected pixels P6(n). The position information of the position correction pixel P6(n) can be corrected based on the position information of P7(1), P7(2)). That is, the apparatus 1 can highly accurately correct the contour line L of the blood vessel extracted from the captured image i1.

Furthermore, according to the embodiment described above, the secondary correction unit 135 selects the correction pixel P4(n) having a relatively large curvature as the position correction pixel P6(n). According to this configuration, the apparatus 1 identifies only the corrected pixel P4(n) whose relative positional information is greatly deviated with respect to the other corrected pixel P4(n) based on the curvature. Only the position information of (n) can be corrected.

Furthermore, according to the embodiment described above, the secondary correction unit 135 selects the correction pixel P4(n) whose curvature is to be calculated from among the plurality of correction pixels P4(n) as the calculation pixel P5(1). Then, the two correction pixels P4(n) arranged on both sides of the calculation pixel P5(1) are selected as the calculation auxiliary pixels P5(2) and P5(3), and the calculation pixel P5(1) and the two calculation auxiliary pixels are selected. The curvature of the calculated pixel P5(1) is calculated based on the radius of the virtual circle C that virtually connects the pixels P5(2) and P5(3). With this configuration, the device 1 can easily calculate the curvature based only on the position information of the corrected pixel P4(n).

●Image processing device (2)●
Next, another embodiment (second embodiment) of the present apparatus will be described, focusing on portions different from the first embodiment described above. The apparatus of the second embodiment differs from the first embodiment in the pixel selection section and correction pixel determination section of the control section.

●Configuration of Image Processing Apparatus (2) FIG. 12 is a functional block diagram showing another embodiment of this apparatus.

The device 1A includes a communication section 11, a storage section 12, and a control section 13A.

The control unit 13A controls the operation of the entire device 1A and executes the method described later. The physical configuration of the controller 13A is the same as the physical configuration of the controller 13 of the first embodiment. The control unit 13A includes a contour line selection unit 131, an extraction unit 132, a pixel selection unit 133A, a correction pixel determination unit 134A, a secondary correction unit 135, and a correction contour line identification unit 136.

133 A of pixel selection parts select two extraction pixels which have a predetermined positional relationship as a selection pixel among several extraction pixels. A specific operation of the pixel selection unit 133A and the predetermined positional relationship will be described later.

The correction pixel determination unit 134A determines the captured image i1 based on the pixel information of each of a plurality of pixels (reference pixels) arranged at positions approximately the same distance from the two selected pixels for each of the two selected pixels. A correction pixel corresponding to the target contour line after correction is determined from among the constituent pixels. A specific operation of the correction pixel determination unit 134A will be described later.

●Operation of Image Processing Apparatus (2) Next, the operation of the apparatus 1A, that is, another embodiment of the method executed by the apparatus 1A will be described with reference to FIG.

FIG. 13 is a flow chart illustrating another embodiment of the method.

The method performs a primary correction process (S1A) and a secondary correction process (S2). In this embodiment, the secondary correction process (S2) is common to the secondary correction process (S2) of the first embodiment.

● Primary Correction Processing First, the device 1A executes the primary correction processing (S1A).

FIG. 14 is a flow chart of the primary correction process (S1A).
FIG. 15 is a schematic diagram showing captured images i1 at each stage of the primary correction processing (S1A).

The "primary correction process (S1A)" is a process of correcting the position information of the contour pixels corresponding to the contour line L associated with the captured image i1, based on the pixel information of the pixels forming the captured image i1. .

First, the contour line selection unit 131 selects one of the first contour line L1 and the second contour line L2 as the target contour line Lt (S101).

Next, the extraction unit 132 extracts a plurality of contour pixels out of the contour pixels corresponding to the target contour line Lt based on the position information of the contour pixels corresponding to the target contour line Lt among the pixels forming the captured image i1. It is extracted as an extraction pixel P1(n) (S102A). That is, for example, the extracting unit 132 extracts a certain number of contour pixels at approximately equal intervals as extracted pixels P1(n) from a plurality of contour pixels. Here, the process (S102A) is common to the process (S103) of the first embodiment. In the following description, it is assumed that the extraction unit 132 has extracted six extraction pixels P1(1) to P1(6).

Next, the pixel selection unit 133A selects the extraction pixel P1(n), which is a candidate for correction, from among the plurality of extraction pixels P1(1) to P1(6) as the candidate pixel P8(n) (S103A). In this method, the candidate pixel P8(n) is selected from among the four extracted pixels P1(2) to P1(5) excluding the starting pixel P1(1) and the ending pixel P1(6). selected. Specifically, the pixel selection unit 133A selects the extracted pixel P1(2), which is closest to the starting point and is not selected as the candidate pixel P8(n), from among the extracted pixels P1(2) to P1(5). Select as candidate pixel P8(1).

Next, the pixel selection unit 133A selects two extracted pixels P1(1) and P1(3) arranged on both sides of the candidate pixel P8(1) as selected pixels P2A(1) and P2A(2) (S104A ). That is, the pixel selection unit 133A selects two extracted pixels P1(n) that have a positional relationship on both sides (skipping one) of the candidate pixel P8(n) among the plurality of extracted pixels P1(1) to P1(6). , P1(n+2) are selected as selected pixels P2A(1) and P2A(2) (S104A).

Here, in the present embodiment, the positional relationship between "two neighboring pixels (skipping one) of the candidate pixel P8(n)" is an example of a predetermined positional relationship in the present invention.

FIG. 16 is a schematic diagram showing a captured image i1 in the middle of the primary correction process (S1A).
For convenience of explanation, this figure shows the vicinity of two selected pixels P2A(1) and PA2(2) in an enlarged manner.

Returning to FIGS. 14-16.
Next, the correction pixel determination unit 134A generates a virtual line segment Lm that connects the two selected pixels P2A(1) and P2A(2), and creates a virtual line segment Lm that is orthogonal to the virtual line segment Lm at the midpoint of the virtual line segment Lm. A perpendicular line Ln is generated (S105). Here, the length of the virtual perpendicular line Ln is set in advance to, for example, a predetermined length (for example, 9 pixels with the midpoint as the central pixel).

Next, the correction pixel determination unit 134A identifies, among the pixels forming the captured image i1, a plurality of pixels corresponding to the virtual perpendicular line Ln as reference pixels P3(n) (S106). That is, for example, when the length of the virtual perpendicular line Ln is 9 pixels, the 9 pixels are determined as the reference pixels P3(1) to P3(9). Here, the pixels corresponding to the imaginary perpendicular line Ln are the pixels arranged at substantially the same distance from the two selected pixels P2A(1) and P2A(2). That is, the pixels corresponding to the imaginary perpendicular line Ln are pixels arranged at predetermined positions in the present invention.

Next, the correction pixel determination unit 134A acquires the luminance values of the reference pixels P3(1) to P3(9) from the storage unit 12, and calculates the luminance values of the plurality of reference pixels P3(1) to P3(9). Based on (S107), the amount of change in luminance value for each of two adjacent reference pixels P3(n) and P3(n+1) is acquired. That is, for example, the correction pixel determination unit 134 calculates the primary luminance gradient and the secondary luminance gradient of each of the reference pixels P3(1) to P3(9).

Next, the correction pixel determination unit 134A selects the reference pixel P3(n), which is close to the target contour line and has the largest change amount among the plurality of reference pixels P3(1) to P3(9), as the correction pixel P4(n). ) (S108).

Note that the correction pixel determination unit in the present invention may perform mask processing similar to the present method of the first embodiment.

Next, the control unit 13A (for example, the correction pixel determination unit 134A) determines whether or not there is an unselected extracted pixel P1(n) as the candidate pixel P8(n) (S109A). When there is an unselected extraction pixel P1(n) ("Y" in S109A), the primary correction process (S1A) returns to the process (S103A). That is, for example, the pixel selection unit 133A selects the extracted pixel P1(3) adjacent to the already selected extracted pixel P1(2) among the unselected extracted pixels P1(3) to P1(5) as the next candidate. Select as pixel P8(2).

On the other hand, when there is no unselected extraction pixel P1(n) (“N” in S109A), the correction pixel determination unit 134A selects all the candidate pixels P8(1) to P8(4) (that is, the extraction pixel P1(2) ) to P1(5)) are corrected to corresponding position information of the corrected pixels P4(1) to P4(4) (S110A). That is, the correction pixel determination unit 134A moves the position of the extraction pixel P1(n) corresponding to the candidate pixel P8(n) to the position of the correction pixel P4(n).

Next, the correction pixel determining unit 134A stores the position information of all the correction pixels P4(2) to P4(5) in the storage unit 12 in association with the captured image i1 as the position information of the correction pixel group (S111A).

Next, the control unit 13A (for example, the correction pixel determination unit 134A) determines whether or not there is an unselected contour line L as the target contour line Lt (S112A). When there is an unselected contour line L ("Y" in S112A), the primary correction process (S1A) returns to the process (S101). On the other hand, when there is no unselected contour line L ("N" in S112A), the primary correction process (S1A) ends. Here, the process (S112A) is common to the process (S111) of the first embodiment.

●Summary (2)
According to the embodiment described above, the pixel selection unit 133 selects the extracted pixel P1(n), which is a candidate for correction, from among the plurality of extracted pixels P1(n) as the candidate pixel P8(n). Two extracted pixels P1(n) arranged on both sides of the pixel P8(n) are selected as selected pixels P2A(1) and P2A(2). The correction pixel determination unit 134 corrects the position information of the candidate pixel P8(n) to the position information of the correction pixel P4(n). According to this configuration, the apparatus 1 can correct the position information itself of the extracted pixel P1(n).

●Summary (Others)
In each of the embodiments described above, the apparatus 1, 1A executes secondary correction processing (S2). Alternatively, if the contour line can be corrected with desired accuracy in the primary correction process, the apparatus does not need to perform the secondary correction process.

Further, the present apparatus may combine the selection of the selected pixels in the first embodiment and the selection of the selected pixels in the second embodiment in the primary correction processing. That is, for example, for the extracted pixels P1(1) to P1(6), the present apparatus extracts the extracted pixel P1(1) corresponding to the starting point and the extracted pixel P1(2) adjacent to the extracted pixel P1(1). ) and are selected as the selected pixels, and then the extracted pixel P1(2) and the extracted pixel P1(4) skipping by one may be selected as the selected pixels.

1 image processing device 12 storage unit 131 contour line selection unit 132 extraction unit 133 pixel selection unit 134 correction pixel determination unit 135 secondary correction unit 136 correction contour line identification unit 1A image processing device 133A pixel selection unit 134A correction pixel determination unit

Claims (15)

An image processing device for correcting a contour line of a blood vessel generated based on a captured image in which the blood vessel is captured,
The outline is
a first contour line indicating the contour of one side in the width direction of the blood vessel;
a second contour line indicating the other contour in the width direction;
including
a storage unit that associates and stores the captured image and position information of contour pixels corresponding to the contour line of the blood vessel captured in the captured image;
a contour selection unit that selects one of the first contour and the second contour as a contour to be corrected;
an extraction unit that extracts a plurality of the contour pixels as extracted pixels from the contour pixels corresponding to the target contour based on the position information of the contour pixels;
a pixel selection unit that selects, as selected pixels, two of the extracted pixels having a predetermined positional relationship from among the plurality of extracted pixels;
For each of the selected two selected pixels, the target after correction from among pixels constituting the captured image based on pixel information of each of a plurality of reference pixels arranged at predetermined positions from the two selected pixels. a correction pixel determination unit that determines correction pixels corresponding to the contour;
a corrected contour identifying unit that identifies the target contour after correction based on the position information of the corrected pixel;
comprising
An image processing apparatus characterized by: The pixel information is
a luminance value of the reference pixel;
including
The correction pixel determination unit
Obtaining an amount of change in the luminance value for each of two adjacent reference pixels based on the luminance value of each of the plurality of reference pixels;
determining the reference pixel having the largest amount of change among the plurality of reference pixels as the correction pixel;
The image processing apparatus according to claim 1. The pixel selection unit
selecting two adjacent extracted pixels from among the plurality of extracted pixels as the selected pixels;
3. The image processing apparatus according to claim 2. The pixel selection unit
selecting the extracted pixel that is a candidate for correction from among the plurality of extracted pixels as a candidate pixel, and selecting two of the extracted pixels arranged on both sides of the candidate pixel as the selected pixels;
3. The image processing apparatus according to claim 2. The correction pixel determination unit corrects the position information of the candidate pixel to the position information of the correction pixel.
5. The image processing apparatus according to claim 4. The correction pixel determination unit
when the first contour line is selected, excluding pixels included in a predetermined region including the second contour line from the target of the correction pixels;
when the second contour is selected, excluding pixels included in a predetermined region including the first contour from the target of the correction pixels;
The image processing apparatus according to claim 1. a secondary correction unit that corrects the position information of the correction pixel;
and
The secondary correction unit is
calculating a curvature corresponding to each of the plurality of correction pixels;
correcting the position information of the correction pixel based on the curvature of each of the correction pixels;
7. The image processing apparatus according to claim 1. The secondary correction unit calculates the curvature corresponding to each of the corrected pixels excluding the corrected pixels corresponding to the start point and the end point of the target contour line after correction among the plurality of corrected pixels,
The image processing apparatus according to claim 7. The secondary correction unit is
selecting the correction pixel to be corrected as a position correction pixel based on the calculated curvature;
selecting two of the correction pixels arranged on both sides of the position correction pixel as reference pixels;
correcting the position information of the position correction pixel based on the position information of the two reference pixels;
9. The image processing apparatus according to claim 8. The secondary correction unit selects the correction pixel having the relatively large curvature as the position correction pixel.
10. The image processing apparatus according to claim 9. The secondary correction unit corrects the position information of the position correction pixel to the position information of the pixel corresponding to the midpoint of the two reference pixels.
10. The image processing apparatus according to claim 9. The secondary correction unit is
Selecting the correction pixel for calculating the curvature as a calculation pixel from among the plurality of correction pixels;
selecting the two correction pixels arranged on both sides of the calculation pixel as calculation auxiliary pixels;
calculating the curvature of the calculated pixel based on the radius of a virtual circle that virtually connects the calculated pixel and two of the calculated auxiliary pixels;
The image processing apparatus according to claim 7. causing a computer to function as the image processing device according to claim 1;
An image processing program characterized by: An image processing method executed by an image processing device for correcting a contour line of a blood vessel generated based on a captured image in which the blood vessel is captured,
The outline is
a first contour line indicating a contour on one side in the width direction of the blood vessel;
a second contour line indicating the other contour in the width direction;
including
The image processing device is
a storage unit that associates and stores the captured image and position information of contour pixels corresponding to the contour line of the blood vessel captured in the captured image;
with
The image processing device is
selecting either one of the first contour and the second contour as a contour to be corrected;
a step of extracting a plurality of the contour pixels as extracted pixels from the contour pixels corresponding to the target contour line based on the positional information of the contour pixels;
a step of selecting two extracted pixels having a predetermined positional relationship from among the plurality of extracted pixels as selected pixels;
For each of the selected two selected pixels, the target after correction from among pixels constituting the captured image based on pixel information of each of a plurality of reference pixels arranged at predetermined positions from the two selected pixels. determining a correction pixel corresponding to the contour;
a step of identifying the corrected target contour line based on the position information of the corrected pixel;
including,
An image processing method characterized by: correcting the position information of the correction pixel;
including,
15. The image processing method according to claim 14.

Images