JP2018000339A - Ultrasonic image processor - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an ultrasonic image processor that can precisely detect a structure to be detected by a simple method.SOLUTION: An ultrasonic image processor comprises: a candidate image portion detecting part 109 which detects a candidate image portion RC assumed to be an image portion showing a structure to be detected, with respect to ultrasonic images of multiple frames; a distance-related information acquiring part 112 which, assuming that one of the multiple frames as a reference frame and a sectional position of the ultrasonic image as a reference position, acquires information related to a distance from the reference position of the sectional position of each of the multiple frames; a candidate image portion determining part 114 which determines whether or not the candidate image portion is included in each of the frames for which the distance from the reference position is within a distance to be detected; a weight score calculating part 115 which calculates a weight score represented as a variable for the distance from the reference position, for each frame; and a structure determining part 116 which determines that the candidate image portion is the structure to be detected when the calculated weight score meets a prescribed condition.SELECTED DRAWING: Figure 5

Description

  The present disclosure belongs to the technical field of ultrasonic image processing, and particularly relates to an ultrasonic image processing apparatus that provides support information for tumor detection based on reflected ultrasonic waves obtained by transmitting ultrasonic waves to a subject.

Ultrasonic image processing apparatuses are widely used for morphological diagnosis of diseases such as rheumatoid arthritis and breast cancer because they are less invasive to a subject and can observe a state of a body tissue with a tomographic image or the like in real time.
In the diagnosis of breast cancer tumors, the presence of structures such as calcification in the breast ducts, duct expansion, cysts, and masses is a major clue in screening tests that look for tumors suspected of being cancerous. Among them, calcification has a fine granular three-dimensional structure and is depicted as a fine bright spot on an ultrasonic B-mode image. Therefore, it is difficult to distinguish from a bright spot indicating a cross section of a streak structure such as a mammary gland or a tissue boundary surface in an ultrasonic B-mode image of one frame.

  As a method of distinguishing from bright spots due to such other structures, inspectors such as doctors and laboratory technicians conventionally make a judgment based on whether or not the bright spot blinks by moving the ultrasonic probe minutely. . In the case of a streak structure, the bright spot does not disappear even if the ultrasonic probe is moved a little, but in the case of a granular structure, if the ultrasonic probe is moved slightly, the bright spot is deviated from the imaging surface. Or disappear.

  Further, Non-Patent Document 1 discloses a method for detecting the shape of a structure by calculation using a Hessian matrix.

“Multiscale vessel enhancement filtering”. in Medical Image Computing and Computer-Assisted Intervention-MICCAI '98, A.I. F. Frangi, W.H. J. et al. Niessen, K.M. L. Vincken, M.M. A. Viagever (1998)

As described above, in the method of discriminating a minute granular structure by moving the ultrasonic probe minutely, there is a problem that a part due to the skill of the inspector's technique is large, and there is a problem that work efficiency is lowered. .
In the method of Non-Patent Document 1, since it is necessary to perform a complicated calculation using a Hessian matrix, it is necessary to mount a CPU having a very high calculation capability, particularly when discriminating a granular structure in real time. There is a problem of high cost. Furthermore, since the method of Non-Patent Document 1 is based on the premise that the detection positions of the respective frames are at equal intervals, there is a problem that it is difficult to cope with frame skipping.

  The present invention has been made in view of the above problems, and an object of the present invention is to provide an ultrasonic image processing apparatus capable of accurately detecting a detection target structure by a simple method.

  An ultrasonic image processing apparatus according to an aspect of the present invention acquires an ultrasonic image signal of a plurality of frames from a subject via an ultrasonic probe, and an ultrasonic image generated based on the acquired ultrasonic image signal Is displayed on a display unit, and there is a candidate image portion that is estimated to be an image portion indicating a detection target structure for each of the ultrasonic images of a plurality of frames. A candidate image part detection unit for detecting a candidate image part, and assuming that one of the plurality of frames is a reference frame and a cross-sectional position of the ultrasonic image is a reference position, a cross-sectional position of the ultrasonic image of each of the plurality of frames A distance-related information acquisition unit that acquires distance-related information related to a distance from the reference position, and a distance from the reference position is a large target detection structure. A weighting target setting unit that sets a frame within a predetermined detection target distance determined based on the weight as a weighting target, and whether or not the candidate image portion is included for a plurality of frames set as the weighting target A candidate image portion determination unit that determines whether or not, a weight score calculation unit that calculates a weight score represented as a variable with respect to a distance from the reference position, for each of a plurality of frames set as the weighting target, and the weighting target And a structure determination unit that determines that the candidate image portion is the detection target structure when the weight scores of the plurality of frames set to the above satisfy a predetermined condition. .

  According to the ultrasonic image processing device of one aspect of the present invention, a weighting score is calculated for a plurality of frames set as weighting targets, and whether or not the calculated weighting scores satisfy a predetermined condition. It is possible to detect the detection target structure by determining whether the candidate image portion actually indicates the detection target structure by a simple method of determination.

1 is a functional block diagram of an ultrasonic diagnostic system 1000 including an ultrasonic image processing apparatus 100 according to Embodiment 1. FIG. (A)-(c) is a schematic diagram which shows an example of the feature-value used for a candidate image part detection process. It is a schematic diagram which shows an example of the Haar-Like feature used for a candidate image part detection process. (A), (b) is explanatory drawing of a candidate image part detection process. (A) is a schematic diagram which shows the sequence of the ultrasonic image of the several flame | frame continuously acquired when the detection target structure is a granular structure. (B) is a graph showing the relationship between the distance from the reference position and the weight score. It is a flowchart which shows the content of the weight score calculation process which concerns on Embodiment 1. FIG. It is a flowchart which shows the content of a granular structure determination process. (A) is a schematic diagram showing a sequence of ultrasonic images of a plurality of frames obtained continuously when the detection target structure is a long structure. (B) is a graph showing the relationship between the distance from the reference position and the weight score. It is an example of a diagnostic image when it determines with a candidate image part being a thing of a granular structure. It is a flowchart which shows the content of the diagnostic image production | generation display process. 6 is a functional block diagram of an ultrasound diagnostic system 2000 including an ultrasound image processing apparatus 200 according to Embodiment 2. FIG. (A) is a schematic diagram which shows the sequence of the ultrasonic image of the several flame | frame acquired continuously in case the detection target structure is a granular structure in Embodiment 2. FIG. (B) is a graph which shows the relationship between the length of time from the reference time in Embodiment 2, and a weight score. 10 is a flowchart showing the contents of a weight score calculation process according to the second embodiment. (A) is a schematic diagram which shows the sequence of the ultrasonic image of the several flame | frame acquired continuously in case the detection target structure is a granular structure in the modification 1. FIG. (B) is a graph showing the relationship between the distance from the reference position and the weight score in Modification 1. It is a figure which shows typically a mode that the probe which concerns on the modification 2 is scanned while rotating on the surface of a subject. FIG. 10 is a functional block diagram of an ultrasonic diagnostic system 3000 including an ultrasonic image processing apparatus 300 according to Modification 2. (A) is a figure which shows typically the several flame | frame set as the weighting object in the modification 3. FIG. (B) is a figure which shows typically the state by which the value of each weight score was allocated to each detection window 301 in the flame | frame shown to (a). (C) is a figure which shows typically the state which overlap | superposed the flame | frame shown in (b). (A) is a schematic diagram which shows the sequence of the ultrasonic image of the several flame | frame continuously acquired in case the detection target structure is a granular structure in the modification 4. FIG. (B) is a graph showing the relationship between the distance from the reference position and the weight score in Modification 4.

Embodiment 1
Hereinafter, an ultrasonic diagnostic system 1000 including the ultrasonic image processing apparatus 100 according to the first embodiment will be described with reference to the drawings.
<Ultrasonic diagnostic system 1000>
(overall structure)
FIG. 1 is a functional block diagram of an ultrasonic diagnostic system 1000 including an ultrasonic image processing apparatus 100 according to the first embodiment. As shown in FIG. 1, the ultrasound diagnostic system 1000 includes an ultrasound image processing apparatus 100 and an ultrasound probe 101 configured to be connectable to the ultrasound image processing apparatus 100 (hereinafter referred to as “probe 101”). , A camera 102, an operation input unit 103, and a display unit 104. FIG. 1 shows a state in which a probe 101, a camera 102, an operation input unit 103, and a display unit 104 are connected to the ultrasonic image processing apparatus 100.

The probe 101, the operation input unit 103, and the display unit 104 may be included in the ultrasonic image processing apparatus 100. Further, in addition to these, the camera 102 may be included in the ultrasonic image processing apparatus 100.
(Each component configuration)
(Probe 101)
The probe 101 includes a plurality of transducers 101a arranged in, for example, a one-dimensional direction (hereinafter referred to as “column direction”). The probe 101 converts a pulsed electric signal (hereinafter referred to as a “transmission signal”) supplied from a transmission unit 105 (described in detail later) of the ultrasonic image processing apparatus 100 into a pulsed ultrasonic wave. To do. The probe 101 transmits an ultrasonic beam composed of a plurality of ultrasonic waves emitted from a plurality of transducers toward a measurement target in a state where the transducer-side outer surface of the probe 101 is in contact with the skin surface of the subject. . The probe 101 receives a plurality of ultrasonic reflected waves (hereinafter referred to as “reflected ultrasonic waves”) from the subject, and each of the reflected ultrasonic waves is converted into an electric signal (hereinafter referred to as “ultrasonic wave”) by a plurality of transducers. Converted to a “sonic image signal”) and supplied to the receiving unit 106 (details will be described later) of the ultrasonic image processing apparatus 100.

(Ultrasonic image processing apparatus 100)
The ultrasonic image processing apparatus 100 causes the probe 101 to transmit and receive ultrasonic waves, receives an output signal from the probe 101, and generates an ultrasonic image. A more detailed description including the internal configuration of the ultrasonic image processing apparatus 100 will be described later.
(Camera 102)
The camera 102 has an image pickup device such as a charge-coupled device (CCD), and picks up an image of a marker (figure) attached to the probe 101 or the subject surface.

(Operation input unit 103)
The operation input unit 103 receives various operation inputs such as various settings and operations on the ultrasonic image processing apparatus 100 from the examiner.
For example, the operation input unit 103 may be a touch panel configured integrally with the display unit 104. In this case, various settings / operations of the ultrasonic image processing apparatus 100 can be performed by performing a touch operation or a drag operation on the operation keys displayed on the display unit 104, and the ultrasonic image processing apparatus 100 can perform this touch panel operation. Is configured to be operable. The operation input unit 103 may be, for example, a keyboard having various operation keys or an operation panel having various operation buttons and levers. Further, a trackball, a mouse, a flat pad, or the like for moving a cursor displayed on the display unit 104 may be used. Alternatively, a plurality of these may be used, or a combination of these may be used.

(Display unit 104)
The display unit 104 is a so-called display device for image display, and displays an image output from a display control unit 117 described later on the screen. As the display unit 104, a liquid crystal display, a CRT, an organic EL display, or the like can be used.
<Ultrasonic image processing apparatus 100>
Next, the ultrasonic image processing apparatus 100 according to the first embodiment will be described.

  The ultrasonic image processing apparatus 100 includes a transmission unit 105, a reception unit 106, a multiplexer unit 107, a B-mode image generation unit 108, a candidate image part detection unit 109, a position related information detection unit 110, a storage unit 111, and a distance related information acquisition unit. 112, a weighting target setting unit 113, a candidate image part determination unit 114, a weight score calculation unit 115, a structure determination unit 116, and a display control unit 117.

(Transmission unit 105)
The transmission unit 105 controls the timing of applying a high voltage to each transducer 101a of the probe 101 in order to transmit ultrasonic waves.
(Receiver 106)
The receiving unit 106 performs reception beam forming based on the reflected ultrasonic wave received by the probe 101 to generate an acoustic line signal.

(Multiplexer 107)
The multiplexer unit 107 selects a transducer to be used for transmission or reception from among the plurality of transducers 101a of the probe 101, and secures input / output for the selected transducer.
(B-mode image generation unit 108)
The B-mode image generation unit 108 generates a B-mode image of a plurality of frames in time series based on the reflection component from the tissue of the subject in the acoustic line signal that is an output signal from the reception unit 106.

(Candidate image part detection unit 109)
The candidate image portion detection unit 109 selects a candidate image portion that is an image portion that may indicate a detection target structure (in this embodiment, a granular structure) in the generated B-mode image (ultrasound image). To detect. Details will be described later.
(Position related information detection unit 110)
The position related information detection unit 110 acquires imaging data from the camera 102 and detects position information of the probe 101 (hereinafter referred to as “probe position information”) based on the acquired imaging data. Specifically, the marker attached to the surface of the probe 101 is imaged by the fixed camera 102, and the position information of the probe 101 is detected by detecting the position of the marker in the imaging screen. The camera 102 is not limited to the case where it is fixed. For example, a figure indicating the position or length of dots or the like arranged at regular intervals on the surface of the subject is attached, the figure is photographed together with the probe 101, and the probe marker and the figure are displayed on the imaging screen. Any method that detects the position information of the probe 101 from the relative positional relationship can be applied to a hand-held camera, a camera installed in a crane, or the like.

(Storage unit 111)
The storage unit 111 acquires a B-mode image from the B-mode image generation unit 108, and obtains probe position information when the reception unit 106 receives an ultrasound image signal corresponding to the B-mode image, and the position related information detection unit 110. Get from. Then, the acquired B-mode image and probe position information are stored in association with each frame.

Also, various setting information received from the inspector via the operation input unit 103 is stored. The various setting information includes information such as the setting distance SD (see FIG. 5), the frame rate, and the scanning speed of the ultrasonic probe.
(Distance related information acquisition unit 112)
The distance related information acquisition unit 112 substitutes the position of the cross section of the subject indicated by the ultrasonic image of one frame with the relative position of the probe 101 with respect to the subject when the ultrasonic image signal of the frame is acquired. The distance from the reference position, which is the distance from the cross-sectional position (reference position), is obtained by calculation. Details of the reference position and the distance from the reference position will be described later.

(Weighting target setting unit 113)
Based on the distance information acquired by the distance related information acquisition unit, the weighting target setting unit 113 specifies a frame whose distance from the reference position is within a predetermined setting distance, and weights all the specified frames. Set the target. Details will be described later.
(Candidate image part determination unit 114)
The candidate image portion determination unit 114 determines whether or not a candidate image portion is included for each of a plurality of frames set as weighting targets.

(Weight score calculation unit 115)
The weight score calculation unit 115 calculates, for each frame, a weight score for determining whether or not the candidate image portion detected by the candidate image portion detection unit 109 is an image portion indicating the detection target structure. Details will be described later.
(Structure determination unit 116)
Based on the weight score calculated by the weight score calculation unit 115, the structure determination unit 116 detects the candidate image portion detected by the candidate image portion detection unit 109 as a detection target structure (in this embodiment, a granular structure It is determined whether or not the image portion indicates a cross section. Details will be described later.

(Display control unit 117)
The display control unit 117 generates a display image and causes the display unit 104 to display the display image.
(Control unit 118)
The control unit 118 controls each component.
Among these, the transmission unit 105, the reception unit 106, the multiplexer unit 107, the B-mode image generation unit 108, the candidate image part detection unit 109, the position related information detection unit 110, the distance related information acquisition unit 112, the weighting target setting unit 113, the candidates The image part determination unit 114, the weight score calculation unit 115, and the structure determination unit 116 constitute an ultrasonic image processing circuit 150.

  Each element constituting the ultrasonic image processing apparatus 100, for example, a transmission unit 105, a reception unit 106, a multiplexer unit 107, a B-mode image generation unit 108, a candidate image part detection unit 109, a position related information detection unit 110, a weight score calculation The unit 115, the structure determination unit 116, the display control unit 117, and the control unit 118 are realized by hardware circuits such as an FPGA (Field Programmable Gate Array) and an ASIC (Application Specific Integrated Circuit), respectively. Alternatively, each of the above-described components may be implemented by a programmable device such as a CPU (Central Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Unit), or a processor, and software. These components can be a single circuit component, or can be an assembly of a plurality of circuit components. In addition, a plurality of components can be combined into one circuit component, or a plurality of circuit components can be assembled.

  The storage unit 111 is a computer-readable recording medium. For example, a flexible disk, a hard disk, an MO, a DVD, a DVD-RAM, a semiconductor memory, or the like can be used. In the present embodiment, the storage unit 111 is built in the ultrasonic image processing apparatus 100. However, the storage unit 111 is not limited to this, and the storage unit 111 is connected to the ultrasonic image processing apparatus 100 from the outside. May be.

The ultrasonic image processing apparatus 100 according to the present embodiment is not limited to the ultrasonic image processing apparatus having the configuration shown in FIG. For example, there may be a configuration in which the multiplexer unit 107 is unnecessary, or a configuration in which the probe unit 101 includes the transmission unit 105, the reception unit 106, or a part thereof.
<Granular structure detection processing>
Next, a series of processes included in the granular structure detection process will be described in more detail below.

<Candidate image part detection processing>
The candidate image portion detection unit 109 selects a candidate image portion that is an image portion that may indicate a detection target structure (in this embodiment, a granular structure) in the generated B-mode image (ultrasound image). To detect. In this embodiment, since the detection target structure is a granular structure, a point-like high-luminance region in the ultrasonic image is detected.

  2A to 2C are schematic diagrams illustrating an example of feature amounts used for candidate image portion detection processing in the candidate image portion detection unit 109. FIG. As a detection method, for example, as shown in FIG. 2 (c), an area where the average luminance of the central pixel (1 to 9) is higher than the average luminance of the peripheral area (A to P) is searched. The magnitudes of the points may be compared, and as a result, if the average luminance at the central portion is greater than a certain threshold value, it may be determined as a point-like high luminance region.

  There is also a method using a machine learning method as a more advanced determination means. In the machine learning method, as a pre-process, ultrasonic image samples for each category to be identified are collected, and a classifier is constructed by learning. In this example, feature values are extracted for images belonging to the two categories of “dotted high-intensity area (positive example)” or “other than that (negative example)”, and learning is performed using this to construct a discriminator. To do. In the detection process, a feature amount related to the point-like high-intensity region is detected from the ultrasonic image as the test image, and compared with the learning data constructed in advance, whether or not the point-like high-intensity region (positive example) is detected. To determine. Here, as the feature amount, a feature amount expressing a light and shade pattern can be used, and for example, a feature amount called Local Binary Pattern (LBP) is used. Specifically, in the image of FIG. 2A, as shown in FIG. 2B, the size of the peripheral pixel value is compared with the peripheral pixel value clockwise from the upper left with respect to the central target pixel. If the pixel value is greater than or equal to the target pixel value, the result is bit-outputted as 1 and if the surrounding pixel value is less than the target pixel value, the result is 0. Then, 8-bit information obtained by arranging the output bits is used as a feature amount. Further, a Haar-Like feature amount may be used as the feature amount. The Haar-Like feature value is a difference obtained by subtracting the sum of the luminance values of the white region from the sum of the luminance values of the black region in the rectangle to be calculated in the search window.

  FIG. 3 is a schematic diagram illustrating an example of a Haar-Like feature used for candidate image part detection processing in the candidate image part detection unit 109. Haar-Like filters HL1 to HL34 shown in FIG. 3 can be used for detection of the dotted high-intensity region by the candidate image portion detection unit 109. Each Haar-Like filter HL1 to HL34 is a filter for detecting a luminance change.

  In the point-like high-intensity region detection in this example, an ultrasonic image is swept by a detection window in which a plurality of Haar-Like filters HL1 to HL34 are arranged at predetermined positions in the detection window. The range of the Haar-Like filter in the search window is an area for calculating the Haar-Like feature quantity for the ultrasonic image. As a result, a calculation result obtained through each filter HL1 to HL34 in the search window is obtained as a feature amount. A predetermined amount (a plurality) of Haar-Like filter feature values are collected in advance for positive examples (dotted high-intensity regions) and negative examples (other regions), and a machine learning classifier is constructed using these. Keep it. The discriminator is a discriminant that discriminates whether or not the input feature quantity is a positive example, or outputs a probability value (similarity) that is a positive example. Using the discriminator, the position of the search window where the similarity is higher than a predetermined reference value with respect to the unknown image can be extracted as a dotted high-intensity region in the ultrasonic image.

  As an algorithm of the machine learning method, for example, SupportVectorMachine, RandomForest, or the like may be used to learn a feature amount pattern peculiar to a dotted high-intensity region and perform the detection. Random Forest is a model identification function that performs group learning using a decision tree that creates a tree-structured graph and performs prediction / classification, and is classified into a weak learner.

By using machine learning in the present invention, a classifier that efficiently extracts and discriminates distinctive patterns from positive and negative learning data can be configured to accurately detect unknown gray patterns. Judgment can be made.
In the detection of candidate image portions in the present embodiment, the feature amount and the identification method are not limited to the above example, and other methods may be used.

The candidate image part detection unit 109 detects a bright spot included in the ultrasound image as a candidate image part rc. The candidate image portion rc is an image region in the ultrasonic image in which a minute granular structure in the subject tissue is shown.
4A to 4E are explanatory diagrams of the candidate image portion detection operation in the candidate image portion detection unit 109. FIG. As shown in FIG. 4A, in this detection operation, a detection window 301 is set on the ultrasonic image, and the similarity (error value, correlation value, etc.) between the template TM1 and the image in the detection window 301 is calculated. To do. In the present embodiment, the template TM1 uses an image pattern in which a rectangular high luminance area is surrounded by a rectangular low luminance area. However, the template TM1 is not limited to a rectangular shape, and uses an image pattern in which a circular, elliptical, polygonal, or irregularly shaped high luminance region is surrounded by a circular, elliptical, or rectangular annular low luminance region. May be. The low luminance region may not continuously surround the entire circumference (360 °) of the high luminance region, and there may be places where the annular portion is interrupted. The size of the template TM1 and the detection window 301 is preferably such that the diameter of the portion corresponding to the high luminance region is in the range of, for example, 0.1 mm to 0.5 mm, but is not limited thereto, and the size of the detection target structure. You may change suitably according to. In the present embodiment, the diameter is 0.2 mm.

  In FIG. 4A, the template TM1 has a lower limit diameter of 0.2 mm, the detection window 301 is moved in the direction of the scan line 303, and the B-mode image portion in the detection window 301 at each position is similar to the template TM1. Calculate the degree. This process is repeated while moving the detection window 301 so that the detection window 301 sweeps the entire ultrasound image, and the detection window 301 at the position of the detection window 301 where the similarity is equal to or higher than the reference value is set as a candidate image portion rc. . A plurality of candidate image portions rc may be detected by one ultrasonic image sweep. FIG. 4B is a schematic diagram showing a state where the candidate image portion rc is detected from the search using the template TM1.

<Target structure judgment operation>
(Overview)
If the distance corresponding to the number of frames of the ultrasonic image in which the candidate image portions appear continuously is known, the length of the structure indicated by the candidate image portion can be known. The distance can be calculated from the position of the cross section of the subject indicated by the ultrasound image in the subject. The position of the cross section of the subject indicated by the ultrasound image can be regarded as the relative position of the ultrasound probe with respect to the subject when the ultrasound image signal of the ultrasound image is acquired.

  Therefore, an ultrasonic image frame corresponding to a cross section within a predetermined distance range from a cross-sectional position of the reference frame (hereinafter referred to as “reference position”) is set as one frame in the sequence of ultrasonic images. Is a target for determining whether or not there is a detection target structure. Here, when the determination is performed in real time while inspecting, the reference frame is a frame of the ultrasonic image acquired most recently. In addition, the frame to be determined as a frame within a predetermined distance range from the reference position does not need to be determined from the reference position to the long distance range in order to detect a minute granular structure, This is because if the number of determination target frames increases more than necessary, the processing load will increase unnecessarily.

  Since the reference position is a determination target position for determining whether or not the detection target structure is present, the granular structure (detection target structure) existing at a position close to the reference position has high importance. The granular structure that exists far from the reference position is less important. Therefore, the closer the cross-sectional position of the frame where the candidate image portion exists to the reference position, the higher the possibility that the granular structure exists at the reference position. Can be said to be low.

In the case of a long structure, candidate image portions continuously appear up to a frame far from the reference position. Therefore, when a candidate image portion exists in a frame that is a predetermined distance or more away from the reference position, the candidate image portion may represent a cross section of another long structure instead of a cross section of the granular structure. high.
Therefore, the possibility of the granular structure that is the detection target structure is quantified, and the granular structure is obtained for each frame of the ultrasonic image that is the target of the determination as to whether or not there is the detection target structure. Weighting is performed according to the possibility of the object, and the value obtained by summing the weight values (weight scores) of each frame is used as a value indicating the possibility that the granular structure exists at the position of the reference frame. Can be used.

Here, in order to positively eliminate the possibility that the long structure is erroneously determined to be a granular structure, the weight score when the candidate image portion exists in a frame that is a predetermined distance or more away from the reference position. Let be a negative value. The predetermined distance is, for example, the size (diameter) of the granular structure to be detected.
In this way, it is possible to specify the granular structure portion (structure that is likely to be) present at the reference position based on the total value of the weight scores.

Subsequently, specific processing contents of the target structure determination operation in the present embodiment will be described below.
(Weighting target setting process)
First, a process for setting a frame of an ultrasonic image corresponding to a cross section within a predetermined distance range from a reference position as a weighting target as a determination target of whether or not a detection target structure exists will be described.

  FIG. 5A schematically illustrates a state in which candidate image portions indicating a cross section of the detection target structure S are detected in some frames of a sequence of ultrasonic images of a plurality of frames acquired continuously. FIG. The sphere shown in the upper part of FIG. 5A indicates the detection target structure S that is a granular structure, and each of the continuous square planes indicates one frame of a continuous ultrasonic image. In the lower part of FIG. 5A, six frames FA, FB, FC, FD, FE, and FF from the right of the continuous ultrasonic image frames in the upper part of FIG. .

  The probe 101 is scanned from left to right in FIG. 5A, and the ultrasonic images are arranged from left to right in the order of acquisition. That is, the rightmost frame is the frame of the ultrasonic image acquired most recently, and the leftmost is the frame of the ultrasonic image acquired the oldest in the figure. In the present embodiment, since it is assumed that a doctor looks at an ultrasound image displayed on a monitor in real time while performing an examination, the rightmost frame fa represents the most recently acquired ultrasound image. It is a frame and is an ultrasonic image corresponding to the position of the probe 101 at the present time during the examination. In the present embodiment, the frame FA is a reference frame.

  FIG. 5B is a graph showing the relationship between the distance d from the reference position PA and the weight score WS. The cross-sectional positions PA, PB, PC, PD, PE, and PF of the ultrasonic images of the six frames FA, FB, FC, FD, FE, and FF shown in the lower part of FIG. As shown in FIG. 5B, the six frames FA, FB, FC, FD, FE, and FF exist within the range of the detection target distance TD from the reference position PA that is the cross-sectional position of the reference position frame FA. Yes. That is, six frames FA, FB, FC, FD, FE, and FF are set as weighting targets WT and are set as weight score assignment targets. The weighting target WT is set by the weighting target setting unit 113 (see FIG. 1).

(Candidate image part determination processing)
Next, the candidate image portion determination unit 114 determines whether or not the candidate image portion RC is included for each of the frames FA, FB, FC, FD, FE, and FF set as the weighting target WT. Here, as shown in the lower part of FIG. 5A, it is determined that the frames FA, FB, FC, and FD include the candidate image portion RC, and the frames FE and FF include the candidate image portion RC. It is determined that it is not.

(Weight score calculation process)
Subsequently, the weight score calculation unit 115 (see FIG. 1) calculates a weight score for each of the frames FA, FB, FC, FD, FE, and FF set as the weighting target WT.
FIG. 6 is a flowchart showing the content of the weight score calculation process in the first embodiment.
First, the reference position PA is acquired from the distance related information acquisition unit 112 (see FIG. 1), and the size of the detection target structure received from the inspector via the operation input unit 103 is acquired from the storage unit 111 (see FIG. 1). (Step S1).

Next, the acquired detection target structure size is set to the set distance SD (step S2).
Subsequently, a position that is a set distance away from the reference position PA is calculated (step S3).
Next, the frame of the ultrasonic image of the cross section located between the position calculated in step S2 and the reference position PA is set as the weighting target WT (step S4).
The distance from the reference position of each frame set as the weighting target WT is acquired from the storage unit 111 (see FIG. 1) (step S5).

Subsequently, information on whether or not each frame set as the weighting target WT includes a candidate image portion is acquired from the candidate image portion determination unit 114 (step S6).
Next, a weight score is calculated for each frame (step SS7).
Here, a specific method for calculating the weight score will be described with reference to FIG. The upper part of FIG. 5B is a graph showing a weighting curve WC that is a curve indicating the value of the weight score when the distance from the reference position PA is the X axis and the weight score is the Y axis. The weighting curve WC is a cubic curve, the distance from the reference position PA intersects the X axis at the set distance SD, and the weight score WS is 0. The weight score WS is a positive value on the side closer to the reference position PA than the installation position SD, and the weight score WS is a negative value on the side farther from the reference position PA than the installation position SD.

The distance from the reference position PA of the cross-sectional position PB of the frame FB is DB, and the weight score WS for the distance DB is 0.9. In FIG. 5A, the weighting score value of each frame is displayed in a square frame.
The distance from the reference position PA of the cross-sectional position PC of the frame FC is DC, and the weight score WS for the distance DC is 0.8. The distance from the reference position PA of the cross-sectional position PD of the frame FD is DD, and the weight score WS for the distance DD is −0.2.

The distance from the reference position PA of the cross-sectional position PE of the frame FE is DE, and the weight score WS for the distance DE is −0.8. However, since the frame FE does not include the candidate image portion RC, the weight score is 0. (Zero). The distance from the reference position PA of the cross-sectional position PF of the frame FF is DF, and the weight score WS for the distance DF is −0.9. However, since the frame FF does not include the candidate image portion RC, the weight score is 0. And In this way, the corresponding weight score is applied as it is to the frame including the candidate image portion RC, and the weight score WC is uniformly set to 0 for the frame not including the candidate image portion RC. A weight score is calculated for each frame set as the weighting target WT by the method described above.
(Step S7 in FIG. 6).

Returning to the flowchart of FIG. 6, subsequently, the sum of the weight scores of all frames set as the weighting target WT is calculated (step S8), and the weight score calculation processing flow ends.
Note that coefficient 1 is assigned to the frame determined to include the candidate image portion RC in step S6 in the flow of FIG. 6, and coefficient 0 is assigned to the frame determined not to include the candidate image portion RC. Regardless of whether or not the image portion is included, the value indicated by the weight curve WC corresponding to the distance from the reference position PA of each frame is calculated as a weight score, and is assigned to the weight score of each frame in step S8. You may calculate the sum total of the value which multiplied the coefficient.

(Granular structure judgment processing)
Subsequently, the structure determination unit 116 determines whether or not the candidate image portion RC included in the ultrasonic image of the frame set as the weighting target WT indicates a granular structure. FIG. 7 is a flowchart showing the contents of the granular structure determination process by the structure determination unit 116.
First, the sum of weight scores is acquired from the weight score calculator 115 (step S11).
Next, it is determined whether or not the sum of the weight scores is equal to or greater than a predetermined threshold. In the present embodiment, the threshold value is 1.5. If the sum of the weight scores is equal to or greater than the threshold, it is determined that the structure is a granular structure (step S12: YES, step S13), and the flow of the granular structure determination process is terminated. If the sum of the weight scores is not equal to or greater than the threshold value, that is, less than the threshold value, it is determined as a granular structure (step S12: NO, step S14), and the flow of the granular structure determination process ends.

In the case shown in FIG. 5, the weight scores of the frames FA, FB, FC, FD, FE, and FF set as the weighting target WT are 1, 0.9, 0.8, −0.2, and 0, respectively. , 0. Accordingly, since the sum of the weight scores is 2.5 and is equal to or greater than the threshold value 1.5, it is determined that the detection target structure S is a granular structure.
Here, FIG. 8 shows a case where the detection target structure is a long structure. Since the detection target structure S is a long structure, the candidate image portion RC is included in all the frames FA, FB, FC, FD, FE, and FF set as the weighting target WT. The weight scores of the frames FA, FB, FC, FD, FE, and FF are 1, 0.9, 0.8, -0.2, -0.8, and -0.9, respectively. Accordingly, since the sum of the weight scores is 0.8 and is less than the threshold value 1.5, it is determined that the detection target structure S is not a granular structure.

  In addition, when the candidate image portion RC is due to speckle noise, the speckle noise usually appears in only one frame among the frames set as weighting targets. Therefore, the weight score of the frame in which speckle noise appears is 1, but the weight score of the other frames is 0. Therefore, the sum of the weight scores is 1, and the threshold value is 1.5 or less. It is determined that it is not.

In this embodiment, the size of the structure to be detected is set as the set distance SD. For example, when detecting a granular structure having a diameter of 0.2 mm, the set distance SD is set to 0.2. The detection target distance TD is determined to be twice the set distance SD.
The set distance SD is not limited to 0.2 mm, and can be set as appropriate according to the size of the detection target structure. The set distance SD is not limited to the size value of the detection target structure. For example, if the size of a structure to be detected is 0.2 mm and a structure slightly larger than that is allowed, the set distance SD is set to be less than 0.2 according to the allowable width. It may be a large value.

  Further, the detection target distance TD is not limited to a value twice the set distance SD, and may be larger or smaller than twice as long as the value is larger than the set distance SD. However, it is not preferable that it is too large even when it is twice or more. The reason is as follows. When a plurality of granular structures exist adjacent to each other, when another granular structure adjacent to the granular structure existing at the reference position exists within the detection target distance TD, the adjacent granular structures are adjacent to each other. A candidate image portion resulting from the granular structure is included in a frame far from the reference frame. As a result, the sum of the weighting scores falls below the threshold value, and the granular structure existing at the reference position is erroneously determined not to be a granular structure. Therefore, if the detection target distance TD is about twice the set distance SD, the possibility that a plurality of adjacent granular structures exist within the detection target distance TD is sufficiently reduced, and the detection probability of the granular structure is increased. It is thought that it can be made high enough.

Furthermore, the threshold value is not limited to 1.5. An appropriate threshold value may be adopted according to the ratio of the detection target distance TD to the set distance SD.
Furthermore, the sign of the weighting curve may be opposite to the weighting curve WC shown in FIG. That is, the smaller side from the reference position PA than the set distance SD may be negative, and the larger side may be positive. In this case, the threshold value is a negative value, and when the threshold value is equal to or less than the threshold value, it is determined to be a granular structure, and when the threshold value is larger than the threshold value, it is determined not to be a granular structure.

<Diagnostic image generation display processing>
Next, when the structure determination unit 116 determines that the structure is a granular structure, the display control unit 117 (see FIG. 1) displays the determined candidate image portion with an easy-to-understand highlight process. An image is generated and displayed on the display unit 104 (see FIG. 1).
FIG. 9 shows an example of a diagnostic image. FIG. 9 shows an ultrasonic image of a frame including candidate image portions RC1 and RC2 that are determined to be granular structures by the structure determination unit 116. Then, for each of the candidate image portions RC1, RC2, a rectangular frame surrounding the periphery is superimposed and displayed. In the present embodiment, the display control unit 117 superimposes and displays a frame surrounding the candidate image portion as highlight processing. In the present embodiment, the frame lines are all lines having the same thickness, but the thickness of the frame line or the color may be changed according to the value of the weight score.

In addition, as long as the ultrasonic image displayed as a diagnostic image contains the candidate image part RC among the frames set to the weighting object WT, you may use any ultrasonic image. In the present embodiment, for example, an ultrasonic image of the reference frame FA is used, but the frame having the largest area of the candidate image portion RC among the frames set as the weighting target WT may be used.
FIG. 10 is a flowchart showing the processing contents of the diagnostic image generation display processing by the display control unit 117 (see FIG. 1).

First, it is determined by the structure determination unit 116 whether or not it is determined to be a granular structure. If it is determined to be a granular structure (step S21: YES), it is determined to be a granular structure. An ultrasonic image of a frame including the candidate image portion RC is acquired from the storage unit 111 (see FIG. 1) (step S22).
Next, a highlight process is performed on the candidate image portion RC of the acquired ultrasonic image (step S23). In the present embodiment, a rectangular frame surrounding the candidate image portion RC is generated.

Then, a diagnostic image is generated by superimposing a frame line surrounding the candidate image portion RC on the ultrasonic image and displayed on the display unit 104 (step S24), and the flow of the diagnostic image generation / display process ends.
If it is not determined in step S21 that the structure is a granular structure, the ultrasound image is displayed as it is (step S21: NO, step S25), and the flow of the diagnostic image generation display process is ended.

As described above, when the ultrasonic image processing apparatus according to the present embodiment is used, when a granular structure exists at the reference position, a candidate image portion that is a portion showing a cross section of the granular structure of the ultrasonic image is displayed. Since the surrounding frame line is superimposed and displayed on the ultrasonic image, the examiner can easily detect the granular structure in real time at the time of ultrasonic diagnosis.
<< Embodiment 2 >>
In the first embodiment, the position information of the ultrasonic probe is directly acquired by photographing the ultrasonic probe with a camera. However, the present invention is not limited to this, and a configuration without a camera may be employed.

  In the second embodiment, information on the frame rate input from the examiner via the operation input unit 103 (see FIG. 1) and the scanning speed (scanning speed) of the probe 101 is acquired, and the interval (time) between adjacent frames is acquired. ) Is calculated, and the distance is substituted with the length of time. Then, from the reference time that is the time when the ultrasonic image signal of the reference frame is acquired, the length of the retroactive time of the time when the ultrasonic image signal of each frame set in the weighting target WT is acquired (that is, each A weight score is calculated based on the elapsed time from the time when the ultrasonic image signal of the frame is acquired to the reference time.

  FIG. 11 is a functional block diagram of an ultrasonic diagnostic system 2000 including the ultrasonic image processing apparatus 200 according to the second embodiment. Among these, the transmission unit 105, the reception unit 106, the multiplexer unit 107, the B-mode image generation unit 108, the candidate image part detection unit 109, the position related information detection unit 210, the distance related information acquisition unit 212, the weighting target setting unit 113, the candidate The image portion determination unit 114, the weight score calculation unit 115, and the structure determination unit 116 constitute an ultrasonic image processing circuit 250.

  FIG. 12A is a cross section of the detection target structure S in several frames of a sequence of ultrasonic images of a plurality of frames continuously acquired using the ultrasonic image processing apparatus according to the second embodiment. It is a figure which shows typically the state in which the candidate image part which shows is detected. As in the case of the first embodiment shown in FIG. 5A, 6 frames FA, FB, FC, FD, FE, and FF are weighted from the right among the frames of the continuous ultrasonic image in the upper stage of FIG. The frame set to the target WT, the rightmost frame FA is the frame of the most recently acquired ultrasonic image, and the leftmost is the frame of the ultrasonic image acquired the oldest in the figure. It is. Also in this embodiment, the frame FA is the reference frame.

As shown in FIG. 12B, in the second embodiment, the weight score WS is expressed as a function of the time (−t) going back in the past with the reference time set to zero. That is, the time going back to the past from the reference time is used as the distance from the reference position in the first embodiment.
As shown in FIG. 12B, the ultrasonic image signal acquisition times of the frames FA, FB, FC, FD, FE, and FF are TA, TB, TC, TD, TE, and TF, respectively. The lengths of the retroactive times from the reference time TA of TB, TC, TD, TE, and TF when the reference time TA is set to zero are tb, tc, td, te, and tf, respectively.

Here, in the present embodiment, it is assumed that the scanning speed of the probe 101 is constant. Therefore, tb, tc, td, te, and tf all have the same length of time.
The set time ST corresponds to the set distance SD in the first embodiment, and is determined based on the size of the detection target structure, the frame rate, and the scanning speed of the probe 101. In this embodiment, for example, since the frame rate is 20 fps, the scanning speed of the probe 101 is 2 cm / sec, and the size of the detection target structure is 0.2 mm, the set time ST is 0.2 seconds. is there. Further, the detection target time TT corresponds to the detection target distance TD of the first embodiment, and is preferably about twice the set time ST as in the first embodiment.

As shown in FIGS. 12A and 12B, also in this embodiment, the weight scores WS of the frames FB, FC, FD, FE, and FF are 1, 0.9, 0.8, and −0, respectively. .2, 0, 0. Therefore, the sum total of the weight scores is 2.5, which is equal to or greater than the threshold value 1.5, and thus is determined to be a granular structure.
FIG. 13 is a flowchart showing the content of the weight score calculation process in the second embodiment.

First, information on the setting scanning speed, frame rate, and detection target structure size of the ultrasonic probe 101 that is set and input from the examiner via the operation input unit 103 (see FIG. 1) and stored in the storage unit 111, Obtained from the storage unit 111 (step S31).
Next, the set time ST and the detection target time TT are calculated based on the set scanning speed, frame rate, and detection target structure size of the ultrasonic probe 101 (step S32).

Subsequently, the time that is the detection target time TT is calculated from the reference time TA (step S33).
Then, the frame of the ultrasonic image signal acquired between the time calculated in step s33 and the reference time TA is set as the weighting target WT (step S34).
Next, the retroactive time from the reference time TA of each frame set as the weighting target WT is calculated (step S35).

Subsequently, information on whether or not the ultrasonic image of each frame set as the weighting target WT includes the candidate image portion RC is acquired from the candidate image portion determination unit 114 (see FIG. 1) (step S36).
Then, a weighted score is calculated for each frame set as the weighting target WT (step S37).

Then, the sum of the weight scores is calculated (step S38), and the flow of the weight score calculation process is terminated.
As described above, the ultrasonic image processing apparatus according to the second embodiment can also determine whether or not the structure is a granular structure based on the frame rate and the scanning speed of the probe 101.

<Modification>
The present invention has been described based on the ultrasonic image processing apparatuses according to the first and second embodiments. However, the present invention is not limited to the above-described embodiment, and the following modifications can be implemented.
In addition, in order to avoid duplication of description, about the same component as Embodiment 1 and 2, the same code | symbol is attached | subjected and the description is abbreviate | omitted.

(Modification 1)
FIG. 14A shows cross sections of the detection target structure S in several frames of a sequence of ultrasonic images of a plurality of frames continuously acquired in the ultrasonic image processing apparatus according to the first modification. It is a figure which shows typically the state in which the candidate image part is detected.
FIG. 14B shows the cross-sectional positions PA, PB, PC, PD, PE, and PF of the ultrasonic images of the six frames FA, FB, FC, FD, FE, and FF shown in the lower part of FIG. ing. The upper part of FIG. 14B is a graph showing a weighting curve WC that is a curve indicating the value of the weight score when the distance from the reference position PA is the X axis and the weight score is the Y axis. In the present modification, the weighting curve WC is a stepped line curve.

In the example illustrated in FIG. 14, the weight scores of the frames FA, FB, FC, FD, FE, and FF are 1, 1, 1, −1, 0, and 0, respectively. Therefore, the sum of the weight scores is 2, which is equal to or greater than the threshold value 1.5, and thus is determined to be a granular structure.
As described above, even if the weighting curve WC of the first modification is used, it can be determined whether or not it is a granular structure.

  Also in this modification, the sign of the weighting curve may be opposite to that of the weighting curve WC shown in FIG. That is, the smaller side from the reference position PA than the set distance SD may be negative, and the larger side may be positive. In this case, the threshold value may be a negative value, and if it is equal to or less than the threshold value, it is determined to be a granular structure, and if the threshold value is greater than the threshold value, it may be determined not to be a granular structure.

(Modification 2)
In the first and second embodiments, it is assumed that the probe 101 is moved linearly on the subject surface during the examination. However, the present invention is not limited to this, and the probe 101 may be applied when it is rotated (including peristaltic) on the surface of the subject.
FIG. 15 is a diagram schematically illustrating a state in which scanning is performed while the probe 101 is rotationally moved on the subject surface SF. FIG. 16 is a functional block diagram of an ultrasonic diagnostic system 3000 including an ultrasonic image processing apparatus 300 according to the second modification. Among these, the transmission unit 105, the reception unit 106, the multiplexer unit 107, the B-mode image generation unit 108, the candidate image part detection unit 109, the position related information detection unit 310, the distance related information acquisition unit 312, the weighting target setting unit 113, the candidates The image part determination unit 114, the weight score calculation unit 115, and the structure determination unit 116 constitute an ultrasonic image processing circuit 350.

The probe 101 according to this modification is integrally provided with an angular velocity sensor 302, and the position related information detection unit 210 acquires angle information of the probe 101 from the angular velocity sensor 302.
The distance related information acquisition unit 212 according to the present modification acquires angle information of the probe 101 from the position related information detection unit 210, and the distance (depth) DP of the detection target structure S from the center of the rotational motion of the probe 101. The arc length AS corresponding to the angle range AA of the probe 101 is calculated, and the calculated arc length AS is output to the weighting target setting unit 113 and the weight score calculation unit 115 as distance information. To do. That is, in this modification, the weight score is calculated by substituting the arc length AS calculated by the distance-related information acquisition unit 212 for the distance in the first embodiment, and whether or not it is a granular structure is determined. Judgment can be made.

The position related information detection unit 210 acquires from the angular velocity sensor 302 is the angle information of the probe 101. However, since the position information is obtained based on the angle information, the angle information is also included in the position information. To do.
(Modification 3)
In the ultrasonic image, the position of the candidate image portion RC may be slightly shifted from frame to frame. Here, when the deviation is large, the candidate image portion RC does not indicate the cross section of the granular structure, and bright spots and speckle noise existing in the cross section of another structure are consecutively located at close positions. There is a possibility that it was detected. In order to reduce the possibility of erroneous determination that such a series of bright spots is a granular structure, it is possible to carry out the following modifications.

  FIG. 17A shows the candidate image portion RC in the frames FA, FB, FC, and FD among the ultrasonic images of the frames FA, FB, FC, FD, FE, and FF set as the weighting targets in the third modification. It is a figure which shows typically the state from which this was detected. In this modified example, as shown in FIG. 17B, for each of the frames FA, FB, FC, and FD in which the candidate image portion RC is detected, each weight score value is assigned to each detection window 301. In FIG. 17B, the hatching of the detection window is changed depending on the assigned weight score value. A weight score of 1 is assigned to the detection window 301a of the frame FA. A weight score of 0.9 is assigned to the detection window 301b of the frame FB. A weight score of 0.8 is assigned to the detection window 301c of the frame FC. A weight score of −0.2 is assigned to the detection window 301d of the frame FD. In the frames FE and FF, since the candidate image portion RC is not detected, there is no detection window for assigning a weight score.

  FIG. 17C shows a superposition of the frames FA, FB, FC, FD, FE, and FF shown in FIG. The overlapped detection windows 301a to 301d are shown circled and enlarged on the right side. As shown in the enlarged view on the right side, the overlapped detection windows 301a to 301d differ in the overlapping detection windows. Then, in this modification, the weight scores assigned to the overlapping detection windows for each part are summed, and when the area of the part where the value of the summed weight score is equal to or greater than a threshold is greater than or equal to a certain value, It is determined that the structure is a granular structure.

  As shown in the enlarged view on the right side of FIG. 17C, the overlapping portion 401 is a portion of only the detection window 301a, and thus the weight score is 1. Note that the total weight score of each overlapping portion is shown in a square frame. Since the overlapped portion 402 is a portion where the detection windows 301a and 301c are overlapped, the total weight score is 1.8. Since the overlapped portion 403 is only the detection window 301c, the total weight score is 0.8. Since the overlapped portion 404 is a portion where the detection windows 301a, 301b, 301c, and 301d are overlapped, the total weight score is 2.5. Since the overlapped portion 405 is a portion where the detection windows 301a, 301b, and 301c are overlapped, the total weight score is 2.7. Since the overlapped portion 406 is a portion where the detection windows 301b and 301c are overlapped, the total weight score is 1.7. Since the overlapped portion 407 is a portion of only the detection window 301b, the weight score is 0.9. Since the overlapped portion 408 is a portion where the detection windows 301a and 301b are overlapped, the total weight score is 1.9. Since the overlapped portion 409 is a portion where the detection windows 301b and 301d are overlapped, the total weight score is 0.7. Since the overlapped portion 410 is a portion where the detection windows 301a, 301b, and 301d are overlapped, the total weight score is 1.7. Since the overlapped portion 411 is a portion where the detection windows 301a, 301c, and 301d are overlapped, the total weight score is 1.6. Since the overlapped portion 412 is a portion where the detection windows 301a and 301d are overlapped, the total weight score is 0.8. Since the overlapped portion 413 is a portion of only the detection window 301d, the weight score is −0.2.

  In the example shown in FIG. 17C, the total weight score of 1.5 or more is the overlapped portions 402, 404, 405, 406, 408, 410, and 411. Is determined to be a granular structure. Note that the value of the predetermined area may be determined as appropriate depending on the size of the detection target structure and the size of the detection window determined accordingly.

Further, the highlight processing method may be changed according to the size of the area of the portion where the total weight score is equal to or greater than the threshold. For example, the larger the area whose total weight score is equal to or greater than the threshold value, the thicker the border line, the darker the color, or the different color.
(Modification 4)
In the first and second embodiments, a case has been described in which the reference frame is the frame from which the latest ultrasonic image signal is acquired and a frame on the past side of the reference frame is set as a weighting target, but is not limited thereto. In the modified example 4, when the candidate image portion RC is detected in the ultrasonic image, the frame is set as a reference frame, and weighting is performed on the frames from which the ultrasonic image signal is acquired after the reference frame as the probe 101 is scanned. I do.

FIG. 18A shows cross sections of the detection target structure S in several frames of a sequence of ultrasonic images of a plurality of frames continuously acquired in the ultrasonic image processing apparatus according to the modification 4. It is a figure which shows typically the state in which the candidate image part is detected.
FIG. 18B shows cross-sectional positions PA, PB, PC, PD, PE, and PF of the ultrasonic images of the six frames FA, FB, FC, FD, FE, and FF shown in the lower part of FIG. ing. The upper part of FIG. 18B is a graph showing a weighting curve WC that is a curve indicating the value of the weight score when the distance from the reference position PA is the X axis and the weight score is the Y axis. The weighting curve WC of the present modification is a cubic curve, the distance from the reference position PA intersects the X axis at the set distance SD, and the weight score WS is 0. The weight score WS is a positive value on the side closer to the reference position PA than the installation position SD, and the weight score WS is a negative value on the side farther from the reference position PA than the installation position SD.

  18A and 18B basically correspond to the diagrams in which the left and right in FIGS. 5A and 5B are reversed. In the present modification, the reference frame FA is the oldest frame among the six frames FA, FB, FC, FD, FE, and FF, and the ultrasonic image signal is acquired. Also in the present modification, as in the first embodiment, the distance from the reference position is calculated for each frame set as the weighting target WT, and the weight score corresponding to each distance is calculated by the weight curve RC. And the sum total of a weight score and a threshold value are compared, and when a sum total is more than a threshold value, it determines with it being a granular structure.

  In the example shown in FIGS. 18A and 18B, when the probe 101 comes to the position of the frame FF, it is determined that the granular structure S exists at the reference position FA. However, since the granular structure is a minute structure having a diameter of about 0.2 mm, the error between the position of the frame FF and the actual position of the granular structure S is negligible. Conceivable. In addition, the candidate image portion is not included in the frame FF corresponding to the position of the probe 101 at the time when it is determined to be the granular structure S. Therefore, in this modification, the frames FA, FB, FC, and FD in which the candidate image portion RC is included in the diagnostic image that the display control unit 117 (see FIG. 1) displays on the display unit 104 (see FIG. 1). Any one of the ultrasonic images is used.

(Modification 4)
In each of the above-described embodiments and modifications, the case where the sum of the weight scores and the threshold value are compared in the granular structure determination process performed by the structure determination unit has been described. However, the present invention is not limited to this, and the average value of the weight scores may be compared with a threshold value. In addition, among the weight scores of each frame set as the weighting target WT, an average value of a weight score having a positive value may be compared with a threshold value, or an average value and a threshold value of a negative value may be compared with each other. May be compared.

(Modification 5)
Further, in the granular structure determination process, it may be determined whether or not the structure is a granular structure by a discriminator from a pattern of a sequence of weight scores arranged in frame order. For example, in the first modification shown in FIGS. 14A and 14B, the numerical sequence in which the weight scores of the frames FA to FF are arranged in order is 1, 1, 1, −1, 0, 0. The discriminator can determine that it is a granular structure when the sequence of weight scores matches the pattern.

(Modification 6)
In the first and second embodiments and the first to third modifications, the reference frame FA is the frame of the ultrasonic image most recently acquired in real time. A granular structure detection process is executed. At this time, if there is a frame in which the candidate image portion is not detected for some reason between the frames in which the candidate image portion is detected, or the scanning speed of the probe 101 (see FIG. 1) is high, the weighting target For example, when the number of frames set to is small, the frame line of the highlight process may blink and become difficult to see. In the ultrasonic image processing device according to the modified example 6, in addition to the highlight processing of the current frame displayed in real time, the highlight processing of a predetermined number of past frames is also superimposed and displayed. As a result, the highlight process of the past frame is superimposed and displayed like an afterimage, so that the blinking of the frame line can be prevented.

(Modification 7)
In each of the above embodiments and modifications, the case where the granular structure is detected in real time while performing an ultrasonic diagnosis has been described. However, the present invention is not limited to this. Ultrasound diagnostic images acquired in past examinations may be read from the storage unit 111 (see FIG. 1) and used as a granular structure detection target frame. In this case, for example, a frame to be displayed may be selected by operating a slider or the like displayed on the display unit 104 (see FIG. 1). The selected frame may be used as a reference frame.

Moreover, you may combine at least one part among the functions of the ultrasonic image processing apparatus which concerns on each embodiment, and its modification. Furthermore, all the numbers used above are exemplified for specifically explaining the present invention, and the present invention is not limited to the illustrated numbers.
Furthermore, the present invention includes various modifications in which the present embodiment is modified within a range conceivable by those skilled in the art.
≪Summary≫
As described above, the ultrasonic diagnostic apparatus according to one aspect of the present invention acquires an ultrasonic image signal of a plurality of frames from a subject via an ultrasonic probe, and is generated based on the acquired ultrasonic image signal. An ultrasonic image processing apparatus that displays a displayed ultrasonic image on a display unit, and for each of the ultrasonic images of a plurality of frames, there is a candidate image portion that is estimated to be an image portion indicating a detection target structure. In this case, a candidate image portion detection unit that detects the candidate image portion and one of the plurality of frames as a reference frame, and a cross-sectional position of the ultrasonic image as a reference position, the ultrasonic waves of the plurality of frames A distance-related information acquisition unit that acquires distance-related information related to the distance from the reference position of the cross-sectional position of the image, and the distance from the reference position is the detection target. A weighting target setting unit that sets a frame within a predetermined detection target distance determined based on the size of the target structure as a weighting target, and the candidate image portion for a plurality of frames set as the weighting target. A candidate image portion determination unit that determines whether or not the image is included, and a weight score calculation unit that calculates a weight score expressed as a variable with respect to a distance from the reference position for each of the plurality of frames set as the weighting targets And a structure determination unit that determines that the candidate image portion is the detection target structure when the weighting scores of the plurality of frames set as the weighting target satisfy a predetermined condition. It is characterized by that.

With such a configuration, a weight score is calculated for a frame set as a weighting target, and whether or not the candidate image portion indicates a detection target structure depending on whether or not the calculated weight score satisfies a predetermined condition. Can be easily determined.
In the ultrasonic image processing device according to another aspect, the ultrasonic image processing apparatus includes a position related information acquisition unit that acquires information related to a position of the ultrasonic probe in the subject, and the distance related information acquisition unit includes the position Information related to the distance from the reference position may be obtained by calculation based on information related to.

With this configuration, by acquiring information related to the position of the ultrasonic probe in the subject, information related to the distance from the reference position can be easily acquired by calculation.
In the ultrasonic image processing device according to another aspect, the position-related information acquisition unit acquires imaging data obtained by imaging the ultrasonic probe and the subject as information related to the position. May be.

With such a configuration, the relative position of the ultrasonic probe with respect to the subject can be easily detected by imaging the ultrasonic probe and the subject together with the camera.
In another aspect, the ultrasonic probe has a plurality of transducers and angular velocity sensors arranged in a one-dimensional direction, and is a virtual plane orthogonal to the rows of the plurality of transducers on the surface of the subject. The position related information acquisition unit uses the information on the relative angle of the ultrasonic probe with respect to the subject as information related to the position, and the angular velocity. The distance-related information acquisition unit is acquired from a sensor, and the angle of the ultrasonic probe when the ultrasonic image signal of the reference frame is acquired and the ultrasonic image signal of each of the plurality of frames are acquired. The distance from the reference position may be obtained by calculation based on the angle difference from the angle of the ultrasonic probe and the distance from the center of the candidate image portion.

With this configuration, the distance from the reference position can be easily acquired by the angular velocity sensor.
In another aspect, the position-related information acquisition unit acquires a scanning speed of the ultrasonic probe and a frame rate in acquiring the ultrasonic image signal from the outside, and the distance-related information acquisition unit acquires the scanning speed. And based on the frame rate, the time corresponding to the distance from the reference position and the time corresponding to the detection target distance are obtained by calculation, the weighting target setting unit is replaced with the distance from the reference position, The time corresponding to the distance from the reference position is set as a weighting target using the time corresponding to the detection target distance instead of the detection target distance, and the weight score calculation unit Instead of the distance, the weight score may be calculated using a time corresponding to the distance from the reference position.

With this configuration, the weight score can be calculated by a simpler method without using a camera or an angular velocity sensor.
In another aspect, the relationship between the distance from the reference position and the weight score is represented by a cubic curve, where the distance from the reference position is on the horizontal axis and the weight score is on the vertical axis, The weight score may be reversed in polarity at a set distance based on the size of the detection target structure.

With such a configuration, it is possible to determine that the structure is a detection target structure by a relatively simple calculation without performing a complicated calculation such as a calculation using a Hessian matrix.
In another aspect, the relationship between the distance from the reference position and the weight score is represented by a curve that increases or decreases in a stepwise manner, with the distance from the reference position on the horizontal axis and the weight score on the vertical axis. In the case of using an axis, the weight score may be reversed in polarity at a set distance based on the size of the detection target structure.

With such a configuration, the detection target structure can be determined by a relatively simple calculation without performing a complicated calculation such as a calculation using a Hessian matrix.
In another aspect, the predetermined condition is that when the weight score when the distance from the reference position is smaller than the set distance is a positive value, the frame set as the weight target The sum or average value of the weight scores of the frames determined to include the candidate image portion is equal to or greater than a predetermined threshold, and the distance from the reference position is greater than the set distance When the weight score in the case of being small is a negative value, the sum or average of the weight scores of the frames determined to include the candidate image portion among the frames set as the weighting targets The value may be a predetermined threshold value or less.

With such a configuration, it can be determined that the structure is a detection target structure by relatively simple calculation such as summation or average.
In another aspect, the weight score calculation unit sets the weight score of a frame that is determined not to include the candidate image portion among frames set as the weight target to be zero. The sum or average of the weight scores of the frames set to may be calculated.

With such a configuration, it can be determined that the structure is a detection target structure by a relatively simple calculation.
In another aspect, the predetermined condition may be that a time-series sequence of the weight scores of each frame set as the weighting target matches a predetermined pattern.
With such a configuration, a discriminator can be used to determine whether or not the structure is a detection target structure.

In another aspect, the detection target structure may be a granular structure.
With such a configuration, a minute granular structure due to calcification in the milk duct can be detected.
In another aspect, the candidate image portion detection unit may detect a dot-like high luminance region in the ultrasound image as the candidate image portion.
With such a configuration, the granular structure can be detected with high accuracy.

  Further, in another aspect, the structure determination unit adds the case where the weight score of the plurality of frames set as the weighting target satisfies a predetermined condition, in addition to the frame set as the weighting target. Of these, the weight image is assigned to each of the candidate image portions of each frame determined to include the candidate image portion, and the respective frames are overlaid by assigning the respective weight scores in units of pixels, and assigned according to the overlap of the candidate image portions. The obtained weight score is added in units of pixels, and when the area where the added weight score is equal to or greater than the predetermined threshold is equal to or greater than a predetermined area, the candidate image portion is determined to be the detection target structure. May be.

With such a configuration, it is possible to reduce the probability that a granular structure is erroneously determined by a bright spot that does not originate from the granular structure.
In another aspect, the ultrasonic image including the candidate image portion determined to be the detection target structure is subjected to highlight processing on the candidate image portion determined to be the detection target structure. You may provide the display control part which gives and displays on the said display part.

With such a configuration, the candidate image portion determined to be the detection target structure can be displayed in an easily understandable manner to the examiner.
In another aspect, the highlighting process may be a process of displaying a line surrounding the candidate image portion.
With such a configuration, the candidate image portion determined to be the detection target structure can be displayed in an easy-to-understand manner to the examiner using a simple method.

In another aspect, the display control unit changes the thickness or color of the surrounding line to the display unit according to the sum of the weight scores or the average value of each frame belonging to the target frame range. It may be displayed.
With such a configuration, the candidate image portion determined to be the detection target structure can be displayed in an easy-to-understand manner to the examiner using a simple method.

<Supplement>
Each of the embodiments described above shows a preferred specific example of the present invention. The numerical values, shapes, materials, constituent elements, arrangement positions and connecting forms of the constituent elements, steps, order of steps, and the like shown in the embodiments are merely examples, and are not intended to limit the present invention. In addition, among the constituent elements in the embodiment, steps that are not described in the independent claims indicating the highest concept of the present invention are described as optional constituent elements constituting a more preferable form.

In addition, for easy understanding of the invention, the scales of the components shown in the above-described embodiments may be different from actual ones. The present invention is not limited by the description of each of the above embodiments, and can be appropriately changed without departing from the gist of the present invention.
Furthermore, in the ultrasonic image processing apparatus, there are circuit components, lead wires, and other members on the substrate, but various modes can be implemented based on ordinary knowledge in the technical field regarding electrical wiring and electrical circuits. The description of the present invention is omitted because it is not directly relevant. Each figure shown above is a schematic diagram, and is not necessarily illustrated strictly.

  The ultrasonic image processing apparatus according to the present disclosure is useful for improving the performance of a conventional ultrasonic image processing apparatus, in particular, improving the visibility of an inspector and improving inspection efficiency.

100, 200, 300 Ultrasonic image processing apparatus 101 Probe 101a Ultrasonic transducer 102 Camera 103 Operation input unit 104 Display unit 109 Candidate image part detection unit 110, 210, 310 Position related information detection unit 111 Storage unit 112, 212, 312 Distance related information acquisition unit 113 Weighted object setting unit 114 Candidate image part determination unit 115 Weight score calculation unit 116 Structure determination unit 117 Display control unit 301, 301a, 301b, 301c, 301d, 301e, 301f Inspection window 401, 402, 403 , 404, 405, 406, 407, 408, 409, 410, 411, 412, 413 Overlapping portion 302 Angular velocity sensor 1000, 2000, 3000 Ultrasonic diagnostic system FA reference frame PA reference position RC, RC1, RC2 candidate Image portion S Detection target structure SD Set distance ST Set time TA Reference time TD Target distance TT Detection target time WC Weighting curve WS Weight score WT Weighting target

Claims (16)

An ultrasonic image processing apparatus for acquiring an ultrasonic image signal of a plurality of frames from a subject via an ultrasonic probe, and displaying an ultrasonic image generated based on the acquired ultrasonic image signal on a display unit,
A candidate image part detecting unit that detects the candidate image part when there is a candidate image part that is estimated to be an image part indicating a detection target structure for each of the ultrasonic images of a plurality of frames;
If one of the plurality of frames is a reference frame and the cross-sectional position of the ultrasonic image is a reference position, distance-related information related to the distance from the reference position of the cross-sectional position of the ultrasonic image of each of the plurality of frames A distance related information acquisition unit for acquiring
A weighting target setting unit configured to set a frame within a predetermined detection target distance in which a distance from the reference position is determined based on a size of the detection target structure as a detection target;
A candidate image part determination unit that determines whether or not the candidate image part is included for a plurality of frames set as the weighting targets;
For each of the plurality of frames set as the weighting target, a weight score calculating unit that calculates a weight score represented as a variable with respect to the distance from the reference position;
A structure determination unit that determines that the candidate image portion is the detection target structure when the weighting scores of the plurality of frames set as the weighting target satisfy a predetermined condition. Image processing device. A position related information acquisition unit for acquiring information related to the position of the ultrasonic probe in the subject;
The ultrasonic image processing apparatus according to claim 1, wherein the distance related information acquisition unit acquires information related to a distance from the reference position by calculation based on information related to the position. The ultrasonic image processing apparatus according to claim 2, wherein the position related information acquisition unit acquires imaging data obtained by imaging the ultrasonic probe and the subject together as information related to the position. The ultrasonic probe has a plurality of transducers and angular velocity sensors arranged in a one-dimensional direction, and the plurality of transducers in a virtual plane orthogonal to the row of the plurality of transducers on the surface of the subject Is rotated around the column of
The position related information acquisition unit acquires information on the relative angle of the ultrasonic probe with respect to the subject as information related to the position from the angular velocity sensor,
The distance related information acquisition unit includes an angle of the ultrasonic probe when the ultrasonic image signal of the reference frame is acquired and an ultrasonic probe of the ultrasonic probe when the ultrasonic image signal of each of the plurality of frames is acquired. The ultrasonic image processing apparatus according to claim 2, wherein a distance from the reference position is obtained by calculation based on an angle difference from an angle and a distance from the center of the candidate image portion. The position related information acquisition unit acquires the scanning speed of the ultrasonic probe and the frame rate in acquiring the ultrasonic image signal from the outside,
The distance related information acquisition unit acquires a time corresponding to the distance from the reference position and a time corresponding to the detection target distance by calculation based on the scanning speed and the frame rate,
The weighting target setting unit performs weighting using a time corresponding to the distance from the reference position instead of the distance from the reference position, and using a time corresponding to the detection target distance instead of the detection target distance. Set the target,
The ultrasonic image processing apparatus according to claim 2, wherein the weight score calculation unit calculates a weight score using a time corresponding to the distance from the reference position instead of the distance from the reference position. The relationship between the distance from the reference position and the weight score is represented by a cubic curve,
2. When the distance from the reference position is on the horizontal axis and the weight score is on the vertical axis, the weight score is reversed in polarity with respect to a set distance based on the size of the detection target structure. 5. The ultrasonic image processing apparatus according to 5. The relationship between the distance from the reference position and the weight score is represented by a curve that increases or decreases in steps,
2. When the distance from the reference position is on the horizontal axis and the weight score is on the vertical axis, the weight score is reversed in polarity with respect to a set distance based on the size of the detection target structure. 5. The ultrasonic image processing apparatus according to 5. The predetermined condition is:
When the weight score when the distance from the reference position is smaller than the set distance is a positive value, the candidate image portion is included in the frame set as the weight target. The sum or average of the weighted scores of the determined frames is equal to or greater than a predetermined threshold;
When the weight score when the distance from the reference position is smaller than the set distance is a negative value, the candidate image portion is included in the frame set as the weight target. The ultrasonic image processing apparatus according to claim 6 or 7, wherein a sum or an average value of the weight scores of the determined frames is equal to or less than a predetermined threshold value. The weight score calculation unit sets the weight score of a frame that is determined not to include the candidate image portion among frames set as the weight target to zero, and the frame set as the weight target The ultrasonic image processing apparatus according to claim 8, wherein a sum or an average of the weight scores is calculated. 8. The ultrasonic image processing according to claim 6, wherein the predetermined condition is that a time-series sequence of the weight scores of each frame set as the weighting target matches a predetermined pattern. 9. apparatus. The ultrasonic image processing apparatus according to claim 1, wherein the detection target structure is a granular structure. The ultrasonic image processing device according to claim 11, wherein the candidate image portion detection unit detects a dotted high-luminance region in the ultrasonic image as the candidate image portion. In addition to the case where the weight scores of the plurality of frames set as the weighting target satisfy a predetermined condition, the structure determination unit includes the candidate image portion of the frame set as the weighting target. Each of the candidate image portions of each frame determined to be included is assigned the weight score in units of pixels to overlap each frame, and the assigned weight score is pixelated according to the overlap of the candidate image portions. The addition is performed in units, and when the region where the added weight score is equal to or greater than the predetermined threshold is equal to or greater than a predetermined area, the candidate image portion is determined to be the detection target structure. The ultrasonic image processing apparatus according to any one of the above. The ultrasonic image including the candidate image portion determined to be the detection target structure is displayed on the display unit by performing a highlight process on the candidate image portion determined to be the detection target structure. The ultrasonic image processing apparatus according to claim 1, further comprising: a display control unit that controls The ultrasonic image processing apparatus according to claim 14, wherein the highlight processing is processing for displaying a line surrounding the candidate image portion. The display control unit changes the thickness or color of the surrounding line according to the sum of the weight scores of each frame belonging to the target frame range or the average value, and causes the display unit to display the frame. Ultrasonic image processing device.