JP7102181B2 - Analyst - Google Patents

Description

Embodiments of the present invention relate to an analyzer.

Conventionally, an ultrasonic diagnostic apparatus displays an ultrasonic image generated in real time or an ultrasonic image generated by past ultrasonic scanning. Further, there is known a CAD (Computer Aided Detection) function that automatically detects a characteristic part in a medical image generated by a medical image diagnostic apparatus.

Japanese Unexamined Patent Publication No. 2006-167289

An object to be solved by the present invention is to provide an analysis device capable of improving diagnostic efficiency.

The analysis device of the embodiment includes an acquisition unit, a search unit, and a search result synthesis unit. The acquisition unit acquires a first image and a second image different from the first image. The search unit searches for a feature portion in the first image by executing the first search process on the first image, and the first search process on the second image. By executing a second search process different from the above, the feature portion in the second image is searched. The search result synthesis unit synthesizes each search result by the search unit.

FIG. 1 is a block diagram showing a configuration example of the ultrasonic diagnostic apparatus according to the first embodiment. FIG. 2 is a diagram for explaining the first embodiment. FIG. 3 is a flowchart showing a procedure of analysis processing by the processing circuit according to the first embodiment. FIG. 4 is a diagram for explaining the first embodiment. FIG. 5 is a diagram for explaining the first embodiment. FIG. 6 is a diagram for explaining the first embodiment. FIG. 7 is a flowchart showing a processing procedure by the search result synthesis function according to the first embodiment. FIG. 8 is a diagram for explaining the processing operation of the search result synthesis function according to the first embodiment. FIG. 9 is a diagram for explaining the first embodiment. FIG. 10 is a diagram for explaining the first embodiment. FIG. 11 is a flowchart showing a processing procedure by the search result synthesis function according to the first modification of the first embodiment. FIG. 12 is a diagram for explaining the processing operation of the search result synthesis function according to the first modification of the first embodiment. FIG. 13 is a diagram for explaining the processing operation of the search result synthesis function according to the first modification of the first embodiment. FIG. 14 is a flowchart showing a processing procedure by the search result synthesis function according to the second modification of the first embodiment. FIG. 15 is a diagram for explaining the processing operation of the search result synthesis function according to the second modification of the first embodiment. FIG. 16 is a diagram for explaining the processing operation of the search result synthesis function according to the third modification of the first embodiment. FIG. 17 is a diagram for explaining the processing operation of the display control function according to the third modification of the first embodiment. FIG. 18 is a diagram for explaining the processing operation of the search result synthesis function according to the third modification of the first embodiment. FIG. 19 is a diagram for explaining the processing operation of the search result synthesis function according to the third modification of the first embodiment. FIG. 20 is a diagram for explaining the processing operation of the display control function according to the fourth modification of the first embodiment. FIG. 21 is a diagram for explaining the processing operation of the display control function 185 according to the fifth modification of the first embodiment. FIG. 22 is a flowchart showing a procedure of display control processing by the display control function according to the second embodiment. FIG. 23 is a diagram for explaining the second embodiment. FIG. 24 is a block diagram showing a configuration example of a medical image processing apparatus according to another embodiment.

Hereinafter, the analysis apparatus according to the embodiment will be described with reference to the drawings.

(First Embodiment)
In the first embodiment, an ultrasonic diagnostic apparatus will be described as an example of the analysis apparatus. FIG. 1 is a block diagram showing a configuration example of the ultrasonic diagnostic apparatus 1 according to the first embodiment. As shown in FIG. 1, the ultrasonic diagnostic apparatus 1 according to the first embodiment includes an apparatus main body 100, an ultrasonic probe 101, an input device 102, and a display 103. The ultrasonic probe 101, the input device 102, and the display 103 are each connected to the device main body 100.

The ultrasonic probe 101 has a plurality of vibrating elements (piezoelectric vibrators). The ultrasonic probe 101 comes into contact with the body surface of the subject P to transmit and receive ultrasonic waves (ultrasonic scanning). The plurality of vibrating elements generate ultrasonic waves based on a drive signal supplied from a transmission circuit 110 included in the device main body 100, which will be described later. The generated ultrasonic waves are reflected by the mismatched surface of the acoustic impedance in the subject P, and are received by a plurality of vibrating elements as reflected wave signals (received echoes) including components scattered by scatterers in the tissue. To. The ultrasonic probe 101 sends the reflected wave signal received by the plurality of vibrating elements to the transmission circuit 110.

In this embodiment, the case where the ultrasonic probe 101 is a two-dimensional ultrasonic probe (also referred to as “2D array probe”) having a plurality of vibrating elements arranged in a matrix (lattice) will be described. However, it is not limited to this. For example, the ultrasonic probe 101 may be a one-dimensional ultrasonic probe (also referred to as “1D array probe”) having a plurality of vibrating elements arranged one-dimensionally in a predetermined direction.

The input device 102 includes a mouse, a keyboard, a button, a panel switch, a touch command screen, a foot switch, a trackball, a joystick, and the like, and receives various setting requests from the operator of the ultrasonic diagnostic device 1 to the device main body 100. Transfer the various setting requests received to the mouse. Further, for example, the input device 102 receives an instruction to execute the CAD process from the operator.

The display 103 displays a GUI (Graphical User Interface) for the operator of the ultrasonic diagnostic apparatus 1 to input various setting requests using the input device 102, ultrasonic image data generated in the apparatus main body 100, and the like. Or display. For example, the display 103 is composed of a liquid crystal display, a CRT (Cathode Ray Tube) display, or the like. Further, the display 103 is appropriately referred to as a monitor. The display 103 is an example of a display unit.

The device main body 100 is a device that generates ultrasonic image data based on the reflected wave signal received by the ultrasonic probe 101. As shown in FIG. 1, the apparatus main body 100 includes, for example, a transmission circuit 110, a reception circuit 120, a B mode processing circuit 130, a Doppler processing circuit 140, an image generation circuit 150, an image memory 160, and a storage circuit. It has 170 and a processing circuit 180. The transmission circuit 110, the reception circuit 120, the B-mode processing circuit 130, the Doppler processing circuit 140, the image generation circuit 150, the image memory 160, the storage circuit 170, and the processing circuit 180 are communicably connected to each other.

The transmission circuit 110 includes a pulsar circuit and the like. The pulser circuit repeatedly generates a rate pulse for forming a transmission ultrasonic wave at a predetermined rate frequency (PRF: Pulse Repetition Frequency), and outputs the generated rate pulse to the ultrasonic probe 101.

Further, the transmission circuit 110 is controlled by the processing circuit 180 and outputs the value of the amplitude of the drive signal output by the pulsar circuit. Further, the transmission circuit 110 is controlled by the processing circuit 180 to transmit the delay amount with respect to the ultrasonic wave transmitted from the ultrasonic probe 101 to the ultrasonic probe 101.

The receiving circuit 120 includes an A / D converter and a receiving beam former. When the receiving circuit 120 receives the reflected wave signal output from the ultrasonic probe 101, the A / D converter first converts the reflected wave signal into digital data, and the receiving beam former converts the reflected wave signal from each of these channels. The digital data is subjected to phasing addition processing to generate reflected wave data, and the generated reflected wave data is transmitted to the B mode processing circuit 130 and the Doppler processing circuit 140.

The B-mode processing circuit 130 receives the reflected wave data output from the receiving circuit 120, performs logarithmic amplification, envelope detection processing, etc. on the received reflected wave data, and expresses the signal strength by the brightness of the brightness. Data to be generated (B mode data) is generated.

The Doppler processing circuit 140 receives the reflected wave data output from the receiving circuit 120, frequency-analyzes the velocity information from the received reflected wave data, extracts the blood flow, tissue, and contrast agent echo component due to the Doppler effect, and averages them. Generate data (Dopla data) that extracts moving object information such as speed, dispersion, and power for multiple points.

The image generation circuit 150 generates ultrasonic image data from the data generated by the B mode processing circuit 130 and the Doppler processing circuit 140. The image generation circuit 150 generates B-mode image data in which the intensity of the reflected wave is represented by brightness from the B-mode data generated by the B-mode processing circuit 130. Further, the image generation circuit 150 generates Doppler image data representing mobile information from the Doppler data generated by the Doppler processing circuit 140. The Doppler image data is speed image data, distributed image data, power image data, or image data in which these are combined. Here, for example, the image generation circuit 150 stores the images generated in time series in real time in the image memory 160.

Here, the image generation circuit 150 generally converts (scan-converts) a scanning line signal string of ultrasonic scanning into a scanning line signal string of a video format typified by a television or the like, and ultrasonic waves for display. Generate image data. Specifically, the image generation circuit 150 generates ultrasonic image data for display by performing coordinate conversion according to the scanning form of ultrasonic waves by the ultrasonic probe 101. In addition to the scan conversion, the image generation circuit 150 also uses, for example, a plurality of image frames after the scan conversion to regenerate an image with an average value of brightness as various image processing (smoothing processing). , Perform image processing (edge enhancement processing) using a differential filter in the image. Further, the image generation circuit 150 synthesizes character information, scales, body marks, and the like of various parameters with the ultrasonic image data.

The image memory 160 is a memory for storing image data (B-mode image data, Doppler image data, etc.) generated by the image generation circuit 150. The image memory 160 can also store the data generated by the B-mode processing circuit 130 and the Doppler processing circuit 140. The B-mode data and Doppler data stored in the image memory 160 can be called by an operator, for example, and become ultrasonic image data for display via the image generation circuit 150.

The storage circuit 170 stores control programs for performing ultrasonic transmission / reception, image processing, display processing, etc., diagnostic information (for example, patient ID, doctor's findings, etc.), and various data such as diagnostic protocols and various body marks. do. The storage circuit 170 is also used for storing image data stored in the image memory 160, if necessary. Further, the data stored in the storage circuit 170 can be transferred to an external device via a communication interface (not shown).

The processing circuit 180 controls the entire processing of the ultrasonic diagnostic apparatus 1. Specifically, the processing circuit 180 is based on various setting requests input from the operator via the input device 102, various control programs and various data read from the storage circuit 170, and the transmission circuit 110 and the reception circuit 120. It controls the processing of the B-mode processing circuit 130, the Doppler processing circuit 140, the image generation circuit 150, and the like. Further, the processing circuit 180 causes the display 103 to display the ultrasonic image data stored in the image memory 160.

Further, as shown in FIG. 1, the processing circuit 180 executes the acquisition function 181 and the search function 182, the search result synthesis function 183, the marker information generation function 184, and the display control function 185. Here, for example, the acquisition function 181 which is a component of the processing circuit 180 shown in FIG. 1, the search function 182, the search result synthesis function 183, the marker information generation function 184, and the display control function 185 are executed. The processing function is recorded in the storage circuit 170 in the form of a program that can be executed by a computer. The processing circuit 180 is a processor that realizes a function corresponding to each program by reading each program from the storage circuit 170 and executing the program. In other words, the processing circuit 180 in the state where each program is read out has each function shown in the processing circuit 180 of FIG. The processing of the acquisition function 181 and the search function 182, the search result synthesis function 183, the marker information generation function 184, and the display control function 185 will be described later.

The transmission circuit 110, the reception circuit 120, the B mode processing circuit 130, the Doppler processing circuit 140, the image generation circuit 150, and the processing circuit 180 built in the apparatus main body 100 include a processor (CPU (Central Processing Unit)) and an MPU (MPU). Micro-Processing Unit), integrated circuit, etc.) hardware.

The ultrasonic diagnostic apparatus 1 configured in this way displays an ultrasonic image generated by ultrasonic scanning in real time and an ultrasonic image generated by past ultrasonic scanning. By the way, there is a CAD (Computer Aided Detection) function that automatically detects a characteristic part in the acquired biological image. Here, as an example of the CAD method, a method of performing a plurality of CAD processes on one input image, synthesizing the results of each process, and displaying it as a final detection result of a feature portion can be considered.

However, in this method, the amount of calculation for one image is large and the processing time becomes long. Therefore, this method is not suitable for real-time processing in, for example, an ultrasonic diagnostic apparatus. Further, if the CAD processing per image is simply reduced, the CAD performance may be deteriorated.

Therefore, the ultrasonic diagnostic apparatus 1 according to the first embodiment disperses the CAD process into a plurality of input images, synthesizes the results of each image, and displays it as the final detection result of one image. .. FIG. 2 is a diagram for explaining the first embodiment. FIG. 2 shows a case where four types of CAD processing are distributed over a four-frame image.

In the example shown in FIG. 2, the detection result of the image of the Nth frame is acquired by using the image of the N-2nd frame, the image of the N-1th frame, the image of the Nth frame, and the image of the N + 1th frame. Will be described. For example, the ultrasonic diagnostic apparatus 1 according to the first embodiment has a method for the image of the N-2nd frame, the image of the N-1th frame, the image of the Nth frame, and the image of the N + 1th frame, respectively. Perform different CAD processing.

More specifically, the ultrasonic diagnostic apparatus 1 according to the first embodiment executes CAD processing using the method A on the image of the N-2nd frame, and the CAD processing result "Result N-2". To get. Further, the ultrasonic diagnostic apparatus 1 according to the first embodiment executes CAD processing using the method B on the image of the N-1th frame to obtain a CAD processing result “Result N-1”. Further, the ultrasonic diagnostic apparatus 1 according to the first embodiment executes CAD processing using the method C on the image at the Nth frame to obtain a CAD processing result “Result N”. Then, the ultrasonic diagnostic apparatus 1 according to the first embodiment executes CAD processing using the method D on the image at the N + 1th frame to obtain the CAD processing result “Result N + 1”.

In each CAD process, the ultrasonic diagnostic apparatus 1 according to the first embodiment searches for a search window, teacher data, and search, which are set for each CAD process, for example, with respect to a search range set by the operator. Using the algorithm as a parameter, the search process of the feature part in the search range is executed. More specifically, the ultrasonic diagnostic apparatus 1 according to the first embodiment moves the search window within the search range, and analyzes the image information of the search window at each position by using the teacher data and the search algorithm. By doing so, the probability that the image information of each search window is the information corresponding to the feature portion is calculated. The ultrasonic diagnostic apparatus 1 according to the first embodiment may calculate another index indicating the degree of similarity instead of the probability. Further, the search window is, for example, a unit of an area within the search range to be compared with the teacher data, the teacher data is, for example, image information that is a sample of a feature part, and the search algorithm is the image information of the search window. It is an algorithm for calculating the similarity between and the teacher data. The teacher data was obtained as a result of performing machine learning based on, for example, the image information of the feature part in the medical image acquired in the past, or the image information of the feature part in each of a plurality of medical images acquired in the past. It may be information. Further, the search algorithm is an algorithm that searches for image information corresponding to a feature portion in an image to be processed by using, for example, a trained network by machine learning, and outputs a search result.

Subsequently, the ultrasonic diagnostic apparatus 1 according to the first embodiment synthesizes each CAD processing result. For example, the ultrasonic diagnostic apparatus 1 according to the first embodiment synthesizes "Result N-2", "Result N-1", "Result N" and "Result N + 1" to obtain an image of the Nth frame. Use as the detection result. Then, the ultrasonic diagnostic apparatus 1 according to the first embodiment displays the detection result of the Nth frame on the display 103, for example, as shown in FIG. In the example of FIG. 2, the ultrasonic diagnostic apparatus 1 according to the first embodiment superimposes marker information indicated by a rectangle on a region detected as a feature portion and displays it on the display 103.

Such analysis processing is realized by the processing circuit 180 executing the acquisition function 181 and the search function 182, the search result synthesis function 183, the marker information generation function 184, and the display control function 185. For example, the acquisition function 181 according to the first embodiment acquires a plurality of input images. In other words, the acquisition function 181 acquires a first image and a second image different from the first image.

The search function 182 according to the first embodiment executes different search processes for a plurality of input images. That is, the search function 182 searches for a feature portion in the first image by executing the first search process on the first image, and performs the first search process on the second image. Searches for a feature portion in the second image by executing a different second search process. Here, the search function 182 sets different parameters for the first search process and the second search process. More specifically, the search function 182 sets a plurality of search windows in the image to be searched as parameters, and calculates the probability of being a feature portion for each search window.

The search result synthesis function 183 according to the first embodiment synthesizes each search result by the search function 182. The marker information generation function 184 according to the first embodiment generates marker information based on the synthesis result by the search result synthesis function 183. The display control function 185 according to the first embodiment displays the medical image on the display 103, and displays the marker based on the marker information on the medical image.

FIG. 3 is a flowchart showing a procedure of analysis processing by the processing circuit 180 according to the first embodiment. FIG. 3 shows a flowchart explaining the operation of the processing circuit 180, and describes which step of the flowchart each component corresponds to. Step S1 and step S7 are steps realized by the acquisition function 181. In step S1, the acquisition function 181 acquires an image. For example, the acquisition function 181 acquires the latest image of the ultrasonic images generated in time series in real time from the image memory 160.

Step S2 and step S3 are steps realized by the search function 182. In step S2, the search function 182 sets the parameters. Here, the search function 182 sets different parameters at predetermined time intervals. In the example shown in FIG. 3, the search function 182 is described as setting different parameters each time the latest image is acquired. In other words, the search function 182 changes the parameters for each frame. The search function 182 may change the parameters by dividing the time, for example, every 5 seconds.

For example, the search function 182 sets a plurality of search windows in the image to be searched as a parameter. Here, the search function 182 sets a plurality of search windows in the image by the first method, the second method, or a method in which they are combined. 4, 5 and 6 are diagrams for explaining the first embodiment.

More specifically, in the first method, the search function 182 sets search windows having different sizes for the first search process and the second search process. FIG. 4 describes a case where the search window set for each frame is changed and three search windows having different sizes are set in the order of the search window 4a, the search window 4b, and the search window 4c.

For example, when the search function 182 sets the search window 4a for the image in the N-1th frame, the search function 182 sets the search window 4b for the image in the Nth frame. The search function 182 sets the search window 4c for the image in the N + 1th frame.

To give another example, in the first method, the search function 182 sets search windows having different relative sizes between the first search process and the second search process. That is, in the search function 182, the size of the search window relative to the first image in the first search process is different from the size of the search window relative to the second image in the second search process. , Set a search window for the first search process and the second search process. FIG. 5 describes a case where the size of the image is changed for each frame and three search windows having the same size are set in the order of the search window 5a, the search window 5b, and the search window 5c.

For example, the search function 182 sets the search window 5a for the image in the N-1th frame. Further, the search function 182 enlarges the image of the Nth frame from the image of the N-1th frame, and sets the search window 5b having the same size as the search window 5a with respect to the image of the Nth frame. Further, the search function 182 enlarges the image of the N + 1th frame more than the image of the N + 1th frame, and sets the search window 5a and the search window 5c having the same size as the search window 5b with respect to the image of the N + 1th frame.

Further, in the second method, the search function 182 sets the search window at different positions in the first search process and the second search process. FIG. 6 describes a case where the position of the search window set for each frame is changed by using the search windows of one type of size. In the second method, the search function 182 sets a search window of the same size in each frame.

For example, the search function 182 sets a search window with the upper left of the search range as the search start position and the lower right of the search range as the search end position for the image in the N-2th frame, and searches within the search range. Move the window. More specifically, the search function 182 superimposes the search window from the upper left to the right of the start position in the search range. For example, FIG. 6 shows a case where the search window is moved to the right so that the search windows are half overlapped. In FIG. 6, the superimposed search windows are shown by broken lines. Then, when the search window is moved to the right side, the search function 182 moves the search window to the lower side and moves the search window from the left side to the right side. In this way, the search function 182 sets the search window in the entire search range by moving the search window in the search range.

Further, for example, the search function 182 sets a search window as a search start position at a position shifted from the upper left to the lower right side of the search range with respect to the image in the N-1th frame, and from the lower right of the search range. Move the search window within the search range with the position shifted to the upper left as the end position of the search. Then, the search function 182 sets the search window by moving the search window from the start position to the end position in the search range of the image in the N-1th frame. In this way, the set position of the search window in the N-1 frame deviates from the set value of the search window in the N-2 frame. As a result, in the N-1th frame, the search window can be set in an area different from that in the N-2th frame.

Then, for example, the search function 182 sets the search window as the search start position at a position shifted further to the lower right from the start position of the image in the N-1th frame with respect to the image in the Nth frame, and N -Move the search window within the search range with the position shifted further to the upper left from the end position of the image in the 1st frame as the end position of the search. Then, the search function 182 sets the search window by moving the search window from the start position to the end position in the search range of the image at the Nth frame. In this way, the set position of the search window in the Nth frame deviates from the set value of the search window in the N-1th frame. As a result, in the Nth frame, the search window can be set in an area different from that in the N-1th frame.

In the example shown in FIG. 6, a case where the combination of the search start position and the search end position is changed in the search range using a search window of one type has been described, but the embodiment is limited to this. It's not something. For example, either the start position or the end position of the search may be changed in the search range. Further, in addition to changing either the start position or the end position of the search, search windows having different sizes may be used.

In step S3, the search function 182 executes a CAD search. For example, when the search function 182 receives an instruction to execute the CAD process from the operator via the input device 102, the search function 182 calculates the probability of being a feature portion for each search window set in step S2. The characteristic site referred to here is, for example, an abnormal (malignant) site such as a tumor. The characteristic site may be a normal (benign) site. For example, a normal (benign) site is a hemangioma (benign) of the liver.

For example, the search function 182 calculates the probability that each search window is a feature part by using the teacher data and the search algorithm for the image for which the search window is set. Then, the search function 182 detects the range surrounded by the search window as the feature portion when the probability of being the feature portion is equal to or greater than a predetermined threshold value. The search function 182 stores the search result for the image in, for example, the storage circuit 170.

Step S4 is a step realized by the search result synthesis function 183. In step S4, the search result compositing function 183 executes the compositing process. For example, the search result synthesis function 183 synthesizes each search result by the search function 182. FIG. 7 is a flowchart showing a processing procedure by the search result synthesis function 183 according to the first embodiment, and FIG. 8 is a diagram for explaining a processing operation of the search result synthesis function 183 according to the first embodiment. Is.

FIG. 7 shows a flowchart illustrating the operation of the search result synthesis function 183. In step S11, the search result compositing function 183 acquires search results for a predetermined number of images. For example, as shown in FIG. 8, the search result composition function 183 obtains the search result for the image in the N-2nd frame, the search result for the image in the N-1th frame, and the search result for the image in the Nth frame. Obtained from the storage circuit 170. Note that FIG. 8 shows the results of searching using the search windows 8a and the search windows 8b having different sizes. Further, FIG. 8 describes a case where the search result of the Nth frame is synthesized by using the image of the N-2nd frame, the image of the N-1th frame, and the image of the Nth frame.

More specifically, in the example shown in FIG. 8, the search result composition function 183 has a feature portion detected in the search window 8a of the image in the N-2nd frame, and the search window of the image in the N-1th frame. The feature portion is detected in 8b, and the search result in which the feature portion is detected in the search window 8a of the image of the Nth frame is acquired. The feature portion T detected in the search window 8a shown in FIG. 8 is a correctly detected feature portion, and the feature portion detected in the search window 8b is an erroneously detected overdetected feature portion. It is assumed that.

In step S12, the search result composition function 183 extracts a search result common to all the images acquired in step 11. For example, in the example shown in FIG. 8, the feature portion T, which is a common search result in the search result in the N-2nd frame and the search result in the Nth frame, is detected. However, in the example shown in FIG. 8, the feature portion T is not detected in the search result of the N-1th frame. That is, in the example shown in FIG. 8, there is no common search result in the three frames. In such a case, the search result synthesis function 183 obtains the result of the synthesis process indicating that the feature portion was not detected in the search result of the Nth frame. As described above, the search result synthesis function 183 sets the feature portion common to the feature portion in the first image and the feature portion in the second image as the synthesis result. As a result, the search result synthesis function 183 can reduce over-detection.

In step S12 shown in FIG. 7, the search result synthesis function 183 may extract all the search results. In such a case, in the example shown in FIG. 8, the search result synthesis function 183 synthesizes the feature portion T detected in the search window 8a and the feature portion detected in the search window 8b as the search result of the Nth frame. As a result of. That is, the search result synthesis function 183 sets the feature portion in the first image and the feature portion in the second image as the synthesis result.

Return to FIG. In step S5, the marker information generation function 184 generates marker information. For example, the marker information generation function 184 generates marker information based on the synthesis result by the search result synthesis function 183. Here, the marker information has a shape corresponding to the search window. For example, the marker information generation function 184 generates marker information having the same shape as the search window. More specifically, the marker information generation function 184 generates marker information having the same shape as the search window shown in FIGS. 4 and 8.

Further, the marker information generation function 184 generates color-coded marker information according to the probability of being a feature portion. For example, the marker information generation function 184 sets the priority to "high" in the search window detected as a result of the synthesis process and the probability of being a feature portion is 80% or more, and generates, for example, red marker information. .. Further, for example, the marker information generation function 184 sets the priority to "medium" for the search window detected as a result of the synthesis processing that the probability of being a feature portion is 60% or more and less than 80%, for example, yellow. Generate marker information. Further, for example, the marker information generation function 184 sets the priority to "low" for the search window detected as a result of the synthesis processing that the probability of being a feature portion is 40% or more and less than 60%, for example, blue. Generate marker information.

In step S6, the display control function 185 causes the display 103 to display the result of the synthesis process. For example, the display control function 185 causes the medical image to be displayed on the display 103, and the marker based on the marker information is displayed on the medical image. The medical image may be the image of the N-2nd frame, the image of the N-1th frame, the image of the Nth frame shown in FIG. 8, or another medical image. Here, as a result of the synthesis process, a plurality of characteristic sites may be obtained. In such a case, a plurality of marker information is generated. Further, when a plurality of marker information is generated, the marker information may overlap. Therefore, the display control function 185 causes the display 103 to display only the outer frame in the shape of the marker information to be superimposed. 9 and 10 are diagrams for explaining the first embodiment.

FIG. 9 shows a case where three types of marker information according to the priority are generated (9a in FIG. 9). Further, FIG. 9 shows a case where nine types of four marker information are generated by combining different priorities (left side of arrows 9b to 9j in FIG. 9). As shown in FIG. 9, when a plurality of marker information is generated in one image, the display control function 185 displays only the outer frame in the shape of the superimposed marker information on the display 103 (9b to FIG. 9). To the right of the 9j arrow). Here, the display control function 185 displays, for example, only the outer frame of the shape of the marker information to be superimposed is color-coded with the color having the highest probability.

More specifically, the display control function 185 color-codes only the outer frame of the shape of the superimposed marker information with the marker information having the highest priority. As an example, when the display control function 185 includes marker information having a priority of "high" among the four marker information, the priority is "" in the outer frame of the shape of the marker information. It is indicated by "red" which is "high" (9b, 9c, 9d, 9g, 9j in FIG. 9). Further, the display control function 185 sets the outer frame of the shape of the marker information to "yellow", which has a priority of "medium", when the marker information having the highest priority among the four marker information is "medium". It is shown (9e, 9f, 9i of FIG. 9). Further, when the marker information having the highest priority among the four marker information is "low", the display control function 185 sets the outer frame of the shape of the marker information to "blue" having the priority "low". It is shown (9h in FIG. 9).

In this way, the display control function 185 causes the display 103 to display the result of the compositing process for each frame. That is, the display control function 185 does not display the marker information in which a plurality of priorities are mixed, but selects the marker information having the highest priority and displays the outer frame in the shape of the marker information. In this way, the display control function 185 causes the display 103 to display the result of the synthesis process with less complexity. This allows, for example, the operator to view the results of the color-coded synthesis process according to the probability of being a featured portion. As a result, the operator can easily grasp the result of the CAD process, for example.

The marker information generation function 184 may generate marker information without color-coding according to the probability of being a feature portion. In such a case, the display control function 185 displays only the outer frame in the shape of the marker information to be superimposed. Further, the display control function 185 may display the marker information in which a plurality of priorities are mixed without selecting the marker information having the highest priority.

Further, the display control function 185 may further display the measured value of the number of marker information on the display 103. In FIG. 10, when 15 marker information having a priority of “high”, 0 marker information having a priority of “medium”, and 0 marker information having a priority of “low” are displayed. Is shown. In such a case, as shown in FIG. 10, the display control function 185 corresponds to the marker information indicating that the priority is “high” 15 and corresponds to the marker information indicating that the priority is “medium”. Add 0, and display 0 in association with the marker information indicating that the priority is "low". As a result, for example, the operator can easily grasp the number of feature parts according to the probability of being a feature part.

Return to FIG. In step S7, the acquisition function 181 determines whether or not the end has been accepted. Here, when the acquisition function 181 does not determine that the end has been accepted (step S7, No), the acquisition function 181 proceeds to step S1 to acquire an image. On the other hand, when it is determined that the acquisition function 181 has accepted the end (step S7, Yes), the acquisition function 181 ends the process.

As described above, in the ultrasonic diagnostic apparatus 1 according to the first embodiment, by executing the first search process on the first image, the feature portion in the first image is searched, and the second image is searched. By executing a second search process different from the first search process on the image, the feature portion in the second image is searched. Then, the ultrasonic diagnostic apparatus 1 according to the first embodiment synthesizes each search result. That is, the ultrasonic diagnostic apparatus 1 according to the first embodiment distributes the CAD process to a plurality of input images, synthesizes the results of each image, and displays it as the final detection result of one image. .. As a result, the ultrasonic diagnostic apparatus 1 according to the first embodiment can reduce the amount of calculation per image while maintaining the performance of CAD processing. As a result, according to the ultrasonic diagnostic apparatus 1 according to the first embodiment, CAD processing can be performed while maintaining real-time performance during ultrasonic scanning. In this way, the ultrasonic diagnostic apparatus 1 according to the first embodiment can improve the diagnostic efficiency.

(First modification of the first embodiment)
In the first embodiment described above, the case where the search result synthesis function 183 uses a feature portion common to the feature portion in the first image and the feature portion in the second image as the synthesis result has been described. However, the synthesis process by the search result synthesis function 183 is not limited to this. For example, the search result synthesis function 183 may copy the feature portion in the first image to the second image. FIG. 11 is a flowchart showing a processing procedure by the search result synthesis function 183 according to the first modification of the first embodiment, and FIGS. 12 and 13 show the first modification of the first embodiment. It is a figure for demonstrating the processing operation of the said search result synthesis function 183.

FIG. 11 shows a flowchart illustrating the operation of the search result synthesis function 183. In step S21, the search result compositing function 183 acquires search results for a predetermined number of images. For example, as shown in FIG. 12, the search result composition function 183 obtains the search result for the image in the N-2nd frame, the search result for the image in the N-1th frame, and the search result for the image in the Nth frame. Obtained from the storage circuit 170. Note that FIG. 12 shows the results of a search using the search windows 12a and the search windows 12b having different sizes. Further, FIG. 12 describes a case where the search result of the Nth frame is synthesized by using the image of the N-2nd frame, the image of the N-1th frame, and the image of the Nth frame.

More specifically, in the example shown in FIG. 12, the search result composition function 183 has a feature portion detected in the search window 12a of the image in the N-2nd frame, and the search window of the image in the N-1th frame. The feature portion is detected in 12b, and the search result in which the feature portion is detected in the search window 12a of the image of the Nth frame is acquired. The feature portion T detected in the search window 12a shown in FIG. 12 is a correctly detected feature portion, and the feature portion detected in the search window 12b is an erroneously detected overdetected feature portion. It is assumed that.

In step S22, the search result compositing function 183 copies the search results between the images. In the example shown in FIG. 12, a case where the search result one frame before is copied will be described. For example, the search result composition function 183 copies the search result in the search window 12a of the image in the N-2nd frame to the image in the N-1th frame. Further, for example, the search result composition function 183 copies the search result in the search window 12b of the image in the N-1th frame to the image in the Nth frame.

In step S23, the search result composition function 183 extracts a search result common to all the images. For example, in the example shown in FIG. 13, the feature portion T, which is a common search result in the search result in the N-2nd frame, the search result in the N-1th frame, and the search result in the Nth frame, is detected. In such a case, the search result synthesis function 183 obtains the result of the synthesis process indicating that the feature portion T is detected in the search result of the Nth frame. As described above, the search result synthesis function 183 copies the feature portion in the first image to the second image, and the feature portion common to the feature portion in the first image and the feature portion in the second image. Is the synthesis result.

In this way, in the first modification of the first embodiment, the feature portion in the first image is copied to the second image, and the feature portion in the first image and the feature portion in the second image are copied. The characteristic sites common to and are used as the synthesis result. As a result, according to the first modification of the first embodiment, it is possible to reduce over-detection and reduce oversight of featured portions.

The search result composition function 183 may copy the search results of the frames before or after a plurality of frames, or may copy the search results for a plurality of frames. For example, the search result composition function 183 may copy the search result two frames before, or may copy the search result of the N-2nd frame to the N-1th frame and the Nth frame.

(Second variant of the first embodiment)
Further, for example, the search result synthesis function 183 sets the feature portion in which the statistical value of the probability of being the feature portion in the first image and the probability of being the feature portion in the second image is equal to or more than a predetermined threshold value as the synthesis result. You may try to do it. FIG. 14 is a flowchart showing a processing procedure by the search result synthesis function 183 according to the second modification of the first embodiment, and FIG. 15 is a search result according to the second modification of the first embodiment. It is a figure for demonstrating the processing operation of the synthesis function 183.

FIG. 14 shows a flowchart illustrating the operation of the search result synthesis function 183. In step S31, the search result compositing function 183 acquires search results for a predetermined number of images. For example, as shown in FIG. 15, the search result composition function 183 includes search results for the image in the N-2nd frame, search results for the image in the N-1th frame, and search results for the image in the Nth frame. The search result for the image in the N + 1th frame is acquired from the storage circuit 170. Note that FIG. 15 shows the results of a search using the search windows 15a and the search windows 15b having different sizes. Further, in FIG. 15, the search result of the Nth frame is synthesized by using the image of the N-2nd frame, the image of the N-1th frame, the image of the Nth frame, and the image of the N + 1th frame. The case will be described.

In step S32, the search result synthesis function 183 calculates the average value of the probabilities that the image information in the search window is the information corresponding to the feature portion in the search window of the same size. For example, in the search result synthesis function 183, as shown in FIG. 15, a feature portion is detected in the search window 15a of the image in the N-2nd frame, and the feature portion is detected in the search window 15b of the image in the N-1th frame. Is detected, a feature portion is detected in the search window 15a of the image in the Nth frame, and nothing is detected in the image in the N + 1th frame. The feature portion T detected in the search window 15a shown in FIG. 15 is a correctly detected feature portion, and the feature portion detected in the search window 15b is an erroneously detected overdetected feature portion. It is assumed that. Further, the search window 15b in the N + 1th frame indicates a position corresponding to the detection location in the N-1th frame.

Then, the search result synthesis function 183 calculates the average value of the probability of being a feature portion in the search window 15a of the image in the N-2nd frame and the probability of being a feature portion in the search window 15a of the image in the Nth frame. .. Further, for example, as shown in FIG. 15, the search result composition function 183 has a probability of being a feature portion in the search window 15b of the image of the N-1th frame and the N-1th frame in the image of the N + 1th frame. The average value with the probability of being a featured part in the search window 15b at the position corresponding to the part where the featured part is detected in the image is calculated. Although the search result synthesis function 183 has described the case where the average value is calculated as a statistical value, the embodiment is not limited to this. For example, the search result synthesis function 183 may calculate a variance value as a statistical value.

In step S33, the search result synthesis function 183 determines whether or not the average value calculated in step S32 is equal to or greater than the threshold value. Here, when the search result synthesis function 183 determines that the average value is equal to or greater than the threshold value (step S33, Yes), the feature portion searched by the search window is extracted as a search result (step S34). On the other hand, when the search result synthesis function 183 does not determine that the average value is equal to or greater than the threshold value (step S33, No), the feature portion searched by the search window is not extracted as the search result (step S35).

More specifically, in the example shown in FIG. 15, the search result composition function 183 has a probability of being a feature portion in the search window 15a of the image in the N-2nd frame and a feature in the search window 15a of the image in the Nth frame. It is determined that the average value with the probability of being a part is equal to or greater than the threshold value. Then, the search result synthesis function 183 extracts the feature portion in the search window 15a as the synthesis result of the N frame. Further, in the example shown in FIG. 15, the search result synthesis function 183 has a probability of being a feature portion in the search window 15b of the image in the N-1 frame and a probability of being a feature portion in the search window 15b of the image in the N + 1 frame. It is determined that the average value of and is less than the threshold value. Then, the search result synthesis function 183 does not extract the feature portion in the search window 15b as the synthesis result of the N frame.

Following step S34 or step S35, the search result compositing function 183 determines whether or not the search windows of all sizes have been determined (step S36). Here, when the search result synthesis function 183 does not determine that the search windows of all sizes have been determined (steps S36 and No), the process proceeds to step S32. On the other hand, when it is determined that the search result composition function 183 has determined the search windows of all sizes (step S36, Yes), the process ends.

In this way, in the second modification according to the first embodiment, the statistical values of the probability of being a feature part in the first image and the probability of being a feature part in the second image are equal to or greater than a predetermined threshold value. The characteristic site is the synthetic result. As a result, according to the second modification according to the first embodiment, over-detection can be reduced more accurately and the feature portion can be detected more accurately.

(Third variant of the first embodiment)
In the above-described embodiment, the case where the feature portion common to the feature portion in the first image and the feature portion in the second image is used as the synthesis result has been described. However, the embodiment is not limited to this. For example, the search result synthesis function 183 may use the part occupied by each feature part as the synthesis result when at least a part of the feature part in the first image and the feature part in the second image are common. good.

FIG. 16 is a diagram for explaining the processing operation of the search result synthesis function 183 according to the third modification of the first embodiment. FIG. 16 shows the result of searching by setting search windows of the same size at different positions for the image of each frame by the search function 182. For example, the search result composition function 183 first acquires the search result for the image in the N-2nd frame, the search result for the image in the N-1th frame, and the search result for the image in the Nth frame from the storage circuit 170. do.

Here, as shown by diagonal lines in FIG. 16, the feature portion in the image of the N-2nd frame, the feature portion in the image of the N-1th frame, and the feature portion in the image of the Nth frame are partially common. It may be extracted to do so. In such a case, the search result synthesis function 183 sets the portion occupied by the feature portion in each frame as the synthesis result. That is, the search result synthesis function 183 sets the feature portion in the image of the N-2nd frame, the feature portion in the image of the N-1th frame, and the feature portion in the image of the Nth frame as the synthesis result of the N frame. Extract. Further, the marker information generation function 184 generates marker information based on the synthesis result by the search result synthesis function 183.

Then, the display control function 185 displays a marker based on the marker information on the medical image. For example, when the marker information is generated based on the synthesis result of FIG. 16, the display control function 185 causes the display 103 to display only the outer frame of the shape of the superimposed marker information as shown in FIG. Note that FIG. 17 is a diagram for explaining the processing operation of the display control function 185 according to the third modification of the first embodiment.

On the other hand, as shown in FIGS. 18 and 19, the feature portion in the image of the N-2nd frame, the feature portion in the image of the N-1th frame, and the feature portion in the image of the Nth frame are It may not have a common part. In this case, the search result composition function 183 uses the feature portion in the image of the N-2nd frame, the feature portion in the image of the N-1th frame, and the feature portion in the image of the Nth frame as the synthesis result of the N frame. Do not extract. Therefore, according to the third modification according to the first embodiment, over-detection can be reduced even when the search windows are set at different positions in the first search process and the second search process. It is possible to reduce oversight of characteristic parts. 18 and 19 are diagrams for explaining the processing operation of the search result synthesis function 183 according to the third modification of the first embodiment.

Further, the search result synthesis function 183 copies the feature portion in the first image to the second image, and then at least a part of the feature portion in the first image and the feature portion in the second image are common. In some cases, the site occupied by each characteristic site may be used as the synthesis result. For example, the search result synthesis function 183 copies the feature portion in the image in the N-2 frame to the image in the N-1 frame in FIG. In such a case, the search result synthesis function 183 assumes that the feature portion in the image of the N-2nd frame, the feature portion in the image of the N-1th frame, and the feature portion in the image of the Nth frame have a common portion. , The part occupied by the characteristic part in each frame is taken as the synthesis result. That is, the search result compositing function 183 uses the feature portion in the image of the N-2 frame and the feature portion copied from the image of the N-2 frame to the image of the N-1 frame as the synthesis result of the N frame. , The feature portion in the image of the N-1th frame and the feature portion in the image of the Nth frame are extracted.

On the other hand, in FIG. 19, when the feature portion in the image of the N-2nd frame is copied to the image of the N-1th frame, the feature portion of each frame has no common portion. In this case, the search result synthesis function 183 does not extract the feature portion in each frame as the synthesis result. Therefore, according to the third modification according to the first embodiment, when the search windows are set at different positions in the first search process and the second search process, over-detection is reduced. , It becomes possible to further reduce the oversight of characteristic parts.

Note that, in FIGS. 16, 18, and 19, search results are shown when search windows of the same size are set for the images of each frame, but the embodiment is not limited to this. .. For example, the search result compositing function 183 acquires search results when search windows of different sizes are set for the first image and the second image, and obtains a feature portion in the first image and a second image. A feature part that is at least partially common to the feature part in the image may be used as the synthesis result.

Further, the search result synthesis function 183 determines the portion occupied by each feature portion when at least a part of the feature portion in the first image and the feature portion in the second image are common according to the area of the common portion. It may be determined whether or not it is a synthesis result. For example, in FIG. 16, the search result composition function 183 calculates the ratio of the area of the search window set for the image at the Nth frame to the area of the common portion. Then, when the calculated ratio exceeds the threshold value, the search result synthesis function 183 determines the feature portion in the image of the N-2nd frame, the feature portion in the image of the N-1th frame, and the feature portion in the image of the N-1th frame as the synthesis result of the N frame. The featured part in the image of the Nth frame is extracted.

(Fourth variant of the first embodiment)
In the above-described embodiment, the display control function 185 has described the case where the result of the composition processing for each frame is displayed on the display 103, but the embodiment is not limited to this. For example, the display control function 185 may display the result of the composition processing for each frame on the display 103 in consideration of the result of the composition processing between consecutive frames. In other words, the display control function 185 causes the display 103 to display only the marker information that is continuous in a plurality of frames among the marker information. FIG. 20 is a diagram for explaining the processing operation of the display control function 185 according to the fourth modification of the first embodiment.

FIG. 20 shows a case where three types of marker information according to the priority are generated (20a in FIG. 20). Further, FIG. 20 shows a composition result of the image of the N-2nd frame, a composition result of the image of the N-1th frame, and a composition result of the image of the Nth frame. In the example shown in FIG. 20, the display control function 185 causes the display 103 to display only the marker information continuous in three frames. That is, the display control function 185 considers the composite result of the image of the N-2nd frame, the composite result of the image of the N-1th frame, and the composite result of the image of the Nth frame with respect to the Nth frame. The result of the synthesis process is displayed on the display 103.

More specifically, as shown in FIG. 20, when the display control function 185 has a frame in which the marker information is not generated in any of the three consecutive frames, the display control function 185 determines the result of the synthesis process for the Nth frame. The marker information is not displayed (20d, 20g, 20j in FIG. 20).

Further, when the marker information is generated in three consecutive frames, the display control function 185 color-codes the marker information having the highest priority as a result of the synthesis processing for the Nth frame. As an example, when the display control function 185 includes marker information having a priority of "high" in the marker information of three frames, the priority is set to the outer frame of the shape of the marker information. It is indicated by "red" which is "high" (20b, 20c in FIG. 20). Further, the display control function 185 sets the outer frame of the shape of the marker information to "yellow", which has a priority of "medium", when the marker information having the highest priority among the marker information of three frames is "medium". (20e, 20f, 20i in FIG. 20). Further, in the display control function 185, when the marker information having the highest priority among the marker information of 3 frames is "low", the outer frame of the shape of the marker information is "blue" having a priority of "low". (20h in FIG. 20).

In this way, in the fourth modification according to the first embodiment, the result of the compositing process for each frame is displayed on the display 103 in consideration of the result of the compositing process between consecutive frames. As a result, the feature portions detected over a plurality of frames can be preferentially displayed on the display 103, and the result of the synthesis process color-coded according to the probability of being the feature portions can be viewed.

(Fifth modification of the first embodiment)
In the above-described embodiment, the case where the display control function 185 displays the result of the synthesis processing of the search results on the medical image targeted for the search processing has been described. However, the embodiment is not limited to this. That is, the display control function 185 may display the result of the composition processing of the search result on the medical image other than the medical image targeted for the search processing. For example, the marker information generation function 184 generates marker information based on the result of the synthesis processing of the search results, and the display control function 185 uses the marker based on the marker information to obtain the image quality of the first image and the second image. Display on different medical images.

For example, the search function 182 acquires image data I1 based on SC data, as shown in FIG. Here, the SC data is, for example, B-mode image data generated by the image generation circuit 150. The image data I1 may be the B-mode image data itself, or may be the data after image processing on the B-mode image data. Next, as shown in FIG. 21, the search function 182 executes a search process (CAD process) on the image data I1. That is, the search function 182 executes the first search process on the first image in the image data I1 and executes the second search process on the second image in the image data I1. .. Note that FIG. 21 is a diagram for explaining the processing operation of the display control function 185 according to the fifth modification of the first embodiment.

For example, the image generation circuit 150 generates image data I1 by performing image processing on the B-mode image data so as to improve the accuracy of the search processing. Specifically, the image generation circuit 150 uses a moving average (smoothing) filter or Gaussian for the B-mode image data so that the image quality is similar to that of the medical image used when generating the teacher data. Image processing such as a filter, a median filter, and a band pass filter is performed to generate image data I1.

That is, in machine learning for generating teacher data, first, image information of a feature portion in a large number of medical images is collected as learning data. Here, it is conceivable to perform machine learning using the image information of the feature portion in the medical image of various image quality as learning data, but it is difficult to collect a large number of medical images of various image quality, and it is relatively relatively. The learning data may be biased due to the large number of collected medical images. Therefore, usually, machine learning is performed using the image information of the feature portion in the medical image that has been image-processed so as to have the same image quality as the learning data. Then, the image generation circuit 150 can improve the accuracy of the search process by the search function 182 by generating the image data I1 so that the image quality is similar to that of the medical image used for machine learning.

However, the image data I1 does not always have an image quality that is easy for the operator to observe. Therefore, the display control function 185 displays the result of the composition processing of the search results on the image data I2 whose image quality is different from that of the image data I1 which is the target of the search processing (CAD processing).

For example, the operator selects arbitrary image processing via the input device 102 so that he / she can easily observe the image. Next, the image generation circuit 150 generates the image data I2 by performing the selected image processing on the B mode image data. Further, the image generation circuit 150 generates image data I1 by performing image processing different from the selected image processing on the B mode image data. For example, the image generation circuit 150 generates image data I1 by performing image processing on the B-mode image data so that the image quality is similar to that of the medical image used for machine learning.

Although the case where the image data I1 and the image data I2 are generated by different image processing has been described, the embodiment is not limited to this. For example, the image generation circuit 150 generates either image data I1 or image data I2 by performing image processing on the B-mode image data, and generates the B-mode image data in a state where the image processing is not performed on the other. It may be used as the image data of. Further, for example, the image generation circuit 150 may generate B-mode image data as image data I1 and generate a superimposed image of B-mode image data and Doppler image data as image data I2.

Then, the display control function 185 displays the result of the synthesis processing of the search results on the image data I2 whose image quality is different from that of the image data I1. For example, the marker information generation function 184 generates marker information based on the result of the synthesis processing of the search results. Then, as shown in FIG. 21, the display control function 185 causes the display 103 to display the image data I3 in which the marker based on the marker information is superimposed on the image data I2 whose image quality is different from that of the image data I1.

The medical images used for machine learning are not limited to ultrasonic image data, and medical images (X-ray CT image, X-ray image, etc.) collected by a medical image diagnostic device other than the ultrasonic diagnostic device. It may be an MRI image or the like). For example, teacher data may be generated by performing machine learning using the image information of the feature portion in the X-ray CT image as learning data.

In this case, the image generation circuit 150 generates the image data I1 so that the image quality is similar to that of the X-ray CT image used for machine learning. For example, the image generation circuit 150 converts the signal of the B-mode image data for the display 103 into coordinates to generate SC data, and processes the SC data so that the image quality is comparable to that of the X-ray CT image. To execute. Specifically, the image generation circuit 150 emphasizes the edges of the SC data and executes image processing so as to make the particle size finer. As a result, the image generation circuit 150 can generate image data I1 having the same image quality as the X-ray CT image used for machine learning, and can improve the accuracy of the search process by the search function 182. Then, the display control function 185 displays the result of the composition processing of the search result on the image data I2 whose image quality is different from that of the image data I1 targeted for the search processing.

(Other modifications according to the first embodiment)
Further, in the first embodiment described above, the search function 182 has been described as setting a plurality of search windows in the image to be searched as a parameter, but the embodiment is not limited to this. .. For example, the search function 182 may set different teacher data as a parameter. In such a case, the search function 182 uses, for example, the same search window and the same search algorithm to change the teacher data at predetermined intervals and execute the CAD process.

Further, for example, the search function 182 may set a different search algorithm as a parameter. In such a case, the search function 182 executes the CAD process by changing the search algorithm at predetermined intervals using, for example, the same search window and the same teacher data.

Further, for example, the search function 182 may execute the CAD process by arbitrarily combining any one of the size of the search window, the position of the search window, the teacher data, and the search algorithm as parameters.

Further, in the search function 182, the scan speed (the speed at which the two-dimensional or three-dimensional scan area is mechanically fluctuated, the speed at which the scan area is electronically fluctuated, or the moving speed of the probe) is equal to or less than a predetermined threshold value. In some cases, change the parameters more frequently. Here, when the scanning speed is equal to or less than a predetermined threshold value, it is considered that the ultrasonic probe 101 is slowly moved in order for the operator to gaze at the area to be scanned. Therefore, for example, the search function 182 makes the size of the search window the same when the scan speed is greater than the predetermined threshold value, and sequentially changes the size of the search window when the scan speed is equal to or less than the predetermined threshold value. The search function 182 may set the frequency of changing the parameters by using a plurality of threshold values.

Further, the search function 182 increases the frequency of changing the parameters when the frame rate is equal to or higher than a predetermined threshold value. Here, when the frame rate is equal to or higher than a predetermined threshold value, the number of frames per unit time increases, so that the CAD processing can be easily distributed to a plurality of input images. Therefore, for example, the search function 182 makes the size of the search window the same when the frame rate is less than a predetermined threshold value, and sequentially changes the size of the search window when the frame rate is equal to or more than a predetermined threshold value. The search function 182 may set the frequency of changing the parameters by using a plurality of threshold values.

(Second embodiment)
In the first embodiment described above, a case where the ultrasonic diagnostic apparatus 1 executes CAD processing on an image generated in real time has been described. By the way, in the ultrasonic diagnostic apparatus 1, it may be desired to observe the result of executing the CAD process in real time after the ultrasonic scanning is completed. Therefore, in the second embodiment, a case of observing the result of executing the CAD process after the completion of the ultrasonic scanning will be described.

In the configuration example of the ultrasonic diagnostic apparatus according to the second embodiment, the ultrasonic diagnostic apparatus 1 according to the first embodiment shown in FIG. 1 except that the display control function 185 further executes an additional function. Is similar to. For example, the display control function 185 according to the second embodiment extracts images including marker information and displays them in a predetermined order.

FIG. 22 is a flowchart showing a procedure of display control processing by the display control function 185 according to the second embodiment, and FIG. 23 is a diagram for explaining the second embodiment. FIG. 22 shows a flowchart illustrating the operation of the display control function 185. Steps S41 to S45 are steps realized by the display control function 185.

In step S41, the display control function 185 acquires the synthesis result. For example, the display control function 185 acquires the synthesis result of the CAD processing of the image obtained by ultrasonic scanning from the storage circuit 170. More specifically, the display control function 185 acquires the synthesis results shown in FIGS. 23 (1) to 23 (9) from the storage circuit 170.

In step S42, the display control function 185 extracts an image in which the feature portion is detected. For example, the display control function 185 is characterized by (2), (4), (5), (6), (8) and (9) among the synthesis results of (1) to (9) shown in FIG. The part is extracted as an image in which it is detected.

In step S43, the display control function 185 determines whether or not the cine reproduction is accepted. Here, when it is not determined that the display control function 185 has received the instruction to execute the cine reproduction from the operator via the input device 102 (steps S43, No), the determination process of step S43 is repeated. On the other hand, when it is determined that the display control function 185 has received the instruction to execute the cine reproduction from the operator via the input device 102 (step S43, Yes), the process proceeds to step S44.

In step S44, the display control function 185 causes the display 103 to display images in a predetermined order. Here, for example, the display control function 185 may switch the images in a predetermined order and perform cine reproduction each time the image is switched from the operator, or automatically cine-reproduce the images in a predetermined order. May be good. More specifically, the display control function 185 determines the order in chronological order. In such a case, the display control function 185 causes the display 103 to display images in the order of (2), (4), (5), (6), (8), and (9) in the example shown in FIG.

Further, for example, the display control function 185 may determine a predetermined order according to the continuity. In such a case, the display control function 185 causes the display 103 to display images in the order of (4), (5), (6), (8), (9), and (2) in the example shown in FIG.

Further, for example, the display control function 185 may determine a predetermined order according to the number of marker information. In such a case, the display control function 185 causes the display 103 to display images in the order of (2), (8), (9), (4), (5), and (6) in the example shown in FIG.

Further, for example, the display control function 185 may determine a predetermined order according to the scanning speed. For example, if the scanning speed is slow, the operator may be scanning the area to be watched. Therefore, the display control function 185 preferentially displays a frame having a slower scanning speed on the display 103.

In step S45, the display control function 185 determines whether or not the end has been accepted. Here, when the display control function 185 does not determine that the end has been accepted (step S45, No), the display control function 185 proceeds to step S43 and executes the determination process. On the other hand, when it is determined that the end is accepted (step S45, Yes), the display control function 185 ends the process.

As described above, in the second embodiment, the images including the marker information are extracted and displayed in a predetermined order. As a result, according to the second embodiment, the frames in which the marker information appears are browsed in chronological order, and the frames in which the marker information appears continuously and the frames in which the marker information is abundant are preferentially browsed. Becomes possible.

In the second embodiment described above, the display control function 185 has been described as extracting an image in which a feature portion is detected, but the embodiment is not limited to this. For example, the display control function 185 causes the composite result acquired in step S41 to be cine-reproduced in the order of (1) to 23 (9), for example, without extracting the image in which the feature portion is detected. You may.

(Modified example of the second embodiment)
In the second embodiment described above, a case where the CAD processing is distributed over a plurality of input images and the result of synthesizing the results of each image is reproduced in a cine manner has been described. However, the display control process described in the second embodiment can be applied even when the CAD process is not distributed to a plurality of input images. The configuration of the ultrasonic diagnostic apparatus according to the modified example of the second embodiment is the same as the configuration of the ultrasonic diagnostic apparatus 1 shown in FIG. 1, except that some of the functions executed by the search function 182 are different. ..

That is, the acquisition function 181 according to the modified example of the second embodiment acquires the first image and the second image different from the first image. Then, the search function 182 according to the modified example of the second embodiment searches for the feature portion by executing the search process on the first image and the second image. That is, the search function 182 according to the modified example of the second embodiment executes the same search process on the first image and the second image without distributing the CAD process to a plurality of input images.

The search result synthesis function 183 according to the modified example of the second embodiment synthesizes each search result by the search function 182. The marker information generation function 184 according to the modified example of the second embodiment generates marker information based on the synthesis result by the search result synthesis function 183. The display control function 185 according to the modified example of the second embodiment displays the medical image on the display 103, and displays the marker based on the marker information on the medical image.

In such a case, the marker information generation function 184 according to the modified example of the second embodiment has the same function as that of the first embodiment described above. That is, the marker information generation function 184 according to the modified example of the second embodiment generates color-coded marker information according to the probability of being a feature portion. Here, the marker information has a shape corresponding to the search window. Then, the display control function 185 according to the modified example of the second embodiment displays only the outer frame of the shape. Here, the display control function 185 displays only the outer frame of the shape by color-coding with the color having the highest probability. Further, the display control function 185 further displays the measured value of the number of marker information. Further, the display control function 185 may display the marker based on the marker information on the first image and the medical image whose image quality is different from that of the second image.

Further, the display control function 185 according to the modified example of the second embodiment extracts images including marker information and displays them in a predetermined order. For example, the display control function 185 determines a predetermined order according to the continuity. Alternatively, the display control function 185 determines a predetermined order according to the number of marker information.

As a result, in the modified example according to the second embodiment, the display control process can be executed even when the CAD process is not distributed to a plurality of input images. As a result, for example, the operator can view the result of the synthesis process color-coded according to the probability of being a feature portion. Further, for example, the operator can easily grasp the number of feature parts according to the probability of being a feature part. Further, for example, the operator can browse the frames in which the marker information appears in chronological order, or preferentially browse the frames in which the marker information appears continuously or the frames in which the marker information is abundant. .. In this way, according to the modified example according to the second embodiment, the diagnostic efficiency can be improved.

(Other embodiments)
The embodiment is not limited to the above-described embodiment.

In the above-described embodiment, the ultrasonic diagnostic apparatus has been described as an example of the analysis apparatus, but the embodiment is not limited thereto. For example, a medical image processing device may be used as an analysis device. FIG. 24 is a block diagram showing a configuration example of the medical image processing apparatus 200 according to another embodiment.

As shown in FIG. 24, the medical image processing device 200 includes an input device 201, a display 202, a storage circuit 210, and a processing circuit 220. The input device 201, the display 202, the storage circuit 210, and the processing circuit 220 are connected to each other so as to be able to communicate with each other.

The input device 201 has a mouse, a keyboard, a button, a panel switch, a touch command screen, a foot switch, a trackball, a joystick, and the like, and receives various setting requests from the operator of the medical image processing device 200. The input device 201 outputs the received various setting requests to the processing circuit 220.

The display 202 displays, for example, medical image data generated by the ultrasonic diagnostic apparatus 1, or displays a GUI (Graphical User Interface) for an operator to input various setting requests using the input device 201. ..

The storage circuit 210 stores various programs for displaying the GUI and information used by the programs. In addition, the storage circuit 210 stores medical image data generated by the ultrasonic diagnostic apparatus 1 and other medical image diagnostic apparatus.

The processing circuit 220 controls the entire processing of the medical image processing apparatus 200. For example, as shown in FIG. 24, the processing circuit 220 executes the acquisition function 221 and the search function 222, the search result synthesis function 223, the marker information generation function 224, and the display control function 225. Here, for example, each processing function executed by the acquisition function 221 and the search function 222, the search result synthesis function 223, the marker information generation function 224, and the display control function 225, which are the components of the processing circuit 220 shown in FIG. 24, is a computer. It is recorded in the storage circuit 210 in the form of a program that can be executed by. The processing circuit 220 is a processor that realizes a function corresponding to each program by reading each program from the storage circuit 210 and executing the program. In other words, the processing circuit 220 in the state where each program is read out has each function shown in the processing circuit 220 of FIG. 24.

The medical image processing device 200 according to the other embodiment acquires, for example, an ultrasonic image generated by the ultrasonic diagnostic device 1, distributes the CAD processing to a plurality of input images, and synthesizes the results of each image. It is displayed as the final detection result of one image. Such analysis processing is realized by the processing circuit 220 executing the acquisition function 221 and the search function 222, the search result synthesis function 223, the marker information generation function 224, and the display control function 225.

The acquisition function 221 executes the same function as the acquisition function 181 according to the first embodiment and the second embodiment. The search function 222 executes the same function as the search function 182 according to the first embodiment and the second embodiment. The search result synthesis function 223 executes the same function as the search result synthesis function 183 according to the first embodiment and the second embodiment. The marker information generation function 224 executes the same function as the marker information generation function 184 according to the first embodiment and the second embodiment. The display control function 225 executes the same function as the display control function 185 according to the first embodiment and the second embodiment.

As described above, the medical image processing apparatus 200 according to the other embodiment searches for the feature portion in the first image by executing the first search process on the first image, and searches for the feature portion in the first image, and the second image. By executing a second search process different from the first search process, the feature portion in the second image is searched for. Then, the medical image processing apparatus 200 according to the other embodiment synthesizes each search result. As a result, the medical image processing apparatus 200 according to the other embodiment can reduce the amount of calculation per image while maintaining the performance of CAD processing.

Further, the medical image processing apparatus 200 according to another embodiment extracts images including marker information and displays them in a predetermined order. As a result, according to the medical image processing apparatus 200 according to the other embodiment, the frames in which the marker information appears are browsed in chronological order, and the frames in which the marker information appears continuously and the frames in which the marker information is abundant are prioritized. It will be possible to browse to. In this way, according to the medical image processing apparatus 200 according to the other embodiment, the diagnostic efficiency can be improved.

In the above-described embodiment, the first image and the second image are described as images having different generation times (frames), but the present invention is not limited to this. For example, the first image and the second image are images of different regions acquired at the same time, images of the same subject acquired at different shooting dates and times (ultrasonic image and X-ray CT image, ultrasonic image and X-ray). Images, ultrasound images and MRI images, or X-ray CT images and MRI images, images of different subjects, different imaging conditions (ultrasonography frequency in ultrasound diagnostic equipment, ultrasound sound pressure, different probes, etc.) , X-ray energy in X-ray CT equipment and X-ray diagnostic equipment, magnetic field strength in magnetic resonance imaging equipment, etc.), images with different image processing, different images acquired by the same medical diagnostic equipment It may be a type of image (B-mode image, contrast image, elastography image, high-definition mode image, etc.).

The term "processor" used in the above description refers to, for example, a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), an application specific integrated circuit (ASIC), or a programmable logic device (for example, an application specific integrated circuit). It means a circuit such as a simple programmable logic device (SPLD), a complex programmable logic device (CPLD), and a field programmable gate array (FPGA). The processor realizes the function by reading and executing the program stored in the storage circuit. Instead of storing the program in the storage circuit, the program may be directly embedded in the circuit of the processor. In this case, the processor realizes the function by reading and executing the program embedded in the circuit. It should be noted that each processor of the present embodiment is not limited to the case where each processor is configured as a single circuit, and a plurality of independent circuits may be combined to form one processor to realize its function. good. Further, the plurality of components in FIG. 1 may be integrated into one processor to realize the function.

In the description of the above-described embodiment, each component of each of the illustrated devices is a functional concept, and does not necessarily have to be physically configured as shown in the figure. That is, the specific form of distribution / integration of each device is not limited to the one shown in the figure, and all or part of the device is functionally or physically dispersed / physically distributed in arbitrary units according to various loads and usage conditions. Can be integrated and configured. Further, each processing function performed by each device may be realized by a CPU and a program analyzed and executed by the CPU, or may be realized as hardware by wired logic.

Further, the image analysis method described in the above embodiment can be realized by executing an image analysis program prepared in advance on a computer such as a personal computer or a workstation. This image analysis program can be distributed via a network such as the Internet. Further, this image analysis program can also be executed by being recorded on a computer-readable recording medium such as a hard disk, flexible disk (FD), CD-ROM, MO, or DVD, and being read from the recording medium by the computer. ..

According to at least one embodiment described above, the diagnostic efficiency can be improved.

Although some embodiments of the present invention have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These embodiments can be implemented in various other forms, and various omissions, replacements, and changes can be made without departing from the gist of the invention. These embodiments and modifications thereof are included in the scope and gist of the invention, as well as in the scope of the invention described in the claims and the equivalent scope thereof.

1 Ultrasonic diagnostic device 100 Device body 180 Processing circuit 181 Acquisition function 182 Search function 183 Search result synthesis function 184 Marker information generation function 185 Display control function

Claims (26)

The first image and the second image, which are time-series images acquired from the same part of the same subject by the same medical image diagnostic apparatus and are the latest images among the images generated in time-series in real time. And the acquisition department that sequentially acquires
By executing the first search process on the newly acquired first image, the feature portion in the first image is searched, and the newly acquired second image is targeted. By executing a second search process different from the first search process, a search unit that searches for a feature portion in the second image, and a search unit.
A search result synthesis unit that synthesizes each search result by the search unit,
Analytical device equipped with. The analysis device according to claim 1, wherein the search unit sets different parameters for the first search process and the second search process. The analysis device according to claim 2, wherein the search unit sets a plurality of search windows in the image to be searched as the parameters, and calculates the probability of being a feature portion for each search window. The analysis device according to claim 3, wherein the search unit sets search windows having different sizes in the first search process and the second search process. The search unit differs in the relative size of the search window in the first search process with respect to the first image and the relative size of the search window in the second search process with respect to the second image. The analysis device according to claim 3, wherein a search window for the first search process and the second search process is set as described above. The analysis device according to claim 3, wherein the search unit sets search windows at different positions in the first search process and the second search process. The analysis device according to claim 2, wherein the search unit sets teacher data as the parameter. The analysis device according to claim 2, wherein the search unit sets a search algorithm as the parameter. The analysis device according to any one of claims 2 to 8, wherein the search unit sets different parameters at predetermined time intervals. The analysis device according to any one of claims 2 to 8, wherein the search unit increases the frequency of changing the parameters when the scan speed is equal to or less than a predetermined threshold value. The analysis device according to any one of claims 2 to 8, wherein the search unit increases the frequency of changing the parameters when the frame rate is equal to or higher than a predetermined threshold value. The search result synthesizing unit according to any one of claims 1 to 11, wherein the search result synthesizing unit uses a feature portion common to the feature portion in the first image and the feature portion in the second image as a synthesis result. Analyst. The analysis device according to any one of claims 1 to 11, wherein the search result synthesizing unit has a feature portion in the first image and a feature portion in the second image as a synthesis result. The search result synthesis unit uses a feature site whose statistical value of the probability of being a feature site in the first image and the probability of being a feature site in the second image is equal to or higher than a predetermined threshold value as a synthesis result. The analyzer according to any one of claims 1 to 11. The claim that the search result synthesizing unit uses a part occupied by each feature part as a synthesis result when at least a part of the feature part in the first image and the feature part in the second image are common. The analyzer according to any one of 1 to 11. The analysis device according to any one of claims 12 to 15, wherein the search result synthesis unit copies a feature portion in the first image to the second image. A marker information generation unit that generates marker information based on the synthesis result of the search result synthesis unit,
A display control unit that displays a medical image on a display unit and displays a marker based on the marker information on the medical image.
The analysis apparatus according to any one of claims 1 to 16. The analysis device according to claim 17, wherein the marker information generation unit generates color-coded marker information according to the probability of being the featured portion. The search unit sets a plurality of search windows in the image to be searched, calculates the probability of being a feature part for each search window, and calculates the probability.
The marker information has a shape corresponding to the search window.
The analysis device according to claim 17, wherein the display control unit displays only the outer frame of the shape of the superimposed marker information. The marker information generation unit generates color-coded marker information according to the probability of being the feature portion.
The analysis device according to claim 19, wherein the display control unit displays only the outer frame of the shape of the superimposed marker information in the color having the highest probability. The display control unit according to any one of claims 17 to 20, wherein the display control unit displays a marker based on the marker information on the first image and the medical image whose image quality is different from that of the second image. Analyst. The analysis device according to claim 18, wherein the display control unit further displays a measured value of the number of marker information. The analysis device according to claim 18, wherein the display control unit extracts images including the marker information and displays them in a predetermined order. The analysis device according to claim 23, wherein the display control unit determines the predetermined order according to continuity. The analysis device according to claim 23, wherein the display control unit determines the predetermined order according to the number of marker information. The analysis device according to claim 23, wherein the display control unit determines the predetermined order according to the scanning speed.

Images