JP6309376B2 - Medical information processing system, medical information processing program, and ultrasonic diagnostic apparatus - Google Patents

Description

  Embodiments described herein relate generally to a medical information processing system, a medical information processing program, and an ultrasonic diagnostic apparatus.

  Conventionally, mammary gland image diagnosis performed by breast cancer screening or the like has been generally performed using a mammography image captured by a mammography apparatus. On the other hand, in recent years, a project called J-START (Japan Strategic Anti-cancer Randomized Trial) has been started, and mammary gland image diagnosis using both mammography images and ultrasound images has been started in breast cancer medical examinations. Yes.

Yoshio Seo, "Experience using the latest mammography system-Utilization of information input function during shooting (Exam-Marker)", Monthly Inner Vision, Vol. 27, no. 1, Inner Vision, January 2012 issue, p. 77 “Comparative study to verify the effectiveness of ultrasonography in J-START breast cancer checkup”, [May 8, 2013 search], Internet <URL: http://www.j-start.org/ >

  The problem to be solved by the present invention is to provide a medical information processing system, a medical information processing program, and an ultrasonic diagnostic apparatus that can improve the accuracy of breast image diagnosis.

  The medical information processing system according to the embodiment includes a storage unit, an area setting unit, a position specifying unit, and an output unit. The storage unit stores a mammography image obtained by imaging the breast of the subject and information indicating the imaging direction of the mammography image. The region setting unit sets a region of interest in the mammography image. The position specifying unit specifies the position information of the region of interest on the schematic diagram schematically representing the breast based on the position information of the region of interest on the mammography image and the information indicating the imaging direction. To do. The output unit outputs position information of the region of interest on the schematic diagram.

FIG. 1 is a diagram illustrating a configuration example of a medical information processing system according to the first embodiment. FIG. 2 is a diagram (1) illustrating a configuration example of the mammography apparatus according to the first embodiment. FIG. 3 is a diagram (2) illustrating a configuration example of the mammography apparatus according to the first embodiment. FIG. 4 is a diagram illustrating a configuration example of the ultrasonic diagnostic apparatus according to the first embodiment. FIG. 5 is a diagram for explaining a mammary gland image diagnosis using both a mammography image and an ultrasound image. FIG. 6 is a diagram illustrating a configuration example of the image processing apparatus according to the first embodiment. FIG. 7 is a diagram illustrating an example of a schematic diagram used by the position specifying unit according to the first embodiment. FIG. 8 is a diagram for explaining the specification of the position information of the region of interest on the schematic diagram performed by the position specifying unit according to the first embodiment. FIG. 9 is a diagram for explaining detection of a breast region in an MLO image performed by the position specifying unit according to the first embodiment. FIG. 10 is a diagram for explaining the specification of the position information of the region of interest on the schematic diagram performed by the position specifying unit according to the present embodiment. FIG. 11 is a diagram for describing an example of region conversion performed by the position specifying unit according to the first embodiment. FIG. 12 is a diagram illustrating an example of display information displayed by the image display apparatus according to the first embodiment. FIG. 13 is a flowchart illustrating a processing procedure of processing performed by the image processing apparatus according to the first embodiment. FIG. 14 is a diagram for explaining the specification of the position information of the region of interest on the schematic diagram performed by the position specifying unit according to the modification of the first embodiment. FIG. 15 is a diagram for explaining a relationship between a mammography image and a schematic diagram according to the first embodiment. FIG. 16 is a diagram illustrating a blind area in photographing an MLO image. FIG. 17 is a diagram illustrating a blind area in CC image capturing. FIG. 18 is a diagram (1) for explaining the display of the blind area performed by the position specifying unit according to the modification of the first embodiment. FIG. 19 is a diagram (2) for explaining the display of the blind area performed by the position specifying unit according to the modification of the first embodiment. FIG. 20 is a diagram (1) for explaining the specification of the position information of the region of interest on the schematic diagram according to the modification of the first embodiment. FIG. 21 is a diagram (2) for explaining the specification of the position information of the region of interest on the schematic diagram according to the modified example of the first embodiment. FIG. 22 is a diagram illustrating a configuration example of an image display apparatus according to the second embodiment. FIG. 23 is a diagram illustrating a detailed configuration example of the ultrasonic diagnostic apparatus according to the third embodiment. FIG. 24 is a diagram illustrating a configuration example of an image display device according to the fourth embodiment. FIG. 25 is a diagram illustrating an example of a reference screen displayed by the display control unit according to the fourth embodiment. FIG. 26 is a diagram (1) illustrating the extraction of the mammary gland parenchymal region performed by the extraction unit according to the fourth embodiment. FIG. 27 is a diagram (2) for explaining the mammary gland parenchymal region extraction performed by the extraction unit according to the fourth embodiment. FIG. 28 is a diagram for describing identification of a region of interest performed by the identifying unit according to the fourth embodiment. FIG. 29 is a diagram for explaining another example of specifying the region of interest performed by the specifying unit according to the fourth embodiment. FIG. 30 is a flowchart illustrating a processing procedure of processing performed by the image display device according to the fourth embodiment. FIG. 31 is a diagram illustrating a configuration example of an image processing device according to the fifth embodiment. FIG. 32 is a diagram illustrating a detailed configuration example of the ultrasonic diagnostic apparatus according to the sixth embodiment.

  Hereinafter, embodiments of a medical information processing system, a medical information processing program, and an ultrasonic diagnostic apparatus will be described with reference to the drawings.

(First embodiment)
FIG. 1 is a diagram illustrating a configuration example of a medical information processing system according to the first embodiment. The medical information processing system according to the present embodiment is installed in a hospital where breast cancer screening is performed, and is used for breast image diagnosis using both a mammography image and an ultrasound image. For example, as illustrated in FIG. 1, the medical information processing system 100 according to the present embodiment includes a mammography apparatus 10, an ultrasonic diagnostic apparatus 20, an image processing apparatus 30, and an image display apparatus 40. The devices are connected to each other via a network 50, and transmit and receive images captured by the mammography device 10 and the ultrasonic diagnostic device 20 to each other.

  The mammography apparatus 10 irradiates a subject's breast with X-rays, detects X-rays transmitted through the breast, and generates a mammography image.

  2 and 3 are diagrams illustrating a configuration example of the mammography apparatus 10 according to the first embodiment. For example, as shown in FIG. 2, the mammography apparatus 10 includes a base 11 and a stand 12. The stand 12 is erected on the base 11 and supports the imaging table 13, the compression plate 14, the X-ray output unit 15, and the X-ray detection unit 16. The imaging table 13, the compression plate 14, and the X-ray detection unit 16 are supported so as to be movable in the vertical direction.

  The imaging table 13 is a table that supports the breast B of the subject, and has a support surface 13a on which the breast B is placed. The compression plate 14 is disposed above the imaging table 13, and is provided so as to be opposed to the imaging table 13 in parallel and to move in a direction in which the imaging plate 13 is in contact with or separated from the imaging table 13. The compression plate 14 compresses the breast B supported on the imaging table 13 when it moves in a direction approaching the imaging table 13. The breast B compressed by the compression plate 14 is thinly spread and the overlap of the mammary glands within the breast B is reduced.

  As shown in FIG. 3, the mammography apparatus 10 includes an operation unit 17a, a lift drive unit 17b, a high voltage generation unit 17c, an image processing unit 17d, an image storage unit 17e, a display unit 17f, and a communication unit. A control unit 17g and a system control unit 17h are included. The operation unit 17a receives various command input operations from the operator. The raising / lowering drive part 17b is connected to the imaging stand 13, and raises / lowers the imaging stand 13 to an up-down direction. Further, the elevating drive unit 17b is connected to the compression plate 14 and raises and lowers the compression plate 14 in the vertical direction (direction in which the imaging plate 13 is in contact with and separated from).

  The X-ray output unit 15 includes an X-ray tube 15a and an X-ray restrictor 15b. The X-ray tube 15a generates X-rays. The X-ray restrictor 15b is disposed between the X-ray tube 15a and the compression plate 14, and controls the irradiation range of X-rays generated from the X-ray tube 15a. The high voltage generator 17c is connected to the X-ray tube 15a and supplies a high voltage for the X-ray tube 15a to generate X-rays.

  The X-ray detection unit 16 includes an X-ray detector 16a and a signal processing unit 16b. The X-ray detector 16a detects X-rays transmitted through the breast B and the imaging table 13 and converts them into electrical signals (transmitted X-ray data). The signal processing unit 16b generates X-ray projection data from the electrical signal converted by the X-ray detector 16a.

  The image processing unit 17d is connected to the signal processing unit 16b and the image storage unit 17e, generates a mammography image based on the X-ray projection data generated by the signal processing unit 16b, and generates the generated mammography image as an image storage unit. Save to 17e. The image processing unit 17d is connected to the display unit 17f and displays the generated mammography image on the display unit 17f. Note that the image processing unit 17d can switch the type of mammography image to be created based on an input operation from the operation unit 17a.

  The communication control unit 17g controls communication performed with other devices via the network 50. For example, the communication control unit 17g transfers the mammography image generated by the image processing unit 17d to another device via the network 50. The mammography image transferred via the network 50 can be subjected to image display or image processing in the transfer destination device.

  The system control unit 17h is connected to the operation unit 17a, the elevation drive unit 17b, the high voltage generation unit 17c, the X-ray diaphragm 15b, the image processing unit 17d, and the communication control unit 17g. Control the whole.

  Returning to FIG. 1, the ultrasound diagnostic apparatus 20 generates an ultrasound image based on the reflected wave data collected by scanning the subject with an ultrasound probe that transmits and receives ultrasound.

  FIG. 4 is a diagram illustrating a configuration example of the ultrasonic diagnostic apparatus 20 according to the first embodiment. As shown in FIG. 4, the ultrasonic diagnostic apparatus 20 according to the present embodiment includes an ultrasonic probe 21, a display unit 23, an input unit 22, and an apparatus main body 24.

  The ultrasonic probe 21 has a plurality of piezoelectric vibrators. The plurality of piezoelectric vibrators generate an ultrasonic pulse based on a drive signal supplied from a transmission / reception unit 241 included in the apparatus main body 24 described later, and receives a reflected wave from the subject P and converts it into an electrical signal. . The ultrasonic probe 21 includes a matching layer provided in the piezoelectric vibrator, a backing material that prevents propagation of ultrasonic waves from the piezoelectric vibrator to the rear, and the like.

  When an ultrasonic pulse is transmitted from the ultrasonic probe 21 to the subject P, the transmitted ultrasonic pulse is reflected one after another on the discontinuous surface of the acoustic impedance in the body tissue of the subject P, and is ultrasonicated as an echo signal. It is received by a plurality of piezoelectric vibrators possessed by the probe 21. The amplitude of the received echo signal depends on the difference in acoustic impedance at the discontinuous surface where the ultrasonic pulse is reflected. The echo signal when the transmitted ultrasonic pulse is reflected by the moving blood flow or the surface of the heart wall depends on the velocity component of the moving object in the ultrasonic transmission direction due to the Doppler effect. , Subject to frequency shift.

  The display unit 23 is a monitor or the like, and displays an GUI (Graphical User Interface) for an operator of the ultrasonic diagnostic apparatus 20 to input various instructions and setting requests using the input unit 22. The generated ultrasonic image and the analysis result are displayed.

  The input unit 22 is a mouse, a keyboard, a button, a panel switch, a touch command screen, a foot switch, a trackball, and the like, and is connected to the apparatus main body 24. The input unit 22 receives various instructions and setting requests from an operator of the ultrasonic diagnostic apparatus 20 and transfers the received various instructions and setting requests to the apparatus main body 24.

  The apparatus main body 24 generates an ultrasonic image based on the reflected wave received by the ultrasonic probe 21. As shown in FIG. 4, the apparatus main body 24 includes a transmission / reception unit 241, a B-mode processing unit 242, a Doppler processing unit 243, an image processing unit 244, an image memory 245, a control unit 248, and an internal storage unit 246. And have.

  The transmission / reception unit 241 includes a trigger generation circuit, a transmission delay circuit, a pulsar circuit, and the like, and supplies a drive signal to the ultrasonic probe 21. The pulsar circuit repeatedly generates a rate pulse for forming an ultrasonic pulse having a predetermined repetition frequency (PRF (Pulse Repetition Frequency)). The PRF is also called a rate frequency. Further, the transmission delay circuit generates a transmission delay time for each piezoelectric vibrator necessary for determining the transmission directivity by converging the ultrasonic pulse generated from the ultrasonic probe 21 into a beam shape. Give for each rate pulse. The trigger generation circuit applies a drive signal (drive pulse) to the ultrasonic probe 21 at a timing based on the rate pulse. That is, the transmission delay circuit arbitrarily adjusts the transmission direction from the piezoelectric vibrator surface by changing the transmission delay time given to each rate pulse.

  The transmission / reception unit 241 has a function capable of instantaneously changing a transmission frequency, a transmission drive voltage, and the like in order to execute a predetermined scan sequence based on an instruction from the control unit 248 described later. In particular, the change of the transmission drive voltage is realized by a linear amplifier type transmission circuit capable of instantaneously switching the value or a mechanism for electrically switching a plurality of power supply units.

  The transmission / reception unit 241 includes an amplifier circuit, an A / D (Analog / Digital) converter, a reception delay circuit, an adder, a quadrature detection circuit, and the like, and various types of reflected wave signals received by the ultrasonic probe 21. Processing is performed to generate reflected wave data. The amplifier circuit amplifies the reflected wave signal for each channel and performs gain correction processing. The A / D converter A / D converts the reflected wave signal whose gain is corrected. The reception delay circuit gives a reception delay time necessary for determining the reception directivity to the digital data. The adder performs addition processing of the reflected wave signal given the reception delay time by the reception delay circuit. By the addition processing of the adder, the reflection component from the direction corresponding to the reception directivity of the reflected wave signal is emphasized.

  The B-mode processing unit 242 receives the reflected wave data from the transmission / reception unit 241 and performs logarithmic amplification, envelope detection processing, and the like to generate data (B-mode data) in which the signal intensity is expressed by brightness. Further, the B mode processing unit 242 generates M mode data to be described later.

  The Doppler processing unit 243 performs frequency analysis on velocity information from the reflected wave data received from the transmission / reception unit 241, extracts blood flow, tissue, and contrast agent echo components due to the Doppler effect, and moving body information such as average velocity, dispersion, and power Is generated for multiple points (Doppler data).

  The image processing unit 244 generates an ultrasound image from the B mode data and M mode data generated by the B mode processing unit 242 and the Doppler data generated by the Doppler processing unit 243. Specifically, the image processing unit 244 generates a B mode image from the B mode data, generates an M mode image from the M mode data, and generates a Doppler image from the Doppler data. Further, the image processing unit 244 converts (scan converts) the scanning line signal sequence of the ultrasonic scan into the scanning line signal sequence of the video format represented by the television or the like by performing coordinate conversion, data interpolation, and the like, and displays them. An ultrasonic image (B mode image, M mode image, Doppler image) as an image is generated.

  The image memory 245 is a memory that stores an ultrasonic image generated by the image processing unit 244 and an image generated by performing image processing on the ultrasonic image. For example, after diagnosis, the operator can call an image recorded during the examination, and can be reproduced as a still image or a moving image using a plurality of images. The image memory 245 may store an image luminance signal after passing through the transmission / reception unit 241, other raw data, image data acquired via the network 50, and the like.

  The internal storage unit 246 stores device control programs for performing ultrasonic transmission / reception, image processing, and display processing, diagnostic information (for example, patient ID, doctor's findings, etc.), various data such as diagnostic protocols and various setting information, and the like. Remember. The internal storage unit 246 may be used for storing images stored in the image memory 245.

  The communication control unit 247 controls communication performed with other devices via the network 50. For example, the communication control unit 247 transfers the ultrasonic image generated by the image processing unit 244 to another device via the network 50. The ultrasonic image transferred via the network 50 can be subjected to image display or image processing in the transfer destination apparatus.

  The control unit 248 controls the entire processing in the ultrasonic diagnostic apparatus 20. Specifically, the control unit 248 is based on various instructions and setting requests input from the operator via the input unit 22, various programs and various setting information read from the internal storage unit 246, and the transmission / reception unit 241 and the B mode. It controls the processing of the processing unit 242, the Doppler processing unit 243, and the image processing unit 244, and controls the display unit 23 to display an ultrasonic image stored in the image memory 245.

  Returning to FIG. 1, the image processing apparatus 30 processes a mammography image generated by the mammography apparatus 10, an ultrasonic image generated by the ultrasonic diagnostic apparatus 20, and the like. The image processing apparatus 30 is mainly used when a mammography examination is performed by a mammography examination engineer. Further, the image processing apparatus 30 receives an input of a finding regarding a mammography image from a mammography examination engineer, and stores information indicating the received finding as finding information. For example, the image processing apparatus 30 is an image storage server or a workstation.

  The image display device 40 acquires and displays mammography images, ultrasonic images, findings information about the mammography images, and the like from the image processing device 30. The image display device 40 is mainly used when an ultrasonic inspection is performed by an ultrasonic inspection engineer. For example, the image display device 40 is a tablet terminal that can be carried by an operator and can be connected to the network 50 via a wireless local area network (LAN). The image display device 40 may be a notebook personal computer, for example.

  Here, conventionally, mammary gland image diagnosis performed by breast cancer screening or the like has been generally performed using a mammography image captured by a mammography apparatus. On the other hand, in recent years, a project called J-START (Japan Strategic Anti-cancer Randomized Trial) has been started, and mammary gland image diagnosis using both mammography images and ultrasound images has been started in breast cancer medical examinations. Yes.

  FIG. 5 is a diagram for explaining a mammary gland image diagnosis using both a mammography image and an ultrasound image. For example, as shown on the left side of FIG. 5, in mammary gland image diagnosis using both a mammography image and an ultrasound image, first, using a mammography apparatus, MLO (Mediolateral-Oblique: Mammographic images in the (inner / outer oblique) direction and CC (Cranio-Caudal) direction are captured. Thereafter, as shown on the right side of FIG. 5, on the same day or another day, referring to the mammography image and its findings, the left and right ultrasound images of the same subject are imaged using the ultrasound diagnostic apparatus. .

  In such mammary gland image diagnosis, in general, an engineer who captures and interprets a mammography image is often different from an engineer who captures and interprets an ultrasound image. Therefore, conventionally, an engineer in charge of ultrasonic examination has been required to have sufficient knowledge and understanding about interpretation and observation of a mammography image in order to perform an ultrasonic examination while referring to the mammography image and its findings. Specifically, "Be careful to perform ultrasonography on high-density areas on mammography images" or "Benign lesions (simple cysts) on mammography images that have clear borders and smooth masses that are obvious by ultrasonography. If it can be confirmed, it is not necessary to conduct a detailed examination. ”If a local asymmetric shadow (FAD) is observed on the mammography image, it is necessary to conduct a detailed examination if it is confirmed by ultrasonic examination that the breast is a normal mammary gland. No "has already been shown as a comprehensive criteria for the Japan Breast and Thyroid Ultrasound Medical Society Breast Cancer Screening Study Group, and the technician in charge of ultrasonography was required to make these judgments appropriately.

  For this reason, in this embodiment, the image processing apparatus 30 sets a region of interest in a mammography image obtained by imaging the breast of the subject, position information of the region of interest on the mammography image, and the mammography image. Based on the information indicating the imaging direction, the position information of the region of interest on the schematic diagram schematically representing the breast is specified and output. Thereby, the technician of the ultrasonic examination can easily grasp the position corresponding to the region of interest set on the mammography image on the schematic diagram, and the accuracy of the mammary gland image diagnosis can be improved. Below, the image display apparatus 40 which concerns on this embodiment is demonstrated in detail.

  FIG. 6 is a diagram illustrating a configuration example of the image processing apparatus 30 according to the first embodiment. As illustrated in FIG. 6, the image processing apparatus 30 includes an input unit 31, a display unit 32, a communication control unit 33, a storage unit 34, and a control unit 35.

  The input unit 31 receives input of various operations and various information from the operator. For example, the input unit 31 is a keyboard, a mouse, a button, a trackball, a touch panel, or the like.

  The display unit 32 displays a GUI for receiving various operations from the operator and various images. For example, the display unit 32 is a liquid crystal display, a CRT (Cathode Ray Tube) display, a touch panel, or the like.

  The communication control unit 33 controls communication performed with other devices via the network 50. For example, the communication control unit 33 is a network card or a network adapter, and communicates with other devices by connecting to the network 50 via an Ethernet (registered trademark) LAN. For example, the communication control unit 33 performs wireless communication with other devices by connecting to the network 50 via a wireless LAN.

  The storage unit 34 is a storage device such as a hard disk or a semiconductor memory, and stores various types of information. Specifically, the storage unit 34 includes an image data storage unit 34a and a finding information storage unit 34b.

  The image data storage unit 34a stores a mammography image obtained by imaging the breast of the subject and information indicating the photographing direction of the mammography image. Specifically, the image data storage unit 34a stores a mammography image and information indicating the shooting direction in association with each image. In this image data storage unit 34a, a mammography image and information indicating the photographing direction are stored by an image data acquisition unit 35a described later.

  For example, the image data storage unit 34a stores an MLO direction mammography image (MLO image) and a CC direction mammography image (CC image). Further, the information indicating the photographing direction here is, for example, position information represented in the apparatus coordinate system of the mammography apparatus, and is given to each image as incidental information when the mammography apparatus generates a mammography image.

  The finding information storage unit 34b stores finding information related to the mammography image of the subject. In the finding information storage unit 34b, finding information is stored by a finding information creating unit 35b described later.

  The control unit 35 includes a CPU (Central Processing Unit) and a memory, and controls the operation of the image processing apparatus 30 by executing various programs using them. Specifically, the control unit 35 includes an image data acquisition unit 35a, a finding information creation unit 35b, a display control unit 35c, an area setting unit 35d, a position specifying unit 35e, and a transmission unit 35f.

  The image data acquisition unit 35a acquires a mammography image obtained by imaging the breast of the subject and information indicating the imaging direction of the mammography image. The image data acquisition unit 35a acquires MLO images and CC images for the left and right breasts of the subject. Specifically, the image data acquisition unit 35a communicates with the mammography apparatus 10 via the communication control unit 33, thereby obtaining a mammography image related to the subject to be diagnosed and information indicating the imaging direction of the mammography image. The acquired mammography image and information indicating the photographing direction are stored in the image data storage unit 34a.

  The finding information creation unit 35b creates finding information related to the mammography image of the subject based on the findings input from the operator. Specifically, the finding information creation unit 35b receives an input of findings regarding the mammography image from the mammography examination engineer via the input unit 31. Then, the finding information creation unit 35b creates finding information indicating the accepted findings and stores the finding information in the finding information storage unit 34b.

  The display control unit 35 c displays a reference screen for referring to the mammography image on the display unit 32. Specifically, when receiving a display request from the operator via the input unit 31, the display control unit 35c reads a mammography image related to the subject to be diagnosed from the image data storage unit 34a, and the subject to be diagnosed The finding information on the information is read from the finding information storage unit 34b. Then, the display control unit 35c displays a reference screen on which the read mammography image and finding information are arranged on the display unit 32.

  The region setting unit 35d sets a region of interest in the mammography image. For example, the region setting unit 35d sets a region of interest in each of the MLO image and the CC image for each of the left and right breasts of the subject. Specifically, the region setting unit 35d is an operation for designating a range of an arbitrary size at an arbitrary position on the mammography image arranged on the reference screen displayed by the display control unit 35c via the input unit 31. Is received from the operator. Then, the region setting unit 35d sets a range designated by the operator as a region of interest.

  For example, the region setting unit 35d may automatically detect a candidate region of a lesion from a mammography image using a computer-aided diagnosis (CAD) function and set the detected region as a region of interest. Good. In addition, for example, the region setting unit 35d accepts an operation in which the operator adjusts the MLO image and the CC image with respect to the region detected by CAD, and sets the adjusted region as a region of interest. Also good.

  The position specifying unit 35e specifies the position information of the region of interest on the schematic diagram schematically representing the breast based on the position information of the region of interest on the mammography image and the information indicating the imaging direction. Specifically, the position specifying unit 35e reads the mammography image of the subject to be examined and information indicating the imaging direction of the mammography image from the image data storage unit 34a, and uses the read mammography image and information indicating the imaging direction. Based on this, the position of the region of interest on the schematic diagram is specified. As a schematic diagram (also referred to as a schema) here, various diagrams can be used as long as they show the positional relationship in the breast.

  FIG. 7 is a diagram illustrating an example of a schematic diagram used by the position specifying unit 35e according to the first embodiment. The example shown in FIG. 7 shows a schematic diagram of a mammary gland region as an example of a schematic diagram schematically representing the breast. For example, as shown in FIG. 7, a schematic diagram of a mammary gland region includes a circular region (hereinafter referred to as a breast region) representing a breast region and a substantially triangular region (hereinafter referred to as an axillary region) for each of the left and right breasts. Axillary region).

  Here, the circular area representing the breast area is divided into four areas “A” to “D”, which are divided into upper and lower and left and right. For example, an area “A” (hereinafter referred to as area A) indicates an upper area inside the breast, and an area “B” (hereinafter referred to as area B) indicates an area below the inner side of the breast. Further, for example, a region “C” (hereinafter referred to as “C region”) indicates an upper region outside the breast, and a region “D” (hereinafter referred to as “D region”) indicates a region below the outer side of the breast. Further, a substantially triangular region “C ′” (hereinafter referred to as C ′ region) representing the axillary region has a shape extending obliquely upward from the region C and becoming narrower as the distance from the region C increases.

  Returning to FIG. 6, for example, the position specifying unit 35e includes position information of a first region of interest that is a region of interest on the MLO image, position information of a second region of interest that is a region of interest on the CC image, Based on the information indicating the imaging direction of the MLO image and the information indicating the imaging direction of the CC image, position information of each of the first region of interest and the second region of interest on the schematic diagram is specified. Here, for example, the position specifying unit 35e specifies the position of the region of interest on the schematic diagram based on the position information represented by the apparatus coordinate system of the mammography apparatus that captured the mammography image.

  FIG. 8 is a diagram for explaining identification of position information of a region of interest on a schematic diagram performed by the position identification unit 35e according to the first embodiment. The example shown in FIG. 8 is an example of a method for converting position information on a mammography image into position information on a schematic diagram based on an imaging direction represented by an apparatus coordinate system of the mammography apparatus. Here, an example in which the schematic diagram of the mammary gland region shown in FIG. 7 is used will be described.

  First, the position specifying unit 35e, based on the apparatus coordinate system of the mammography apparatus, position information of the region corresponding to the breast region (A to D region) in the schematic diagram of the mammary gland region and the region of interest set in the mammography image. Specify location information. For example, using the MLO image and the CC image, the position specifying unit 35e specifies the position information of the region corresponding to the breast region and the position information of the region of interest set in each of the MLO image and the CC image.

First, the position specifying unit 35e sets an _{X_CC} axis, a _{Y_CC} axis, and an origin on the detector in the CC image. For example, the position specifying unit 35e sets the vertex corresponding to the upper side of the breast among the two vertices arranged on the subject side among the four vertices of the rectangular detection surface of the detector as the origin. . In addition, the position specifying unit 35e detects the skin surface _{S_CC} depicted in the CC image. Here, as a method for detecting the skin surface _{S_CC} , various generally known image detection methods can be used.

The position specifying unit 35e, of the two points and _{S _CC} axis and _{X _CC} axis detected contacts a point closer to the origin and _{CC _1,} to a point farther from the origin and _{CC _2.} The position specifying unit 35e receives an operation to specify a CC _{_1} and CC _{_2} each position on the CC image from the operator, may set the respective points based on the operation. Then, the position specifying unit 35e generates a straight line _{L _CC1} by projecting from point _{CC _1} in the direction of the tube, to generate a straight line _{L _CC2} by projecting from point _{CC _2} in the direction of the tube.

In addition, the position specifying unit 35e sets the _{X_MLO} axis, the _{Y_MLO} axis, and the origin on the detector in the MLO image. For example, the position specifying unit 35e sets the vertex corresponding to the outside of the breast among the two vertices arranged on the subject side among the four vertices of the rectangular detection surface of the detector as the origin. . Further, the position specifying unit 35e detects the skin surface _{S_MLO} depicted in the MLO image. Here, as a method for detecting the skin surface S _{_MLO} may like the detection of the skin surface S _{_CC,} using commonly known various image detection method. Thereafter, the position specifying unit 35e detects a breast region in the MLO image.

FIG. 9 is a diagram for explaining detection of a breast region in an MLO image performed by the position specifying unit 35e according to the first embodiment. As illustrated in FIG. 9, the position specifying unit 35e defines a straight line _{L_S1} represented by a linear function _{Y_MLO} = _{aX_MLO} + b (a is an arbitrary positive number), and the value of b is changed from a sufficiently small negative value. The point where the straight line first intersects with the skin surface _{S_MLO} is detected by increasing the size sequentially. Then, the position specifying unit 35e sets a point obtained by projecting the point perpendicular to the _{X_MLO} axis as _{MLO_upper} .

Further, the position specifying unit 35e is a linear function _{Y _MLO = -aX _MLO + b (} a is an arbitrary positive number) define a straight line _{L _S2} represented by the same manner, sequentially the value of b from a sufficiently small negative value The point where the straight line first intersects with the skin surface _{S_MLO} is detected by increasing. Then, the position specifying unit 35e sets a point obtained by projecting the point perpendicular to the _{X_MLO} axis as _{MLO_lower} .

  Note that the values of a and b in each linear function used here may be set in advance and stored in the storage unit 34 or the like, and may be changed according to an instruction from the operator. For example, the position specifying unit 35e may detect the position of the nipple depicted in the MLO image, and use the slope of a straight line passing through the detected nipple position and the origin of the detector as the value of a.

Position specifying unit 35e is thus the region between the _{MLO _Upper} and point _{MLO _Lower} points specified is detected as the breast area. The position specifying unit 35e receives an operation to specify the operator from over MLO image of each point _{MLO _Upper} and point _{MLO _Lower} position may be set each point based on the operation.

Returning to FIG. 8, further, the position specifying unit 35e generates, linear _{L _MLO_l} generate a straight line _{L _MLO_u} by projecting from point _{MLO _Upper} towards Kan sphere by projecting from point _{MLO _Lower} towards Kan sphere To do. The position specifying unit 35e linearly _{L _CC1,} linear _{L _CC2,} calculates an ellipse E which is inscribed in a region surrounded by a straight line _{L _MLO_u} and linear _{L _MLO_l.} Note that the ellipse E may be a perfect circle.

Then, the position specifying unit 35e is a region of interest set in the MLO image (ROI: Region Of Interest) and _{ROI _MLO} a point obtained through projection perpendicular to the _{X _MLO} axis from interest set in the CC image A point projected perpendicularly to the _{X_MLO} axis from the region (ROI) is _{defined as ROI_CC} . Further, the position specifying unit 35e is the area where the straight line obtained by projecting the straight line and the point _{ROI _CC} projected from point _{ROI _MLO} the direction of tube in the direction of the tube crosses the M. Here, when the region of interest set in each of the MLO image and the CC image is a point, the region M becomes a point, and at least one of the regions of interest set in each of the MLO image and the CC image is a range. In some cases, the region M is a region having an area.

The position specifying unit 35e sets the center position of the ellipse E as the nipple position N. The position specifying unit 35e extracts the nipple position _{N_MLO} from the MLO image, extracts the nipple position _{N_CC} from the CC image, and projects the straight line projected from the nipple position _{N_MLO in the} tube direction to the tube position from the nipple position _{N_CC.} An intersection point with a straight line projected in the spherical direction may be set as a nipple position in the ellipse E. Here, as a method for detecting the nipple position N, various generally known image detection methods can be used.

  Next, the position specifying unit 35e specifies position information of an area corresponding to the C ′ area in the schematic diagram of the mammary gland area based on the apparatus coordinate system of the mammography apparatus. For example, the position specifying unit 35e sets A as the point on the side closer to the detector among the two intersections where the straight line passing through the tube position and the nipple position N when the CC image is captured and the ellipse E intersect, Of the two intersections where the straight line passing through the tube position and the nipple position N when the MLO image is captured and the ellipse E intersect, let B be the point closer to the detector.

In addition, the position specifying unit 35e is parallel to the _{X_MLO} axis and the straight line projected in the tube direction from the origin of the _{X_MLO} axis, which is the end point of the detector when the MLO image is captured, and in the ellipse E Let C be the point where the straight line passing through the nipple position N intersects. Then, the position specifying unit 35e converts the area A-B-C surrounded by the line segment connecting the point A and the point C, the line segment connecting the point B and the point C, and the ellipse E to the C ′ area. It is specified as an area corresponding to.

  Thereafter, the position specifying unit 35e uses the position information of the specified area of the ellipse E, the position information of the area corresponding to the C ′ area, and the position information of the area M indicating the region of interest as position information on the schematic diagram, respectively. Is converted into the position information of the region of interest on the schematic diagram.

  FIG. 10 is a diagram for explaining the specification of the position information of the region of interest on the schematic diagram performed by the position specifying unit 35e according to the present embodiment. For example, as shown from the upper left to the middle left of FIG. 10, the position specifying unit 35e is a schematic diagram of a mammary gland region, which includes an ellipse E region, a region corresponding to the C ′ region, and a region M indicating a region of interest. The position information of each region is converted into position information on the schematic diagram by enlarging or reducing the size so as to match the shape.

  First, the position specifying unit 35e converts the position information of the area of the ellipse E into position information of a circular area representing the breast area in the schematic diagram of the mammary gland area. Here, as a method of converting the position information of the area of the ellipse E into the position information of the circular area, various generally known coordinate conversion methods can be used. For example, the position specifying unit 35e uses a method of converting ellipse position coordinates into circle position coordinates using a predetermined conversion matrix.

FIG. 11 is a diagram for describing an example of region conversion performed by the position specifying unit 35e according to the first embodiment. For example, as shown in FIG. 11, the position coordinates of the ellipse before conversion are represented by (X, Y) = (X ₁ , X ₂ ), and the position coordinates of the circle after conversion are (X ′, Y ′) = ( When expressed by X ₁ ′, X ₂ ′), the position coordinates of the ellipse can be expressed by the following equation (1) using the transformation matrix A _ks .

Here, when the symmetric matrix F _ks is represented by b, c, and d as in the following equation (2), the transformation matrix A _ks in the equation (1) can be represented as in the following equation (3). it can.

That is, in this example, the ellipse is converted into a circle of a desired size by setting the values of b, c, and d appropriately according to the size of the circle after conversion and _obtaining the conversion matrix _Aks. be able to. Therefore, the position specifying unit 35e uses this method to appropriately set b, c, and d according to the size of the schematic diagram of the mammary gland region, thereby changing the area of the ellipse E to the schematic diagram of the mammary gland region. It can be converted into a circular area corresponding to the breast area (A to D area).

Thereafter, the position specifying unit 35e converts the C ′ region and the region M, respectively, using the same conversion matrix as the conversion matrix A _ks used for the conversion of the ellipse E, thereby obtaining the position information of each of the C ′ region and the region M. The position information in the schematic diagram of the mammary gland region is converted.

  For example, when the region of interest is not set in the CC image, the position specifying unit 35e, as the region of interest on the schematic diagram, is a linear region obtained by projecting the region of interest set in the MLO image onto the ellipse E. Is identified. Similarly, for example, when the region of interest is not set in the MLO image, the position specifying unit 35e is a straight line obtained by projecting the region of interest set in the CC image onto the ellipse E as the region of interest on the schematic diagram. Identify the area.

  Returning to FIG. 10, the position specifying unit 35 e converts the position information of each of the area of the ellipse E, the area corresponding to the C ′ area, and the area M indicating the region of interest into position information on the schematic diagram. For example, as shown from the lower left to the lower right in FIG. 10, display information is generated in which a region M indicating a region of interest is arranged on a template of a schematic diagram of a mammary gland region based on position information of each region on the schematic diagram. . The display information here is generated as image data in a format such as JPEG (Joint Photographic Experts Group), GIF (Graphics Interchange Format), or bitmap.

  Then, the position specifying unit 35e stores the generated display information in the storage unit 34 in association with the corresponding mammography image. For example, the position specifying unit 35e attaches the generated display information as supplementary information to the mammography image, and stores it in the image data storage unit 34a together with the image data of the mammography image. Alternatively, the position specifying unit 35e stores the generated display information in the image data storage unit 34a in association with the patient ID assigned to the subject.

  For example, when generating the display information using the template of the schematic diagram, the position specifying unit 35e not only indicates the position information of the region of interest on the schematic diagram, but also the region where the breast density is higher than a predetermined value. Alternatively, a calcified region, a tumor region, or the like may be extracted from a mammography image and displayed on a schematic diagram. For example, the position specifying unit 35e displays a portion corresponding to these areas on the schematic diagram in a color different from the color of the schematic diagram for each type of area, or marks determined in advance for each type of area. Or display.

  The display information generated by the position specifying unit 35e in this manner is transmitted to the image display device 40 or the ultrasonic diagnostic device 20 by, for example, a transmission unit 35f described later. Then, the image display device 40 or the ultrasonic diagnostic device 20 that has received the display information outputs position information of the region of interest on the schematic diagram to the display unit 32 in a predetermined display format based on the received display information. To do.

  For example, the image display apparatus 40 or the ultrasonic diagnostic apparatus 20 outputs position information of the region of interest on the schematic diagram as reference information at the time of ultrasonic diagnosis. Further, for example, the image display device 40 or the ultrasonic diagnostic device 20 outputs position information on a schematic diagram corresponding to the patient ID of the examinee receiving the ultrasonic diagnosis.

  FIG. 12 is a diagram illustrating an example of display information displayed by the image display device 40 according to the first embodiment. For example, as illustrated in FIG. 12, the image display device 40 displays a reference screen on which the mammography images 61 to 64, schematic diagrams 65 and 66 of a mammary gland region, a patient ID region 67, and a finding display region 68 are arranged on the display unit 42. indicate.

  Here, for example, the mammography image 61 is a CC image relating to the left breast of the subject, and the mammography image 62 is an MLO image relating to the left breast of the subject. Further, the mammography image 63 is an MLO image relating to the right breast of the subject, and the mammography image 64 is a CC image relating to the right breast of the subject. The patient ID area 67 is an area for displaying a patient ID, and the finding display area 68 is an area for displaying finding information related to the mammography image of the subject.

  A schematic diagram 65 shows a mammary gland region in the left breast of the subject, and a schematic diagram 66 shows a mammary gland region in the right breast of the subject. Here, each of the schematic diagrams 65 and 66 is a schematic diagram of a mammary gland region displayed based on display information transmitted from the image display device 40. For example, a rectangular mark indicating a region of interest on the schematic diagram R is displayed.

  Returning to FIG. 6, the transmission unit 35 f transmits the display information generated by the position specifying unit 35 e to the image display device 40 or the ultrasonic diagnostic device 20 in accordance with an instruction from the operator. Specifically, the transmission unit 35 f receives an instruction to transmit display information from the operator of the image processing apparatus 30 or the ultrasonic diagnostic apparatus 20 via the input unit 31. When receiving the display information transmission instruction, the transmission unit 35 f reads the display information designated by the operator from the storage unit 34 and transmits the display information to the image display device 40 or the ultrasonic diagnostic device 20.

  For example, the transmission unit 35f transmits the display information generated by the position specifying unit 35e to the image display device 40 or the ultrasonic diagnostic device 20 in response to a request from the image display device 40 or the ultrasonic diagnostic device 20. May be. In this case, for example, the transmission unit 35 f receives a request for display information from the image display device 40 or the ultrasonic diagnostic device 20 via the communication control unit 33. And the transmission part 35f will read the requested | required display information from the memory | storage part 34, if the request | requirement of display information is received. For example, when the display information is stored in the storage unit 34 in association with the patient ID, the transmission unit 35f receives the patient ID as a request from the image display device 40 or the ultrasonic diagnostic device 20, and receives the patient ID. Read the display information associated with the ID. Then, the transmission unit 35f transmits the read display information to the image display apparatus 40 or the ultrasonic diagnostic apparatus 20 that transmitted the request.

  FIG. 13 is a flowchart illustrating a processing procedure of processing performed by the image processing apparatus 30 according to the first embodiment. As illustrated in FIG. 13, when the image processing apparatus 30 receives an instruction to display a mammography image and finding information from the operator (step S101, Yes), the image data acquisition unit 35a displays the left and right breasts of the subject. , An MLO direction mammography image and a CC direction mammography image are acquired (steps S102 and S103). In addition, the finding information creation unit 35b creates finding information about the breast of the subject (step S104).

  Subsequently, the display control unit 35c displays a reference screen for referring to the mammography image on the display unit 32 (step S105). Thereafter, the region setting unit 35d receives an operation for designating a desired range on the mammography image in the MLO direction arranged on the reference screen via the input unit 31 from the operator, and sets the designated range as a region of interest ( Step S106). Further, the region setting unit 35d receives an operation for designating a desired range on the mammography image in the CC direction arranged on the reference screen via the input unit 31, and sets the designated range as a region of interest ( Step S107).

  Then, when a region of interest is set in each of the mammography image in the MLO direction and the mammography image in the CC direction (step S108, Yes), the position specifying unit 35e sets the position information of the region of interest set in each image and each image. The position information of the region of interest on the schematic diagram is specified based on the information indicating the imaging direction (step S109). Thereafter, the position specifying unit 35e outputs the position information of the region of interest on the specified schematic diagram to, for example, the image display device 40, the ultrasonic diagnostic device 20, and the like (step S110).

(Modification of the first embodiment)
In the first embodiment described above, the position specifying unit 35e specifies the position of the region of interest on the schematic diagram based on the position information represented by the apparatus coordinate system of the mammography apparatus that captured the mammography image. Although an example of the case has been described, the embodiment is not limited thereto. For example, the position specifying unit 35e may specify the position information of the region of interest on the schematic diagram from the mammography image by predetermined image processing.

FIG. 14 is a diagram for explaining the specification of the position information of the region of interest on the schematic diagram performed by the position specifying unit 35e according to the modification of the first embodiment. For example, as illustrated in FIG. 14, the position specifying unit 35e _divides the MLO image into a region on the abdomen side and a region on the head side by a straight line _{L_MLO} passing through the nipple. The position specifying unit 35e is parallel to the straight line _{L _MLO,} and a region having a width that includes the set region of interest MLO image (region of dashed circle shown in FIG. 14) _is set as _{R _MLO.}

Furthermore, the position specifying unit 35e _divides the CC image into an inner area (area corresponding to the _AB area) and an outer area (area corresponding to the _CD area) along the straight line _{L_CC} passing through the nipple. To do. In addition, the position specifying unit 35e sets, as _{R_CC} , a region including the region of interest set in the CC image (the dotted circle region illustrated in FIG. 14) among the two divided regions.

Then, the position specifying unit 35e, based on the region _{R _MLO} and region _{R _CC} respective position information to locate information of the overlapped portion between the region _{R _MLO} and the region _{R _CC} in diagram schematic. And the position specific | specification part 35e produces _| generates the display information which has arrange _| positioned the overlapping part of area _| region _{R_MLO} and area _| region _{R_CC} on the template of the schematic diagram of a mammary gland area _| region based on the specified positional information.

  For example, when the display information is generated using the template of the schematic diagram, the position specifying unit 35e may display the depth information of the region of interest as the position information of the region of interest on the schematic diagram. Good. For example, the position specifying unit 35e divides the CC image into N equal parts in the Y direction (N is an arbitrary integer), and displays the position of the region of interest by the N quantile method. As an example, for example, the position specifying unit 35e divides the CC image into four equal parts in the Y direction, and when the region of interest is set in the second part counting from the side opposite to the chest wall, Is displayed.

  As described above, in the first embodiment, the image processing apparatus 30 sets a region of interest in a mammography image obtained by imaging the breast of the subject, position information of the region of interest on the mammography image, and the mammography image. The position information of the region of interest on the schematic diagram schematically representing the breast is specified and output based on the information indicating the imaging direction. Thereby, for example, a mammography examination engineer sets a region of interest in a mammography image using the image processing device 30, and an ultrasonic examination technician uses the image display device 40 or the ultrasonic diagnostic device 20 to perform mammography examination. The position corresponding to the region of interest set by the technician can be easily grasped on the schematic diagram, and the accuracy of the mammary gland image diagnosis can be improved.

  FIG. 15 is a diagram for explaining a relationship between a mammography image and a schematic diagram according to the first embodiment. For example, as shown in FIG. 15, just by looking at the mammography image, the region of interest set in the MLO image and the CC image (the circle region shown by the broken line in FIG. 15) is located at which position during breast ultrasound imaging. In some cases, it may be difficult to grasp whether it corresponds. Even in such a case, according to the first embodiment, the mark R indicating the region of interest is displayed on the schematic diagram schematically representing the breast. The position of the breast to be examined can be easily grasped.

  In the first embodiment described above, an example in which a region of interest is displayed on a schematic diagram based on the region of interest set in each of the MLO image and the CC image has been described. However, the embodiment is not limited to this. I can't. For example, in imaging with a mammography device, it is known that a blind area occurs due to the curvature of the chest wall or the relationship between fixed tissue and movable tissue, etc., but when a lesion area exists in this blind area, There may be a case where the region of interest is not set on the mammography image.

  FIG. 16 is a diagram illustrating a blind area in capturing an MLO image, and FIG. 17 is a diagram illustrating a blind area in capturing a CC image. For example, as shown in FIG. 16, when an MLO image is taken, blind areas 301 are likely to occur on the upper inner side (see the left diagram in FIG. 16) and lower portion (see the right diagram in FIG. 16). Also, for example, as shown in FIG. 17, in CC image capturing, a blind area 302 is formed on the upper part of the breast (see the right figure in FIG. 17) and the outer side near the axilla (see the left figure in FIG. 17). Prone to occur.

  Here, for example, when a lesion is present in a blind area generated by MLO image capturing or CC image capturing, a region of interest may not be set in at least one of the MLO image and the CC image. For example, as shown in the left diagram of FIG. 16, when the lesioned part 303 exists in the blind area inside the upper part of the breast, the lesioned part 303 is not drawn in the MLO image, and as a result, the MLO image is displayed. A region of interest may not be set. Further, for example, as shown in the right diagram of FIG. 17, when the lesioned part 304 exists in the blind area at the upper part of the breast, the lesioned part 304 is not drawn in the CC image, and as a result, the CC image The region of interest may not be set.

  For this reason, for example, when a region of interest is not set in at least one of the MLO image and the CC image, information indicating the blind area may be further displayed on the schematic diagram of the breast. In this case, for example, in the image processing apparatus 30, the position specifying unit 35e further specifies whether or not a blind area has occurred based on the mammography image. Then, the position specifying unit 35e generates display information indicating the position information of the blind area on the schematic diagram in addition to the position information of the region of interest on the schematic diagram based on the specified result. Then, for example, the image display device 40 or the ultrasonic diagnostic device 20 determines the blind area on the schematic diagram along with the position information of the region of interest on the schematic diagram based on the display information generated by the position specifying unit 35e. The position information is further output.

  For example, when specifying the region of interest by the method illustrated in FIGS. 8 to 11, the position specifying unit 35 e first detects whether the region of interest is set in each of the MLO image and the CC image. Then, when the region of interest is set in the CC image, but the region of interest is not set in the MLO image, the position specifying unit 35e specifies that a blind area has occurred in the MLO image. In addition, when the region of interest is set in the MLO image but the region of interest is not set in the CC image, the position specifying unit 35e specifies that a blind area is generated in the CC image. Then, the position specifying unit 35e generates display information that displays the position information of the blind area together with the position information of the region of interest on the template of the schematic diagram of the mammary gland region based on the specified result.

  18 and 19 are diagrams for explaining the display of the blind area performed by the position specifying unit 35e according to the modified example of the first embodiment. For example, when the position specifying unit 35e specifies that a blind area has occurred in the MLO image, the area A in the schematic diagram shown in FIG. 7 is displayed differently from the other areas as shown in FIG. Display in a manner. At this time, for example, the position specifying unit 35e displays the A area in a different color from other areas or displays it in a different pattern. 18 shows an example of changing the display mode of the entire area A, for example, the display mode of a part of the area A may be changed. For example, the display mode may be changed for a portion near the outer periphery of the region A.

  Here, as described above, in the first embodiment, when the region of interest is not set in the MLO image, the position specifying unit 35e is set in the CC image as the region of interest on the schematic diagram. A straight line region on which the region of interest is projected is specified. As a result, for example, as shown in FIG. 18, a linear region of interest R is displayed on the schematic diagram. According to such a display, for example, an ultrasonic examination engineer determines that it is necessary to carefully perform an ultrasonic examination on a part corresponding to the A area in the linear region of interest R. be able to.

  Further, for example, when the position specifying unit 35e specifies that a blind area is generated in the CC image, as shown in FIG. 19, instead of the schematic diagram shown in FIG. A template 305 of a schematic diagram of the seen shape is used. At this time, for example, when the blind area is generated in the CC image of the right breast, the position specifying unit 35e uses the schematic diagram for the right breast, and the blind area is generated in the CC image of the left breast. In this case, a schematic diagram for the left breast is used. Here, for example, the schematic diagram for the right breast and the schematic diagram for the left breast are defined to be symmetrical to each other in the left-right direction.

  And the position specific | specification part 35e produces | generates the display information which showed the area | region showing the general shape of the blind area in imaging | photography of CC image with the positional information on a region of interest on the template 305 of a schematic diagram. For example, on the schematic diagram, the position specifying unit 35e displays a region 306 indicating a blind area generated in the upper part of the breast in a display mode different from other regions. At this time, for example, the position specifying unit 35e displays the region 306 in a color different from other regions or displays it in a different pattern.

  Here, as described above, in the first embodiment, when the region of interest is not set in the CC image, the position specifying unit 35e is set in the MLO image as the region of interest on the schematic diagram. A straight line region on which the region of interest is projected is specified. As a result, for example, as shown in FIG. 19, a linear region of interest R is displayed on the schematic diagram. According to such a display, for example, an ultrasonic examination engineer determines that it is necessary to carefully perform an ultrasonic examination in the linear region of interest R, particularly on the side closer to the upper part of the breast. be able to.

  Here, an example in which the position specifying unit 35e automatically displays the position information of the blind area on the schematic diagram when the region of interest is not set in one of the MLO image and the CC image will be described. However, the embodiment is not limited to this. For example, the position specifying unit 35e may display an area to be displayed as a blind area based on an instruction from the operator. In this case, for example, when the region of interest is not set in one of the MLO image and the CC image, the position specifying unit 35e is a region to be displayed as a blind area on the mammography image in which the region of interest is not set. An operation to specify the range of the is accepted from the operator. And the position specific | specification part 35e displays a blind area on the template of a schematic diagram based on the range designated by the operator.

  Further, here, an example has been described in which a blind area is displayed on a schematic diagram of a breast when a region of interest is not set in at least one of an MLO image and a CC image, but the embodiment is not limited thereto. Absent.

  For example, the position specifying unit 35e may specify the position information of the region of interest on the schematic diagram based on the region of interest set in only one of the MLO image and the CC image. In this case, for example, when specifying the region of interest by the method shown in FIG. 14, the position specifying unit 35e first detects whether the region of interest is set in each of the MLO image and the CC image. And the position specific | specification part 35e produces | generates the display information which has arrange | positioned the region of interest on the schematic diagram according to the detection result of the region of interest.

20 and 21 are diagrams for explaining the specification of the position information of the region of interest on the schematic diagram according to the modification of the first embodiment. For example, when the region of interest (the dashed circle region shown in FIG. 20) is set in the CC image, but the region of interest is not set in the MLO image, the position specifying unit 35e As shown, the CC image is _{divided into} an inner area (area corresponding to the _AB area) and an outer area (area corresponding to the _CD area) along the straight line _{L_CC} passing through the nipple. Of the two regions, a region including the region of interest set in the CC image is set as _{R_CC} . Further, the position specifying unit 35e specifies the position information of the region _{R_CC} on the schematic diagram based on the position information of the region _{R_CC} expressed in the device coordinate system of the mammography apparatus. And the position specific | specification part 35e produces _| generates the display information which has arrange _| positioned area _| region _{R_CC} on the template of the schematic diagram of a mammary gland area _| region based on the specified position information. For example, when the region of interest is set in the inner region of the CC image, the position specifying unit 35e displays a frame line indicating the region _{R_CC} as the region of interest in the _AB region on the schematic diagram. For example, when the region of interest is set in the outer region of the CC image, the position specifying unit 35e displays a frame line indicating the region _{R_CC} as the region of interest in the _CD region on the schematic diagram. To do.

In addition, for example, the position specifying unit 35e displays a region of interest (broken circle area shown in FIG. 21) in the MLO image, but a region of interest is not set in the CC image. As shown in FIG. 21, the MLO image is divided into an abdominal region and a head region by a straight line _{L_MLO} passing through the nipple. The position specifying unit 35e is parallel to the straight line _{L _MLO,} and a region having a width that includes the set region of interest MLO image (region of dashed circle shown in FIG. 21) _is set as _{R _MLO.} The position specifying unit 35e, based on the position information of the region _{R _MLO,} identifies the position information of the region _{R _MLO} in diagram schematic. And the position specific | specification part 35e produces _| generates the display information which has arrange _| positioned area _| region _{R_MLO} on the template of the schematic diagram of a mammary gland area _| region based on the specified position information. Note that the position specifying unit 35e may set the entire region including the region of interest set in the MLO image among the region on the abdomen side and the region on the head side as _{R_MLO} . For example, when the region of interest is set in the abdominal region in the MLO image, the position specifying unit 35e displays a frame line indicating the region _{R_MLO} as the region of interest in the _DB region on the schematic diagram. . In addition, for example, when the region of interest is set in the region on the head side in the MLO image, the position specifying unit 35e has a frame line indicating the region _{R_MLO} as the region of interest in the area _AC in the schematic diagram. Is displayed.

  As described above, even when the region of interest is not set in one of the MLO image and the CC image, the position specifying unit 35e is based on the mammography image in which the region of interest is set, and the region of interest on the schematic diagram. Specify location information. Thereby, even if a blind area has occurred in the MLO image or the CC image, the position of the region of interest set in at least one of the mammography images is shown on the schematic diagram. it can.

(Second Embodiment)
In the first embodiment described above, an example has been described in which the image processing apparatus 30 sets a region of interest in a mammography image and specifies position information of the region of interest on the schematic diagram. In contrast, in the second embodiment, an example in which the image display device 40 sets a region of interest in a mammography image and specifies position information of the region of interest on the schematic diagram will be described. The configuration of the apparatus included in the medical information processing system according to the second embodiment is the same as that shown in FIG.

  FIG. 22 is a diagram illustrating a configuration example of the image display device 40 according to the second embodiment. As illustrated in FIG. 22, the image display device 40 includes an input unit 41, a display unit 42, a communication control unit 43, a storage unit 44, and a control unit 45.

  The input unit 41 receives various operations and various information inputs from the operations. For example, the input unit 41 is a button, a touch panel, a pen tablet, or the like.

  The display unit 42 displays a GUI and various images for receiving various operations from the operator. For example, the display unit 42 is a liquid crystal display or a touch panel.

  The communication control unit 43 controls communication performed with other devices via the network 50. For example, the communication control unit 43 performs wireless communication with other devices by connecting to the network 50 via a wireless LAN.

  The storage unit 44 is a storage device such as a hard disk or a semiconductor memory, and stores various types of information. Specifically, the storage unit 44 includes an image data storage unit 44a and a finding information storage unit 44b.

  The image data storage unit 44a stores a mammography image obtained by imaging the breast of the subject and information indicating the photographing direction of the mammography image. Specifically, like the image data storage unit 34a described in the first embodiment, the image data storage unit 44a stores a mammography image and information indicating a shooting direction in association with each image. A mammography image obtained by imaging the breast of the subject is stored. A mammography image is stored in the image data storage unit 44a by an image data acquisition unit 45a described later.

  The finding information storage unit 44b stores finding information related to the mammography image of the subject. In the finding information storage unit 44b, finding information is stored by a finding information acquisition unit 45b described later.

  The control unit 45 includes a CPU and a memory, and controls the operation of the image display device 40 by executing various programs using them. Specifically, the control unit 45 includes an image data acquisition unit 45a, a finding information acquisition unit 45b, a display control unit 45c, an area setting unit 45d, and a position specifying unit 45e.

  The image data acquisition unit 45a acquires a mammography image obtained by imaging the breast of the subject and information indicating the imaging direction of the mammography image. The image data acquisition unit 45a acquires a mammography image in the MLO direction and a mammography image in the CC direction for each of the left and right breasts of the subject.

  Specifically, the image data acquisition unit 45a communicates with the image processing device 30 via the communication control unit 43, so that the mammography image regarding the subject to be diagnosed and information indicating the imaging direction of the mammography image are displayed. And the acquired mammography image and information indicating the photographing direction are stored in the image data storage unit 44a. Note that the image data acquisition unit 45a communicates with the mammography apparatus 10 via the communication control unit 43 to acquire a mammography image related to the subject to be diagnosed and information indicating the imaging direction of the mammography image. Also good.

  The finding information acquisition unit 45b acquires finding information related to the breast of the subject. Specifically, the finding information acquisition unit 45b communicates with the image processing apparatus 30 via the communication control unit 43, thereby acquiring finding information of the mammography image regarding the subject to be diagnosed. Then, the finding information acquisition unit 45b stores the acquired finding information in the finding information storage unit 44b.

  The display control unit 45c displays a reference screen for referring to the mammography image on the display unit 42. Specifically, when the display control unit 45c receives a display request from the operator via the input unit 41, the display control unit 45c reads a mammography image related to the subject to be diagnosed from the image data storage unit 44a, and the subject to be diagnosed The finding information on the information is read from the finding information storage unit 44b. Then, the display control unit 45c displays a reference screen on which the read mammography image and finding information are arranged on the display unit 42.

  The region setting unit 45d sets a region of interest in the mammography image. Specifically, the region setting unit 45d sets a region of interest in the mammography image by the same method as the region setting unit 35d described in the first embodiment. For example, the region setting unit 45d sets a region of interest in each of the MLO image and the CC image for each of the left and right breasts of the subject.

  The position specifying unit 45e specifies the position information of the region of interest on the schematic diagram schematically representing the breast based on the position information of the region of interest on the mammography image and the information indicating the imaging direction. Specifically, the position specifying unit 45e specifies position information of the region of interest on the schematic diagram by the same method as the position specifying unit 35e described in the first embodiment.

  Thereafter, the position specifying unit 45e generates display information in which a region indicating a region of interest is arranged on a template of a schematic diagram of a mammary gland region, similarly to the position specifying unit 35e described in the first embodiment. And the position specific | specification part 45e displays the reference screen shown in FIG. 12 on the display part 42 based on the produced | generated display information, for example.

  As described above, in the second embodiment, the image display device 40 sets a region of interest in a mammography image obtained by imaging the breast of the subject, position information of the region of interest on the mammography image, and the mammography image. The position information of the region of interest on the schematic diagram schematically representing the breast is specified and output based on the information indicating the imaging direction. Accordingly, for example, an ultrasonic examination engineer can easily grasp the position corresponding to the region of interest set on the mammography image on the schematic diagram using the image display device 40, and the mammary gland The accuracy of image diagnosis can be increased.

(Third embodiment)
In the first and second embodiments described above, an example in which the image processing device 30 or the image display device 40 sets a region of interest in a mammography image and specifies position information of the region of interest on the schematic diagram will be described. did. In contrast, in the third embodiment, an example in which the ultrasound diagnostic apparatus 20 sets a region of interest in a mammography image and specifies position information of the region of interest on the schematic diagram will be described. Note that the configuration of the apparatus included in the medical information processing system according to the third embodiment is the same as that shown in FIG.

  FIG. 23 is a diagram illustrating a detailed configuration example of the ultrasonic diagnostic apparatus 20 according to the sixth embodiment. Note that FIG. 23 illustrates the input unit 22, the display unit 23, the apparatus main body 24, the image memory 245, the internal storage unit 246, and the communication control in the configuration of the ultrasonic diagnostic apparatus 20 illustrated in FIG. The part 247 and the control part 248 are shown.

  As shown in FIG. 23, for example, the image memory 245 includes an image data storage unit 245a, and the internal storage unit 246 includes a finding information storage unit 246a. Note that the image data storage unit 245a may be included in the internal storage unit 246, and the finding information storage unit 246a may be included in the image memory 245.

  The image data storage unit 245a stores a mammography image obtained by imaging the breast of the subject. In this image data storage unit 245a, a mammography image is stored by an image data acquisition unit 248a described later.

  The finding information storage unit 246a stores finding information related to the mammography image of the subject. In the finding information storage unit 246a, finding information is stored by a finding information acquisition unit 248b described later.

  As shown in FIG. 23, for example, the control unit 248 includes an image data acquisition unit 248a, a finding information acquisition unit 248b, a display control unit 248c, a region setting unit 248d, and a position specifying unit 248e.

  The image data acquisition unit 248a acquires a mammography image obtained by imaging the breast of the subject. Note that the image data acquisition unit 248a acquires a mammography image in the MLO direction and a mammography image in the CC direction for each of the left and right breasts of the subject.

  Specifically, the image data acquisition unit 248a communicates with the image processing device 30 via the communication control unit 247, thereby acquiring a mammography image related to the subject to be diagnosed, and storing the acquired mammography image in the image data storage. Stored in the unit 245a. The image data acquisition unit 248a may acquire a mammography image related to the subject to be diagnosed by communicating with the mammography apparatus 10 via the communication control unit 247.

  The finding information acquisition unit 248b acquires finding information related to the breast of the subject. Specifically, the finding information acquisition unit 248b communicates with the image processing apparatus 30 via the communication control unit 247, thereby acquiring the finding information of the mammography image regarding the subject to be diagnosed. Then, the finding information acquisition unit 248b stores the acquired finding information in the finding information storage unit 246a.

  The display control unit 248c displays a reference screen for referring to the mammography image on the display unit 23. Specifically, when the display control unit 248c receives a display request from the operator via the input unit 22, the display control unit 248c displays the mammography image and the observation information as in the display control unit 35c described in the first embodiment. The arranged reference screen is displayed on the display unit 23. At this time, the display control unit 248c uses the mammography image of the subject stored in the image data storage unit 245a and the finding information of the subject stored in the finding information storage unit 246a.

  The region setting unit 248d sets a region of interest in the mammography image. Specifically, the region setting unit 248d sets a region of interest in the mammography image by the same method as the region setting unit 35d described in the first embodiment. For example, the region setting unit 248d sets a region of interest in each of the MLO image and the CC image for each of the left and right breasts of the subject.

  The position specifying unit 248e specifies the position information of the region of interest on the schematic diagram schematically representing the breast based on the position information of the region of interest on the mammography image and the information indicating the imaging direction. Specifically, the position specifying unit 248e specifies position information of the region of interest on the schematic diagram by the same method as the position specifying unit 35e described in the first embodiment.

  Thereafter, the position specifying unit 248e generates display information in which a region indicating a region of interest is arranged on a template of a schematic diagram of a mammary gland region, similarly to the position specifying unit 35e described in the first embodiment. Then, for example, the position specifying unit 248e displays a reference screen similar to the reference screen illustrated in FIG. 12 on the display unit 23 based on the generated display information.

  As described above, in the third embodiment, the ultrasound diagnostic apparatus 20 sets a region of interest in a mammography image obtained by imaging the breast of the subject, position information of the region of interest on the mammography image, and the mammography. Based on the information indicating the imaging direction of the image, the position information of the region of interest on the schematic diagram schematically representing the breast is specified and output. Thereby, for example, an ultrasonic examination engineer can easily grasp the position corresponding to the region of interest set on the mammography image on the schematic diagram using the ultrasonic diagnostic apparatus 20, The accuracy of breast image diagnosis can be increased.

(Fourth embodiment)
In the first to third embodiments described above, the example in which the region of interest is set based on the operation by the operator has been described, but the embodiment is not limited thereto. For example, a mammary gland parenchymal region may be extracted from a mammography image, and a region of interest in the breast may be automatically set based on the extracted mammary gland parenchymal region.

  Therefore, in the fourth embodiment, the image display device 140 extracts a mammary gland parenchyma region from a mammography image obtained by imaging the subject's breast, and identifies and displays a region of interest in the breast based on the extracted mammary gland parenchyma region. An example of the case will be described. Accordingly, an ultrasonic examination engineer can perform an ultrasonic examination with particular attention to an area where the density of the mammary gland is high on the mammography image, and the accuracy of the mammary gland image diagnosis can be improved. Hereinafter, the image display device 140 according to the present embodiment will be described in detail.

  FIG. 24 is a diagram illustrating a configuration example of the image display device 140 according to the fourth embodiment. As illustrated in FIG. 24, the image display device 140 includes an input unit 141, a display unit 142, a communication control unit 143, a storage unit 144, and a control unit 145.

  The input unit 141 receives various operations and various information inputs from the operations. For example, the input unit 141 is a button, a touch panel, a pen tablet, or the like.

  The display unit 142 displays a GUI for receiving various operations from the operator and various images. For example, the display unit 142 is a liquid crystal display, a touch panel, or the like.

  The communication control unit 143 controls communication performed with other devices via the network 50. For example, the communication control unit 143 performs wireless communication with other devices by connecting to the network 50 via a wireless LAN.

  The storage unit 144 is a storage device such as a hard disk or a semiconductor memory, and stores various types of information. Specifically, the storage unit 144 includes an image data storage unit 144a and a finding information storage unit 144b.

  The image data storage unit 144a stores a mammography image obtained by imaging the breast of the subject. A mammography image is stored in the image data storage unit 144a by an image data acquisition unit 145a described later.

  The finding information storage unit 144b stores finding information related to the mammography image of the subject. In the finding information storage unit 144b, finding information is stored by a finding information acquisition unit 145b described later.

  The control unit 145 includes a CPU and a memory, and controls the operation of the image display device 140 by executing various programs using them. Specifically, the control unit 145 includes an image data acquisition unit 145a, a finding information acquisition unit 145b, a display control unit 145c, an extraction unit 145d, and a specification unit 145e.

  The image data acquisition unit 145a acquires a mammography image obtained by imaging the breast of the subject. The image data acquisition unit 145a acquires a mammography image in the MLO direction and a mammography image in the CC direction for each of the left and right breasts of the subject.

  Specifically, the image data acquisition unit 145a communicates with the image processing apparatus 30 via the communication control unit 143, thereby acquiring a mammography image related to the subject to be diagnosed, and storing the acquired mammography image as image data. Stored in the unit 144a. The image data acquisition unit 145a may acquire a mammography image related to the subject to be diagnosed by communicating with the mammography apparatus 10 via the communication control unit 143.

  The finding information acquisition unit 145b acquires finding information related to the breast of the subject. Specifically, the finding information acquisition unit 145b communicates with the image processing device 30 via the communication control unit 143, thereby acquiring the finding information of the mammography image related to the subject to be diagnosed. Then, the finding information acquisition unit 145b stores the acquired finding information in the finding information storage unit 144b.

  The display control unit 145c displays a reference screen for referring to the mammography image on the display unit 142. Specifically, when the display control unit 145c receives a display request from the operator via the input unit 141, the display control unit 145c reads a mammography image related to the subject to be diagnosed from the image data storage unit 144a, and the subject to be diagnosed The finding information on the information is read from the finding information storage unit 144b. Then, the display control unit 145c displays a reference screen on which the read mammography image and finding information are arranged on the display unit 142.

  FIG. 25 is a diagram illustrating an example of a reference screen displayed by the display control unit 145c according to the fourth embodiment. For example, as illustrated in FIG. 25, the display control unit 145 c displays a reference screen on which the mammography images 161 to 164, schematic diagrams 165 and 166 of the mammary gland region, a processing start button 167, and a finding display region 168 are arranged on the display unit 142. indicate.

  Here, the mammography image 161 is a mammography image in the CC direction regarding the left breast of the subject, and the mammography image 162 is a mammography image in the MLO direction regarding the left breast of the subject. The mammography image 163 is a mammography image in the MLO direction regarding the right breast of the subject, and the mammography image 164 is a mammography image in the CC direction regarding the right breast of the subject. A schematic diagram 165 shows a mammary gland region in the left breast of the subject, and a schematic diagram 166 shows a mammary gland region in the right breast of the subject. The process start button 167 is a button for receiving an instruction to start the mammary gland parenchymal region extraction process from the operator. The finding display area 168 is an area for displaying the finding information regarding the mammography image of the subject.

  Returning to FIG. 24, the extraction unit 145d extracts a mammary gland parenchymal region from a mammography image obtained by imaging the breast of the subject. Then, the extraction unit 145d displays the extracted mammary gland parenchyma region superimposed on the mammography image on the reference screen displayed by the display control unit 145c. Note that the extraction unit 145d extracts a mammary gland parenchyma region from each of the mammography image in the MLO direction and the mammography image in the CC direction for each of the left and right breasts of the subject.

  Specifically, the extraction unit 145d reads and reads out a mammography image related to the subject to be diagnosed from the image data storage unit 144a when the operator of the image display device 140 presses the processing start button 167 on the reference screen. Mammary gland parenchyma region is extracted from the obtained mammography image. The extraction unit 145d may extract a mammary gland parenchyma region from the mammography image when the mammography image of the subject is acquired by the image data acquisition unit 145a.

  For example, the extraction unit 145d extracts the mammary gland parenchyma region by threshold processing based on the luminance value distribution in the mammography image.

  26 and 27 are diagrams for explaining extraction of the mammary gland parenchymal region performed by the extraction unit 145d according to the fourth embodiment. In general, mammary parenchyma, pectoralis major muscle and fat have different X-ray transmission coefficients, so it is known that mammary gland parenchyma and pectoral muscle are depicted white and mammary gland parenchyma and fat are depicted black. Yes. Therefore, it is possible to extract a region estimated to be mammary gland parenchyma by analyzing the feature of the luminance value distribution using the luminance value histogram in the mammography image.

  For example, as illustrated in FIG. 26, the extraction unit 145d includes a threshold A for distinguishing between the luminance value of fat and the luminance value of the mammary gland, and a threshold value for distinguishing between the luminance value of the mammary gland and the luminance value of the chest wall. Using B, the luminance values of the pixels included in the mammography image are divided into three ranges. As described above, the extraction unit 145d uses the lower limit threshold A and the upper limit threshold B related to the luminance values corresponding to the mammary gland parenchyma to extract an area having a high mammary gland density as a mammary gland parenchymal area.

  For example, the extraction unit 145d assigns a pixel value “0” to a pixel having a luminance value equal to or less than the threshold A, and assigns a pixel value “1” to a pixel whose luminance value is greater than the threshold A and equal to or less than the threshold B. A pixel value “2” is assigned to a pixel whose luminance value is larger than the threshold value B. As a result, as shown in the left and center diagrams of FIG. 27, the mammography image to be processed is ternarized, and the region 171 having the pixel value “0”, the region 172 having the pixel value “1”, and the pixel value It is divided into “2” area 173. Here, the region 171 is a region indicating fat, the region 172 is a region indicating mammary gland parenchyma, and the region 173 is a region indicating the chest wall. Thereafter, the extraction unit 145d extracts a mammary gland substantial region by extracting the region 172 having the pixel value “1”. Here, the extraction unit 145d may extract the outline (boundary) of the region having the pixel value “1” as the mammary gland substantial region. Then, as illustrated on the right side of FIG. 27, the extraction unit 145d superimposes and displays the extracted mammary gland actual region on the mammography image on the reference screen displayed by the display control unit 145c.

  Note that the extraction unit 145d may extract the mammary gland parenchymal region using another edge detection method. For example, the extraction unit 145d may extract the mammary gland parenchymal region using a differential edge detection method based on the luminance value distribution in the mammography image. The differential edge detection method is a technique for detecting the zero crossing of the second derivative in the gradient direction in the gradient of the luminance value with respect to the change of the luminance value, and the edge position can be obtained with the accuracy of the subpixel. .

Further, the extraction unit 145d may extract the mammary gland parenchymal region using a Sobel operator based on the distribution of luminance values in the mammography image. The Sobel operator is a method for obtaining the intensity (differential value) of the gradient of the luminance value by local product-sum operation with respect to the change of the luminance value. For example, the Sobel operator, the following equation (4) the coefficient matrices f _x and f _y shown in is used. f _x is the coefficient matrix for detecting the longitudinal edges, f _y is the coefficient matrix for detecting the lateral edges.

  And the intensity | strength | ▽ f | of the gradient of a luminance value is calculated | required by Formula (5) shown below.

  Further, the edge direction θ is obtained by the following equation (6).

_{^{θ = tan -1 (f x /}} f y) ··· (6)

  Returning to FIG. 24, the specifying unit 145e specifies a region of interest in the breast of the subject based on the mammary gland parenchymal region extracted by the extracting unit 145d. Then, the specifying unit 145e displays the specified region of interest on the schematic diagram of the mammary gland region on the reference screen displayed by the display control unit 145c. The specifying unit 145e specifies a region of interest in the breast based on the mammary gland parenchyma regions extracted from the MLO-direction mammography image and the CC-direction mammography image by the extraction unit 145d for the left and right breasts of the subject, respectively. .

  For example, the specifying unit 145e sets the region set by the operator based on the mammary parenchyma region extracted from the mammography image in the MLO direction and the operator sets based on the mammary parenchymal region extracted from the mammography image in the CC direction. The overlapping part with the region is specified as the region of interest. In this way, by specifying the overlapping portion of the regions set in the mammography images in different directions as the region of interest, the region to be inspected with particular attention when the sonographer performs the ultrasound examination, It becomes possible to present more accurately.

  FIG. 28 is a diagram for describing identification of a region of interest performed by the identifying unit 145e according to the fourth embodiment. Here, an example in which the region of interest in the right breast of the subject is specified will be described, but the region of interest can also be specified in the same way for the left breast.

  As shown on the upper left side of FIG. 28, the mammary gland parenchymal region 181 extracted from the mammography image in the MLO direction by the extraction unit 145d is displayed on the mammography image 163 in the MLO direction on the reference screen. Further, as shown in the lower left side of FIG. 28, the mammary gland parenchyma region 182 extracted from the CC direction mammography image by the extraction unit 145d is displayed on the CC direction mammography image 164 on the reference screen.

  When each mammary gland parenchymal region is displayed by the extraction unit 145d, the specifying unit 145e receives an operation of setting an area on the schematic diagram 166 of the mammary gland region from the operator via the input unit 141. Specifically, as shown in the upper center of FIG. 28, the specifying unit 145e moves the operator onto the schematic diagram 166 of the mammary gland region based on the mammary gland parenchymal region 181 displayed on the mammography image 163 in the MLO direction. An operation for setting the region 183 is accepted. In addition, as shown in the lower center of FIG. 28, the specifying unit 145e allows the operator to input an area 184 on the schematic diagram 166 of the mammary gland area based on the mammary gland actual area 182 displayed on the mammography image 164 in the CC direction. Accepts an operation to set. Then, when the two regions 183 and 184 are set by the operator, the specifying unit 145e specifies an overlapping portion of each region as a region of interest 185 as illustrated on the right side of FIG.

  For example, the specifying unit 145e may automatically set the region of interest instead of receiving the region setting from the operator. For example, the specifying unit 145e sets one length of the region of interest from the size of the mammary gland parenchyma region extracted from the mammography image in the MLO direction, and determines the size of the mammary gland parenchyma region extracted from the mammography image in the CC direction. The region of interest is specified by setting the other length of the region of interest.

  FIG. 29 is a diagram for explaining another example of specifying a region of interest performed by the specifying unit 145e according to the fourth embodiment. For example, as shown in the upper side of FIG. 29, the specifying unit 145e sets in advance the correspondence between the position of the region in the mammography image and the position of the region in the schematic diagram of the mammary gland region. 29, the specifying unit 145e calculates a length x in the inner-outer direction of the mammary gland parenchyma region in the mammography image in the CC direction, and determines the region of interest according to the calculated length x. The length x ′ in the inner-outer direction is set. The specifying unit 145e sets the upper-lower direction length y ′ of the region of interest according to the upper-lower direction length of the mammary gland parenchyma region in the mammography image in the MLO direction.

  FIG. 30 is a flowchart illustrating a processing procedure of processing performed by the image display device 140 according to the fourth embodiment. As illustrated in FIG. 30, when the image display device 140 receives an instruction to display a mammography image and finding information from the operator (Yes in step S201), the image data acquisition unit 145a causes the left and right breasts of the subject to be displayed. , An MLO direction mammography image and a CC direction mammography image are acquired (steps S202 and S203). In addition, the finding information acquisition unit 145b acquires finding information regarding the breast of the subject (step S204).

  Subsequently, the display control unit 145c displays a reference screen for referring to the mammography image on the display unit 142 (step S205). Thereafter, when the processing unit start button 167 on the reference screen is pressed by the operator (step S206, Yes), the extraction unit 145d extracts the mammary gland parenchymal region from each of the mammography image in the MLO direction and the mammography image in the CC direction of the subject. Extract (steps S207 and S208).

  Subsequently, the specifying unit 145e specifies a region of interest in the breast based on the mammary gland parenchymal region extracted from each of the mammography image in the MLO direction and the mammography image in the CC direction by the extraction unit 145d (step S209). Then, the specifying unit 145e displays the specified region of interest on the reference screen displayed by the display control unit 145c (Step S210).

  As described above, in the fourth embodiment, the image display device 140 extracts a mammary gland region from a mammography image obtained by imaging the subject's breast, and identifies a region of interest in the breast based on the extracted mammary gland region. And display. Accordingly, an ultrasonic examination engineer can perform an ultrasonic examination with particular attention to an area where the density of the mammary gland is high on the mammography image, and the accuracy of the mammary gland image diagnosis can be improved.

(Fifth embodiment)
In the fourth embodiment described above, an example has been described in which the image display device 140 extracts a mammary gland parenchymal region from a mammography image of a subject, identifies a region of interest, and displays the identified region of interest. On the other hand, in the fifth embodiment, the image processing device 130 extracts the mammary gland parenchyma region from the mammography image of the subject and specifies the region of interest, and the image display device 40 is specified by the image processing device 130. An example of displaying a region of interest will be described. The configuration of the apparatus included in the medical information processing system according to the fifth embodiment is the same as that shown in FIG.

  FIG. 31 is a diagram illustrating a configuration example of an image processing apparatus 130 according to the fifth embodiment. As illustrated in FIG. 31, the image processing apparatus 130 includes an input unit 131, a display unit 132, a communication control unit 133, a storage unit 134, and a control unit 135.

  The input unit 131 receives input of various operations and various information from the operator. For example, the input unit 131 is a keyboard, a mouse, a button, a trackball, a touch panel, or the like.

  The display unit 132 displays a GUI and various images for receiving various operations from the operator. For example, the display unit 132 is a liquid crystal display, a CRT (Cathode Ray Tube) display, a touch panel, or the like.

  The communication control unit 133 controls communication performed with other devices via the network 50. For example, the communication control unit 133 is a network card or a network adapter, and communicates with other devices by connecting to the network 50 via an Ethernet (registered trademark) LAN. Further, for example, the communication control unit 133 performs wireless communication with other devices by connecting to the network 50 via a wireless LAN.

  The storage unit 134 is a storage device such as a hard disk or a semiconductor memory, and stores various types of information. Specifically, the storage unit 134 includes an image data storage unit 134a and a finding information storage unit 134b.

  The image data storage unit 134a stores a mammography image obtained by imaging the breast of the subject. In this image data storage unit 134a, a mammography image is stored by an image data acquisition unit 135a described later.

  The finding information storage unit 134b stores finding information related to the mammography image of the subject. In the finding information storage unit 134b, finding information is stored by a finding information creating unit 135b described later.

  The control unit 135 includes a CPU and a memory, and controls the operation of the image processing apparatus 130 by causing them to execute various programs. Specifically, the control unit 135 includes an image data acquisition unit 135a, a finding information creation unit 135b, a display control unit 135c, an extraction unit 135d, a specification unit 135e, and a transmission unit 135f.

  The image data acquisition unit 135a acquires a mammography image obtained by imaging the breast of the subject. Note that the image data acquisition unit 135a acquires a mammography image in the MLO direction and a mammography image in the CC direction for each of the left and right breasts of the subject. Specifically, the image data acquisition unit 135a acquires a mammography image related to the subject to be diagnosed by communicating with the mammography apparatus 10 via the communication control unit 133.

  The finding information creation unit 135b creates finding information related to the mammography image of the subject based on the findings input by the operator. Specifically, the finding information creation unit 135b receives an input of a finding related to a mammography image from a mammography examination engineer via the input unit 131. Then, the finding information creation unit 135b creates finding information indicating the accepted finding and stores it in the finding information storage unit 134b.

  The display control unit 135c displays a reference screen for referring to the mammography image on the display unit 132. Specifically, when the display control unit 135c receives a display request from the operator via the input unit 131, the display control unit 135c displays the mammography image and the observation information as in the display control unit 145c described in the fourth embodiment. The arranged reference screen is displayed on the display unit 132. At this time, the display control unit 135c uses the mammography image of the subject stored in the image data storage unit 134a and the finding information of the subject stored in the finding information storage unit 134b.

  The extraction unit 135d extracts a mammary gland parenchymal region from a mammography image obtained by imaging the breast of the subject. Then, the extraction unit 135d displays the extracted mammary gland parenchyma region superimposed on the mammography image on the reference screen displayed by the display control unit 135c. Note that the extraction unit 135d extracts a mammary gland parenchyma region from each of the mammography image in the MLO direction and the mammography image in the CC direction for each of the left and right breasts of the subject.

  Specifically, the extraction unit 135d reads out a mammography image related to the subject to be diagnosed from the image data storage unit 134a when the operator of the image processing apparatus 130 presses the processing start button on the reference screen, The mammary gland parenchyma region is extracted from the read mammography image by the same method as the extraction unit 145d described in the embodiment. The extraction unit 135d may extract a mammary gland parenchyma region from the mammography image when the mammography image of the subject is acquired by the image data acquisition unit 135a.

  The specifying unit 135e specifies a region of interest in the subject's breast based on the mammary gland parenchymal region extracted by the extracting unit 135d. Then, the specifying unit 135e displays the specified region of interest on the schematic diagram of the mammary gland region on the reference screen displayed by the display control unit 135c. The specifying unit 135e specifies a region of interest in the breast based on the mammary gland parenchyma regions extracted from the mammography image in the MLO direction and the mammography image in the CC direction by the extraction unit 135d for the left and right breasts of the subject, respectively. .

  Specifically, the specifying unit 135e specifies a region of interest in the subject's breast based on the mammary gland parenchymal region extracted by the extracting unit 135d in the same manner as the specifying unit 145e described in the fourth embodiment. Then, the identified region of interest is displayed on the reference screen.

  The transmission unit 135f transmits information indicating the region of interest identified by the identification unit 135e to the image display device 40 in accordance with an instruction from the operator. Specifically, the transmission unit 135 f receives a display information transmission instruction from the operator of the image processing apparatus 130 via the input unit 131. And the transmission part 135f will generate | occur | produce the display information for displaying the screen containing the information similar to the reference screen after the region of interest is specified by the specific | specification part 135e, if the transmission instruction | indication of display information is received, It transmits to the image display apparatus 40. For example, the transmission unit 135f superimposes the mammary gland actual region extracted by the extraction unit 135d on the mammography image on the reference screen displayed by the display control unit 135c, and sets the region of interest specified by the specifying unit 135e to the mammary gland region Display information for displaying the screen shown in the schematic diagram is generated and transmitted to the image display device 40.

  For example, the transmission unit 135f may transmit information indicating the region of interest specified by the specifying unit 135e to the image display device 40 in response to a request from the image display device 40. In this case, for example, the transmission unit 135f receives a request for display information from the image display device 40 via the communication control unit 133. When receiving a request for display information, the transmission unit 135f generates the display information described above and transmits the display information to the image display device 40 that has transmitted the request.

  As described above, in the fifth embodiment, the image processing apparatus 130 extracts a mammary gland region from a mammography image obtained by imaging the subject's breast, and identifies a region of interest in the breast based on the extracted mammary gland region. To do. Then, the image display device 40 displays information indicating the region of interest specified by the image processing device 130. Thus, for example, a mammography examination engineer specifies an area of interest using the image processing device 130, and an ultrasonic examination engineer uses the image display device 40 to identify an area of interest specified by the mammography examination engineer. Ultrasonography can be performed with particular care. As described above, the mammographic examination engineer can specify the region of interest, thereby further improving the accuracy of the mammary gland image diagnosis.

  In the fifth embodiment, an example has been described in which the extraction unit 135d extracts a mammary gland parenchyma region from a mammography image when the processing start button on the reference screen is pressed by the operator of the image processing apparatus 130. . On the other hand, for example, the extraction unit 135d may extract the mammary gland parenchyma region from the mammography image when a processing start request is received from the image display device 40. In this case, for example, in response to a request from the image display device 40, the display control unit 135c transmits to the image display device 40 information for displaying a reference screen on which the mammography image and the finding information are arranged. . When the operator of the image display device 40 instructs to start processing via the reference screen, the extraction unit 135d receives a request for processing start from the image display device 40 via the network 50.

(Sixth embodiment)
In the first and fifth embodiments described above, the example in which the image display device 40 displays the region of interest in the breast of the subject has been described. In contrast, in the sixth embodiment, an example will be described in which the ultrasound diagnostic apparatus 20 extracts a mammary gland parenchymal region from a mammography image of a subject, identifies the region of interest, and displays the identified region of interest. . The configuration of the apparatus included in the medical information processing system according to the sixth embodiment is the same as that illustrated in FIG.

  FIG. 32 is a diagram illustrating a detailed configuration example of the ultrasonic diagnostic apparatus 20 according to the sixth embodiment. Note that FIG. 32 illustrates the input unit 122, the display unit 123, the apparatus main body 124, the image memory 1245, the internal storage unit 1246, and the communication control in the configuration of the ultrasonic diagnostic apparatus 20 illustrated in FIG. A unit 1247 and a control unit 1248 are shown.

  As shown in FIG. 32, for example, the image memory 1245 includes an image data storage unit 1245a, and the internal storage unit 1246 includes a finding information storage unit 1246a. Note that the image data storage unit 1245a may be included in the internal storage unit 1246, and the finding information storage unit 1246a may be included in the image memory 1245.

  The image data storage unit 1245a stores a mammography image obtained by imaging the breast of the subject. In this image data storage unit 1245a, a mammography image is stored by an image data acquisition unit 1248a described later.

  The finding information storage unit 1246a stores finding information related to the mammography image of the subject. In the finding information storage unit 1246a, finding information is stored by a finding information acquisition unit 1248b described later.

  As illustrated in FIG. 32, for example, the control unit 1248 includes an image data acquisition unit 1248a, a finding information acquisition unit 1248b, a display control unit 1248c, an extraction unit 1248d, and a specification unit 1248e.

  The image data acquisition unit 1248a acquires a mammography image obtained by imaging the breast of the subject. The image data acquisition unit 1248a acquires a mammography image in the MLO direction and a mammography image in the CC direction for each of the left and right breasts of the subject.

  Specifically, the image data acquisition unit 1248a communicates with the image processing device 30 via the communication control unit 1247 to acquire a mammography image related to the subject to be diagnosed, and stores the acquired mammography image in the image data storage. Stored in the unit 1245a. Note that the image data acquisition unit 1248a may acquire a mammography image related to the subject to be diagnosed by communicating with the mammography apparatus 10 via the communication control unit 1247.

  The finding information acquisition unit 1248b acquires finding information related to the breast of the subject. Specifically, the finding information acquisition unit 1248b communicates with the image processing apparatus 30 via the communication control unit 1247 to acquire the finding information of the mammography image related to the subject to be diagnosed. Then, the finding information acquisition unit 1248b stores the acquired finding information in the finding information storage unit 1246a.

  The display control unit 1248c displays a reference screen for referring to the mammography image on the display unit 123. Specifically, when the display control unit 1248c receives a display request from the operator via the input unit 122, the display control unit 1248c displays the mammography image and the observation information as in the display control unit 145c described in the fourth embodiment. The arranged reference screen is displayed on the display unit 123. At this time, the display control unit 1248c uses the mammography image of the subject stored in the image data storage unit 1245a and the finding information of the subject stored in the finding information storage unit 1246a.

  The extraction unit 1248d extracts a mammary gland parenchyma region from a mammography image obtained by imaging the breast of the subject. Then, the extraction unit 1248d superimposes the extracted mammary gland region on the mammography image on the reference screen displayed by the display control unit 1248c. Note that the extraction unit 1248d extracts a mammary gland parenchyma region from each of the mammography image in the MLO direction and the mammography image in the CC direction for each of the left and right breasts of the subject.

  Specifically, the extraction unit 1248d reads out a mammography image related to the subject to be diagnosed from the image data storage unit 1245a when the operator presses the processing start button on the reference screen, and will be described in the fourth embodiment. The mammary gland parenchyma region is extracted from the read mammography image by the same method as the extraction unit 145d. Note that the extraction unit 1248d may extract the mammary gland parenchyma region from the mammography image when the mammography image of the subject is acquired by the image data acquisition unit 1248a.

  The specifying unit 1248e specifies a region of interest in the subject's breast based on the mammary gland parenchymal region extracted by the extracting unit 1248d. Then, the specifying unit 1248e displays the specified region of interest on the schematic diagram of the mammary gland region on the reference screen displayed by the display control unit 1248c. The specifying unit 1248e specifies the region of interest in the breast based on the mammary gland parenchyma regions extracted from the MLO mammographic image and the CC mammographic image by the extraction unit 1248d for the left and right breasts of the subject, respectively. .

  Specifically, the specifying unit 1248e specifies a region of interest in the subject's breast based on the mammary gland parenchymal region extracted by the extracting unit 1248d in the same manner as the specifying unit 145e described in the fourth embodiment. Then, the identified region of interest is displayed on the reference screen.

  As described above, in the sixth embodiment, the ultrasound diagnostic apparatus 20 extracts a mammary gland parenchyma region from a mammographic image obtained by imaging the subject's breast, and based on the extracted mammary gland parenchyma region, the region of interest in the breast is extracted. Identify and display. Thereby, even if it does not use the image display apparatus 40, the area | region which should be test | inspected can be shown through the ultrasonic diagnostic apparatus 20 especially with respect to the technician of ultrasonic examination.

  In the sixth embodiment, an example has been described in which the ultrasound diagnostic apparatus 20 extracts a mammary gland parenchyma region from a mammography image, identifies the region of interest, and displays the identified region of interest. As described in the fifth embodiment, the image processing apparatus 30 may extract the mammary gland parenchyma region from the mammography image and specify the region of interest.

  In this case, for example, in the ultrasonic diagnostic apparatus 20, the communication control unit 1247 transmits, via the network 50, a mammography image obtained by imaging the breast of the subject and information on the region of interest for the mammography image to the image processing apparatus 30. Receive from. Further, the image processing unit 244 generates an ultrasonic image based on the data collected by the ultrasonic probe 21. Then, the display control unit 1248c displays the mammography image received by the communication control unit 1247, the region of interest, and the ultrasound image generated by the image processing unit 244 on the display unit 123.

  In the first to sixth embodiments described above, an example in which a mammography image captured by the mammography apparatus 10 is used has been described, but the embodiment is not limited thereto. For example, an image captured by a screen film system may be used instead of a mammography image.

  In the first to sixth embodiments described above, an example in which an MLO direction mammography image and a CC direction mammography image are used has been described. However, the embodiment is not limited thereto. For example, a mammography image captured from a direction other than the MLO direction and the CC direction may be used.

  The function of the control unit 35 included in the image processing apparatus 30 described in the above embodiment, the function of the control unit 45 included in the image display apparatus 40, and the function of the control unit 248 included in the ultrasonic diagnostic apparatus 20 are respectively software. Can also be realized. For example, the function of each control unit is realized by causing a computer to execute a medical information processing program that defines the processing procedure described as being performed by each control unit. In this case, the medical information processing program is stored in, for example, a hard disk or a semiconductor memory device, and is read and executed by a processor such as a CPU or MPU. The medical information processing program can be recorded and distributed on a computer-readable recording medium such as a CD-ROM (Compact Disc-Read Only Memory), an MO (Magnetic Optical disk), or a DVD (Digital Versatile Disc).

  According to at least one embodiment described above, the accuracy of breast image diagnosis can be improved.

  Although several embodiments of the present invention have been described, these embodiments are presented by way of example and are not intended to limit the scope of the invention. These embodiments can be implemented in various other forms, and various omissions, replacements, and changes can be made without departing from the spirit of the invention. These embodiments and their modifications are included in the scope and gist of the invention, and are also included in the invention described in the claims and the equivalents thereof.

DESCRIPTION OF SYMBOLS 100 Medical information processing system 20 Ultrasound diagnostic apparatus 30 Image processing apparatus 34 Storage part 34a Image data storage part 35 Control part 35d Area setting part 35e Position specification part 40 Image display apparatus

Claims (20)

A storage unit that stores a mammography image obtained by imaging the breast of the subject, and information indicating the imaging direction of the mammography image;
A region setting unit for setting a region of interest in the mammography image;
A position specifying unit for specifying position information of the region of interest on a schematic diagram schematically representing a breast based on position information of the region of interest on the mammography image and information indicating the imaging direction; ,
A medical information processing system comprising: an output unit that outputs position information of the region of interest on the schematic diagram. The storage unit stores a first mammography image obtained by imaging the breast from a first imaging direction and a second mammography image obtained by imaging the breast from a second imaging direction,
The region setting unit sets a first region of interest in the first mammography image, sets a second region of interest in the second mammography image,
The position specifying unit includes position information of the first region of interest on the first mammography image, position information of the second region of interest on the second mammography image, and the first imaging direction. And position information of each of the first region of interest and the second region of interest on the schematic diagram based on the information indicating the information and the information indicating the second imaging direction,
The medical information processing system according to claim 1. The position specifying unit is an overlapping portion of the first region of interest and the second region of interest on the schematic diagram based on position information of each of the first region of interest and the second region of interest. Identify location information for
The medical information processing system according to claim 2. The position specifying unit specifies the position of the region of interest on the schematic diagram based on position information represented by a device coordinate system of a mammography apparatus that has captured the mammography image.
The medical information processing system according to any one of claims 1 to 3. The position specifying unit specifies a position of the region of interest on the schematic diagram by predetermined image processing from the mammography image;
The medical information processing system according to any one of claims 1 to 3. The region setting unit extracts a mammary gland parenchyma region from a mammography image obtained by imaging the breast of the subject, and sets the region of interest based on the mammary gland parenchymal region.
The medical information processing system according to claim 1. The region setting unit extracts a first mammary gland region from a mammography image obtained by imaging the breast from a first imaging direction, and a second mammary gland region from a mammography image obtained by imaging the breast from a second imaging direction. And setting the region of interest based on the first mammary gland region and the second mammary gland region.
The medical information processing system according to claim 6. The region setting unit sets one length of the region of interest from the size of the first mammary parenchymal region and sets the other length of the region of interest from the size of the second mammary parenchymal region. To set the region of interest,
The medical information processing system according to claim 7. The region setting unit sets, as the region of interest, an overlapping portion between a region set by the operator based on the first mammary parenchymal region and a region set based on the second mammary parenchymal region,
The medical information processing system according to claim 7. The region setting unit extracts the mammary gland parenchyma region by threshold processing based on a luminance value distribution in the mammography image.
The medical information processing system according to any one of claims 6 to 9. The region setting unit extracts the mammary gland parenchymal region using a differential edge detection method based on a distribution of luminance values in the mammography image.
The medical information processing system according to any one of claims 6 to 9. The region setting unit extracts the mammary gland parenchymal region using a Sobel operator based on a luminance value distribution in the mammography image.
The medical information processing system according to any one of claims 6 to 9. The output unit outputs position information of the region of interest on the schematic diagram as reference information at the time of ultrasonic diagnosis.
The medical information processing system according to any one of claims 1 to 12. The output unit outputs position information on a schematic diagram corresponding to a patient undergoing ultrasonic diagnosis,
The medical information processing system according to any one of claims 1 to 13. The output unit further outputs position information of a blind area on the schematic diagram;
The medical information processing system according to any one of claims 1 to 14. An image display device and an image processing device connected to each other via a network;
The image display device includes the output unit,
The image processing apparatus includes:
The storage unit;
The region setting unit;
The position specifying unit;
The medical information processing according to claim 1, further comprising: a transmission unit that transmits information indicating the region of interest to the image display device in response to an instruction from an operator. system. An image display device and an image processing device connected to each other via a network;
The image display device includes the output unit,
The image processing apparatus includes:
The storage unit;
The region setting unit;
The position specifying unit;
The medical device according to any one of claims 1 to 15, further comprising: a transmission unit that transmits information indicating the region of interest to the image display device in response to a request from the image display device. Information processing system. A procedure for setting a region of interest in a mammography image obtained by imaging a breast of a subject;
Based on the position information of the region of interest on the mammography image and the information indicating the imaging direction with reference to a storage unit that stores the mammography image and information indicating the imaging direction of the mammography image. A procedure for identifying position information of the region of interest on a schematic diagram schematically representing
A medical information processing program causing a computer to execute a procedure for outputting position information of the region of interest on the schematic diagram. An acquisition unit for acquiring a mammography image obtained by imaging the breast of the subject, and information indicating a photographing direction of the mammography image;
A region setting unit for setting a region of interest in the mammography image;
A position specifying unit for specifying position information of the region of interest on a schematic diagram schematically representing a breast based on position information of the region of interest on the mammography image and information indicating the imaging direction; ,
An ultrasonic diagnostic apparatus comprising: an output unit that outputs position information of the region of interest on the schematic diagram. A communication control unit that receives, from the image processing device, a mammography image obtained by imaging the breast of the subject and information indicating a region of interest set in the mammography image via the network;
An ultrasonic probe;
An image processing unit that generates an ultrasonic image based on data collected by the ultrasonic probe;
An ultrasonic diagnostic apparatus comprising: a display control unit configured to display the mammography image, the region of interest, and the ultrasonic image on a display unit.