JP5460547B2 - Medical image diagnostic apparatus and control program for medical image diagnostic apparatus - Google Patents

Description

  The present disclosure relates to a medical image diagnostic apparatus that performs a diagnosis using a medical image, and a control program for the medical image diagnostic apparatus.

  In recent years, medical image diagnostic apparatuses that diagnose a subject using a medical image obtained by photographing the subject have become widespread. As an example of a medical image diagnostic apparatus, an ultrasonic wave is transmitted from an ultrasonic transducer incorporated in an ultrasonic probe to a subject, and a reflected wave generated in the subject (hereinafter simply referred to as an echo signal). There is an ultrasonic diagnostic apparatus that receives an ultrasonic wave and generates an ultrasonic image based on an echo signal. 2. Description of the Related Art In recent years, scanning a three-dimensional space by arranging ultrasonic transducers built in an ultrasonic probe in a two-dimensional manner or by oscillating ultrasonic transducers arranged in a row by a motor. An ultrasonic diagnostic apparatus capable of performing the above has been put into practical use.

  In such a medical image diagnostic apparatus that processes a 3D medical image, a technique for performing projection processing on volume data obtained by scanning a three-dimensional space and generating a projection image has been put into practical use. By setting the projection direction when generating the projection image to an arbitrary direction, the scan range of the subject can be observed from a desired direction.

JP 2004-283373 A

  Consider a case where an operator makes a diagnosis using a projection image in the medical image diagnostic apparatus as described above. When the operator finds a specific site such as a disease in the projection image, the operator adds a character string (hereinafter simply referred to as supplementary information) representing clinical information regarding the specific site to the projection image. Adding incidental information to the projected image is useful for grasping previously discovered clinical information when looking back at the ultrasound image at a later date or when a different operator refers to the projected image.

  Consider a case where the incidental information is applied to volume data in a three-dimensional space. When additional information is added to volume data in a three-dimensional space, a specific part to be indicated by the additional information also has a three-dimensional positional relationship. Since there was no means for easily grasping the positional relationship of the incidental information, for example, when a plurality of volume data is added to the volume data, the operator who refers to the medical image has which additional information among the plurality of incidental information in front. It was difficult to grasp whether it was on the side or the back side.

  Therefore, an object of the present disclosure is to easily grasp the three-dimensional positional relationship of incidental information on a medical image.

  In order to solve the above problems, in the medical image diagnostic apparatus according to the embodiment, medical image acquisition means for acquiring a medical image obtained by projecting three-dimensional volume data from a predetermined viewpoint, and the three-dimensional coordinates of the volume data are associated with each other. Attached information acquisition means for acquiring a plurality of incidental information, a distance reading means for reading a distance between the viewpoint and each of the plurality of incidental information, and controlling transparency or luminance of the plurality of incidental information based on the distance And ancillary information display means for superimposing and displaying the plurality of incidental information on the medical image.

  Further, in order to solve the above problem, in the control program of the medical image diagnostic apparatus according to the embodiment, a step of acquiring a medical image obtained by projecting three-dimensional volume data from a predetermined viewpoint and the three-dimensional coordinates of the volume data are associated. Obtaining a plurality of supplementary information, calculating a distance between the viewpoint and each of the plurality of supplementary information, and controlling the plurality of supplementary information or luminance based on the distance, And displaying information superimposed on the medical image.

The block diagram which shows the internal structure of the medical image diagnostic apparatus which concerns on embodiment. The figure which shows the projection image which concerns on embodiment. The figure which shows a mode that the annotation was superimposed on the projection image which concerns on embodiment. The figure which shows a mode when the display area of the annotation which concerns on embodiment overlaps. 6 is a flowchart showing a flow of processing for superimposing and displaying an annotation in a projection image according to the embodiment. The figure which shows the annotation which has angle information which concerns on embodiment. The figure which shows another annotation which has angle information which concerns on embodiment. The figure which shows the coloring display of the annotation and volume data which concern on embodiment. The figure which shows a mode that the annotation outside a screen which concerns on embodiment is displayed. The figure which shows a mode that the display color of the annotation which concerns on embodiment is changed.

(Configuration of medical image diagnostic apparatus 1)
Hereinafter, embodiments of the present disclosure will be described with reference to the drawings. FIG. 1 is a block diagram illustrating an internal configuration of a medical image diagnostic apparatus 1 according to each embodiment. In this embodiment, as an example of the medical image diagnostic apparatus 1, an ultrasonic diagnostic apparatus that captures an ultrasonic image using ultrasonic waves will be described, and the medical image diagnostic apparatus captures an ultrasonic image as one type of medical image. State as what you want to do. However, the medical image diagnostic apparatus 1 is not limited to an ultrasonic diagnostic apparatus, and may be various types such as an X-ray diagnostic apparatus, an X-ray CT (Computed Tomography) apparatus, a magnetic resonance imaging apparatus, a nuclear medicine diagnostic apparatus, and an endoscopic imaging apparatus. It may be a device for taking a medical image. Further, in the present embodiment, the medical image diagnostic apparatus 1 is not limited to the one provided with a medical image photographing unit such as the ultrasonic probe 20. For example, a workstation that receives volume data, incidental information, medical images, and the like from an external terminal and performs processing is also included in the medical image diagnostic apparatus 1 referred to in the present embodiment.

  The medical image diagnostic apparatus 1 is configured by combining a system control unit 10 and an ultrasonic probe 20, for example. The system control unit 10 includes an input unit 11, a storage unit 12, a display unit 13, an incidental information generation unit 14, a medical image generation unit 15, a volume data generation unit 16, a transmission / reception unit 17, a B mode processing unit 18, and a Doppler processing unit 19. Consists of The system control unit 10 outputs a control command to each component and controls the medical image diagnostic apparatus 1 in an integrated manner. In addition, the structure of the system control part 10 in this embodiment is not restricted to this. Appropriate components may be added, or a processing device connected to the outside of the medical image diagnostic apparatus 1 may play several roles of the components.

  For example, when the medical image diagnostic apparatus 1 is an X-ray diagnostic apparatus, an X-ray irradiation unit and an X-ray detection are used instead of the transmission / reception unit 17, the B-mode processing unit 18, the Doppler processing unit 19, and the ultrasonic probe 20. Etc. are provided. Further, when the medical image diagnostic apparatus 1 is a workstation that processes medical images, the medical image diagnostic apparatus 1 is connected to the medical image diagnostic apparatus 1 instead of the transmission / reception unit 17, the B-mode processing unit 18, the Doppler processing unit 19, and the ultrasonic probe 20. An interface unit for exchanging information with the connected terminal is provided. The interface unit acquires and outputs volume data, incidental information, medical images, and the like via a terminal connected to the medical image diagnostic apparatus 1.

  The ultrasonic probe 20 is connected to the system control unit 10. The ultrasonic probe 20 includes an ultrasonic transducer 21 and controls the ultrasonic transducer 21 to be driven based on the drive signal output from the transmission / reception unit 17.

  The ultrasonic transducer 21 is a member that generates an ultrasonic wave based on a drive signal input via the ultrasonic probe 20 and receives the ultrasonic wave reflected from the subject and converts it into an electric signal. The ultrasonic transducer 21 outputs an electrical signal (hereinafter simply referred to as an echo signal) generated by receiving the ultrasonic wave to the transmission / reception unit 17.

  Note that the ultrasonic probe 20 in the present embodiment is configured by, for example, a mechanical 4D probe (mechanical three-dimensional probe) or a 2D array probe (matrix array probe), and directivity of ultrasonic waves transmitted and received by the ultrasonic transducer 21. Is configured to control the two-dimensional direction. The mechanical 4D probe is a probe in which the ultrasonic transducers 21 are arranged in a one-dimensional direction and a motor that swings the ultrasonic transducers 21 in a direction orthogonal to the arrangement direction is attached. The mechanical 4D probe controls the directivity of the ultrasonic wave in a two-dimensional direction by driving the ultrasonic vibrator 21 while driving the motor. On the other hand, the 2D array probe is a probe in which the ultrasonic transducers 21 are arranged on a plane in a grid pattern. The 2D array probe controls the directivity of the ultrasonic wave in a two-dimensional direction by controlling the drive timing of each ultrasonic transducer 21. Note that the configuration of the present embodiment is not limited to this, and the ultrasonic probe 20 may scan a three-dimensional space by an arbitrary method different from the mechanical 4D probe and the 2D array probe. For example, the three-dimensional space may be scanned by arranging the ultrasonic transducers 21 in a one-dimensional direction and performing scanning while moving the ultrasonic probe 20 manually.

  The transmission / reception unit 17 includes an amplifier circuit that processes an echo signal received by the ultrasonic probe 20, an A / D converter, an adder, and the like. The amplifier circuit amplifies the echo signal received by the ultrasonic probe 20 and outputs it to the A / D converter. The A / D converter gives a delay time necessary for determining the reception directivity of the amplified echo signal to the echo signal and outputs it to the adder. The adder adds echo signals given a delay time to obtain an echo signal corresponding to the scan line for transmitting the ultrasonic wave. The transmission / reception unit 21 outputs an echo signal corresponding to the scan line to the B-mode processing unit 18 or the Doppler processing unit 19. The transmission / reception unit 17 outputs a drive signal for driving the ultrasonic transducer 21 and transmitting ultrasonic waves to the ultrasonic ultrasonic probe 20.

  The B-mode processing unit 18 generates a B-mode signal that changes according to the amplitude intensity of the echo signal output from the transmission / reception unit 17. The B mode processing unit 18 outputs the generated B mode signal to the volume data generating unit 16.

  The Doppler processing unit 19 detects the frequency transition of the echo signal and generates a Doppler signal obtained by extracting the moving speed of the tissue or blood flow. The Doppler processing unit 19 outputs a Doppler signal to the volume data generation unit 16.

  The volume data generation unit 16 generates three-dimensional volume data based on the B mode signal and the Doppler signal output from the B mode processing unit 18 or the Doppler processing unit 19. Volume data is data configured by combining three-dimensional unit areas called voxels. When the ultrasonic probe 20 is a mechanical 4D probe, the volume data generation unit 16 sets the swing angle of the ultrasonic transducer 21 at the time of receiving the echo signal and the distance in the depth direction corresponding to the echo signal. In response, the three-dimensional coordinates are specified, and the B-mode signal or the Doppler signal is mapped to the voxels at the specified three-dimensional coordinates. On the other hand, when the ultrasonic probe 20 is a 2D array probe, the volume data generation unit 16 determines 3 according to the arrangement position of the ultrasonic transducers 21 that received the echo signal and the distance in the depth direction corresponding to the echo signal. Dimensional coordinates are specified, and B-mode signals or Doppler signals are mapped to voxels at the specified three-dimensional coordinates. When the volume data generation unit 16 generates the volume data, the volume data generation unit 16 outputs the volume data to the medical image generation unit 15 or the storage unit 12.

  The incidental information generation unit 14 generates incidental information regarding volume data based on an input operation of the input unit 11 described later. When the accompanying information generating unit 14 generates the accompanying information, the accompanying information is output to the medical image generating unit 15 or the storage unit 12 in association with the volume data or the ultrasonic image. In the present embodiment, the incidental information generation unit 14 is described as generating incidental information in association with volume data. However, the configuration of the embodiment is not limited to this, and incidental information may be generated in association with an ultrasonic image. In the present embodiment, supplementary information refers to clinical information (annotation) added to volume data or an ultrasound image. The clinical information indicates, for example, at which position in the volume data or the ultrasonic image a specific site such as a disease is present, and information such as character information such as findings and information such as a graphic is displayed at the indicated position. The incidental information includes, for example, an “accompanying information ID” that is an identification number of the incidental information, a “display name” that determines the contents of a character string or a graphic displayed on the screen, and any position in the three-dimensional space of the volume data. It consists of three pieces of information of “coordinates” indicating whether the information is associated. These pieces of information can be arbitrarily set based on the input operation received by the input unit 11. Further, the name of information included in the supplementary information is not limited to this, and the number of information included in the supplementary information is not limited thereto. For example, the information of “accompanying information ID” may be omitted, and the incidental information may be configured from two pieces of information of “display name” and “coordinates”. “Angle”, “length”, and “color” to be described later It may have various information such as.

  The medical image generation unit 15 is a processing unit that generates a projection image obtained by rendering the volume data and the accompanying information as an ultrasonic image. The projection image is generated by the following procedure, for example. First, the medical image generation unit 15 sets a viewpoint and a line-of-sight direction in a three-dimensional space of volume data. The viewpoint and the line-of-sight direction are set by an input operation received by the input unit 11, for example. Further, the line of sight and the direction of the line of sight may be automatically set based on the parameters stored in the storage unit 12. When the viewpoint and the line-of-sight direction are set, the medical image generation unit 15 calculates the projection plane of the projection image based on the viewpoint and the line-of-sight direction, and sets the projection plane in the three-dimensional space of the volume data. Next, the medical image generation unit 15 examines the voxel value of each voxel on the straight line connecting the viewpoint and the projection plane, and extracts the voxel value of the voxel closest to the viewpoint from the voxels exceeding the predetermined transmission threshold value. Map to the projection plane. By performing this voxel value extraction and mapping for each pixel on the projection plane, the medical image generation unit 15 generates a projection image. Through the above processing, the projection image is obtained as an ultrasonic image of the structure 100 having a value exceeding the transmission threshold in the three-dimensional space of the volume data as viewed along the line-of-sight direction from an arbitrary viewpoint. When the medical image generation unit 15 finishes the volume data projection process, the medical image generation unit 15 performs a process of superimposing and displaying the auxiliary information output by the auxiliary information generation unit 14 on the projection image. The operation for displaying the accompanying information in a superimposed manner and the operation for calculating the distance between the accompanying information and the viewpoint will be described in detail later.

  Note that the ultrasonic image generated by the medical image generation unit 15 in the embodiment is not limited to the projection image described here. For example, the medical image generation unit 15 may perform a so-called maximum value projection in which the maximum voxel value is mapped to the projection plane among the voxel values of each voxel on the straight line based on the viewpoint and the line-of-sight direction. You may perform what is called minimum value projection which maps a value on a projection surface. Alternatively, an MPR image may be generated by performing MPR (Multi Planar Information) processing in which voxel values on a predetermined cross section in volume data are projected onto a two-dimensional plane and mapped.

  The display unit 13 is configured by an arbitrary display such as an LCD (Lucid Crystal Display) display, an organic EL (Electro Luminescence) display, or a cathode ray tube display. The display unit 13 displays the ultrasonic image output from the medical image generation unit 15. Alternatively, display parameters when the medical image generation unit 15 displays an ultrasound image, parameters when the ultrasound probe 20 transmits and receives ultrasound, and the like are displayed. The display unit 13 is not limited to the display described here. For example, the display unit 13 may be configured as a so-called 3D display that is stereoscopically visible to the operator by displaying two images with parallax with different polarizations. Alternatively, a goggle capable of projecting video may be provided separately from the display that displays the ultrasonic image, and incidental information may be projected onto the goggle, and the display unit 13 may be configured by integrating the display and the goggles. Absent.

  The storage unit 12 is a storage medium including, for example, a ROM (Read Only Memory), a RAM (Random Access Memory), a flash memory, an HDD (Hard Disk Drive), and the like. The storage unit 12 stores volume data generated by the volume data generation unit 16, auxiliary information generated by the auxiliary information generation unit 14, or ultrasonic images generated by the medical image generation unit 15.

  The input unit 11 is configured using various operation devices such as mechanical buttons, dials, trackballs, joysticks, sliders, and wheels, for example, and accepts operator input operations and converts the input operations into electrical signals. This is a member that is output to the system control unit 10. The input unit 11 outputs, for example, an instruction signal for instructing the transmission / reception unit 17 to start / stop transmission / reception of ultrasonic waves according to the received input operation, or a viewpoint / line-of-sight direction in which the medical image generation unit 15 generates an ultrasonic image. A signal for setting the display direction and setting the incidental information generated by the incidental information generation unit 14 is output to the system control unit 10.

(Coordinate input for incidental information)
The incidental information generation unit 14 sets incidental information according to the input operation received by the input unit 11. For example, the operator operates the keyboard of the input unit 11 to set “accompanying information ID”, “display name”, and “coordinates” in the incidental information. The “coordinate” can be set by various methods described below in addition to the method of directly inputting the numerical value of each coordinate using the keyboard.

  For example, the input unit 11 includes a trackball that can be pushed. The incidental information generation unit 14 changes the x coordinate and the y coordinate among the three coordinates x, y, and z according to the operation of rotating the trackball. On the other hand, the incidental information generation unit 14 changes the z coordinate according to the operation of pushing the trackball.

  Further, for example, the input unit 11 is provided with a dial that receives a rotation operation around the trackball. The incidental information generation unit 14 changes the x coordinate and the y coordinate according to the operation of rotating the trackball. On the other hand, the incidental information generation unit 14 changes the z coordinate according to an operation of rotating the dial.

  Further, for example, the input unit 11 includes a joystick that is a rod-shaped member that can be tilted or pushed. The incidental information generation unit 14 changes the x coordinate and the y coordinate according to the operation of tilting the joystick. On the other hand, the incidental information generation unit 14 changes the z coordinate according to the operation of pushing the joystick.

  For example, the input unit 11 includes a remote controller with a built-in position sensor. The incidental information generation unit 14 transforms the x coordinate, the y coordinate, and the z coordinate according to the positional relationship between the remote controller and the display detected by the position sensor. The operator holds the remote control in his hand and sets each coordinate by moving the remote control relative to the display.

  For example, the input unit 11 includes a touch panel with a built-in pressure sensor. The incidental information generation unit 14 changes the x-coordinate and the y-coordinate according to the coordinate information that has received the pressing operation on the touch panel. On the other hand, the incidental information generation unit 14 changes the z coordinate according to the pressure detected by the touch panel.

  In addition to setting the x, y, and z coordinates, the incidental information generation unit 14 may set the coordinates using volume data or an ultrasonic image displayed on the display unit 13. For example, the incidental information generation unit 14 may specify each coordinate by moving the cursor in the ultrasonic image onto which the volume data is projected. By moving the cursor while changing the viewpoint / line-of-sight direction S on which the volume data is projected, the operator moves the cursor to an arbitrary position in the three-dimensional space and designates each coordinate.

  Further, for example, the incidental information generation unit 14 may set a cursor on any one of the tomographic images obtained by reconstructing the volume data with a plurality of cross sections having different positions, and move the cursor to designate each coordinate. Good.

(Attached information display processing)
Using each of the components described above, the medical image diagnostic apparatus 1 displays an ultrasound image on which supplementary information is superimposed. FIG. 2 is a flowchart showing a process when displaying an ultrasound image in which supplementary information is superimposed and displayed. The ultrasonic image display process will be described below with reference to FIG. It is assumed that generation of volume data by the volume data generation unit 16 and generation of incidental information by the incidental information generation unit 14 have been completed prior to the display processing of the ultrasonic image.

  First, when the operator of the medical image diagnostic apparatus 1 starts display processing (step 1000), the system control unit 10 reads volume data that is a target for generating an ultrasound image from the storage unit 12 (step 1001). Subsequently, the system control unit 10 reads incidental information associated with the volume data read in step 1001 from the storage unit 12 (step 1002).

  When the system control unit 10 reads the volume data and the incidental information, the system control unit 10 waits for setting of a viewpoint and a line-of-sight direction when generating an ultrasound image (step 1003). When the operator operates the input unit 11 to set the viewpoint and line-of-sight direction, the medical image generation unit 15 generates an ultrasound image in which volume data is projected from the set viewpoint and line-of-sight direction (step 1004). FIG. 3 shows the relationship between the structure 100 scanned by the ultrasonic probe 20 and the ultrasonic image generated by the medical image generation unit 15. For example, assuming that the structure 100 has a substantially rectangular parallelepiped shape, the operator views a viewpoint / line of sight S in an obliquely upward direction of the structure 100 (in the drawings after FIG. 3, an arrow S that integrally represents the viewpoint and the line of sight is If it is set as “line of sight S”, the medical image generation unit 15 generates an ultrasound image of the structure 100 viewed from the viewpoint / line of sight S.

  When the medical image generation unit 15 generates an ultrasonic image, the medical image generation unit 15 then superimposes additional information on the ultrasonic image (step 1005). FIG. 4 shows the relationship between the incidental information associated with the volume data and the ultrasonic image on which the incidental information is superimposed. Here, the incidental information is assumed to be composed of three pieces of information of “accompanying information ID”, “display name”, and “coordinates”. In the example of FIG. 4, the incidental information A1 is added to the upper surface of the structure 100 as coordinates (x1, y1, z1). The display name of the incidental information A1 is “symtom A1”. The incidental information A2 is added to the front surface of the structure 100 as coordinates (x2, y2, z2). The accompanying information A1 is expressed as “symtom A2”. The medical image generation unit 15 superimposes the incidental information on the ultrasonic image generated in step 1004. Specifically, the medical image generation unit 15 superimposes “symtom A1” in the ultrasound image so as to indicate the coordinates (x1, y1, z1) on the top surface of the structure 100, and also coordinates (x2) on the front surface of the structure 100. , Y2, z2), “symtom A2” is superimposed. In the present embodiment, an example is shown in which a character string is displayed as supplementary information and the coordinates to which the supplementary information is attached and the character string are connected by a line segment, but the configuration of the embodiment is not limited to this. For example, a figure or a medical image may be superimposed as the auxiliary information instead of a character string. In addition, a character string may be superimposed at a position corresponding to the coordinates to which the supplementary information is attached with the line segment omitted. Further, the coordinates with the accompanying information and the character string may be overlapped by connecting them with an arbitrary method such as a curve or an arrow.

  When the medical image generation unit 15 superimposes the supplementary information on the ultrasonic image, the medical image generation unit 15 determines whether or not a plurality of superimposed information display areas overlap in the generated ultrasonic image (step 1006). ). In the present embodiment, “overlap” refers to a state in which character strings or line segment display areas displayed as supplementary information overlap between the supplementary information. FIG. 5 shows an ultrasonic image in a state where the auxiliary information is overlapped. As shown in FIG. 5, when the viewpoint / line-of-sight direction S is set to look up at the structure 100 from the lower surface, the medical image generation unit 15 generates an ultrasound image corresponding to the viewpoint / line-of-sight direction S. At this time, if the viewpoint / gaze direction S is set so that the supplementary information A1 is on the back side of the surface to which the supplementary information A2 is attached, the display areas of the supplementary information attached to the respective surfaces overlap.

  When the medical image generation unit 15 determines that the display areas of the auxiliary information overlap (Yes in Step 1006), the medical image generation unit 15 extracts the auxiliary information where the display areas overlap (Step 1007). In the example of FIG. 5, the incidental information A1 and incidental information A2 are extracted as overlapping display areas.

  When the medical image generation unit 15 extracts the incidental information that overlaps the display areas, the medical image generation unit 15 calculates the distance between the extracted incidental information and the viewpoint. The medical image generation unit 15 may read the distance information from the storage unit 12 instead of calculating the distance. Then, the medical image generation unit 15 controls the additional information farther from the viewpoint among the extracted additional information so that the transparency on the display is increased (step 1008). In this embodiment, “transparency increases” means that the incidental information is displayed in a semi-transparent manner when displaying an ultrasonic image, and the ultrasonic image or incidental information in which the incidental information overlaps the display area is transmitted and displayed. Point to. In the example of FIG. 5, the auxiliary information farther than the set viewpoint, that is, the auxiliary information A1 associated with the deeper coordinates on the ultrasound image is displayed with higher transparency. . Note that the configuration of the present embodiment is not limited to that described here. For example, instead of increasing the transparency of the incidental information, the medical image generation unit 15 may control the luminance on the display of the incidental information to approach the background. In this case, if the background of the ultrasonic image is black, the medical image generation unit 15 performs control so as to reduce the luminance of the incidental information. Alternatively, the medical image generation unit 15 may reset the viewpoint / line-of-sight direction S to a position where the incidental information does not overlap, instead of increasing the transparency of the incidental information. In this case, the medical image generation unit 15 rotates the viewpoint / gaze direction S around the center of the volume data, for example, until the position where the display areas of the auxiliary information A1 and the auxiliary information A2 do not overlap is reached. Further, for example, when the display unit 13 is a 3D display, the medical image generation unit 15 displays the incidental information associated with the coordinates close to the viewpoint as if they were closer, and the incidental information associated with the coordinates far from the viewpoint. Information may be displayed as if it is in a farther position, and the transparency control process described above may be performed on the incidental information with the display position overlapping.

  When the medical image generation unit 15 changes the transparency of the incidental information or determines that there is no additional information overlapping the display area (No in step 1006), the medical image generation unit 15 superimposes the incidental information on the display unit 13. An ultrasonic image is output, and the ultrasonic image is displayed on the display unit 13 (step 1009).

  When the display unit 13 displays the ultrasound image, the system control unit 10 waits for an input for changing the viewpoint and line-of-sight direction when generating the ultrasound image (step 1011). When the operator performs an operation of resetting the viewpoint / gaze direction S using the input unit 11 (Yes in step 1011), the system control unit 10 returns to the process of step 1003 and repeats the process. On the other hand, when the operator does not perform an operation of resetting the viewpoint / line-of-sight direction S (No in Step 1011), the system control unit 10 ends the process (Step 1012).

  Through the above processing, the medical image diagnostic apparatus 1 according to the embodiment controls the additional information overlapping the display area so that the transparency on the display of the auxiliary information at a position far from the viewpoint is lowered. By this operation, the operator can easily grasp the three-dimensional positional relationship of the incidental information. Further, even when the display areas of the supplementary information overlap, the operator can easily grasp which supplementary information is attached to the coordinates close to the viewpoint among the superimposed supplementary information. In addition, the operator can easily visually check incidental information attached to coordinates close to the viewpoint.

(First modification)
Next, a first modification of the medical image diagnostic apparatus 1 will be described. The difference from the first embodiment in the first modification is that an arrow having angle information is displayed instead of a line segment connecting a character string and coordinates as shown in FIG. Hereinafter, the first modification will be described in detail with reference to FIG.

  In the first modification, the incidental information includes five pieces of information of “angle” and “length” in addition to “accompanying information ID”, “display name”, and “coordinate”. The information on “angle” and “length” is used to display an arrow connecting a coordinate with attached information and a character string. The “angle” is configured by combining two pieces of angle information, for example, (θ1, φ1). θ represents an inclination angle of an arrow with respect to the z axis, where the z axis direction is 0 degree on the volume data. On the other hand, φ represents the inclination angle of the arrow with the x axis as 0 degree and the z axis as the rotation center on the volume data. By defining the two angle information of θ and φ, the direction of the arrow in the three-dimensional space is determined. The “length” is defined by one scalar quantity, for example, R1. The length of the arrow in the three-dimensional space is determined as the “length” value.

  The medical image generation unit 15 uniquely determines the position, direction, and length of the arrow in the three-dimensional space based on the above-described three pieces of information “coordinate”, “angle”, and “length”. The medical image generation unit 15 generates an ultrasonic image by projecting the structure 100 and this arrow from the set viewpoint / gaze direction S. As shown in FIG. 6, the incidental information is displayed by combining a character string and an arrow. The medical image generation unit 15 displays the supplementary information in a manner in which the coordinate with the supplementary information is a tip and a character string is attached to the other end. Since the position, direction, and coordinates of the arrow are uniquely determined in the three-dimensional space, the arrow is displayed at a fixed angle with respect to the structure 100 even if the viewpoint / gaze direction S is changed. Become. By this operation, the operator can easily grasp the three-dimensional positional relationship of the incidental information. Further, the operator can easily grasp to which surface of the structure 100 the incidental information is attached.

  The first modified example can be implemented in combination with the transparency control process described in the first embodiment. That is, when the display area of the arrow or the character string of the incidental information A1 and the incidental information A2 overlaps due to the setting of the viewpoint / line-of-sight direction S, the medical image generation unit 15 increases the transparency of the incidental information at a position far from the viewpoint. To control. As a result, the character string and the arrow of the incidental information at a position far from the viewpoint are displayed in the ultrasonic image with high transparency.

(Second modification)
Next, a second modification of the medical image diagnostic apparatus 1 will be described. The difference from the first embodiment in the second modification is that the character string and line segment have angle information as shown in FIG. 7, and are displayed tilted according to the viewpoint / line-of-sight direction S. . Hereinafter, the second modification will be described in detail with reference to FIG.

  In the second modification, the incidental information includes four information of “angle” in addition to “accompanying information ID”, “display name”, and “coordinate”. The “angle” information is used to determine the display direction of the character string and line segment. As described in the first modification, the “angle” is configured by combining two pieces of angle information, for example (θ1, φ1). By defining two angle information of θ and φ, the display direction in the three-dimensional space of the character string and the line segment is determined.

  Based on the information of “coordinates” and “angles” described above, the medical image generation unit 15 uniquely determines the positions of character strings and line segments in the three-dimensional space, and the display direction. As shown in the supplementary information A2 in FIG. 7, the display angle of the supplementary information changes according to the setting of the viewpoint / line-of-sight direction S. On the other hand, as shown in the supplementary information A1 of FIG. 7, when the viewpoint / line-of-sight direction S is set at a position orthogonal to the display direction of the supplementary information, the supplementary information A1 does not tilt with respect to the ultrasound image. Displayed from the front.

  By this operation, the operator can easily grasp the three-dimensional positional relationship of the incidental information. Further, the operator can easily grasp to which surface of the structure 100 the incidental information is attached. In addition, the angle information of the incidental information is set so that the incidental information can be seen from the front in the viewpoint / line-of-sight direction S where the specific part is easily observed. Thus, when the viewpoint / gaze direction S is set later and an ultrasonic image is generated, the viewpoint / gaze direction S is set so that the incidental information can be seen from the front, so that the specific part can be easily observed at the same time. Can be displayed.

  The second modified example can be implemented in combination with the transparency control process described in the first embodiment. That is, when the line segment or the character string display area of the incidental information A1 and the incidental information A2 overlaps due to the setting of the viewpoint / line-of-sight direction S, the medical image generation unit 15 increases the transparency of the incidental information at a position far from the viewpoint. To control. As a result, the line segment and the character string of the incidental information at a position far from the viewpoint are displayed in the ultrasonic image with high transparency.

(Third Modification)
Next, a third modification of the medical image diagnostic apparatus 1 will be described. The difference from the first embodiment in the third modification is that the structure 100 and the accompanying information are displayed in color as shown in FIG. Hereinafter, the third modification will be described in detail with reference to FIG. In the third modification, the structure 100 is described as being composed of two structures having different voxel data.

  In the third modification, the incidental information includes four information of “color” in addition to “accompanying information ID”, “display name”, and “coordinate”. The “color” information is used for coloring and displaying incidental information in the ultrasonic image and the structure 100. For example, in FIG. 8, the “color” of the auxiliary information A1 is set as Blue. In this case, the medical image generation unit 15 displays the color of the supplementary information in blue in the ultrasonic image. As a method of coloring the incidental information, for example, the background of the character string may be colored blue, or the display color of the character string itself may be colored blue. Moreover, you may give the same coloring to a line segment. Further, the medical image generation unit 15 applies blue coloring to the structure 100 to which the auxiliary information A1 is attached. Specifically, the medical image generation unit 15 extracts voxel data having a voxel value similar to the voxel value of the voxel data at the coordinates to which the supplementary information A1 is attached, and performs blue coloring on the voxel data. Perform projection processing. Whether or not the voxel values are similar is determined, for example, depending on whether or not the absolute value difference between the voxel value at the coordinates to which the auxiliary information A1 is attached and the voxel value to be determined is equal to or less than a predetermined threshold value. Is called. For example, it is assumed that the structure 100 includes an L-shaped structure having a certain voxel value and a rectangular parallelepiped structure having another voxel value. Assuming that the supplementary information A1 is attached to a certain coordinate in the L-shaped structure and the supplementary information A2 is attached to a certain coordinate in the rectangular parallelepiped structure, the medical image generation unit 15 has the L-shaped structure. An ultrasonic image is generated by projecting an object in blue corresponding to the color information of the incidental information A1 and projecting a rectangular parallelepiped structure in red corresponding to the color information of the incidental information A2.

  In the present embodiment, the medical image generation unit 15 is described as coloring the voxel data having similar voxel values. However, the configuration of the embodiment is not limited thereto. For example, coloration may be applied only to voxel data having similar voxel values and within a predetermined distance range from the coordinates with the accompanying information. Alternatively, coloring may be applied to all voxel data having a voxel value larger than the voxel value at the coordinate with the accompanying information. Alternatively, the voxel data in an arbitrary range may be colored based on the input operation received by the input unit 11 starting from the coordinates with the accompanying information.

  By this operation, the operator can easily grasp the three-dimensional positional relationship of the incidental information. In addition, the operator can easily grasp which area on the structure 100 the incidental information indicates. This operation is effective in clearly distinguishing the area indicated by the accompanying information when attaching the accompanying information to a complicated structure such as a coronary artery connected to the heart.

  The third modification can be implemented in combination with the transparency control process described in the first embodiment. That is, when the line segment or the character string display area of the incidental information A1 and the incidental information A2 overlaps due to the setting of the viewpoint / line-of-sight direction S, the medical image generation unit 15 increases the transparency of the incidental information at a position far from the viewpoint. To control. As a result, the line segment and the character string of the incidental information at a position far from the viewpoint are displayed in the ultrasonic image with high transparency.

(Fourth modification)
Next, a fourth modification of the medical image diagnostic apparatus 1 will be described. The difference from the first embodiment in the fourth modification is that incidental information outside the display area of the ultrasonic image is displayed as shown in FIG. Hereinafter, the fourth modification will be described in detail with reference to FIG.

  In the fourth modification, the medical image generation unit 15 displays incidental information outside the display area of the ultrasonic image. Specifically, an arrow indicating the direction in which the coordinates with the accompanying information are relative to the display surface is displayed on the ultrasonic image, and the “display name” attached to the accompanying information is displayed together with the arrow. To do. As shown in FIG. 9, when the viewpoint / line-of-sight direction S is set at a position close to the structure 100, the structure 100 is displayed large in the ultrasonic image and attached to the upper surface of the structure 100. The incidental information A1 may not fit within the display area of the ultrasonic image. In such a case, the medical image generation unit 15 calculates the direction in which the coordinates of the auxiliary information A1 are in relation to the display surface of the ultrasonic image determined by the viewpoint / gaze direction S. When the direction is calculated, the medical image generation unit 15 displays an arrow indicating the direction in the frame of the ultrasonic image. Further, the medical image generation unit 15 displays “symtom A1” corresponding to the “display name” of the incidental information A1 together with the arrow. The configuration of the embodiment is not limited to this. For example, instead of calculating the direction from the relationship between the display surface of the ultrasonic image determined by the viewpoint and the line-of-sight direction S and the coordinates of the auxiliary information A1, the medical image generation unit 15 determines the position in the space attached in the ultrasonic probe 20. The direction may be calculated from the relationship between the position sensor to be acquired and the coordinates to which the auxiliary information A1 is attached.

  By this operation, the operator can easily grasp the three-dimensional positional relationship of the incidental information. In addition, even when the display area of the incidental information is located outside the display area of the ultrasonic image, the operator can grasp in which direction the incidental information is with respect to the display surface of the ultrasonic image.

  The fourth modification can be implemented in combination with the transparency control process described in the first embodiment. That is, when the display area of the supplementary information A2 and the supplementary information A1 displayed together with the arrow overlaps due to the setting of the viewpoint / line-of-sight direction S, the medical image generation unit 15 displays the supplementary information displayed together with the arrow. Control to increase the transparency of A1. As a result, the incidental information outside the display area of the ultrasonic image is displayed in the ultrasonic image with high transparency.

(Fifth modification)
Next, a fifth modification of the medical image diagnostic apparatus 1 will be described. The difference from the first embodiment in the fifth modified example is that the display unit 13 includes an HMD (head mounted display) that displays graphics and characters in addition to a display that displays a planar image. The image display surface of the HMD is made of a transparent material, and an operator wearing the HMD can visually recognize the display, the input unit 11 of the medical image diagnostic apparatus 1 and the like through the HMD. The medical image generation unit 15 controls to display a projection image obtained by projecting the volume data on the display constituting the display unit 13, and controls to project and display incidental information on the image display surface of the HUD. .

  In the fifth modification, the HMD includes a position sensor that detects the position and direction of the image display surface of the HMD and the position and direction of the image display surface of the display. When the position sensor detects that there is a display that displays a flat image on the display surface of the HMD, the medical image generation unit 15 displays the display information of the auxiliary information in the HMD so that the auxiliary information overlaps the image display surface of the display. To control.

  By this operation, an operator wearing the HMD can visually recognize an ultrasonic image in which volume data is projected on a display and auxiliary information related to the volume data. On the other hand, for a patient who does not wear the HMD, only the ultrasonic image on which the volume data is projected is visually recognized on the display, and the incidental information cannot be visually recognized. Therefore, incidental information necessary for treatment can be displayed only to the operator of the medical image diagnostic apparatus 1, and a situation in which unnecessary information is displayed for a patient who does not wear the HMD can be avoided.

  The fifth modification can be implemented in combination with the transparency control process described in the first embodiment. That is, when the line segment or the character string display area of the auxiliary information A1 and the auxiliary information A2 overlaps due to the setting of the viewpoint / line-of-sight direction S, the medical image generation unit 15 is at a position far from the viewpoint in the display surface of the HUD. Control to increase the transparency of incidental information. As a result, the character string and the arrow of the incidental information at a position far from the viewpoint are displayed in the ultrasonic image with high transparency.

(Sixth Modification)
Next, a sixth modification of the medical image diagnostic apparatus 1 will be described. The difference from the first embodiment in the sixth modification is that the display color of the auxiliary information is changed according to the display color of the ultrasonic image. Hereinafter, the sixth modification will be described in detail with reference to FIG.

  In the sixth modification, the medical image generation unit 15 changes the display color of the ultrasonic image obtained by projecting the volume data in accordance with the input operation of the input unit 11. This display color change is performed, for example, by reversing the black and white of the ultrasonic image in order to highlight a cavity or the like in the volume data. Here, the medical image generation unit 15 reverses the display color of the incidental information in black and white in accordance with the ultrasonic image that has been reversed in black and white. In this case, as shown in FIG. 10, the supplementary information is displayed with the display color of the line segment and the character string as white.

  By this operation, the medical image generation unit 15 can superimpose and display the supplementary information in a color that is easy to see by changing the display color of the supplementary information in accordance with the change of the display color of the ultrasonic image. The configuration of the embodiment is not limited to this. For example, when the display color of the ultrasonic image is changed, the medical image generation unit 15 changes the display color of only the line segment of the auxiliary information instead of changing the display color of the auxiliary information, and the character string of the auxiliary information is changed. You may move outside the display area of an ultrasonic image. Alternatively, when the display color of the ultrasonic image is changed, the medical image generation unit 15 moves the display area of the character string to an area where the display color of the character string is different from the color of the pixel in the ultrasonic image. You may display. It should be noted that the display color of the ultrasonic image with the display color changed and the accompanying information displayed in a superimposed manner can be returned to the display color before the change according to the input operation received by the input unit 11.

  Note that the sixth modification can be implemented in combination with the transparency control process described in the first embodiment. That is, when the display area of the arrow or the character string of the incidental information A1 and the incidental information A2 overlaps due to the setting of the viewpoint / line-of-sight direction S, the medical image generation unit 15 increases the transparency of the incidental information at a position far from the viewpoint. To control. As a result, the character string and the arrow of the incidental information at a position far from the viewpoint are displayed in the ultrasonic image with high transparency.

  In addition, although embodiment and the modification of this invention were described above, the medical image diagnostic apparatus 1 may operate | move combining each embodiment and modification. Moreover, according to the input operation which the input part 11 received, you may select which operation | movement to perform among 1st Embodiment or each modification. In the embodiment, the ultrasonic image is generated based on the B-mode signal or the Doppler signal. However, the volume data is generated based on various signals such as a THI (Third Harmonic Imaging) signal, an elastography signal, and a strain signal. It may be generated.

Although several embodiments of the present invention have been described above, these embodiments are presented as examples and are not intended to limit the scope of the invention. These novel embodiments can be implemented in various other forms, and various omissions, replacements, and changes can be made without departing from the scope of the invention. These embodiments and modifications thereof are included in the scope and gist of the invention, and are included in the invention described in the claims and the equivalents thereof.

DESCRIPTION OF SYMBOLS 1 Medical image diagnostic apparatus 10 System control part 11 Input part 12 Storage part 13 Display part 14 Additional information generation part 15 Medical image generation part 16 Volume data generation part 17 Transmission / reception part 18 B mode processing part 19 Doppler processing part 20 Ultrasonic probe 21 Ultrasonic vibrator 100 Structure S View point / Gaze direction

Claims (6)

Medical image acquisition means for acquiring a medical image obtained by projecting three-dimensional volume data from a predetermined viewpoint;
Incidental information acquisition means for acquiring a plurality of incidental information associated with the three-dimensional coordinates of the volume data;
Distance reading means for reading the distance between the viewpoint and each of the plurality of incidental information;
A medical image diagnostic apparatus comprising: additional information display means for controlling transparency or luminance of the plurality of auxiliary information based on the distance and displaying the plurality of auxiliary information superimposed on the medical image. . The incidental information display means includes
2. The medical image diagnosis according to claim 1, wherein the luminance or the transparency is controlled for the plurality of supplementary information in which at least a part of a display area overlaps in the medical image among the plurality of supplementary information. apparatus. The incidental information display means includes
3. The control according to claim 1, wherein the control is performed such that the transparency of the incidental information having a large distance to the viewpoint is increased or the luminance is decreased as compared to the incidental information having a small distance to the viewpoint. Medical image diagnostic equipment. The incidental information acquisition means acquires the plurality of incidental information associated with angle information with respect to a three-dimensional coordinate system in addition to the three-dimensional coordinates,
2. The supplementary information display means controls the display direction of the supplementary information to be tilted based on the viewpoint and the angle information, and superimposes the plurality of supplementary information on the medical image. 4. The medical image diagnostic apparatus according to any one of items 1 to 3. The supplementary information acquisition means acquires the plurality of supplementary information associated with color information in addition to the three-dimensional coordinates,
The medical image acquisition means acquires the medical image colored in a color corresponding to the color information based on the three-dimensional coordinates associated with the supplementary information,
5. The medical image diagnosis according to claim 1, wherein the supplementary information display unit colors the plurality of supplementary information according to the color information and superimposes the supplementary information on the medical image. apparatus. Obtaining a medical image obtained by projecting three-dimensional volume data from a predetermined viewpoint;
Obtaining a plurality of incidental information associated with the three-dimensional coordinates of the volume data;
Calculating a distance between the viewpoint and each of the plurality of incidental information;
A control program for a medical image diagnostic apparatus, comprising: controlling the plurality of supplementary information or luminance based on the distance and displaying the plurality of supplementary information superimposed on the medical image.

Images