JP6622114B2 - Ultrasound diagnostic device, camera terminal and alignment support program - Google Patents

Description

  Embodiments described herein relate generally to an ultrasonic diagnostic apparatus, a camera-equipped terminal, and an alignment support program.

  As medical image diagnostic apparatuses for examining a subject, an X-ray CT (Computed Tomography) apparatus, an MRI (Magnetic Resonance Imaging) apparatus (magnetic resonance diagnostic apparatus), an ultrasonic diagnostic apparatus, and the like are known.

  In recent examinations and treatments, for example, tomographic images of X-ray CT images and MRI images are displayed on a display device, and the same tomographic images as the X-ray CT images and MRI images are simultaneously displayed as ultrasonic images. Has been done. In this case, the alignment of the tomographic image is performed with reference to a characteristic part of the body in the tomographic image of the X-ray CT image or the MRI image.

  However, even if alignment is performed using characteristic parts of the body of the tomographic image of the X-ray CT image or MRI image, in order to perform diagnosis using an ultrasonic image, the region of interest to be observed in the ultrasonic image after the alignment Had to be set.

JP2013-63253A Japanese Patent Laid-Open No. 9-24035

  SUMMARY OF THE INVENTION The problem to be solved by the present invention is an ultrasonic diagnosis capable of performing alignment between an ultrasonic image and a captured image captured by a diagnostic apparatus other than the ultrasonic diagnostic apparatus in a simple and intuitive manner. An apparatus, a camera-equipped terminal, and an alignment support program are provided.

  The ultrasound diagnostic apparatus according to the present embodiment is an ultrasound diagnostic apparatus including a camera-equipped terminal that captures a body surface image of a subject, and the camera-equipped terminal captures a first body surface image of the subject. A first camera for imaging, a display for displaying a body surface image of the subject, and a second body surface of the subject imaged by a second camera in an image diagnostic apparatus different from the ultrasonic diagnostic apparatus Receiving an image and an input of an in-vivo image related to the second body surface image by a spatial positional relationship; and projecting the in-vivo image in the direction of the second body surface image, A generating unit that generates a projection reference image of the in-vivo image viewed from the imaging direction of the second camera, the projection reference image, the first body surface image, and the second body surface image; A display control unit that superimposes and displays them together, Serial includes a guide for imaging an ultrasound image of the subject, and the first body image, together with the projected reference image, a guide display unit for displaying on the display, the.

1 is a schematic configuration diagram illustrating an example of a schematic configuration of an ultrasound diagnostic apparatus including a camera-equipped terminal according to the present embodiment. The ultrasonic diagnostic apparatus according to the present embodiment displays the CT body surface image of the subject received from the image storage circuit and the UL body surface image captured by the camera so that the images can be aligned, The flowchart which shows the operation | movement of the guide display process which displays the guide for imaging a sound wave image. The conceptual diagram which showed the concept of CT body surface image and in-vivo image which the processing circuit of the ultrasound diagnosing device which concerns on this embodiment acquires from a memory circuit. The conceptual diagram which showed the concept that the processing circuit of the ultrasound diagnosing device which concerns on this embodiment produces | generates a projection reference image from an in-vivo image. Explanatory drawing which showed the condition where the camera of the ultrasound diagnosing device which concerns on this embodiment images the UL body surface image of the subject P. FIG. The conceptual diagram which showed the concept which the processing circuit of the ultrasound diagnosing device which concerns on this embodiment superimposes and displays a projection reference image, UL body surface image, and CT body surface image so that alignment is possible. In the display of the camera-equipped terminal of the ultrasonic diagnostic apparatus according to the present embodiment, a display example after alignment is shown including a projection reference image that is superimposed and displayed so that the CT body surface image matches the UL body surface image. Conceptual diagram. The guide example which showed four examples which the processing circuit of the ultrasound diagnosing device which concerns on this embodiment guides the place which presses an ultrasound probe with respect to the projection reference image displayed on a display. The schematic block diagram which showed an example of the schematic structure of an ultrasound diagnosing device provided with the terminal with a camera which concerns on 2nd Embodiment. The example of a guide which guides the place where the light source of the ultrasound diagnosing device which concerns on 2nd Embodiment presses an ultrasound probe with respect to the projection reference image displayed on a display. The ultrasonic diagnostic apparatus which concerns on 3rd Embodiment is the schematic block diagram which showed the structure further provided with the correction function in the processing circuit of the terminal with a camera. The conceptual diagram which showed the concept which a processing circuit correct | amends so that a CT body surface image may correspond with UL body surface image with the change of UL body surface image in the display of the ultrasound diagnosing device which concerns on 3rd Embodiment. .

(First embodiment)
Hereinafter, an ultrasonic diagnostic apparatus including a camera-equipped terminal according to the present embodiment will be described with reference to the accompanying drawings. In the present embodiment, when performing an ultrasonic image diagnosis, as an example, a description will be given of a case in which a CT image captured by an X-ray CT (Computed Tomography) apparatus and an ultrasonic image are aligned. Is not limited to this. For example, the present invention can also be applied to alignment of an MRI image picked up by an MRI (Magnetic Resonance Imaging) apparatus and an ultrasonic image.

[Schematic configuration]
FIG. 1 is a schematic configuration diagram illustrating an example of a schematic configuration of an ultrasonic diagnostic apparatus 100 including a camera-equipped terminal 200 according to the present embodiment.

  As shown in FIG. 1, the ultrasonic diagnostic apparatus 100 according to the present embodiment includes an ultrasonic probe 110, an ultrasonic diagnostic apparatus main body 120, and a camera-equipped terminal 200.

  The ultrasonic probe 110 includes an ultrasonic vibration element. The ultrasonic vibration element has a function of converting an electrical drive signal into a transmission ultrasonic wave during transmission and converting an ultrasonic reflected wave (reception ultrasonic wave) into an electrical reception signal during reception.

  The ultrasonic diagnostic apparatus main body 120 includes a transmission / reception circuit 130, a signal processing circuit 140, an image processing circuit 150, a display 160, an input circuit 170, a processing circuit 180, and a storage circuit 190.

  The transmission / reception circuit 130 includes a transmission circuit and a reception circuit. The transmission circuit has a function of supplying a drive signal for radiating transmission ultrasonic waves to the ultrasonic probe 110 in a predetermined direction of the subject. The receiving circuit includes a phase detection circuit and a beam forming circuit. The phase detection circuit converts the received signals of the plurality of channels received from the ultrasonic probe 110 into a baseband in-phase signal (I signal, I: In-phase) and a quadrature signal (Q signal, Q: Quadrature- phase) and further converted into a digital signal. The beam forming circuit forms a beam by adding a predetermined delay to each channel signal (I signal and Q signal). The beam-formed signal output from the beam forming circuit is also composed of an I signal and a Q signal. Hereinafter, the beam-formed signal output from the beam forming circuit is referred to as an “IQ signal”. The transmission / reception circuit 120 outputs the IQ signal to the signal processing circuit 140.

  The signal processing circuit 140 receives an IQ signal from the transmission / reception circuit 130, and generates image data of a continuous wave Doppler image using the FFT method (Fast Fourier Transform method) on the IQ signal. In the pulse transmission mode, the signal processing circuit 140 detects the envelope of the IQ signal to generate a B-mode image, or generates a color Doppler image from the IQ signal.

  Hereinafter, the B-mode image generated by the signal processing circuit 140, the image data of the color Doppler image, the image data of the continuous wave Doppler image, and the like are also referred to as ultrasonic image data.

  The image processing circuit 150 has a function of performing coordinate conversion processing for aligning the coordinate system with the imaging cross section of the subject on the ultrasound image data generated by the signal processing circuit 140. For example, the image processing circuit 150 converts ultrasonic image data from a scan-type coordinate system to a television-type coordinate system. Further, the image processing circuit 150 has a function of performing gradation setting suitable for image display and image processing for changing resolution or frame rate on the ultrasonic image data subjected to coordinate conversion.

  The display 160 has a function of displaying ultrasonic image data of the subject imaged by the ultrasonic probe 110 as an image. The display 160 is constituted by, for example, a liquid crystal display or a monitor.

  The input circuit 170 is a circuit that inputs a signal from an input device such as a pointing device (such as a mouse) or a keyboard that can be operated by an operator such as a doctor or a laboratory technician. Here, the input device itself is also the input circuit 170. Shall be included. In this case, an input signal according to the operation is sent from the input circuit 170 to the processing circuit 180.

  The processing circuit 180 has a function of comprehensively controlling the ultrasonic diagnostic apparatus main body 120. For example, the processing circuit 180 is a processor that implements a function corresponding to each program by reading the program from the storage circuit 190 and executing the program.

  The memory circuit 190 includes a ROM (Read Only Memory), a RAM (Random Access Memory), and the like as memories. The storage circuit 190 stores the above program, IPL (Initial Program Loading), BIOS (Basic Input / Output System) data, and the work memory of the processing circuit 180 and temporary storage of data. Or

  Next, the camera-equipped terminal 200 will be described. The camera-equipped terminal 200 according to this embodiment includes a processing circuit 210, a camera 220, an input circuit 230, a display 240, a storage circuit 250, a network interface 260, and an internal bus 270.

  The camera 220 has a function of capturing a body surface image of a subject. An image captured by the camera 220 is referred to as a UL body surface image.

  The display 240 has a function of displaying a body surface image of the subject. The display 240 is configured by, for example, a liquid crystal display or a monitor.

  The processing circuit 210 is a processor that implements a function corresponding to each program by reading the program from the storage circuit 250 and executing the program. In other words, the processing circuit 210 can read each program and realize a reception function, a generation function, a display control function, and a guide control function.

  Here, the reception function is an image diagnostic apparatus different from the ultrasonic diagnostic apparatus 100, for example, a CT body surface image of a subject imaged by a CT-side camera in an X-ray CT apparatus, and the CT body surface image and space. This is a function that accepts an input with an in-vivo image related by a specific positional relationship. The in-vivo image is, for example, a tomographic image in the three-dimensional volume data of the same subject acquired by an X-ray CT apparatus or an MRI apparatus.

  The generation function is a function of projecting the in-vivo image in the direction of the CT body surface image and generating a projection reference image of the in-vivo image viewed from the imaging direction of the CT-side camera.

  The display control function is a function that superimposes and displays the projection reference image, the UL body surface image, and the CT body surface image so that they can be aligned.

  The guide control function is a function for displaying a guide for capturing an ultrasonic image of a subject on the display 240 together with a UL body surface image and a projection reference image.

  Note that the reception function, generation function, display control function, and guide control function in the present embodiment are examples of the reception unit, generation unit, display control unit, and guide display unit in the claims.

  The term “processor” used in the above description is, for example, a dedicated or general-purpose CPU (Central Processing Unit), an application specific integrated circuit (ASIC), or a programmable logic device (for example, simple It means circuits such as a programmable logic device (SPLD), a complex programmable logic device (CPLD), and a field programmable gate array (FPGA).

  The processor implements each function by reading and executing a program stored in the memory or directly incorporated in the circuit of the processor. When a plurality of processors are provided, the memory for storing the program may be provided for each processor individually, or, for example, the storage circuit 190 or the storage circuit 250 in FIG. 1 corresponds to the function of each processor. The program to be stored may be stored.

  The input circuit 230 is a circuit that inputs a signal from an input device such as a pointing device (such as a mouse) or a keyboard that can be operated by an operator such as a doctor or a laboratory technician. Here, the input device itself is also input to the input circuit 20. Shall be included. In this case, an input signal according to the operation is sent from the input circuit 230 to the processing circuit 210.

  The camera-equipped terminal 200 can be realized by, for example, a portable information processing apparatus, and can be configured by a touch panel capable of inputting signals and displaying images, with the input circuit 230 and the display 240 integrated.

  The storage circuit 250 includes a storage device including a ROM, a RAM, an HDD, and the like. The storage circuit 250 stores IPL, BIOS and data, is used as a work memory for the processing circuit 210, or is used when data is temporarily stored. The HDD is a storage device that stores programs installed in the camera-equipped terminal 200 (including application programs as well as an OS (Operating System)) and image data. In addition, a graphic user interface (GUI) that allows basic operations to be performed by the input circuit 230 using graphics frequently for displaying information on the display 240 for operators such as doctors and laboratory technicians is provided to the OS. You can also.

  In the present embodiment, the storage circuit 250 receives and stores the CT body surface image of the subject and its in-vivo image from the image storage circuit 300 via the network interface 260. That is, the storage circuit 250 stores the CT body surface image of the subject and the in-vivo image thereof.

  The network interface 260 performs communication control according to the communication standard, and connects the camera-equipped terminal 200 to the network through, for example, a wireless LAN (Local Area Network) compliant with the IEEE802.11 series, near field communication, or a telephone line. It has a function to do. For example, the network interface 260 receives the CT body surface image of the subject and the in-vivo image from the image storage circuit 300.

  The internal bus 270 is connected to each component so that the processing circuit 210 is centrally controlled by the processing circuit 210.

  Next, the image storage circuit 300 provided outside the ultrasonic diagnostic apparatus 100 and connected via a network will be described.

  The image storage circuit 300 includes a ROM, a RAM, and the like as memories. The image storage circuit 300 stores, for example, a CT body surface image of a subject acquired by an X-ray CT apparatus, and an in-vivo image related to the CT body surface image by a spatial positional relationship. The in-vivo image is a tomographic image in the three-dimensional volume data of the same subject acquired by the X-ray CT apparatus described above.

[motion]
Hereinafter, a detailed operation of the ultrasonic diagnostic apparatus 100 including the camera-equipped terminal 200 according to the present embodiment will be described using a flowchart.

(Guide display processing)
FIG. 2 shows that the ultrasonic diagnostic apparatus 100 according to the present embodiment displays the CT body surface image of the subject received from the image storage circuit 300 and the UL body surface image captured by the camera 220 in a position-alignable manner. It is a flowchart which shows the operation | movement of the guide display process which displays the guide for imaging the ultrasonic image of a subject about an image.

  As shown in FIG. 2, first, the processing circuit 210 of the ultrasonic diagnostic apparatus 100 according to the present embodiment, the CT body surface image captured by the CT side camera in the X-ray CT apparatus, and the CT The body surface image and the in-vivo image related by the spatial positional relationship are acquired, and the input is received (step ST001).

  FIG. 3 is a conceptual diagram showing the concepts of the CT body surface image CB and the in-vivo image VD acquired from the storage circuit 250 by the processing circuit 210 of the ultrasonic diagnostic apparatus 100 according to the present embodiment.

  FIG. 3A shows a concept of acquiring a CT body surface image CB of the subject P using a CT-side camera CC provided in an X-ray CT apparatus (not shown). FIG. 3B shows a concept of acquiring an in-vivo image VD that is volume data of the subject P using an imaging device provided in an X-ray CT apparatus (not shown).

  Note that the CT camera CC is provided, for example, in the gantry of the X-ray CT apparatus, so that the position of the CT camera CC can be specified, and position information regarding the position of the CT camera CC is included in the in-vivo image VD. It is possible to associate the CT body surface image CB with position information indicating a relative position. Further, since the CT body surface image CB can be captured from the CT-side camera CC, it can have position information indicating the relative position of the CT-side camera CC. The in-vivo image VD and the CT body surface image CB may be associated with each other using, for example, a spatial position at the bed position.

  Next, the processing circuit 210 projects the in-vivo image VD in the direction of the CT body surface image CB, and generates a projection reference image of the in-vivo image VD viewed from the imaging direction of the CT-side camera CC (step ST003).

  FIG. 4 is a conceptual diagram showing a concept that the processing circuit 210 of the ultrasonic diagnostic apparatus 100 according to the present embodiment generates a projection reference image from the in-vivo image VD.

  FIG. 4A shows a concept of extracting a reference cross section from CT volume data in the subject P. For example, reference slices D1, D2, and D3 extracted from volume data are illustrated as tomographic images in CT volume data.

  In FIG. 4B, the shape of the reference cross sections D1, D2, and D3 extracted in FIG. 4A as viewed from the direction of the CT side camera CC, for example, is a trapezoidal concept. It is the conceptual diagram which displayed. For example, the reference cross section D1 shown in FIG. 4A corresponds to the shape of the trapezoid J1 in FIG. Similarly, the reference cross section D2 shown in FIG. 4A corresponds to the shape of the trapezoid J2 in FIG. 4B, and the reference cross section D3 shown in FIG. 4A is the trapezoid in FIG. 4B. Corresponds to the shape of J3. Note that the trapezoids J1, J2, and J3 correspond to the projection reference images J1, J2, and J3.

  As described above, the processing circuit 210 projects the in-vivo image VD in the direction of the CT body surface image CB, thereby generating a projection reference image of the in-vivo image VD viewed from the imaging direction of the CT-side camera CC (step ST003). .

  Next, the processing circuit 210 captures the body surface image of the subject P with the camera 220 provided in the camera-equipped terminal 200, and acquires the UL body surface image UB (step ST005). The camera 220 is also referred to as a UL side camera.

  FIG. 5 is an explanatory diagram showing a situation in which the camera 220 of the ultrasonic diagnostic apparatus 100 according to the present embodiment images the UL body surface image UB of the subject P.

  As shown in FIG. 5, since the camera-equipped terminal 200 according to the present embodiment includes the camera 220 and the display 240, the UL body surface image UB of the subject P can be captured. In addition, since the camera-equipped terminal 200 is fixed to a movable arm, the operator can capture the UL body surface image UB displayed on the display 240 of the camera-equipped terminal 200 without having the camera-equipped terminal 200. Can do.

  In FIG. 5, the camera-equipped terminal 200 is fixed to a movable arm so that the operator can view the UL body surface image UB displayed on the display 240. The form is not limited to this. For example, the operator may hold the camera-equipped terminal 200 with his / her hand and watch the UL body surface image UB while holding the terminal with his / her hand.

  Further, when the camera-equipped terminal 200 is fixed by, for example, an arm that fixes the camera-equipped terminal 200, position information indicating the relative position between the subject P and the camera-equipped terminal 200 is displayed as the UL body surface image UB. Can have. In this case, position information indicating the relative position between the camera-equipped terminal 200 and the bed and position information indicating the position of the subject P on the bed may be associated with the UL body surface image UB.

  Next, the processing circuit 210 superimposes and displays the projection reference image, the UL body surface image UB, and the CT body surface image CB so that they can be aligned (step ST007).

  FIG. 6 shows a concept in which the processing circuit 210 of the ultrasonic diagnostic apparatus 100 according to the present embodiment superimposes and displays the projection reference image J2, the UL body surface image UB, and the CT body surface image CB so that they can be aligned. It is the conceptual diagram shown.

  6A shows a display example when the CT body surface image CB received by the camera-equipped terminal 200 and the generated projection reference image J2 are displayed on the display 240. FIG. Since the CT body surface image CB is a body surface image obtained by photographing the body surface of the subject P with the CT side camera CC, it is a two-dimensional image. Further, since the projection reference image J2 is an image obtained by projecting the reference cross section D2 onto the CT body surface image CB and viewed from the CT side camera CC, the three-dimensional shape is displayed as a two-dimensional trapezoid. The CT body surface image CB and the projection reference image J2 can be integrated by projection.

  FIG. 6B shows a UL body surface image UB in which the camera-equipped terminal 200 images the body surface of the subject P with the camera 220. On display 240, UL body surface image UB imaged by camera 220 in step ST005 is displayed.

  FIG. 6C shows a case where the projection reference image J2, the UL body surface image UB, and the CT body surface image CB are superimposed and displayed, and the UL body surface image UB and the CT body surface image CB are aligned. An example is shown. For example, the camera-equipped terminal 200 integrates the CT body surface image CB and the projection reference image J2 into the UL body surface image UB displayed on the display 240, and the CT body surface image CB matches the UL body surface image UB. Align so that

  FIG. 7 shows the display 240 of the camera-equipped terminal 200 of the ultrasonic diagnostic apparatus 100 according to the present embodiment in which the CT body surface image CB is superimposed on the UL body surface image UB so as to coincide with the projection body image J2. It is the conceptual diagram which showed the example of a display after position alignment.

  As shown in FIG. 7, the processing circuit 210 can align the CT body surface image CB and the projection reference image J2 together with the UL body surface image UB on the display 240. Further, for example, the relative position at which the UL body surface image UB of the subject P can be photographed using the position information regarding the position of the CT camera CC and the relative position information indicating the relative position of the CT body surface image CB. You may make it set the terminal 200 with a camera to. For example, the camera-equipped terminal 200 performs fine adjustment on the arm and adjusts the range in which the UL body surface image UB is picked up so that the CT body surface image CB and the UL body surface image UB are aligned. It can be performed. This alignment may be manual or automatic.

  Further, the projection reference image J2 can be synchronized with the reference section D2. For example, when the operator selects the reference section D1 in FIG. 4A, the projection reference image J1 is converted into the UL body surface image UB. When the operator selects the reference cross section D3 in FIG. 4A, the projection reference image J3 can be displayed superimposed on the UL body surface image UB.

  Next, the processing circuit 210 displays a guide for capturing an ultrasound image of the subject P on the display 240 together with the UL body surface image UB and the projection reference image J2 (step ST009). In the present embodiment, four examples are assumed as examples of displaying a guide for capturing an ultrasonic image of the subject P, for example.

  FIG. 8 shows four examples in which the processing circuit 210 of the ultrasonic diagnostic apparatus 100 according to the present embodiment guides the place where the ultrasonic probe 110 is pressed against the projection reference image J2 displayed on the display 240. This is a guide example.

  FIG. 8A shows an example of a guide when a location for capturing a sound wave image is displayed as a point as a guide for the projection reference image J2. A point P1 indicates a pressing point, and the operator can use the point P1 as a guide when operating the ultrasonic probe 110.

  FIG. 8B shows an example of a guide when a place for capturing a sound wave image is displayed as a line as a guide for the projection reference image J2. A line P2 indicates a line corresponding to the scanning direction, and the operator can use it as a guideline when operating the ultrasonic probe 110 with the line P2. 30 degrees indicates, for example, an angle at which the ultrasonic probe 110 is tilted. For example, the tip of the ultrasonic probe 110 is tilted 30 degrees from the vertical direction of the display 240 to the projection reference image J2 and imaged. This shows that an ultrasonic image corresponding to the reference section D2 can be obtained.

  In addition, the display method by the numerical value of 30 degrees is not limited to this. As a guide for the projection reference image J2, any information that can visually display angle information indicating an angle to which the ultrasonic probe is applied is applicable. For example, angle information using a protractor may be visually displayed.

  FIG. 8C shows an example of a guide where a location for capturing a sound wave image is displayed as a guide for the projection reference image J2. A region P3 indicates a region where the ultrasonic probe 110 is pressed, and the operator can use the region P3 as a guide when operating the ultrasonic probe 110.

  In FIG. 8D, as a guide for the projection reference image J2, a guide for displaying a location for capturing a sound wave image by using a virtual image simulating an ultrasonic probe and using the virtual probe. It is an example. The virtual probe P4 indicates a place where the operator presses the ultrasonic probe 110, and the operator can use it as a guide when operating the ultrasonic probe 110 with the virtual probe P4. Further, the virtual probe P4 can be displayed with the virtual probe P4 tilted, for example, in accordance with the angle at which the reference cross section D2 in FIG. In this case, the scanning angle of the ultrasonic probe 110 can be intuitively recognized not only by the numerical value but also by the inclined shape of the virtual probe P4.

  The ultrasound diagnostic apparatus 100 according to the present embodiment displays such a guide on the display 240 and ends the guide display process when the operator presses the ultrasound probe 110 against the subject P.

  As described above, the ultrasonic diagnostic apparatus 100 according to the present embodiment includes the camera-equipped terminal 200, and the processing circuit 210 of the camera-equipped terminal 200 receives inputs of the CT body surface image CB and the in-vivo image VD, The in-vivo image VD is projected onto the CT body surface image CB to generate a projection reference image. The camera-equipped terminal 200 can superimpose and display the projection reference image, the UL body surface image UB, and the CT body surface image CB on the display 240 so as to be aligned, and captures an ultrasonic image of the subject P. Is displayed on the display 240.

  Thereby, the ultrasonic diagnostic apparatus 100 according to the present embodiment operates a position adjustment between the captured image captured by the diagnostic apparatus other than the ultrasonic diagnostic apparatus and the ultrasonic image by a simple and intuitive method. Can be performed.

  Further, the camera-equipped terminal 200 displays the projection reference image according to the angle of the cross section indicating the region of interest in the in-vivo image VD while keeping the positions where the UL body surface image UB and the CT body surface image CB are displayed. The displayed display form can be changed.

  For example, by selecting any one of the reference cross sections D1, D2, and D3 shown in FIG. 4, the corresponding projection reference image can be displayed on the display 240. Specifically, when the reference cross section D1 shown in FIG. 4A is selected, it can be displayed as a projected reference image J1 by the shape of the trapezoid J1 in FIG. 4B. When the reference cross section D3 shown in FIG. 4A is selected, it can be displayed as a projection reference image J3 by the shape of the trapezoid J3 in FIG.

  For example, when the reference cross section is changed from the reference cross section D1 to the reference cross section D3, the projection reference image J1 can be changed to the projection reference image J3, and the display position of the guide that displays the guide in synchronization is also changed. Can do.

  In FIG. 7, the projected reference image after alignment is displayed by the projected reference image J2. However, the projected reference image is not limited to this. For example, the reference cross section shown in FIG. 4A may be corrected three-dimensionally and the projected cross-sectional image may be fitted as a projected reference image by a cross-sectional image obtained by graphic processing.

(Second Embodiment)
The ultrasonic diagnostic apparatus 100 according to the first embodiment, in the camera-equipped terminal 200, for example, as shown in FIG. It was displayed as a guide for the projected reference image.

  In the second embodiment, the camera-equipped terminal 200 further includes a light source 280 that emits a light beam having a visual effect, and the processing circuit 210 captures an ultrasound image using the light source 280 as a guide for the projection reference image. The point to do is displayed.

  FIG. 9 is a schematic configuration diagram illustrating an example of a schematic configuration of an ultrasonic diagnostic apparatus 100A including the camera-equipped terminal 200A according to the second embodiment.

  As illustrated in FIG. 9, the ultrasonic diagnostic apparatus 100 </ b> A according to the second embodiment includes a camera-equipped terminal 200 </ b> A including a light source 280. The difference from the ultrasonic diagnostic apparatus 100 of the first embodiment is that a light source 280 is provided.

  FIG. 10 shows a place where the light source 280 of the ultrasonic diagnostic apparatus 100A according to the second embodiment presses the ultrasonic probe 110 against the projection reference image J2 displayed on the display 240 using a light beam. It is an example of a guide.

  As illustrated in FIG. 10, the point SP indicates a point irradiated with laser on the UL body surface image UB by the light source 280, and indicates a pressing point where the ultrasonic probe 110 is pressed. For example, by using the light source 280 to irradiate and point to the point SP that is the pressing point, the operator can more clearly recognize the point to which the ultrasonic probe 110 is applied.

(Third embodiment)
In the third embodiment, the processing circuit 210 of the camera-equipped terminal 200 further includes a correction function for correcting the display positions of the CT body surface image CB and the UL body surface image UB.

  FIG. 11 is a schematic configuration diagram illustrating a configuration in which an ultrasonic diagnostic apparatus 100B according to the third embodiment further includes a correction function in the processing circuit 210B of the camera-equipped terminal 200B.

  When the position where the UL body surface image UB is displayed is changed by imaging by the camera 220, the camera-equipped terminal 200B determines whether the CT body surface image CB and the projection reference image are based on the changed position. The display position is corrected, and the UL body surface image UB and the CT body surface image CB are displayed on the display 240 so as to coincide with each other.

  FIG. 12 shows that in the display 240 of the ultrasonic diagnostic apparatus 100B according to the third embodiment, the processing circuit 210B causes the CT body surface image CB to coincide with the UL body surface image UB as the UL body surface image UB changes. It is the conceptual diagram which showed the concept corrected as follows.

  FIG. 12A shows that, for example, when the position of the camera-equipped terminal 200B is moved, the position where the UL body surface image UB is displayed is changed with the movement of the camera-equipped terminal 200B. On the other hand, in FIG. 12B, the CT body surface image CB and the projection reference image J2 follow the movement of the position of the camera-equipped terminal 200B, and the CT body surface image UB is displayed at the changed display position of the UL body surface image UB. A display example when the image CB and the projection reference image are corrected is shown.

  In the case of the third embodiment, even when the camera-equipped terminal 200B moves, the position of the CT body surface image CB and the projection reference image J2 can be corrected by the correction function of the processing circuit 210B. The operator can view the projection reference image J2 without stopping the inspection even when the camera-equipped terminal 200 moves, and can continuously hit the ultrasonic probe 110.

  Note that the correction method is an example, and the present invention is not limited to this. For example, when the camera-equipped terminal 200B is fixed, the body position of the subject P may also move. In this case, the CT body surface image CB and the projection reference image accompany the movement of the body position of the subject P. The position of J2 can be corrected.

  According to at least one embodiment described above, it is possible to perform alignment of an ultrasonic image and a captured image captured by a diagnostic apparatus other than the ultrasonic diagnostic apparatus 100 by a simple and intuitive method. .

  The camera 220 in the present embodiment is an example of a first camera in the claims, and the CT-side camera CC in the present embodiment is an example of a second camera in the claims. The UL body surface image UB in the present embodiment is an example of the first body surface image in the claims, and the CT body surface image CB in the present embodiment is the second body surface image in the claims. It is an example of an image.

  Moreover, although the camera-equipped terminal 200 includes the camera 220 and the display 240 as one body, the present invention is not limited to this. For example, in the camera-equipped terminal 200, at least one of the camera 220 and the display 240 may be configured separately.

  Although several embodiments of the present invention have been described, these embodiments are presented by way of example and are not intended to limit the scope of the invention. These embodiments can be implemented in various other forms, and various omissions, replacements, and changes can be made without departing from the spirit of the invention. These embodiments and their modifications are included in the scope and gist of the invention, and are also included in the invention described in the claims and the equivalents thereof.

  Further, in the embodiment of the present invention, each step of the flowchart shows an example of processing that is performed in time series in the order described. The process to be executed is also included.

DESCRIPTION OF SYMBOLS 100 ... Ultrasonic diagnostic apparatus 110 ... Ultrasonic probe 120 ... Ultrasonic diagnostic apparatus main body 130 ... Transmission / reception circuit 140 ... Signal processing circuit 150 ... Image processing circuit 160 ... Display 170 ... Input circuit 180 ... Processing circuit (processor)
190 ... Memory circuit 200 ... Terminal 210 with camera ... Processing circuit (processor)
220 ... Camera 230 ... Input circuit 240 ... Display 250 ... Storage circuit 260 ... Network interface 270 ... Internal bus 300 ... Image storage circuit

Claims (9)

An ultrasonic diagnostic apparatus including a camera-equipped terminal that captures a body surface image of a subject,
The camera terminal is
A first camera that captures a first body surface image of the subject;
A display for displaying a body surface image of the subject;
The second body surface image of the subject imaged by a second camera in an image diagnostic apparatus different from the ultrasonic diagnostic apparatus, and the second body surface image and the second body surface image are related by a spatial positional relationship. A reception unit that accepts input with in-vivo images;
A generation unit that projects the in-vivo image in the direction of the second body surface image and generates a projection reference image of the in-vivo image viewed from the imaging direction of the second camera;
A display control unit that superimposes and displays the projection reference image, the first body surface image, and the second body surface image so as to be aligned;
A guide display unit for displaying a guide for imaging an ultrasonic image of the subject on the display together with the first body surface image and the projection reference image;
An ultrasonic diagnostic apparatus comprising: The guide display unit
The ultrasonic diagnostic apparatus according to claim 1, wherein a place for capturing the ultrasonic image is displayed by a point, a line, or a region as the guide for the projection reference image. The guide display unit
The ultrasonic diagnostic apparatus according to claim 1, wherein angle information indicating an angle at which an ultrasonic probe is applied is visually displayed as the guide with respect to the projection reference image. Comprising a light source that emits a light beam having a visual effect;
The guide display unit
The ultrasound diagnostic apparatus according to any one of claims 1 to 3, wherein a spot for capturing the ultrasound image using the light beam is displayed as the guide for the projection reference image. The guide display unit
5. The location for capturing the ultrasonic image is displayed using a virtual image imitating the shape of an ultrasonic probe as the guide for the projection reference image. Ultrasound diagnostic equipment. The display control unit
Display in which the projection reference image is displayed according to an angle of a cross section indicating a region of interest in the in-vivo image while the display positions of the first body surface image and the second body surface image are aligned. The ultrasonic diagnostic apparatus according to any one of claims 1 to 5, wherein the form is changed. A correction unit that corrects display positions of the second body surface image and the projection reference image;
The display control unit
When the position where the first body surface image is displayed is changed by imaging with the first camera, based on the changed position, the second body surface image, the projection reference image, 7. The display position is corrected so that the first body surface image, the second body surface image, and the projection reference image are fixedly displayed on the display. The ultrasonic diagnostic apparatus of Claim 1. A first camera that captures a first body surface image of a subject;
A display for displaying a body surface image of the subject;
A second body surface image of the subject imaged by a second camera in an image diagnostic apparatus different from the ultrasound diagnostic apparatus, and a body related to the second body surface image by a spatial positional relationship An accepting unit that accepts input of an image;
A generation unit that projects the in-vivo image in the direction of the second body surface image and generates a projection reference image of the in-vivo image viewed from the imaging direction of the second camera;
A display control unit that superimposes and displays the projection reference image, the first body surface image, and the second body surface image so as to be aligned;
A guide display unit for displaying a guide for imaging an ultrasonic image of the subject on the display together with the first body surface image and the projection reference image;
A camera-equipped terminal equipped with an alignment support function. A computer comprising: a first camera that captures a first body surface image of a subject; and a display that displays the body surface image of the subject.
A second body surface image of the subject imaged by a second camera in an image diagnostic apparatus different from the ultrasound diagnostic apparatus, and the body related to the second body surface image by a spatial positional relationship The ability to accept input with images,
A function of projecting the in-vivo image in the direction of the second body surface image and generating a projection reference image of the in-vivo image viewed from the imaging direction of the second camera;
A function of superimposing and displaying the projection reference image, the first body surface image, and the second body surface image so that they can be aligned;
A function for displaying a guide for imaging an ultrasonic image of the subject on the display together with the first body surface image and the projection reference image;
Alignment support program for realizing

Images