JP7333338B2 - ULTRASOUND DIAGNOSTIC SYSTEM AND CONTROL METHOD OF ULTRASOUND DIAGNOSTIC SYSTEM - Google Patents

Description

The present invention relates to an ultrasonic diagnostic apparatus equipped with a head-mounted display and a control method for the ultrasonic diagnostic apparatus.

Conventionally, an ultrasonic diagnostic apparatus is known as a device for obtaining an image of the inside of a subject. An ultrasonic diagnostic apparatus generally includes an ultrasonic probe provided with a transducer array in which a plurality of elements are arranged. In a state in which the ultrasonic probe is in contact with the body surface of the subject, an ultrasonic beam is transmitted from the transducer array toward the inside of the subject, and an ultrasonic echo from the subject is received by the transducer array and the element Data is obtained. Furthermore, the ultrasonic diagnostic apparatus electrically processes the obtained element data to generate an ultrasonic image of the relevant part of the subject.

Such an ultrasonic diagnostic apparatus is sometimes used to observe the puncture needle inserted into the subject when the puncture needle is inserted into the subject. At this time, the user checks the ultrasound image displayed on the monitor while inserting the puncture needle into the subject while keeping the ultrasound probe in contact with the subject. The puncture needle is inserted into the subject while observing the puncture needle.
Here, since the monitor on which the ultrasound image is displayed is usually placed at a position distant from the ultrasound probe, such as a bedside, the user should not be able to see the space between the puncture needle and the ultrasound probe at hand and the monitor. It is necessary to move the line of sight alternately. In order to reduce such movement of the user's line of sight, an ultrasonic diagnostic apparatus equipped with a so-called head-mounted display, such as that disclosed in Patent Document 1, has been developed. In this ultrasonic diagnostic apparatus, an ultrasonic image showing a puncture needle is displayed on the display section of the head-mounted display.

JP 2011-200533 A

By the way, when inserting the puncture needle into the subject, the user preferably grasps the length of the part of the puncture needle inserted into the subject from the viewpoint of safety. In the ultrasonic diagnostic apparatus of Patent Document 1, the user can confirm the puncture needle on the ultrasonic image while reducing the movement of the line of sight by viewing the display unit of the head-mounted display. It has been difficult to accurately grasp the length of the portion of the puncture needle inserted into the subject only by confirming the puncture needle on the image. In addition, in order to accurately grasp the length of the portion of the puncture needle inserted into the subject, for example, the length of the puncture needle may be measured on the ultrasound image. Usually, the user often manually measures the length, and in fact, it is impossible to measure the length of the puncture needle on the ultrasonic image while the user is inserting the needle into the subject. It was difficult and difficult to measure the exact length.

The present invention has been made to solve such conventional problems, and enables the user to accurately and simply grasp the length of the portion of the puncture needle inserted into the subject. An object of the present invention is to provide an ultrasonic diagnostic apparatus and a control method for the ultrasonic diagnostic apparatus.

In order to achieve the above object, an ultrasonic diagnostic apparatus according to the present invention is an ultrasonic diagnostic apparatus that visualizes an ultrasonic image of a puncture needle inserted into a subject, and is mounted on a user's head, A head-mounted display having a camera unit that acquires a visual field image that captures a field of view in front of the user, and when at least a part of the puncture needle is captured by the camera unit, the puncture needle is recognized by image analysis of the visual field image. and a puncture needle length calculator for calculating the length of the puncture needle in the visual field image.

The ultrasonic diagnostic apparatus includes a total length acquiring unit that acquires the actual total length of the puncture needle that is not inserted into the subject, and a visual field image that captures the entire puncture needle that is not inserted into the subject. Obtaining a correspondence relationship between the length of the puncture needle in the visual field image calculated by the length calculation unit and the actual total length of the puncture needle obtained by the total length obtaining unit, and calculating the length of the puncture needle based on the obtained correspondence relationship It is preferable to further include an actual length estimation unit that estimates the actual length of the puncture needle captured in the visual field image from the length of the puncture needle in the visual field image calculated by the calculation unit.

At this time, when the camera unit captures an image of the entire puncture needle in which a plurality of grooves arranged at predetermined intervals are formed and is not inserted into the subject, the total length acquisition unit obtains a plurality of groove can be used to obtain the actual total length of the puncture needle.
Alternatively, when the length information recording member in which the length information representing the actual total length of the puncture needle is recorded is photographed by the camera unit, the total length acquisition unit obtains the length information recorded in the length information recording member. It is also possible to obtain the actual total length of the puncture needle by reading .
Alternatively, when the entire puncture needle that is not inserted into the subject and the scale for measuring the length are photographed in parallel by the camera unit, the total length acquisition unit performs the puncture using the scale. It is also possible to obtain the actual total length of the needle.

Also, the actual length estimation unit can display the estimated actual length of the puncture needle by superimposing it on the ultrasound image.
Further, when the camera unit captures an image of the puncture needle partially inserted into the subject, the ultrasonic diagnostic apparatus measures the actual total length of the puncture needle acquired by the total length acquisition unit and the actual length of the puncture needle. an insertion length estimating unit for estimating the actual length of the portion of the puncture needle inserted into the subject based on the actual length of the uninserted portion of the puncture needle appearing in the visual field image of the puncture needle estimated by the estimating unit; is preferably further provided.
At this time, the insertion length estimating unit can superimpose and display the estimated actual length of the portion of the puncture needle inserted into the subject on the ultrasound image.

In addition, when the target site of puncture and the puncture needle are visualized in the ultrasound image, the ultrasonic diagnostic apparatus recognizes the target site and the puncture needle by image analysis of the ultrasound image, and determines the target site and the insertion length. A distance calculator for calculating the distance between the target site and the tip of the puncture needle based on the actual length of the portion of the puncture needle inserted into the subject estimated by the length estimator. is preferred.
At this time, the distance calculation unit can display the calculated value of the distance between the target site and the tip of the puncture needle by superimposing it on the ultrasound image.

The ultrasonic diagnostic apparatus can further include a notification unit that notifies the user when the distance calculated by the distance calculation unit is equal to or less than a predetermined value.
Also, the head-mounted display can have a display that displays an ultrasound image.
Alternatively, the ultrasonic diagnostic apparatus can further include a display monitor that displays the sound wave image.

A control method for an ultrasonic diagnostic apparatus according to the present invention is a method for controlling an ultrasonic diagnostic apparatus for depicting a puncture needle inserted into a subject in an ultrasonic image, wherein the ultrasonic diagnostic apparatus is mounted on the user's head and includes a camera unit. A head-mounted display having a head-mounted display acquires a visual field image that captures the field of view in front of the user, and when at least a part of the puncture needle is captured by the camera unit, the puncture needle is recognized by image analysis of the visual field image. and calculating the length of the puncture needle in the visual field image.

According to the present invention, when at least a part of the puncture needle is imaged by the head-mounted display mounted on the user's head and having a camera unit that captures a field of view image in front of the user, and the camera unit and a puncture needle length calculator for recognizing the puncture needle by image analysis of the visual field image and calculating the length of the puncture needle in the visual field image. It is possible to accurately and simply grasp the length of the portion where the portion is located.

1 is a block diagram showing the configuration of an ultrasonic diagnostic apparatus according to Embodiment 1 of the present invention; FIG. 1 is a block diagram showing the configuration of an ultrasonic probe according to Embodiment 1 of the present invention; FIG. FIG. 2 is a block diagram showing the configuration of a receiving section according to Embodiment 1 of the present invention; FIG. It is a figure which shows the example of the head mounted display in Embodiment 1 of this invention. 1 is a block diagram showing the configuration of a head mounted display according to Embodiment 1 of the present invention; FIG. 1 is a block diagram showing the configuration of a main body of a diagnostic device according to Embodiment 1 of the present invention; FIG. FIG. 4 is a diagram showing an example of a puncture needle; FIG. 4 is a diagram schematically showing a puncture needle inserted into a subject; 4 is a flow chart showing the operation of the ultrasonic diagnostic apparatus according to Embodiment 1 of the present invention; FIG. 4 is a diagram schematically showing an acquired ultrasound image displaying the actual full length of the puncture needle. FIG. 4 is a diagram schematically showing an ultrasound image in which the actual total length of the acquired puncture needle and the estimated length of the puncture needle are displayed. FIG. 4 is a diagram schematically showing a packaging bag for a puncture needle having a bar code; FIG. 4 is a diagram schematically showing how a scale and a puncture needle are arranged side by side; FIG. 4 is a block diagram showing the configuration of an apparatus main body according to Embodiment 2 of the present invention; FIG. 4 is a diagram schematically showing a puncture needle inserted into a subject; FIG. 4 is a diagram schematically showing an ultrasound image in which the actual total length of the acquired puncture needle and the length of the portion of the puncture needle inserted into the subject are displayed. FIG. 11 is a block diagram showing the configuration of a main body of a diagnostic device according to Embodiment 3 of the present invention; FIG. 4 is a diagram schematically showing an ultrasound image in which the actual total length of the puncture needle that has been acquired, the estimated length of the puncture needle, and the distance between the target site and the tip of the puncture needle are displayed. FIG. 12 is a block diagram showing the configuration of a main body of a diagnostic device according to Embodiment 4 of the present invention; FIG. 11 is a block diagram showing the configuration of an ultrasonic diagnostic apparatus according to Embodiment 5 of the present invention;

BEST MODE FOR CARRYING OUT THE INVENTION Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings.
The description of the constituent elements described below is based on representative embodiments of the present invention, but the present invention is not limited to such embodiments.
In this specification, a numerical range represented by "-" means a range including the numerical values before and after "-" as lower and upper limits.
Also, in this specification, the terms "perpendicular" and "parallel" include the range of error that is permissible in the technical field to which the present invention belongs. For example, "perpendicular" and "parallel" means within a range of less than ±10° with respect to strict perpendicularity or parallelism, and the error with respect to strict perpendicularity or parallelism is 5° or less is preferable, and 3° or less is more preferable.
As used herein, the terms "same" and "same" shall include the margin of error generally accepted in the technical field. In addition, in this specification, when "all", "all" or "whole surface" is used, in addition to the case of 100%, the error range generally accepted in the technical field is included, for example, 99% or more, 95% or more, or 90% or more shall be included.

Embodiment 1
FIG. 1 shows the configuration of an ultrasonic diagnostic apparatus 1 according to Embodiment 1 of the present invention. The ultrasonic diagnostic apparatus 1 includes an ultrasonic probe 2, a head-mounted display 3, and a diagnostic apparatus main body 4. The ultrasonic probe 2 and the head-mounted display 3 are each wirelessly connected to the diagnostic apparatus main body 4. there is The head-mounted display 3 is a display device that is worn on the user's head and is visually recognized by the user wearing it. An ultrasound image or the like obtained from the ultrasonic wave is displayed on the head-mounted display 3 .

As shown in FIG. 2, the ultrasonic probe 2 includes a transducer array 11 to which a transmitter 12 and a receiver 13 are connected. A signal processing unit 14 and a probe-side wireless communication unit 15 are sequentially connected to the receiving unit 13 . A communication control unit 16 is connected to the probe-side wireless communication unit 15 . A probe control unit 17 is connected to the transmission unit 12, the reception unit 13, the signal processing unit 14, the probe-side wireless communication unit 15, and the communication control unit 16. , the communication control unit 16 and the probe control unit 17 constitute a probe processor 18 . In addition, the ultrasonic probe 2 incorporates a battery 19 .

The transducer array 11 of the ultrasonic probe 2 shown in FIG. 2 has a plurality of transducers arranged one-dimensionally or two-dimensionally. These transducers transmit ultrasonic waves in accordance with drive signals supplied from the transmission unit 12, receive ultrasonic echoes from the subject, and output signals based on the ultrasonic echoes. Each vibrator includes, for example, a piezoelectric ceramic typified by PZT (Lead Zirconate Titanate), a polymeric piezoelectric element typified by PVDF (Poly Vinylidene Di Fluoride), and PMN-PT ( Lead Magnesium Niobate-Lead Titanate: Electrodes are formed at both ends of a piezoelectric body made of a piezoelectric single crystal or the like represented by lead magnesium niobate-lead titanate solid solution).

The transmitter 12 of the probe processor 18 includes, for example, a plurality of pulse generators, and generates a plurality of vibrations of the transducer array 11 based on a transmission delay pattern selected according to a control signal from the probe controller 17. Each drive signal is supplied to a plurality of transducers by adjusting the delay amount so that ultrasonic waves transmitted from the transducers form ultrasonic beams. Thus, when a pulsed or continuous wave voltage is applied to the electrodes of the transducers of the transducer array 11, the piezoelectric body expands and contracts, and pulsed or continuous wave ultrasonic waves are generated from the respective transducers. , an ultrasonic beam is formed from the composite wave of these ultrasonic waves.

The transmitted ultrasonic beam is reflected by an object such as a site of the subject and propagates toward the transducer array 11 of the ultrasonic probe 2 . The ultrasonic waves propagating toward the transducer array 11 in this way are received by the respective transducers forming the transducer array 11 . At this time, each transducer constituting the transducer array 11 receives the propagating ultrasonic echo and expands and contracts to generate an electric signal, and outputs these electric signals to the receiving unit 13 .

The receiver 13 of the probe processor 18 processes the signal output from the transducer array 11 according to the control signal from the probe controller 17 . As shown in FIG. 3, the receiver 13 has a configuration in which an amplifier 20, an AD (Analog Digital) converter 21, and a beamformer 22 are connected in series.

The amplification unit 20 amplifies signals input from the respective transducers forming the transducer array 11 and transmits the amplified signals to the AD conversion unit 21 . The AD converter 21 converts the signal transmitted from the amplifier 20 into digital data, and transmits the data to the beamformer 22 . The beamformer 22 converts each data converted by the AD converter 21 into respective A so-called reception focus process is performed by giving a delay and adding. By this reception focusing process, each data converted by the AD converter 21 is phased and added, and a reception signal in which the focus of the ultrasonic echo is narrowed down is obtained.

The signal processing unit 14 of the probe processor 18 corrects the received data generated by the receiving unit 13 for attenuation due to distance according to the depth of the reflection position of the ultrasonic wave, and then performs envelope detection processing. , to generate ultrasound image signals, which are tomographic image information about tissues in the subject.

The probe-side wireless communication unit 15 of the ultrasonic probe 2 includes an antenna for transmitting and receiving radio waves, and modulates a carrier based on the ultrasonic image signal generated by the signal processing unit 14 to generate ultrasonic waves. A transmission signal representing the image signal is generated. The probe-side wireless communication unit 15 supplies a transmission signal representing the ultrasonic image signal thus generated to the antenna and transmits radio waves from the antenna, thereby sequentially wirelessly transmitting the ultrasonic image signal to the diagnostic apparatus main body 4. Send. Carrier modulation methods include ASK (Amplitude Shift Keying), PSK (Phase Shift Keying), QPSK (Quadrature Phase Shift Keying), 16QAM (16 Quadrature Amplitude Modulation: 16 quadrature amplitude modulation), etc. are used. The probe-side wireless communication unit 15 also receives instruction information and the like for instructing the operation of the ultrasonic probe 2 from the diagnostic apparatus main body 4 and inputs the received instruction information and the like to the probe control unit 17 .

The communication control unit 16 of the probe processor 18 controls the probe-side wireless communication unit 15 so that the ultrasonic image signal is transmitted with the transmission radio wave intensity set by the probe control unit 17 .
The probe control section 17 of the probe processor 18 controls each section of the ultrasonic probe 2 based on a program stored in advance and instruction information wirelessly transmitted from the diagnostic apparatus main body 4 .
The battery 19 of the ultrasonic probe 2 is built in the ultrasonic probe 2 and supplies electric power to each circuit of the ultrasonic probe 2 .

A probe processor 18 having a transmitting unit 12, a receiving unit 13, a signal processing unit 14, a communication control unit 16, and a probe control unit 17 is a CPU (Central Processing Unit), and performs various processes on the CPU. FPGA (Field Programmable Gate Array), DSP (Digital Signal Processor), ASIC (Application Specific Integrated Circuit), It may be configured using other ICs (Integrated Circuits), or may be configured by combining them. Further, the transmitting unit 12, the receiving unit 13, the signal processing unit 14, the communication control unit 16, and the probe control unit 17 can be partially or wholly integrated into one CPU or the like.

The head-mounted display 3 is a display device worn on the user's head and visually recognized by the user wearing it, and has a so-called eyeglass-like shape, as shown in FIG. The head-mounted display 3 includes two display portions 31A and 31B, which are connected to each other by a bridge portion B, and tendons are attached to the ends of the two display portions 31A and 31B, respectively. A is connected. For example, the head-mounted display 3 is fixed to the subject's head by hooking the bridge portion B on the user's nose and the two tendrils A on the user's ears. At this time, the two display units 31A and 31B face the left and right eyes of the user, respectively.

A camera section 32 having a photographing lens F arranged in front thereof is attached to the connecting portion between the display section 31B on the left side and the vine section A. As shown in FIG. In addition, in the vine portion A connected to the display portion 31A on the right side, a storage portion D in which various circuits necessary for the operation of the head mounted display 3, a battery, and the like are stored is arranged.

FIG. 5 shows the configuration of the head mounted display 3. As shown in FIG. The head mounted display 3 has a head mounted display side wireless communication section 33 , and the camera section 32 , the communication control section 34 and the display control section 35 are connected to the head mounted display side wireless communication section 33 . Also, the display unit 31 is connected to the display control unit 35 . A head mounted display control section 36 is connected to the camera section 32 , the head mounted display side wireless communication section 33 , the communication control section 34 and the display control section 35 .
For the sake of explanation, the two display sections 31A and 31B in FIG. 4 are collectively called the display section 31. FIG.

A head mounted display processor 37 is configured by the communication control section 34 , the display control section 35 and the head mounted display control section 36 . Also, the head mounted display 3 incorporates a battery 38 . The head mounted display side wireless communication section 33 , the head mounted display processor 37 and the battery 38 are housed in the housing section D of the head mounted display 3 .

The camera unit 32 of the head-mounted display 3 generates a field-of-view image obtained by photographing the field of view in front of the user through the imaging lens F. Although not shown, the camera unit 32 has an image sensor that captures a field of view in front of the user through the photographing lens F, acquires a field of view image signal that is an analog signal, and amplifies the field of view image signal acquired by the image sensor. It incorporates an analog signal processing circuit that converts to a digital signal and a digital signal processing circuit that performs various corrections such as gain correction on the converted digital signal to generate a visual field image.
Note that the analog signal processing circuit and the digital signal processing circuit can also be incorporated in the head mounted display processor 37 .

The head-mounted display-side wireless communication unit 33 of the head-mounted display 3, like the probe-side wireless communication unit 15 of the ultrasonic probe 2, includes an antenna for transmitting and receiving radio waves, and is generated by the camera unit 32. The visual field image obtained is transmitted to the main body 4 of the diagnostic apparatus. In addition, it receives an ultrasonic image or the like transmitted from the diagnostic apparatus main body 4 and transmits the received ultrasonic image or the like to the display section 31 via the display control section 35 . The head mounted display side wireless communication unit 33 also receives instruction information and the like for instructing the operation of the head mounted display 3 from the diagnostic apparatus main body 4 and outputs the received instruction information and the like to the head mounted display control unit 36. .

The display unit 31 of the head mounted display 3 has translucency in order to ensure the user's field of vision when the user wears the head mounted display 3 . Further, the display unit 31 is a display for displaying an ultrasound image or the like. Since the display unit 31 has such a configuration, it can display an ultrasonic image or the like transmitted from the diagnostic apparatus main body 4, for example.

The display control unit 35 of the head-mounted display processor 37 performs predetermined processing on the ultrasonic image or the like transmitted from the diagnostic apparatus main body 4 under the control of the head-mounted display control unit 36, and displays the ultrasonic image on the display unit 31. etc. is displayed.
The communication control unit 34 of the head-mounted display processor 37 controls the head-mounted display-side wireless communication unit so that the field-of-view image is transmitted and the ultrasonic image and the like are received at the transmission/reception radio wave intensity set by the head-mounted display control unit 36. 33.

The head-mounted display control unit 36 of the head-mounted display processor 37 controls each unit of the head-mounted display 3 based on a program stored in advance and instruction information wirelessly transmitted from the diagnostic apparatus main body 4 .
A battery 38 of the head-mounted display 3 is built in the head-mounted display 3 and supplies power to each circuit of the head-mounted display 3 .

The head-mounted display processor 37 having the communication control unit 34, the display control unit 35, and the head-mounted display control unit 36 is composed of a CPU and a control program for causing the CPU to perform various processes. , DSP, ASIC, GPU (Graphics Processing Unit), other ICs, or a combination thereof. Also, the communication control unit 34, the display control unit 35, and the head-mounted display control unit 36 can be partially or wholly integrated into one CPU or the like.

As shown in FIG. 6 , the diagnostic apparatus main body 4 has a main body side wireless communication section 41 , and a communication control section 42 is connected to the main body side wireless communication section 41 . Also, an ultrasonic image processing unit 43, a display control unit 44, and a display monitor 45 are connected to the main unit side wireless communication unit 41 in this order. Further, a puncture needle length calculator 46 and a full length acquisition unit 47 are connected to the body side wireless communication unit 41 . An actual length estimation unit 48 is connected to the puncture needle length calculation unit 46 , and a display control unit 44 and an actual length estimation unit 48 are connected to the total length acquisition unit 47 . Also, the display control unit 44 is connected to the actual length estimation unit 48 . A body control unit 49 is connected to the communication control unit 42, the ultrasonic image processing unit 43, the display control unit 44, the puncture needle length calculation unit 46, the total length acquisition unit 47, and the actual length estimation unit 48, A storage unit 50 and an input unit 51 are connected to the body control unit 49 . The body control unit 49 and the storage unit 50 are connected so as to be able to exchange information bidirectionally.

In addition, the diagnostic device main body processor 52 is controlled by the communication control unit 42, the ultrasonic image processing unit 43, the display control unit 44, the puncture needle length calculation unit 46, the total length acquisition unit 47, the actual length estimation unit 48, and the main body control unit 49. is configured.

The body-side wireless communication unit 41 of the diagnostic apparatus body 4 transmits and receives radio waves in the same manner as the probe-side wireless communication unit 15 of the ultrasonic probe 2 and the head-mounted display-side wireless communication unit 33 of the head-mounted display 3. , and transmits instruction information, etc., and receives ultrasonic image signals, visual field images, etc. to the probe-side wireless communication unit 15 and the head-mounted display-side wireless communication unit 33 by wireless communication.

The ultrasonic image processing unit 43 of the diagnostic apparatus main body processor 52 receives the ultrasonic image signal transmitted from the ultrasonic probe 2 via the main body side wireless communication unit 41, and generates an ultrasonic image based on the received ultrasonic image signal. to generate At this time, the ultrasonic image processing unit 43 first converts the ultrasonic image signal into an image signal (raster conversion) in accordance with a normal television signal operation method to generate an ultrasonic image. Further, the ultrasonic image processing unit 43 performs various necessary image processing such as gradation processing on the generated ultrasonic image, and then transmits the processed ultrasonic image to the main body side wireless communication unit 41 and the Output to the display control unit 44 . Here, the ultrasonic image output to the main body side wireless communication unit 41 is transmitted to the head mounted display 3 by wireless communication.

The puncture needle length calculation unit 46 of the diagnostic apparatus main body processor 52 receives the visual field image transmitted from the head mounted display 3 via the main body side wireless communication unit 41, and inserts the needle into the subject in the received visual field image. When at least part of the puncture needle is captured, the puncture needle is recognized by image analysis of the visual field image, and the length of the puncture needle in the visual field image is calculated.
Here, the puncture needle refers to, for example, the one shown in FIG. 7, and one end of the puncture needle N is sharply pointed for puncturing the subject, and the other end is attached to a so-called syringe cylinder or the like. is connected. Such a puncture needle N is generally used for the purpose of collecting a specimen from inside a subject, injecting a drug into the subject, and the like.

Further, when recognizing the puncture needle N shown in the visual field image, the puncture needle length calculation unit 46 stores, for example, typical pattern data of the puncture needle as a template in advance, and searches the visual field image using the template. While calculating the similarity to the pattern data, it can be considered that the puncture needle N exists at a place where the similarity is greater than or equal to the threshold and reaches the maximum.
In addition to simple template matching, for similarity calculation, for example, Csurka et al.: Visual Categorization with Bags of Keypoints, Proc. of ECCV Workshop on Statistical Learning in Computer Vision, pp.59-74 (2004) described machine learning methods, or in general using deep learning as described in Krizhevsk et al.: ImageNet Classification with Deep Convolutional Neural Networks, Advances in Neural Information Processing Systems 25, pp.1106-1114 (2012) An image recognition technique or the like can be used.

The total length acquisition unit 47 of the diagnostic apparatus main body processor 52 receives the visual field image transmitted from the head mounted display 3 via the main body side wireless communication unit 41, and calculates the actual total length of the puncture needle N based on the received visual field image. get. For example, as shown in FIG. 7, when a plurality of grooves G are formed in the puncture needle N at predetermined intervals LG and the entire puncture needle N is shown in the visual field image, , the total length acquisition unit 47 acquires the actual total length LA of the puncture needle N using the interval LG between the plurality of grooves by image analysis of the visual field image. At this time, the total length acquiring unit 47 stores, for example, in advance the actual length of the interval LG between the plurality of grooves G, and refers to the stored actual length of the interval LG to obtain the puncture needle N can be obtained. Also, the value of the interval LG between the plurality of grooves G can be input in advance by the user via the input section 51 .

The actual length estimating unit 48 of the diagnostic apparatus main body processor 52 calculates the visual field of the puncture needle N calculated by the puncture needle length calculating unit 46 from the visual field image of the entire puncture needle N not inserted into the subject. A correspondence relationship between the length in the image and the actual total length of the puncture needle N acquired by the total length acquisition unit 47 is acquired, and the puncture needle is calculated by the puncture needle length calculation unit 46 based on the acquired correspondence relationship. From the length of N, the actual length of the puncture needle N photographed in the visual field image is estimated. For example, as shown in FIG. 8, when the puncture needle N is inserted into the subject S and the image of the puncture needle N is captured by the camera unit 32 of the head-mounted display 3, the outside of the subject S is shown in the visual field image. A portion where the puncture needle N is not inserted is shown. In this case, the actual length estimating unit 48 calculates the length of the puncture needle N in the visual field image calculated by the puncture needle length calculating unit 46 based on the actual total length LA of the puncture needle N. Estimate the actual length LB of the uninserted portion.

Further, the actual length estimating unit 48 sends the estimated value of the actual length of the puncture needle N to the main body side wireless communication unit 41 and the display control unit 44 . The value of the actual length of the puncture needle N sent to the body side wireless communication unit 41 is sent to the head mounted display 3 by wireless communication.

The display control unit 44 of the diagnostic apparatus main body processor 52 displays the ultrasonic image generated by the ultrasonic image processing unit 43 and the actual length of the puncture needle N estimated by the actual length estimating unit 48 under the control of the main body control unit 49 . is subjected to predetermined processing, and the ultrasonic image and the estimated actual length of the puncture needle N are displayed on the display monitor 45 .
The display monitor 45 of the diagnostic apparatus main body 4 displays images under the control of the display control unit 44. For example, a display such as an LCD (Liquid Crystal Display) or an organic EL display (Organic Electroluminescence Display) is used. Including equipment.

A body control unit 49 of the diagnostic device body processor 52 controls each part of the diagnostic device body 4 based on a program pre-stored in the storage unit 50 or the like and a user's input operation via the input unit 51 .
The communication control section 42 of the diagnostic apparatus main body processor 52 controls the main body side wireless communication section 41 so that data is transmitted and received at the transmission/reception radio wave intensity set by the main body control section 49 .
The input unit 51 of the diagnostic apparatus main body 4 is used by the user to perform input operations, and can be configured by including a keyboard, mouse, trackball, touch pad, touch panel, and the like.

A diagnostic apparatus main body processor 52 having a communication control section 42, an ultrasonic image processing section 43, a display control section 44, a puncture needle length calculation section 46, a total length acquisition section 47, an actual length estimation section 48, and a main body control section 49. is composed of a CPU and a control program for causing the CPU to perform various processes, but may be configured using FPGA, DSP, ASIC, GPU, and other ICs, or may be configured by combining them may be In addition, the communication control unit 42, the ultrasonic image processing unit 43, the display control unit 44, the puncture needle length calculation unit 46, the total length acquisition unit 47, the actual length estimation unit 48, and the main body control unit 49 are partially or entirely It is also possible to configure them to be practically integrated into one CPU or the like.

Next, the operation of the ultrasonic diagnostic apparatus 1 according to Embodiment 1 of the present invention will be described using the flowchart of FIG.
First, in step S1, the camera unit 32 of the head-mounted display 3 worn on the user's head captures the forward field of view of the user to generate a field-of-view image.

In subsequent step S2, it is determined whether or not the entire puncture needle N has been imaged in step S1, that is, whether or not the entire puncture needle N is shown in the visual field image generated in step S1. At this time, for example, if the puncture needle length calculation unit 46 of the diagnostic apparatus main body 4 performs recognition processing of the puncture needle N on the visual field image captured in step S1, and the entire puncture needle N is recognized. , it is determined that the whole puncture needle N is shown in the visual field image, and if the whole puncture needle N is not recognized, it is judged that the whole puncture needle N is not shown in the visual field image.

When it is determined in step S1 that the entire puncture needle N is not photographed, the process returns to step S1, a new visual field image is photographed by the camera section 32 of the head mount display 3, and the process proceeds to step S2. In this manner, the processes of steps S1 and S2 are repeated until it is determined in step S2 that the entire puncture needle N has been photographed in step S1.
If it is determined in step S2 that the entire puncture needle N has been imaged in step S1, the process proceeds to step S3.

In step S3, the puncture needle length calculator 46 calculates the total length of the puncture needle N in the visual field image based on the visual field image acquired in step S1. At this time, the puncture needle length calculator 46 performs image analysis on the visual field image acquired in step S1, thereby recognizing the entire puncture needle N shown in the visual field image. Calculate the total length on the visual field image.

In subsequent step S4, the actual total length of the puncture needle N is obtained by the total length obtaining section 47 of the diagnostic device main body 4. FIG. For example, as shown in FIG. 7, when a plurality of grooves G are formed in the puncture needle N at predetermined intervals LG and the entire puncture needle N is shown in the visual field image, , the total length acquisition unit 47 acquires the actual total length LA of the puncture needle by image analysis of the visual field image using the interval LG between the plurality of grooves. At this time, the total length acquiring unit 47 stores, for example, in advance the actual length of the interval LG between the plurality of grooves G, and refers to the stored actual length of the interval LG to obtain the puncture needle N Obtain the actual total length LA of .

In step S5, the actual total length LA of the puncture needle N obtained in step S4 is displayed on the display unit 31 of the head-mounted display 3 and the display monitor 45 of the diagnostic apparatus main body 4. FIG. Here, the actual total length LA of the puncture needle N acquired in step S4 continues to be displayed on the display unit 31 of the head-mounted display 3 and the display monitor 45 of the diagnostic device main body 4 even after that. This allows the user to easily grasp the actual total length of the puncture needle N being used.

In step S6, the actual length estimating unit 48 of the diagnostic device main body 4 uses the total length of the puncture needle N in the visual field image calculated in step S3 and the actual total length of the puncture needle N acquired in step S4. , the correspondence relationship between the length of the puncture needle N on the visual field image calculated by the puncture needle length calculation unit 46 and the actual length of the puncture needle N photographed by the camera unit 32 of the head mounted display 3 is obtained. be. For example, the actual length estimating unit 48 may obtain, as this correspondence relationship, the ratio of the actual total length of the puncture needle N obtained in step S4 to the total length of the puncture needle N in the visual field image calculated in step S3. can. The actual length estimation unit 48 calculates the actual length of the puncture needle N captured in the visual field image from the length of the puncture needle N calculated by the puncture needle length calculation unit 46 by using the acquired correspondence relationship. Length can be estimated.

In step S7, an ultrasonic beam is transmitted from the transducer array 11 into the subject S while the user keeps the ultrasonic probe 2 in contact with the subject S, and the ultrasonic echoes from the subject S are generated by the transducers. Received by array 11 to generate a received signal. When the signal processing unit 14 of the ultrasonic probe 2 processes the received signal and generates an ultrasonic image signal, the ultrasonic image processing unit 43 of the diagnostic apparatus main body 4 generates an ultrasonic image. The ultrasonic image thus generated is displayed on the display unit 31 of the head-mounted display 3 and the display monitor 45 of the main body 4 of the diagnostic apparatus.
It should be noted that even after step S8, ultrasonic images are sequentially generated, and the generated ultrasonic images are displayed on the display unit 31 of the head-mounted display 3 and the display monitor 45 of the diagnostic apparatus main body 4. FIG. The user touches the subject S with the ultrasound probe 2 so as to visualize the location where the puncture needle N is inserted into the subject S in the ultrasound image while the ultrasound images are being sequentially generated. , the puncture needle N is inserted into the subject S;

Further, when the ultrasonic image generated in step S8 is displayed on the display unit 31 of the head-mounted display 3 and the display monitor 45 of the diagnostic device main body 4, the actual total length of the puncture needle N acquired in step S4 is , superimposed on the ultrasonic image generated in step S8 and displayed on the display unit 31 of the head-mounted display 3 and the display monitor 45 of the main body 4 of the diagnostic apparatus. For example, in a state in which the puncture needle N is not inserted into the subject S, the actual total length of the puncture needle N obtained in step S4 is the puncture needle length V1 as shown in FIG. superimposed on the On the ultrasonic image U illustrated in FIG. 10, the fact that the puncture needle length V1 is 70 mm is superimposed, and the target site T to be punctured is depicted.

In the subsequent step S8, the camera section 32 of the head-mounted display 3 captures a field-of-view image in the same manner as in step S1.
In step S9, it is determined whether or not at least part of the puncture needle N has been captured in step S8, that is, whether or not at least part of the puncture needle N is captured in the visual field image captured in step S8. If it is determined in step S9 that at least part of the puncture needle N has not been captured in step S8, the process returns to step S8, a new visual field image is captured by the camera unit 32 of the head mounted display 3, and step S9 is performed. proceed to In this way, the processes of steps S8 and S9 are repeated until it is determined in step S9 that at least part of the puncture needle N has been photographed in step S8.

If it is determined in step S9 that at least part of the puncture needle N has been imaged in step S8, the process proceeds to step S10.
In step S10, the length of the puncture needle N in the visual field image captured in step S7 is calculated by the puncture needle length calculator 46 of the diagnostic apparatus main body 4. FIG. For example, as shown in FIG. 8, when the puncture needle N is partially inserted into the subject S, the length of the uninserted portion of the puncture needle N that exists outside the subject S in the visual field image is is calculated.

In step S11, the actual length estimating section 48 of the diagnostic apparatus main body 4 estimates the actual length of the puncture needle N photographed in the visual field image. At this time, the actual length estimating unit 48 uses the correspondence relationship between the total length of the puncture needle N in the visual field image and the actual total length of the puncture needle N acquired in step S6 to calculate the puncture length calculated in step S10. From the length of the needle N, the actual length of the puncture needle N photographed in the visual field image is estimated. For example, as shown in FIG. 8, when the puncture needle N is partially inserted into the subject S, the actual length estimation unit 48 calculates the actual length LA of the puncture needle N as a reference. Estimate the actual length LB of the uninserted portion of .

In subsequent step S12, the actual length of the puncture needle N estimated in step S11 is superimposed on the ultrasonic image U generated in step S7 by the actual length estimating unit 48, and displayed on the head mounted display 3. It is displayed on the unit 31 and the display monitor 45 of the diagnostic device main body 4 . For example, when part of the puncture needle N is inserted into the subject S, the actual length estimator 48 calculates the actual length of the puncture needle N estimated in step S11 as shown in FIG. is superimposed on the ultrasonic image U as an estimated length V2. On the ultrasonic image U illustrated in FIG. 10, the information that the estimated length V2 is 50 mm is superimposed, and the puncture needle N and the target site T are depicted.
By confirming the length of the uninserted portion of the puncture needle N displayed on the display unit 31 of the head-mounted display 3 or the display monitor 45 of the diagnostic apparatus main body 4 in this way, the user can confirm the current subject S It is possible to easily grasp how much the puncture needle N is inserted inside.

Finally, in step S13, it is determined whether or not the operation of the ultrasonic diagnostic apparatus 1 is finished. Although not shown, for example, the display unit 31 of the head-mounted display 3 and the display monitor 45 of the diagnostic apparatus main body 4 display an end button for ending the operation of the ultrasonic diagnostic apparatus 1 . When the end button is pressed by the user via the input unit 51 or the like, it is determined that the operation of the ultrasonic diagnostic apparatus 1 is to end. If the end button is not pressed, the operation of the ultrasonic diagnostic apparatus 1 must be ended. be judged.
If it is determined in step S13 that the operation of the ultrasonic diagnostic apparatus 1 is not to end, the processing of steps S8 to S13 is performed again. In this manner, the processes of steps S8 to S13 are repeated until it is determined that the operation of the ultrasonic diagnostic apparatus 1 is finished. When it is determined in step S13 that the operation of the ultrasonic diagnostic apparatus 1 is finished, the operation of the ultrasonic diagnostic apparatus 1 is finished.

As described above, according to the ultrasonic diagnostic apparatus 1 according to Embodiment 1 of the present invention, when at least part of the puncture needle N is photographed by the camera unit 32 of the head-mounted display 3, the length of the puncture needle N The length of the puncture needle N photographed is calculated by the calculation unit 46, and the actual length of the puncture needle N is estimated by the actual length estimation unit 48 based on the calculated length of the puncture needle N. , the estimated actual length of the puncture needle N is superimposed on the ultrasonic image U and displayed on the display unit 31 of the head-mounted display 3 and the display monitor 45 of the diagnostic apparatus main body 4, so that the user can perform the puncture. The length of the portion of the needle N inserted into the subject S can be accurately and easily grasped.

In Embodiment 1, as shown in FIG. 7, the puncture needle N is exemplified with a plurality of grooves G arranged at predetermined intervals LG. , and some are further subjected to so-called engraving, in which fine grooves are formed over a certain length from the tip. For example, when the camera unit 32 of the head-mounted display 3 captures an image of the entire puncture needle N that has been engraved, the total length acquisition unit 47 of the diagnostic device main body 4 determines the interval LG between the plurality of grooves G. Alternatively, the length of the engraved portion can be used to obtain the actual total length LA of the puncture needle N. FIG.

Further, although the total length acquisition unit 47 acquires the actual total length LA of the puncture needle N based on the interval LG between the plurality of grooves G formed in the puncture needle N, the total length LA of the puncture needle N The method of obtaining the actual full-length LA of is not limited to this.
For example, as shown in FIG. 12, the packaging bag P of the puncture needle N may be printed with a bar code BC in which length information indicating the actual total length LA of the puncture needle N is recorded. When the barcode BC is photographed by the camera unit 32 of the head-mounted display 3, the total length acquisition unit 47 reads the length information recorded in the barcode BC appearing in the visual field image, thereby obtaining the actual length of the puncture needle N. of full-length LA can be obtained. Thus, the total length acquisition unit 47 can also acquire the actual total length LA of the puncture needle N by reading the length information recorded in the length information recording member such as the barcode BC.

Further, as shown in FIG. 13, the total length acquiring unit 47 acquires the entire puncture needle N not inserted into the subject S and the scale SC for measuring the length by the camera unit 32 of the head mounted display 3. are taken in parallel with each other, the actual total length LA of the puncture needle N can be calculated based on the graduations of the scale SC.

Further, in Embodiment 1, the ultrasonic image U generated by the ultrasonic image processing unit 43 of the diagnostic apparatus main body 4 and the actual length of the puncture needle N estimated by the actual length estimating unit 48 are combined into a head mount. Although it is described that the information is displayed on both the display unit 31 of the display 3 and the display monitor 45 of the diagnostic device main body 4, either the display unit 31 of the head mounted display 3 or the display monitor 45 of the diagnostic device main body 4 may be displayed in Even in this case, the user can confirm the estimated actual length of the puncture needle N, and can accurately and easily grasp the length of the portion of the puncture needle N inserted into the subject S. .

Moreover, although the ultrasonic probe 2, the head-mounted display 3, and the diagnostic apparatus main body 4 are connected to each other by wireless communication, they may be connected by wire instead of using wireless communication. For example, although not shown, the ultrasonic probe 2 and the diagnostic device main body 4 can be connected by wire, and the head mounted display 3 and the diagnostic device main body 4 can be connected by wire.

In addition, if the head mounted display 3 is equipped with a display unit 31 that can be visually recognized by the user and a camera unit 32 for capturing a field of view in front of the user and can be worn on the user's head, the head mounted display 3 is not limited to the shape shown in FIG. For example, the head-mounted display 3 may have only one of the two display sections 31A and 31B instead of having both of the two display sections 31A and 31B. A single display may be provided.

Further, although the diagnostic apparatus main body 4 is provided with the ultrasonic image processing section 43 , the ultrasonic probe 2 may be provided with the ultrasonic image processing section 43 instead of being provided in the diagnostic apparatus main body 4 . In this case, an ultrasonic image U is generated in the ultrasonic probe 2 and the generated ultrasonic image U is transmitted to the diagnostic apparatus main body 4 via the probe-side wireless communication unit 15 . The ultrasonic image U transmitted to the diagnostic apparatus main body 4 is input to the display control section 44 via the main body side wireless communication section 41 of the diagnostic apparatus main body 4, and is transmitted from the main body side wireless communication section 41 to the head mounted display 3. sent to. The ultrasonic image U transmitted to the head mounted display 3 is input to the display control section 35 via the head mounted display side wireless communication section 33 of the head mounted display 3 . At this time, the ultrasonic image U may be transmitted directly from the ultrasonic probe 2 to the head-mounted display 3 without going through the diagnostic apparatus main body 4 .

Embodiment 2
In Embodiment 1, the actual length estimation unit 48 of the diagnostic device main body 4 estimates the actual length of the puncture needle N photographed by the camera unit 32 of the head mounted display 3. Based on the length of the puncture needle N, the length of the portion of the puncture needle N inserted into the subject S can also be estimated.

The ultrasonic diagnostic apparatus according to the second embodiment has the ultrasonic probe 2 and the head mount display 3 according to the first embodiment connected to a diagnostic apparatus main body 4A shown in FIG. A diagnostic device main body 4A according to the second embodiment has a main body control section 49A instead of the main body control section 49 in the diagnostic device main body 4 shown in FIG. 6, and an insertion length estimation section 53 is added.

In the diagnostic apparatus main body 4A, an insertion length estimation section 53 is connected to the total length acquisition section 47 and the actual length estimation section 48, and the insertion length estimation section 53 is connected to the main body side wireless communication section 41 and the distance calculation section 54. It is In addition, the communication control unit 42, the ultrasound image processing unit 43, the display control unit 44, the puncture needle length calculation unit 46, the total length acquisition unit 47, the storage unit 50, the input unit 51, and the insertion length estimation unit 53 are provided with a main body control unit. 49A is connected.
Also, a communication control unit 42, an ultrasound image processing unit 43, a display control unit 44, a puncture needle length calculation unit 46, a total length acquisition unit 47, an actual length estimation unit 48, a main body control unit 49A, and an insertion length estimation unit 53 The diagnosis device main body processor 52A is configured by the above.

As shown in FIG. 15, the insertion length estimating unit 53 of the diagnostic apparatus main body processor 52A uses the camera unit 32 of the head mounted display 3 to capture an image of the puncture needle N partially inserted into the subject S. In this case, based on the actual total length LA of the puncture needle N acquired by the total length acquisition unit 47 and the actual length LB of the uninserted portion of the puncture needle N estimated by the actual length estimation unit 48, the puncture is performed. Estimate the length LC of the portion of the needle N inserted into the subject S. In addition, the insertion length estimating unit 53 sends the estimated value of the length LC to the main body side wireless communication unit 41 and the display control unit 44, and They are displayed on the display monitor 45 respectively.

At this time, the insertion length estimating unit 53 sets the estimated length LC of the portion of the puncture needle N inserted into the subject S as the insertion length V3, for example, as shown in FIG. , superimposed on the ultrasonic image U and displayed. In the ultrasonic image U illustrated in FIG. 16, the actual total length of the puncture needle N acquired by the total length acquisition unit 47, that is, the puncture needle length V1 is 70 mm, and the insertion length V3 is 20 mm. is superimposed, and the puncture needle N inserted into the subject S and the target site T to be punctured are depicted.

As described above, according to the ultrasonic diagnostic apparatus according to the second embodiment, the length LC of the portion of the puncture needle N inserted into the subject S is estimated by the insertion length estimating unit 53. Since the length LC is superimposed on the ultrasonic image U and displayed on the display unit 31 of the head-mounted display 3 and the display monitor 45 of the diagnostic apparatus main body 4A, the user inserts the puncture needle N into the subject S. It is possible to accurately and simply ascertain the length of the portion that is held.

In the second embodiment, the actual length LB of the puncture needle N photographed in the visual field image estimated by the actual length estimating unit 48 of the diagnostic device main body 4A is the display unit 31 of the head mounted display 3. Instead of being displayed on the display monitor 45 of the diagnostic apparatus main body 4A, the length LC of the portion of the puncture needle N inserted into the subject S estimated by the insertion length estimation unit 53 is displayed. Both the actual length LB of the puncture needle N and the length LC of the portion of the puncture needle N inserted into the subject S are displayed on the display unit 31 of the head-mounted display 3 and the display monitor 45 of the diagnostic apparatus main body 4A. may be displayed. In this case as well, the user can accurately and easily grasp the length of the portion of the puncture needle N inserted into the subject S.

Embodiment 3
In the second embodiment, the insertion length estimation unit 53 estimates the length LC of the portion of the puncture needle N inserted into the subject S. The distance to the tip of the puncture needle N can also be calculated.

The ultrasonic diagnostic apparatus according to the third embodiment is obtained by connecting the ultrasonic probe 2 and the head mount display 3 according to the first embodiment to a diagnostic apparatus main body 4B shown in FIG. A diagnostic device main body 4B according to the third embodiment has a main body control section 49B instead of the main body control section 49A in the diagnostic device main body 4A according to the second embodiment shown in FIG. 14, and a distance calculation section 54 is added. is.

In the diagnostic apparatus main body 4B, the ultrasonic image processing section 43 and the insertion length estimating section 53 are connected to the distance calculating section 54, and the main body side wireless communication section 41 and the display control section 44 are connected to the distance calculating section 54. there is Communication control unit 42, ultrasonic image processing unit 43, display control unit 44, puncture needle length calculation unit 46, total length acquisition unit 47, actual length estimation unit 48, storage unit 50, input unit 51, insertion length A body control unit 49B is connected to the estimation unit 53 and the distance calculation unit 54 .
Communication control unit 42 , ultrasonic image processing unit 43 , display control unit 44 , puncture needle length calculation unit 46 , total length acquisition unit 47 , actual length estimation unit 48 , body control unit 49B, insertion length estimation unit 53 and the distance calculation unit 54 constitute a diagnostic apparatus main body processor 52B.

As shown in FIG. 18, the distance calculation unit 54 of the diagnostic apparatus main body processor 52B performs image analysis of the ultrasonic image U when the puncture target site T and the puncture needle N are depicted in the ultrasonic image U. to recognize the target site T and the puncture needle N, and the target site T and the actual length LC of the portion of the puncture needle N inserted into the subject S estimated by the insertion length estimating unit 53. , the distance DT between the target site T and the tip of the puncture needle N is calculated. Here, the distance calculation unit 54 calculates, for example, the distance between the target site T and the tip of the puncture needle N in the extending direction of the puncture needle N as the distance DT between the target site T and the tip of the puncture needle N. .
Further, the distance calculation unit 54 transmits the distance DT between the target site T and the tip of the puncture needle N calculated in this way to the main body side wireless communication unit 41 and the display control unit 44, and The information is displayed on the display unit 31 of the mount display 3 and the display monitor 45 of the diagnostic device main body 4A.

At this time, the distance calculation unit 54, for example, as shown in FIG. It is superimposed on the sound wave image U and displayed. The ultrasound image U exemplified in FIG. The length of the portion of the puncture needle N inserted into the subject S, that is, the insertion length V3 is 20 mm, and the distance V4 to the target site is 8 mm. , a puncture needle N inserted into a subject S and a target site T are depicted.

As described above, according to the ultrasonic diagnostic apparatus according to Embodiment 3, the distance DT between the target site T, which is the target of puncture, and the tip of the puncture needle N is calculated by the distance calculation unit 54. Since the measured distance DT is displayed on the display unit 31 of the head-mounted display 3 and the display monitor 45 of the diagnostic apparatus main body 4A, the user can grasp the positional relationship of the puncture needle N with respect to the target site T accurately.

Embodiment 4
In Embodiment 3, the distance DT between the target site T and the tip of the puncture needle N is calculated by the distance calculator 54. However, when the calculated distance DT is short, that is, when the puncture needle When the tip of N approaches the target site T, the user can be informed of that.

The ultrasonic diagnostic apparatus according to the third embodiment is obtained by connecting the ultrasonic probe 2 and the head mount display 3 according to the first embodiment to a diagnostic apparatus main body 4C shown in FIG. A diagnostic device main body 4C according to the fourth embodiment has a main body control section 49C instead of the main body control section 49B in the diagnostic device main body 4B according to the third embodiment shown in FIG. be.

In the diagnostic apparatus main body 4C, the distance calculation section 54 is connected to the notification section 55, and the main body side wireless communication section 41 and the display control section 44 are connected to the notification section 55. FIG. Communication control unit 42, ultrasonic image processing unit 43, display control unit 44, puncture needle length calculation unit 46, total length acquisition unit 47, actual length estimation unit 48, storage unit 50, input unit 51, insertion length A body control unit 49C is connected to the estimation unit 53, the distance calculation unit 54, and the notification unit 55. FIG.
Communication control unit 42 , ultrasonic image processing unit 43 , display control unit 44 , puncture needle length calculation unit 46 , total length acquisition unit 47 , actual length estimation unit 48 , main body control unit 49C, insertion length estimation unit 53 , the distance calculation unit 54 and the notification unit 55 constitute a diagnostic apparatus main body processor 52C.

When the value of the distance DT between the target site T and the tip of the puncture needle N, which is calculated by the distance calculation unit 54, is equal to or less than a predetermined value, the notification unit 55 of the diagnostic apparatus main body processor 52C The user is notified to that effect. For example, although not shown, the notification unit 55 superimposes a text, an image, or the like indicating that the tip of the puncture needle N is approaching the target site T on the ultrasound image U, and displays it on the head-mounted display 3. The user can be notified by displaying on the unit 31 and the display monitor 45 of the diagnostic apparatus main body 4C.

As described above, according to the ultrasonic diagnostic apparatus according to the fourth embodiment, when the value of the distance DT between the target site T and the tip of the puncture needle N becomes equal to or less than a predetermined value, the notification unit 55 notifies the user that the tip of the puncture needle N is approaching the target site T, so that the user can grasp the positional relationship of the puncture needle N with respect to the target site T accurately and more clearly. can.

Although not shown, for example, when the ultrasonic probe 2, the head-mounted display 3, and the diagnostic device main body 4C are provided with a sound source such as a speaker, the notification unit 55 emits a notification sound through the sound source. Alternatively, the user can be notified by voice or the like. Further, for example, when the head mounted display 3 is provided with a vibrating section such as a motor for vibrating the head mounted display 3, the notification section 55 causes the vibrating section to vibrate the head mounted display 3 to notify the user of the vibration. can also be notified.

Embodiment 5
In the ultrasonic diagnostic apparatus 1 of Embodiment 1, the ultrasonic probe 2 and the head-mounted display 3 are connected to the diagnostic apparatus main body 4 by wireless communication, and the display monitor 45 and the input unit 51 are connected to the diagnostic apparatus of the diagnostic apparatus main body 4. Although it has a configuration that is directly connected to the main body processor 52, for example, the ultrasonic probe 2, the head mounted display 3, the display monitor 45, the input unit 51, and the diagnostic apparatus main body processor 52 are connected via a network. It can also be connected indirectly.

As shown in FIG. 20, in the ultrasonic diagnostic apparatus 1D according to the fifth embodiment, the ultrasonic probe 2, the head mounted display 3, the display monitor 45, and the input unit 51 are connected to the diagnostic apparatus main body 4D via the network NW. It is a thing. The diagnostic device main body 4D is the same as the diagnostic device main body 4 according to Embodiment 1 shown in FIG.

Even when the ultrasonic diagnostic apparatus 1D has such a configuration, at least part of the puncture needle N is imaged by the camera section 32 of the head-mounted display 3 as in the ultrasonic diagnostic apparatus 1 of the first embodiment. , the length of the puncture needle N photographed is calculated by the puncture needle length calculator 46, and the actual length of the puncture needle N is calculated by the actual length estimator 48 based on the calculated length of the puncture needle N. N is estimated, and the estimated actual length of the puncture needle N is superimposed on the ultrasonic image U and displayed on the display unit 31 of the head-mounted display 3 and the display monitor 45 of the diagnostic apparatus main body 4. Is displayed. Therefore, according to the ultrasonic diagnostic apparatus 1D, the user can accurately and easily grasp the length of the portion of the puncture needle N inserted into the subject S.

Further, since the ultrasonic probe 2, the head-mounted display 3, the display monitor 45, and the input unit 51 are connected to the diagnostic device main body 4D via the network NW, the diagnostic device main body 4D can be used as a so-called remote server. can. As a result, for example, the user wears the head-mounted display 3 on the head and prepares the ultrasonic probe 2, the display monitor 45, and the input unit 51 at hand, so that the inside of the subject S can be viewed using the ultrasonic image U. Therefore, the convenience of observing the inside of the subject S can be improved.
Further, for example, when a portable thin computer called a tablet is used as the display monitor 45 and the input unit 51, the user can observe the inside of the subject S more easily.

Although it has been described that the aspect of the fifth embodiment is applied to the first embodiment, it can be similarly applied to the second to fourth embodiments.

1, 1A, 1B, 1C, 1D ultrasonic diagnostic device, 2 ultrasonic probe, 3 head mounted display, 4, 4A, 4B, 4C, 4D diagnostic device body, 11 transducer array, 12 transmitter, 13 receiver, 14 signal processing unit, 15 probe-side wireless communication unit, 16, 34, 42 communication control unit, 17 probe control unit, 18 probe processor, 19, 38 battery, 20 amplification unit, 21 AD conversion unit, 22 beam former, 31, 31A, 31B display section, 32 camera section, 33 head mounted display side wireless communication section, 35, 44 display control section, 36 head mounted display control section, 37 head mounted display processor, 41 main body side wireless communication section, 43 ultrasonic image processing unit, 45 display monitor, 46 puncture needle length calculation unit, 47 total length acquisition unit, 48 actual length estimation unit, 49, 49A, 49B, 49C main body control unit, 50 storage unit, 51 input unit, 52, 52A, 52B, 52C diagnostic apparatus main body processor, 53 insertion length estimating unit, 54 distance calculating unit, 55 reporting unit, A: vine, B: bridge, BC barcode, D: storage unit, DT, V4: distance, E: mounting unit, F Imaging lens, G groove, LA total length, LB, LC length, LG interval, P packaging bag, N puncture needle, NW network, S subject, SC scale, T target site, U ultrasound image, V1 puncture needle length , V2 estimated length, V3 inserted length.

Claims (19)

An ultrasonic diagnostic apparatus that visualizes a puncture needle inserted into a subject in an ultrasonic image,
a head-mounted display that is mounted on the user's head and has a camera unit that acquires a field-of-view image that captures a field of view in front of the user;
A puncture needle that recognizes the puncture needle by image analysis of the visual field image when at least part of the puncture needle is photographed by the camera unit, and calculates the length of the puncture needle in the visual field image. An ultrasonic diagnostic apparatus comprising: a length calculator; An ultrasonic diagnostic apparatus that visualizes a puncture needle inserted into a subject in an ultrasonic image,
a head-mounted display that is mounted on the user's head and has a camera unit that acquires a field-of-view image that captures a field of view in front of the user;
A puncture needle that recognizes the puncture needle by image analysis of the visual field image when at least part of the puncture needle is photographed by the camera unit, and calculates the length of the puncture needle in the visual field image. a length calculator;
a total length acquisition unit that acquires the actual total length of the puncture needle that is not inserted into the subject;
The length of the puncture needle in the visual field image calculated by the puncture needle length calculation unit from the visual field image in which the entire puncture needle that is not inserted into the subject is captured, and the total length acquisition unit acquires the length of the puncture needle. obtained from the length of the puncture needle in the visual field image calculated by the puncture needle length calculator based on the obtained correspondence relationship, and an actual length estimating unit that estimates the actual length of the puncture needle captured in the visual field image. When the camera unit captures an image of the entire puncture needle in which a plurality of grooves arranged at predetermined intervals are formed and the puncture needle is not inserted into the subject, 3. The ultrasonic diagnostic apparatus according to claim 2, wherein a plurality of grooves are used to obtain the actual total length of the puncture needle. The total length acquisition unit obtains the length information recorded in the length information recording member when the length information recording member in which the length information representing the actual total length of the puncture needle is recorded is photographed by the camera unit. 3. The ultrasonic diagnostic apparatus according to claim 2, wherein the actual total length of the puncture needle is obtained by reading length information. When the camera unit images the entire puncture needle that is not inserted into the subject and a scale for measuring the length in parallel, the total length acquisition unit captures the scale. 3. The ultrasonic diagnostic apparatus according to claim 2, wherein the actual length of the puncture needle is obtained by using. An ultrasonic diagnostic apparatus that visualizes a puncture needle inserted into a subject in an ultrasonic image,
a head-mounted display that is mounted on the user's head and has a camera unit that acquires a field-of-view image that captures a field of view in front of the user;
A puncture needle that recognizes the puncture needle by image analysis of the visual field image when at least part of the puncture needle is photographed by the camera unit, and calculates the length of the puncture needle in the visual field image. a length calculator;
When the entirety of the puncture needle that is not inserted into the subject and is formed with a plurality of grooves arranged at predetermined intervals is captured by the camera unit, using the plurality of grooves, An ultrasonic diagnostic apparatus comprising: a total length acquisition unit that acquires the actual total length of the puncture needle that is not inserted into the subject. An ultrasonic diagnostic apparatus that visualizes a puncture needle inserted into a subject in an ultrasonic image,
a head-mounted display that is mounted on the user's head and has a camera unit that acquires a field-of-view image that captures a field of view in front of the user;
A puncture needle that recognizes the puncture needle by image analysis of the visual field image when at least part of the puncture needle is photographed by the camera unit, and calculates the length of the puncture needle in the visual field image. a length calculator;
reading the length information recorded in the length information recording member when the length information recording member in which the length information representing the actual total length of the puncture needle is recorded is photographed by the camera unit; and a total length acquisition unit that acquires the actual total length of the puncture needle that is not inserted into the subject. The ultrasonic diagnostic apparatus according to claim 2 , wherein the actual length estimating unit displays the estimated actual length of the puncture needle by superimposing it on the ultrasonic image. When the camera unit captures an image of the puncture needle partially inserted into the subject, the actual total length of the puncture needle acquired by the total length acquiring unit and the actual length estimating unit an insertion length for estimating the actual length of the portion of the puncture needle inserted into the subject based on the actual length of the uninserted portion of the puncture needle appearing in the visual field image estimated by The ultrasonic diagnostic apparatus according to claim 2 or 8, further comprising an estimator. The ultrasound according to claim 9, wherein the insertion length estimating unit superimposes and displays the estimated actual length of the portion of the puncture needle inserted into the subject on the ultrasound image. diagnostic equipment. When the target site for puncture and the puncture needle are depicted in the ultrasonic image, the target site and the puncture needle are recognized by image analysis of the ultrasonic image, and the target site and the insertion length are determined. a distance calculation unit for calculating the distance between the target site and the tip of the puncture needle based on the actual length of the portion of the puncture needle inserted into the subject estimated by the estimation unit; The ultrasonic diagnostic apparatus according to claim 9 or 10, further comprising: 12. The ultrasonic diagnostic apparatus according to claim 11, wherein the distance calculation unit superimposes and displays the calculated value of the distance between the target site and the tip of the puncture needle on the ultrasonic image. 13. The ultrasonic diagnostic apparatus according to claim 11, further comprising a notification unit that notifies a user when the distance calculated by said distance calculation unit is equal to or less than a predetermined value. The ultrasonic diagnostic apparatus according to any one of claims 1 to 13, wherein the head mounted display has a display section for displaying the ultrasonic image. The ultrasonic diagnostic apparatus according to any one of claims 1 to 1113, further comprising a display monitor for displaying said ultrasonic image. A control method for an ultrasonic diagnostic apparatus that includes a head-mounted display that is worn on the head of a user and has a camera section, and a puncture needle length calculation section, and visualizes a puncture needle inserted into a subject on an ultrasonic image. and
The head-mounted display acquires a field-of-view image of a field of view in front of the user,
When at least part of the puncture needle is photographed by the camera unit, the puncture needle length calculation unit analyzes the visual field image to recognize the puncture needle, and the puncture needle in the visual field image is recognized. A control method for an ultrasonic diagnostic apparatus that calculates the length of a needle. A head-mounted display mounted on the user's head and having a camera unit, a puncture needle length calculation unit, a total length acquisition unit, and an actual length estimation unit. A control method for an ultrasonic diagnostic apparatus that renders a sound image, comprising:
The head-mounted display acquires a field-of-view image of a field of view in front of the user,
When at least part of the puncture needle is photographed by the camera unit, the puncture needle length calculation unit analyzes the visual field image to recognize the puncture needle, and the puncture needle in the visual field image is recognized. Calculate the length of the needle,
the total length acquisition unit acquires the actual total length of the puncture needle that is not inserted into the subject;
The actual length estimating unit calculates the length of the puncture needle in the visual field image from the visual field image in which the entire puncture needle that is not inserted into the subject is captured, and the acquired puncture needle. and the actual length of the puncture needle photographed in the visual field image is obtained from the length of the puncture needle in the visual field image based on the obtained correspondence relationship. A method of controlling an ultrasonic diagnostic apparatus for estimating Ultrasound diagnosis that includes a head-mounted display that is worn on the head of a user and has a camera unit, a puncture needle length calculation unit, and a total length acquisition unit, and that visualizes the puncture needle inserted into the subject on an ultrasound image. A device control method comprising:
The head-mounted display acquires a field-of-view image of a field of view in front of the user,
When at least part of the puncture needle is photographed by the camera unit, the puncture needle length calculation unit analyzes the visual field image to recognize the puncture needle, and the puncture needle in the visual field image is recognized. Calculate the length of the needle,
When the entire puncture needle, in which a plurality of grooves are formed at predetermined intervals and are not inserted into the subject, is photographed by the camera unit, the full length acquiring unit may A control method for an ultrasonic diagnostic apparatus that acquires the actual total length of the puncture needle that is not inserted into the subject using a plurality of grooves. Ultrasound diagnosis that includes a head-mounted display that is worn on the head of a user and has a camera unit, a puncture needle length calculation unit, and a total length acquisition unit, and that visualizes the puncture needle inserted into the subject on an ultrasound image. A device control method comprising:
The head-mounted display acquires a field-of-view image of a field of view in front of the user,
When at least part of the puncture needle is photographed by the camera unit, the puncture needle length calculation unit analyzes the visual field image to recognize the puncture needle, and the puncture needle in the visual field image is recognized. Calculate the length of the needle,
When the length information recording member in which the length information representing the actual total length of the puncture needle is recorded is photographed by the camera unit, the total length acquisition unit detects the length information recorded in the length information recording member. A control method for an ultrasonic diagnostic apparatus for acquiring the actual total length of the puncture needle that is not inserted into the subject by reading length information.