JP2013208413A - Image processing apparatus and image processing method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To enable intuitive checking of a diagnostic position.SOLUTION: A plurality of simple display images are displayed in a three-dimensionally distorted state in such a manner as to surround a circle indicating an diagnostic object. A position of the simple display image is determined on the basis of angle information detected from an angle sensor of an ultrasonic probe. In other words, the plurality of simple display images are displayed by being generated (in an interlocking position) in such a manner as to interlock with a rotational operation (an angle obtained from the angle sensor) of the ultrasonic probe. The present disclosure can be applied, for example, to an image processing system which generates an ultrasonic image from a signal from the probe taking the ultrasonic image and displays it.

Description

  The present disclosure relates to an image processing apparatus and method, and more particularly, to an image processing apparatus and method capable of intuitively confirming a diagnosis position.

  In an ultrasonic device that takes an ultrasonic image, it is possible to display an image of a diagnostic (observation) object and an ultrasonic image side by side, and roughly display where the ultrasonic image corresponds to the diagnostic object. Existed for a long time.

  For example, Patent Document 1 describes that the position or movement of an ultrasonic probe is detected, and the locus of the ultrasonic probe at the examination site of the subject is expressed based on the detected position or movement of the ultrasonic probe. Yes.

JP 2008-86742 A

  However, there is no ultrasonic device that actually displays the positional relationship with the diagnostic object while accurately taking the positional relationship with the diagnostic object in conjunction with the operation.

  This indication was made in view of such a situation, and can confirm a diagnostic position intuitively.

  An image processing apparatus according to an aspect of the present disclosure includes an image generation unit that generates a plurality of simple display images corresponding to a plurality of ultrasonic images input from a probe, and a plurality of simple display images generated by the image generation unit. Is arranged at a position interlocked with the rotation operation of the probe and is displayed on a display screen.

  The display control unit can arrange and display a plurality of simple display images generated by the image generation unit at a position surrounding a diagnosis target of ultrasonic diagnosis.

  The display control unit may display a circumference in which a plurality of simple display images generated by the image generation unit surround a diagnosis target of the ultrasonic diagnosis in conjunction with a size of the diagnosis target of the ultrasonic diagnosis. it can.

  The display control unit superimposes and displays an image to be diagnosed in the ultrasonic diagnosis and a plurality of simple display images arranged at positions interlocked with the rotation operation of the probe.

  The display control unit can arrange and display a plurality of simple display images generated by the image generation unit at positions distorted three-dimensionally.

  The image generation unit can generate a plurality of simple display images based on a rotation angle or a rotation angular velocity from a sensor included in the probe.

  The display control unit can widely display the arrangement intervals of the plurality of simple display images when the rotation angle or the rotation angular velocity from the sensor included in the probe is large.

  The display control unit can display the arrangement intervals of the plurality of simple display images narrowly when the rotation angle or the rotation angular velocity from the sensor included in the probe is small.

  The probe further includes a support portion provided at an angle orthogonal to a beam direction, a rotation mechanism provided between the probe and the support portion, and a guide for supporting the rotation operation of the probe by the rotation mechanism. Can be provided.

  In the image processing method according to one aspect of the present disclosure, an image processing apparatus generates a plurality of simple display images corresponding to a plurality of ultrasonic images input from a probe, and the generated plurality of simple display images are used as the probe. Place it at a position linked to the rotation operation of and display it on the display screen.

  In one aspect of the present disclosure, a plurality of simplified display images corresponding to a plurality of ultrasonic images input from the probe are generated. Then, the generated plurality of simple display images are arranged at positions linked with the rotation operation of the probe and displayed on the display screen.

  According to the present disclosure, the diagnostic position can be intuitively confirmed.

It is a figure which shows the structural example of the external appearance of the ultrasonic probe to which this technique is applied. It is a figure explaining operation | movement of an ultrasonic probe. It is a figure explaining operation | movement of an ultrasonic probe. It is a figure explaining operation | movement of an ultrasonic probe. It is a figure explaining operation | movement of an ultrasonic probe. It is a figure which shows the structural example of the other external appearance of the ultrasonic probe to which this technique is applied. It is a figure which shows the structural example of the further another external appearance of the ultrasonic probe to which this technique is applied. It is a figure which shows the structural example of the other external appearance of the ultrasonic probe to which this technique is applied. It is a figure which shows the structural example of the further another external appearance of the ultrasonic probe to which this technique is applied. It is a figure explaining the rotation direction of a ball joint. It is a figure which shows the usefulness of the jig | tool in a probe. It is a block diagram showing an example of composition of an image processing system to which this art is applied. It is a flowchart explaining the imaging | photography process of an image processing system. It is a flowchart explaining the example of the simple display image group production | generation process of an image processing system. It is a flowchart explaining the other example of the simple display image group production | generation process of an image processing system. It is a figure which shows the example of a simple display image group and its virtual space arrangement | positioning. It is a figure which shows the example of a simple display image group. It is a figure which shows the example of a simple display image group. It is a figure which shows the example of a simple display image group. It is a figure which shows the example of virtual space arrangement | positioning. It is a figure which shows the other structural example of the ultrasonic probe to which this technique is applied. It is a figure explaining the image surface of an array vibrator. It is a figure explaining the acoustic lens in an ultrasonic probe. It is a figure explaining the influence to the x-axis direction of an acoustic lens. It is a figure explaining the influence to the z-axis direction of an acoustic lens. It is a figure explaining calculation of the amount of movement of the probe in an image processing system. It is a figure explaining application to the two-dimensional array probe of this art. FIG. 26 is a block diagram illustrating another configuration example of an image processing system to which the present technology is applied. It is a block diagram which shows the structural example of the probe unit in the case of performing a receiving side process. It is a block diagram which shows the structural example of the probe unit in the case of performing a transmission side process. It is a flowchart explaining the example of the ultrasonic reception process of a probe unit. It is a flowchart explaining the example of the reception display process of a reception display apparatus. It is a flowchart explaining the example of the ultrasonic transmission process of a probe unit. It is a flowchart explaining the example of the pre-process of imaging | photography of an image processing system. It is a flowchart explaining the example of the imaging | photography process of an image processing system. It is a block diagram which shows the structural example of a computer.

Hereinafter, modes for carrying out the present disclosure (hereinafter referred to as embodiments) will be described. The description will be given in the following order.
1. First embodiment (ultrasonic probe)
2. Second embodiment (image processing system)
3. Third embodiment (user interface)
4). Fourth embodiment (other configuration of ultrasonic probe)
5. Fifth embodiment (other configuration of image processing system)
6). Sixth embodiment (computer)

<First Embodiment>
[External configuration example of ultrasonic probe]
FIG. 1 is a diagram illustrating an external configuration example of an ultrasonic probe to which the present technology is applied. 1A is a side view of the ultrasonic probe, FIG. 1B is a top view of the ultrasonic probe when the ultrasonic probe shown in FIG. 1A is viewed from above, and FIG. FIG. 2 is a front view of the ultrasonic probe as viewed from the left side of the ultrasonic probe shown in FIG.

  In the example of FIG. 1, the ultrasonic probe 11 is configured to include a probe (main body) 21, a base 22, a rotating shaft 23, a handle 24, a guide 25, and a joint portion 26. The ultrasonic probe 11 in the example of FIG. 1 has a structure convenient for turning around a relatively thin object such as a finger to be diagnosed.

  The probe 21 is a sector probe, for example, but may be a probe having another structure. The probe 21 has a contact portion 21a formed of a flexible member that comes into contact with the diagnostic object on the sensor surface. For example, the probe 21 is configured by arranging vibrators in the vertical direction of the contact portion 21a in FIG.

  Under the probe 21, a pedestal 22 having a built-in angle sensor (an angle sensor 241 in FIG. 12 described later), a rotating shaft 23 that is a rotating mechanism attached to the angle sensor, and a handle 24 are provided. . The rotating shaft 23 and the probe 21 are fixed by, for example, the joint portion 26, but this fixing method is not limited.

  Although it is possible to attach a ground axis sensor or an acceleration sensor instead of the angle sensor, by attaching this angle sensor, the angle information of the rotation of the probe 21 can be obtained more accurately than the ground axis sensor or the acceleration sensor. .

  For example, when the ultrasonic probe 11 rotates around the finger, the rotation probe 23 is rotated around the rotation axis 23 (axis β orthogonal to the beam direction) in FIG. It is provided between the probe 21 and the handle 24 so as to rotate in the left-right direction of the contact portion 21a. A handle is a support part for a to-be-photographed person to operate. The rotation shaft 23 and the handle 24 are provided at an angle orthogonal to the beam direction indicated by the tip of the center axis α of the sensor surface of the probe 21. Thereby, the probe 21 can be stably rotated around the diagnostic object without intervention of a human hand.

  In the present specification, the definition of the orthogonal angle includes not only strict 90 degrees but also an angle having this operation function. However, realistically, it is preferable to carry out at 90 degrees.

  In addition, on the side surface of the probe 21 (on the left side of the contact portion 21a in FIG. 1C), the ultrasonic beam of the probe 21 is always sent from the vertical direction to the diagnostic object when the ultrasonic probe 11 rotates. A guide 25 for supporting the rotation operation is provided. The guide 25 is composed of a panel or the like, and is provided in the probe 21 and the joint portion 26 so that the guide 25 and the same plane γ as the sensor surface of the probe 21 form a right angle. In the present specification, the definition of the right angle does not need to be 90 degrees unless the center axis is shifted and the function hits firmly and does not impair the operational feeling.

  For example, a diagnostic object such as a finger indicated by a dotted line is applied to the contact portion 21a and the guide 25 on the sensor surface and rotated in the direction indicated by the white arrow, that is, in the direction opposite to the sensor surface. By doing so, the contact portion 21a on the sensor surface and the guide 25 fix the diagnostic object, so that the ultrasonic beam always hits the diagnostic object almost vertically, as shown by the dotted line. The circumference of the diagnostic object can be stably rotated.

  The guide 25 can be rotated in the direction opposite to the direction indicated by the white arrow. However, since the diagnostic object may float with respect to the contact portion 21a and the guide 25 on the sensor surface, the white arrow It is preferable to turn in the direction indicated by.

  In the example of FIG. 1, the sensor surface (same plane γ) is shown as not contacting the diagnostic object, but actually, the contact portion 21 a and the diagnostic object are elastic, Furthermore, since the gel is applied and imaging is performed at the time of diagnosis, the sensor surface and the diagnostic object are in close contact with each other. This also applies to the subsequent drawings.

[Operation of ultrasonic probe]
Next, the rotation operation of the ultrasonic probe 11 will be described with reference to FIGS. In the examples of FIGS. 2 to 5, the hand of the subject having the handle 24 is omitted.

  Hereinafter, although described as a rotation operation, photographing is also performed while rotating. The rotation position of the ultrasonic probe 11 is set in advance to a predetermined position of the diagnostic object (for example, the back of the finger shown in FIG. 2).

  As shown in FIG. 2, the person to be photographed starts shooting by placing the contact portion 21a on the sensor surface of the probe 21 and the guide 25 on the back of the finger that is the start position, and in the depth direction in the figure, The ultrasonic probe 11 is rotated to make a round around the finger.

  That is, the probe 21 passes through the back of the finger shown in FIG. 2, the left side of the finger shown in FIG. 3, the belly of the finger shown in FIG. 4, and the right side of the finger shown in FIG. The ultrasonic image is taken.

  During the rotation operation around the finger, the contact portion 21a on the sensor surface of the guide 25 and the probe 21 does not leave the finger surface, and the position does not shift. Therefore, the ultrasonic probe 11 can be stably rotated while the ultrasonic beam is transmitted in the vertical direction with respect to the finger.

[Configuration example of other appearance of ultrasonic probe]
FIG. 6 is a diagram illustrating another external configuration example of the ultrasonic probe to which the present technology is applied. 6A is a side view of the ultrasonic probe, FIG. 6B is a top view of the ultrasonic probe when the ultrasonic probe shown in FIG. 6A is viewed from above, and FIG. FIG. 7 is a front view of the ultrasonic probe when the ultrasonic probe shown in FIG. 6A is viewed from the left side.

  In the example of FIG. 6, the ultrasonic probe 51 is common to the ultrasonic probe 11 of FIG. 1 in that it includes a probe (main body) 21, a base 22, a rotating shaft 23, a handle 24, and a guide 25. The ultrasonic probe 51 is different from the ultrasonic probe 11 in FIG. 1 in that the bonding portion 26 is replaced with the bonding portion 61.

  That is, the joint portion 26 shown in FIG. 1 is fixed to the probe 21, but the joint portion 61 shown in FIG. 6 has a gap between the guide 25 and the probe 21. The length is adjustable (variable).

  With this joint portion 61, the guide 25 can be provided on the probe 21 so that the distance between the guide 25 and the center axis α of the sensor surface of the probe 21 is equal to the radius of the diagnostic object. That is, the guide 25 can be provided in the probe 21 so that the center axis α of the sensor surface of the probe 21 is equal to the center position of the diagnostic object.

  Thereby, it is possible to deal with a thick cylindrical diagnostic object as indicated by a dotted line.

  That is, as in the case of FIG. 1, a diagnostic object such as an elbow joint indicated by a dotted line is applied to the contact portion 21a and the guide 25 on the sensor surface, and the direction indicated by the white arrow, that is, the sensor surface Rotate in the opposite direction. By doing in this way, even if it is a thick cylindrical shape, the contact part 21a and the guide 25 on a sensor surface can fix a diagnostic target. Therefore, the circumference of the cylindrical diagnostic object as shown by the dotted line can be stably rotated so that the ultrasonic beam always hits the diagnostic object almost vertically.

[Configuration example of other appearance of ultrasonic probe]
FIG. 7 is a diagram illustrating a configuration example of another appearance of an ultrasonic probe to which the present technology is applied. 7A is a side view of the ultrasonic probe, FIG. 7B is a top view of the ultrasonic probe when the ultrasonic probe shown in FIG. 7A is viewed from above, and FIG. FIG. 8 is a front view of the ultrasonic probe when the ultrasonic probe shown in FIG. 7A is viewed from the left side.

  In the example of FIG. 7, the ultrasonic probe 81 is common to the ultrasonic probe 11 of FIG. 1 in that it includes a probe (main body) 21, a base 22, a rotating shaft 23, a handle 24, and a joint portion 26. . The ultrasonic probe 81 is different from the ultrasonic probe 11 of FIG. 1 in that the guide 25 is replaced with the guide 91.

  That is, the guide 91 is provided on the same direction as the sensor surface of the probe 21 on the moving direction (rotation direction) side of the probe 21 at an angle orthogonal to the beam direction pointed to by the center axis α of the sensor surface of the probe 21. Although the guide may be one surface in the traveling direction, as shown in FIG. 7B, the structure extends not only in the traveling direction (upward direction in the figure) but also in the reverse direction (downward direction in the figure). However, it is easier to stabilize the rotation. In the present specification, the definition of the orthogonal angle includes not only a strict 90 degrees, but also an angle that allows the ultrasonic beam to hit the diagnostic object almost vertically. However, 90 degrees is preferable in practice.

  In the example of FIG. 7, the guide 91 has the same length in the traveling direction and the reverse direction, but the guide in the traveling direction may be longer than the guide in the reverse direction.

  Here, in the configuration of the ultrasonic probe 11 in FIG. 1 and the ultrasonic probe 51 in FIG. 6 described above, the effect is high when the cross section of the diagnostic object is close to a perfect circle. Thus, it is difficult to apply when the cross section of the diagnostic object is elliptical. Therefore, when the cross section of the diagnostic object is elliptical, the configuration of the ultrasonic probe 81 in FIG. 7 is suitable.

  In the case of the example in FIG. 7, a diagnostic object such as an elbow joint indicated by a dotted line is applied to the guide 91 (including the contact portion 21a on the sensor surface) and rotated in the direction indicated by the white arrow. By doing so, the guide 91 fixes the object to be diagnosed even in the case of an elliptical cylindrical shape, so that an ellipse as shown by the dotted line is always shown so that the ultrasonic beam strikes the object to be diagnosed almost vertically. The periphery of the cylindrical diagnostic object can be rotated stably.

  In the example of FIG. 7, the guide 91 (sensor surface) and the diagnostic object are shown not in contact with each other, but actually, the contact portion 21 a and the diagnostic object are elastic, Since the gel is applied and imaging is performed at the time of diagnosis, the guide 91 and the diagnostic object are in close contact with each other.

[Configuration example of other appearance of ultrasonic probe]
FIG. 8 is a diagram illustrating a configuration example of another external appearance of an ultrasonic probe to which the present technology is applied. 8A is a side view of the ultrasonic probe, FIG. 8B is a top view of the ultrasonic probe when the ultrasonic probe shown in FIG. 8A is viewed from above, and FIG. FIG. 9 is a front view of the ultrasonic probe as seen from the left side of the ultrasonic probe shown in FIG.

  In the example of FIG. 8, the ultrasonic probe 111 is common to the ultrasonic probe 81 of FIG. 7 in that it includes a probe (main body) 21, a base 22, a rotating shaft 23, a handle 24, a joint portion 26, and a guide 91. doing. The ultrasonic probe 111 is different from the ultrasonic probe 81 of FIG. 7 in that a movement amount sensor 121 is added to the probe 21.

  That is, in the example of FIG. 8, the movement amount sensor 121 is provided below the probe 21 and on the guide 91. The installation position of the movement amount sensor 121 is not limited to this position. The movement amount sensor 121 is composed of, for example, an optical movement amount sensor used for an optical mouse or the like, and detects a movement amount on the body surface such as a side slip other than the rotation of the ultrasonic probe 111.

  In the ultrasonic probe 111, when the object to be diagnosed is an elliptical cylinder or the like, an operation such as skidding is performed in addition to rotation. In the above-described ultrasonic probes 11, 51, and 81, only the angle sensor is provided. However, in order to reconstruct the three-dimensional volume under this condition, there is a restriction that the circumference is moved at a constant speed. Necessary.

  In the case of the ultrasonic probe 111, this restriction can be eliminated by providing the movement amount sensor 121.

  In the example of FIG. 8, the example in which the movement amount sensor 121 is provided in the ultrasonic probe 81 of FIG. 7 has been described. Of course, other than the ultrasonic probe 11 of FIG. 1 or the ultrasonic probe 51 of FIG. It is also possible to provide the ultrasonic probe having the configuration described above.

[Configuration example of other appearance of ultrasonic probe]
FIG. 9 is a diagram illustrating a configuration example of another external appearance of the ultrasonic probe to which the present technology is applied. 9A and 9B are side views below the joint portion 26 of the ultrasonic probe. In the example of FIG. 9, the portions above the joint portion 26 are not shown in order to focus on the handle portion.

  In the example of FIG. 9, the ultrasonic probe 141 is common to the ultrasonic probe 11 of FIG. 1 in that it includes a probe (main body) 21, a base 22, a rotating shaft 23, a guide 25, and a joint portion 26. The ultrasonic probe 141 is different from the ultrasonic probe 11 of FIG. 1 in that the handle 24 is replaced with the handle 151.

  In the example of FIG. 9, the handle 151 is configured to include an auxiliary operation unit 153 having a ball joint 152. As shown in FIG. 9A, the handle 151 is provided substantially perpendicular to the ultrasonic beam direction of the probe 21, similarly to the handle 24 of FIG. 1. Further, as shown in FIG. 9B, the handle 151 can make the auxiliary operation unit 153 have an angle with respect to the ultrasonic beam direction of the probe 21 by the ball joint 152.

  In addition, you may comprise the handle 151 so that the auxiliary | assistant operation part 153 which has the ball joint 152 may be added to the handle 24 of FIG. In this case, the auxiliary operation unit 153 having the ball joint 152 may be detachable.

  Although the operability of the handle 24 of the ultrasonic probe 11 of FIG. 1 is made flexible by the rotating shaft 23, the data acquired by the angle sensor is always the rotation angle of the probe 21. If you do, it tends to be strange. Therefore, by providing the ball joint 152, for example, when measuring parts such as elbows and knees, the procedure of turning the probe 21 on the circumference of those parts can be facilitated.

  In addition, as shown in FIG. 10, in order to obtain the rotation angle of the Y-axis center indicated by the axis β orthogonal to the beam direction with the existing rotation axis 23, it is below the joint portion 26 of the ultrasonic probe 111. The Y-axis rotation is prohibited in the pole joint 152 located at. Therefore, the pole joint 152 has only degrees of freedom for only the X-axis rotation and the Z-axis rotation.

  9 and 10, the ball joint 152 is used. However, any other type may be used as long as the auxiliary operation unit 153 has an angle with respect to the axis β. It is not limited to.

  As described above, the image around the joint can be stably captured by simply attaching at least one of the plurality of jigs as described above to the probe 21.

  FIG. 11 shows the usefulness of the elements (jigs) having the above-described configuration, which are summarized for each region to be diagnosed. Double circles indicate usefulness.

  The rotation angle sensor is useful for fingers, wrists, elbows, shoulders, knees, and torso around the hand. A configuration having a narrow joint guide (eg, guide 25 in FIG. 1) is useful for the fingers of the hand. That is, the narrow joint guide is a guide specialized for fingers of the hand.

  A configuration having a thick joint guide (eg, guide 91 in FIG. 7) is useful for fingers, wrists, elbows, knees, and girth around the hand. The circles shown on the fingers of the hand do not have to be used on the fingers of the hand, but indicate that the fine joint guide is more useful.

  The movement amount sensor is useful for the wrist, elbow, knee, and waistline. Ball joints are useful for wrists, elbows, knees, and waistline. Although it is useful for the wrist, it is not necessary.

  In the diagnosis of rheumatism, toes, shoulders, ankles, and the like are also objects of diagnosis, but in the present technology, these parts are not included.

  As described above, it can be realized only by attaching a jig to an existing probe, for example, as a probe support device.

  In particular, when a handle having a rotation axis with a built-in angle sensor is attached as a jig so as to be orthogonal to the probe, it is possible to detect the exact angle of the probe that goes around the cylindrical subject.

  Further, when a guide is attached as a jig, it is possible to always transmit and receive an ultrasonic beam from the vertical direction to the subject and to easily rotate around the subject. Therefore, an ultrasonic image for acquiring the three-dimensional structure of the target part can be acquired accurately and easily.

  Further, when a ball joint is provided as a jig separately from the rotation shaft, the operational feeling can be improved.

  Next, the configuration of an image processing system as an ultrasonic processing apparatus including the above-described ultrasonic probe will be described below. Although the ultrasonic probe having any configuration described above may be used, the ultrasonic probe 111 in FIG. 8 will be described as an example.

<Second Embodiment>
[Image processing system configuration example]
FIG. 12 is a block diagram illustrating a configuration example of an image processing system 201 to which the present technology is applied.

  The image processing system 201 is a system that generates and displays a cross-sectional image showing at least a part of a cross section of a subject using ultrasonic waves. The image processing system 201 is used, for example, as an ultrasonic diagnostic apparatus when photographing and inspecting a cross-sectional image of each part such as a person's abdomen.

  The image processing system 201 is configured to include the ultrasonic probe 111, the image processing device 212, the recording devices 213a to 213d, and the display 214 shown in FIG.

  The ultrasonic probe 111 is configured to include an ultrasonic transmission / reception unit 221 and a detection unit 222.

  The ultrasonic transmission / reception unit 221 is provided at the tip of the ultrasonic probe 111, for example, and transmits / receives ultrasonic waves under the control of the ultrasonic control unit 251 of the image processing apparatus 212. The ultrasonic transmission / reception unit 221 is configured to include an ultrasonic generator 231 and an ultrasonic receiver 232.

  The ultrasonic generator 231 generates ultrasonic waves under the control of the ultrasonic control unit 251. More specifically, for example, the ultrasonic generator 231 oscillates pulsed ultrasonic waves at a predetermined interval and scans the ultrasonic waves.

  An arbitrary method can be adopted as the ultrasonic scanning method. For example, scanning may be performed in a radial manner or in parallel. When scanning radially, a fan-shaped ultrasonic image can be acquired, and when scanning in parallel, a rectangular ultrasonic image can be acquired.

  The ultrasonic receiver 232 receives the reflected ultrasonic wave generated by the ultrasonic generator 231 under the control of the ultrasonic controller 251. Then, the ultrasonic receiver 232 measures the intensity of the received reflected wave and, for example, data indicating a time-series measurement result of the intensity of the reflected wave (hereinafter referred to as ultrasonic measurement data) The sound wave image generation unit 252 is supplied.

  The detection unit 222 detects the state (for example, an angle or a position) of the ultrasonic probe 111. The detection unit 222 is configured to include the angle sensor 241 and the movement amount sensor 121 of FIG. The angle sensor 241 detects the rotation angle of the ultrasonic probe 111, for example. The movement amount sensor 121 detects the movement amount of the ultrasonic probe 111. As shown in the figure, an angular velocity sensor 242 constituted by a gyro or the like may be included.

  Then, each sensor of the detection unit 222 supplies sensor data indicating the detection result to the sensor information acquisition unit 253 of the image processing device 212.

  The image processing device 212 performs processing for generating an image of a cross section of the subject and displaying it on the display 214. The image processing apparatus 212 includes an ultrasonic control unit 251, an ultrasonic image generation unit 252, a sensor information acquisition unit 253, a probe state detection unit 254, a cross-sectional image generation unit 255, a display control unit 256, and a simple display image generation unit 257. Configured to include.

  The ultrasonic control unit 251 controls the ultrasonic generator 231 and the ultrasonic receiver 232 to control transmission / reception of ultrasonic waves of the ultrasonic probe 111.

  The ultrasonic image generation unit 252 generates an ultrasonic image based on the ultrasonic measurement data supplied from the ultrasonic receiver 232.

  Therefore, the ultrasonic generator 231, the ultrasonic receiver 232, the ultrasonic controller 251, and the ultrasonic image generator 252 generate ultrasonic waves, receive the reflected waves, and based on the received reflected waves. Processing for generating an ultrasound image, that is, imaging of an ultrasound image is performed.

  The ultrasonic image generation unit 252 stores ultrasonic image data indicating the generated ultrasonic image in the recording device 213a.

  The sensor information acquisition unit 253 acquires information indicating a state such as an angle and a position of the ultrasonic probe 111. Specifically, the sensor information acquisition unit 253 samples the detection value of each sensor at a predetermined interval based on sensor data supplied from each sensor of the ultrasonic probe 111. And the sensor information acquisition part 253 preserve | saves in the recording device 213b as sensor information with the time which sampled the detection value of each sensor.

  The probe state detection unit 254 detects the state of the ultrasonic probe 111 that is taking an ultrasonic image based on the sensor information stored in the recording device 213b, and supplies the detection result to the cross-sectional image generation unit 255. The probe state detection unit 254 selects and uses information used for detecting the state of the ultrasonic probe 111 from the sensor information stored in the recording device 213b.

  The cross-sectional image generation unit 255 converts an ultrasonic image into a three-dimensional image in the display area based on the ultrasonic image stored in the recording device 213a and the state of the ultrasonic probe 111 that is capturing the ultrasonic image. By performing placement and volume interpolation, a cross-sectional image (three-dimensional volume data) of the subject is generated. The cross-sectional image generation unit 255 stores cross-sectional image data indicating the generated cross-sectional image in the recording device 213c.

  The display control unit 256 displays a cross-sectional image of the subject on the display 214 based on the cross-sectional image data stored in the recording device 213c. Based on the simple display image group data stored in the recording device 213d, the display control unit 256 arranges a plurality of simple display images (preview images) in a three-dimensional manner and makes a simple display such that the three-dimensional curved images are displayed. The image group is displayed on the display 214.

  Further, when one of a plurality of simple display images in the displayed simple display image group is selected, the display control unit 256 displays an ultrasonic image corresponding to the simple display image and information related to the ultrasonic image, respectively. The data is read from the recording device 213a and the recording device 213b and displayed on the same screen.

  That is, the simple display image group is displayed on the display 214 as an index for viewing the ultrasonic image.

  The simplified display image generation unit 257 uses a plurality of ultrasound image data stored in the recording device 213a in conjunction with the rotation operation of the ultrasound probe 111, and a plurality of simplified displays corresponding to the plurality of ultrasound images. A simple display image group is generated in which images are arranged in a three-dimensional manner. The simple display image generation unit 257 stores the simple display image group data indicating the generated simple display image group in the recording device 213d.

  For example, the simple display image generation unit 257 arranges the ultrasound image data stored in the recording device 213a in the display area in conjunction with the rotation operation of the ultrasound probe 111. At that time, the ultrasonic image data is arranged in the display area so as to be arranged and displayed in three dimensions and to be displayed in a three-dimensional curve.

  Then, the simple display image generation unit 257 generates a plurality of simple display images corresponding to the plurality of ultrasonic images, which are arranged in the display area in conjunction with the rotation operation of the ultrasonic probe 111 as described above. A simple display image group is generated. As a result, the display control unit 256 can display the plurality of simple display images generated by the simple display image generation unit 257 on the display screen by arranging them at positions linked with the rotation operation of the probe.

  In addition, an ultrasonic image (simple display image) is arranged and displayed in a three-dimensional manner in a display region, and is displayed in a three-dimensional curved manner so that the user can recognize the ultrasonic image in a spatial state. In consideration, the display area can also be defined as a space (or a virtual space).

  The recording device 213a is composed of, for example, a cine memory, and stores ultrasonic image data indicating an ultrasonic image generated by the ultrasonic image generation unit 252. The recording device 213b stores the detected values of each sensor as sensor information together with the sampling time.

  The recording device 213c stores cross-sectional image data indicating the cross-sectional image generated by the cross-sectional image generation unit 255. The recording device 213d stores the simplified display image group data indicating the simplified display image group generated by the simplified display image generation unit 257.

  The display 214 displays an image under the control of the display control unit 256.

  12 shows an example in which the recording devices 213a to 213d are configured separately from the image processing device 212. However, the recording devices 213a to 213d are configured in the image processing device 212. May be.

[Shooting process]
Next, imaging processing executed by the image processing system 201 will be described with reference to the flowchart of FIG. Note that this processing is started, for example, when a shooting start command is input via an operation unit (not shown) of the image processing system 201.

  Hereinafter, a case where a cross section of a human joint is photographed using the image processing system 201 will be described as an example. In this case, for example, as shown in FIGS. 2 to 5, the person to be imaged applies the ultrasonic probe 111 substantially perpendicular to the joint in order to photograph a cross section of the joint such as a finger or an elbow. Cycle around the joint.

  The ultrasonic probe 111 can be made to make a round by being applied substantially perpendicularly to the joint by the jig described above.

  A person other than the person to be imaged may operate the ultrasonic probe 111 or may perform a remote operation using a robot arm or the like.

  In step S <b> 11, the ultrasonic generator 231 starts generating ultrasonic waves under the control of the ultrasonic control unit 251. For example, the ultrasonic generator 231 oscillates pulsed ultrasonic waves at a predetermined interval and scans the ultrasonic waves in a predetermined direction.

  In step S <b> 12, the ultrasonic receiver 232 starts receiving the reflected wave of the ultrasonic wave generated by the ultrasonic generator 231 under the control of the ultrasonic controller 251. Then, the ultrasonic receiver 232 measures the intensity of the received reflected wave, and supplies ultrasonic measurement data indicating the measurement result to the ultrasonic image generation unit 252.

  In step S13, the sensor information acquisition unit 253 starts acquisition of sensor information. Specifically, the sensor information acquisition unit 253 samples the detection value of each sensor at a predetermined interval based on sensor data supplied from each sensor of the ultrasonic probe 111. And the sensor information acquisition part 253 preserve | saves in the recording device 213b as sensor information with the time which sampled the detection value of each sensor.

  In step S <b> 14, the ultrasonic image generation unit 252 generates an ultrasonic image based on the ultrasonic measurement data supplied from the ultrasonic reception device 232. That is, the ultrasonic image generation unit 252 generates a two-dimensional ultrasonic image showing an internal cross section near the position where the ultrasonic probe 111 of the joint of the subject is applied. The ultrasonic image generation unit 252 stores ultrasonic image data indicating the generated ultrasonic image in the recording device 213a together with the photographing time.

  In addition, the generation method of an ultrasonic image is not limited to a specific method, Arbitrary methods can be employ | adopted.

  In step S15, the simple display image generation unit 257 executes simple display image group generation processing. The details of the simplified display image group generation processing will be described later with reference to FIG. 14, but the plurality of simplified display images corresponding to the plurality of ultrasonic images are arranged in a three-dimensionally curved manner by the processing in step S15. A simple display image group to be generated is generated.

  In step S16, the simple display image generation unit 257 stores the simple display image group data indicating the generated simple display image group in the recording device 213d.

  In step S <b> 17, the image processing system 201 displays a simple display image group. Specifically, the display control unit 256 reads the simplified display image group data from the recording device 213d. Then, the display control unit 256 causes the display 214 to display a simple display image group based on the read simple display image group data. As a result, a simple display image group, which will be described later with reference to FIG.

  In step S18, the image processing system 201 determines whether or not to continue shooting. If it is determined to continue shooting, the process returns to step S14.

  Thereafter, the processes in steps S14 to S18 are repeatedly executed until it is determined in step S18 that the photographing is not continued. That is, a process of capturing an ultrasonic image, generating a simple display image, and displaying it is continuously performed.

  Note that the interval at which an ultrasonic image is taken and a simple display image or a cross-sectional image is generated is determined in consideration of the processing capacity of the system, the capacity of the recording devices 213a to 213d, and the like. Further, when the image processing system 201 is driven by a battery, the capacity of the battery or the like may be taken into consideration.

  On the other hand, in step S18, for example, when an instruction to end shooting is input via an operation unit (not shown) of the image processing system 201, the image processing system 201 determines that shooting is not continued, and shooting processing is performed. finish.

[Details of Simple Display Image Group Generation Processing]
Next, details of the simplified display image group generation processing in step S15 in FIG. 13 will be described with reference to the flowchart in FIG.

  In step S31, the probe state detection unit 254 detects the state of the ultrasonic probe 111 at the time of imaging based on the sensor information stored in the recording device 213b.

  Specifically, the probe state detection unit 254 obtains the change (trajectory) of the position and orientation (angle) of the ultrasonic probe 111 up to the present based on the detection results of the angle sensor 241 and the movement amount sensor 121.

  As described above, the detection value of each sensor is obtained discretely at a predetermined sampling interval. Therefore, the probe state detection unit 254 obtains changes in the position and angle of the ultrasonic probe 111 by interpolating the detection values of each sensor as necessary.

  Note that the interpolation method used at this time is not limited to a specific method, and for example, linear interpolation, spline interpolation, or the like is performed on the assumption that the motion of the ultrasonic probe 111 during imaging is smooth. .

  The probe state detection unit 254 generates an ultrasonic wave and receives a reflected wave in order to capture the latest ultrasonic image based on the change in the position and angle of the ultrasonic probe 111. 111 position and angle are detected.

  Then, the probe state detection unit 254 supplies a simple display image generation unit 257 with the detection result of the state of the ultrasonic probe 111 at the time of imaging.

  In step S <b> 32, the simple display image generation unit 257 obtains the position and orientation (angle) at which the ultrasonic image is captured. Specifically, the simple display image generation unit 257, based on the position and angle of the ultrasonic probe 111 at the time of shooting the latest ultrasonic image, the position (shooting position) and angle (shooting position) of the latest ultrasonic image. (Shooting angle) is calculated. Note that the imaging start position and angle of the ultrasonic probe 111 are set in advance, and the relationship between the position and angle of the ultrasonic probe 111 and the localization position of the ultrasonic image (imaging position and imaging angle) is known. It shall be.

  In step S33, the simple display image generation unit 257 places the ultrasonic image in the display area. Specifically, for example, the simple display image generation unit 257 reads the latest ultrasonic image from the recording device 213a. Then, the simple display image generation unit 257 updates the latest ultrasonic image in the display area (space) in which the ultrasonic image up to the previous frame is arranged three-dimensionally based on the imaging position and imaging angle of the latest ultrasonic image. Place an ultrasound image. In addition, the relative positional relationship of each ultrasonic image can be calculated | required by the imaging position and imaging angle of each ultrasonic image.

  Moreover, the simple display image generation unit 257 adjusts the position where the ultrasonic image is arranged based on the information of the ultrasonic image.

  For example, the simple display image generation unit 257 detects a feature point of the latest ultrasonic image. Then, the simple display image generation unit 257 adjusts the position where each ultrasonic image is arranged by tracking the trajectory of the feature points of the ultrasonic image so far. This specific example will be described later with reference to FIG. 15, for example.

  For example, in general, a sensor is not good at detecting movement in the translation direction, and tends to have a large detection error. Therefore, when only sensor information is used, the accuracy of ultrasonic image alignment may deteriorate. Therefore, by using not only the sensor information but also the information of the ultrasonic image, the accuracy of the alignment of the ultrasonic image is improved.

  On the other hand, since an ultrasonic image is generally noisy, it is difficult to accurately position an ultrasonic image using only information of the ultrasonic image. Therefore, by using not only the information of the ultrasonic image but also the sensor information, the accuracy of the alignment of the ultrasonic image is improved.

  If the finger joint is photographed, if it is arranged in a three-dimensional manner in the display area and the arrangement is viewed from above, it is a perfect circle (B in FIG. 16 described later). The arrangement can be performed only with the angle information detected by the sensor 241. In contrast, when the wrist, elbow, torso, etc. are photographed, when the three-dimensional arrangement in the display area is viewed from above, the arrangement becomes an elliptical shape (FIG. 20 described later). Information on the amount of movement from the movement amount sensor 121 is also required.

  However, even if it is an ellipse when viewed from the top when it is arranged three-dimensionally in the display area, it is possible to arrange only the angle information detected by the angle sensor 241 as long as constant velocity motion is assumed. Can be done. That is, in this case, it is necessary to rotate the ultrasonic probe 111 around the diagnostic object at substantially constant speed.

  In step S34, the simple display image generation unit 257 generates a simple display image group. That is, the simple display image generation unit 257 generates a simple display image corresponding to the arrangement position from the ultrasonic image arranged three-dimensionally in the display area. Then, the simple display image generation unit 257 is arranged such that a plurality of corresponding simple display images are arranged three-dimensionally at the position where the ultrasonic image is arranged, in other words, three-dimensionally curved. A simple display image group is generated.

  Note that, in the process of step S34, when the ultrasonic probe 111 has not yet made a round around the joint, a simple display image group indicating a progress in the middle of capturing the ultrasonic image may be generated. Alternatively, the simple display image group may be generated after all shooting is completed.

[Another example of simplified display image group generation processing]
Further, a modified example of the simplified display image group generation process of step S15 of FIG. 13 will be specifically described with reference to the flowchart of FIG.

  For example, as shown in FIG. 2 to FIG. 5 described above, the guide 91 of the ultrasonic probe 111 is vertically applied to the joint, and the ultrasonic probe 111 is rotated about the rotation shaft 23 to thereby rotate the joint. A case will be described in which an ultrasonic image is taken by making a round around the periphery. Further, for example, it is assumed that an ultrasonic image is captured so that the imaging ranges of adjacent frames overlap by setting a sufficiently high frame rate or moving the ultrasonic probe 111 slowly.

  In step S51, the probe state detection unit 254 detects the angle of the ultrasonic probe 111 at the time of imaging based on the detection result of the angle sensor 241 stored in the recording device 213b.

  In step S52, the probe state detection unit 254 obtains the amount of change in the angle of the ultrasonic probe 111 with respect to the previous frame based on the detection result of the angle of the ultrasonic probe 111.

  The probe state detection unit 254 supplies information indicating the obtained change amount of the angle of the ultrasonic probe 111 to the simple display image generation unit 257.

  In step S <b> 53, the simple display image generation unit 257 rotates the ultrasonic image of the previous frame based on the change amount of the angle of the ultrasonic probe 111.

  In step S54, the simple display image generation unit 257 detects local feature points of the ultrasonic images of the previous frame and the current frame. More precisely, the simple display image generation unit 257 detects local feature points of the rotated image of the previous frame and the ultrasonic image of the current frame.

  Any type of local feature point and detection method may be employed. For example, a Harris Corner that is resistant to deformation of an object to be imaged and suitable for a flexible human tissue is used as a local feature point.

  In step S55, the simple display image generation unit 257 tracks the movement of the local feature point between the previous frame and the current frame. More precisely, the simple display image generation unit 257 tracks the movement of local feature points between the rotated image of the previous frame and the ultrasonic image of the current frame.

  An arbitrary method can be adopted as a method of tracking the movement of local feature points. For example, the Optical Flow Lucas-Kanade method, which is resistant to deformation of a subject to be imaged and is suitable for a flexible human tissue, is used.

  In step S56, the simple display image generation unit 257 obtains a translation vector between frames based on the tracking result.

  In step S <b> 57, the simple display image generation unit 257 obtains the translation vector of the ultrasonic probe 111. Specifically, the simple display image generation unit 257 obtains the inverse vector obtained by inverting the translation vector T obtained in the process of step S56 as the translation vector of the ultrasonic probe 111. This translation vector represents the movement in the translation direction of the ultrasound probe 111 from the time when the previous ultrasound image is captured until the next ultrasound image UI is captured.

  In step S <b> 58, the simple display image generation unit 257 obtains a drawing position based on the orientation of the ultrasonic probe 111 and the translation vector. Specifically, the simple display image generation unit 257 arranges ultrasonic images up to one frame before based on the angle of the ultrasonic probe 111 detected by the angle sensor 241 and the detected translation vector of the ultrasonic probe 111. The drawing position of the ultrasonic image of the current frame in the three-dimensional virtual space is obtained.

  For example, the simple display image generation unit 257 calculates the relative change amount of the drawing position between the ultrasonic image of the previous frame and the ultrasonic image of the current frame based on the angle of the ultrasonic probe 111 and the translation vector. Ask. Then, the simplified display image generation unit 257 obtains the drawing position of the ultrasonic image of the current frame in the display area based on the obtained change amount.

  In step S59, the simple display image generation unit 257 generates a simple display image group. That is, the simple display image generation unit 257 arranges the simple display image corresponding to the ultrasonic image of the current frame at the obtained drawing position, and combines the simple display image group up to the current frame with the simple display image group. Generate.

  Thereafter, the simplified display image group generation process ends.

  Note that in steps S15 to S17, FIG. 14, and FIG. 15 in FIG. 13 described above, an example in which a simple display image group is generated, stored, and displayed has been described. However, information from the probe state detection unit 254 is a cross section. It is also supplied to the image generation unit 255. Therefore, a cross-sectional image can be generated instead of the simple display image group.

  In the case of a cross-sectional image, the cross-sectional image generation unit 255 can generate a cross-sectional image (3D volume data) by performing volume interpolation using an ultrasonic image arranged three-dimensionally in the display area.

  In the above description, an example in which a simple display image group is generated using an ultrasonic image has been described. However, the present invention is not limited to this. For example, as described above, it is also possible to generate a simple display image group after generating a cross-sectional image (3D volume data) by volume interpolation using ultrasonic images arranged in three dimensions in the display area. It is. Thereby, it becomes possible to browse from an arbitrary cross section.

  Furthermore, both the simple display image group and the cross-sectional image may be generated and displayed side by side on the screen.

  Further, when the obtained sequential ultrasonic image is stored in the recording device 214a, which is a cine memory, the position in the memory corresponds to the angle, so the ultrasonic image in the desired direction is searched while moving in the memory with a slider. be able to. In such a case, for example, it is more preferable that a simple display image group described below is displayed as an index.

  In the above description, an example of generating and displaying a cross-sectional image in real time while capturing an ultrasonic image has been shown. However, after capturing all ultrasonic images first, a cross-sectional image is generated and displayed. You may make it do.

  Further, the imaging of ultrasonic images and the acquisition of sensor information may be synchronized or may not be synchronized. When not synchronizing, it is desirable to record the time when the ultrasonic image was captured and the time when the sensor information was acquired so that the correspondence relationship can be understood later.

  Furthermore, the kind of sensor provided in the ultrasonic probe 111 described above is an example, and it is possible to add another type or use another type as necessary.

  In the present technology, for example, a simple display image group can be generated from an ultrasonic image using only sensor information without using information of the ultrasonic image.

<Third Embodiment>
[Example of simple display images]
FIG. 16 shows an example of a simple display image group displayed in step S17 of FIG. 13 described above. 16A shows the simple display image group 271 displayed on the display 214, and FIG. 16B shows the arrangement image 272 showing the arrangement when the simple display image group 271 is viewed from above, that is, FIG. The figure which looked at what arranged the ultrasonic image used as the basis of the simple display image group of A to the display field from the top is shown. The arrangement image 272 may also be displayed together with the simple display image group 271.

  The simple display image group 271 includes a plurality of simple display images 281 corresponding to a plurality of ultrasonic images. In addition, in the example of FIG. 16, it is comprised by 12 simple display images 281, However, The number of constituents is not limited to 12.

  The plurality of simple display images 281 are displayed in a three-dimensionally distorted manner so as to surround a circle 282 representing the diagnostic object.

  The position of the simple display image 281 is obtained based on angle information detected from the angle sensor 241 of the ultrasonic probe 111. That is, the plurality of simple display images 281 are generated and displayed in conjunction with the rotation operation of the ultrasonic probe 111 (the angle obtained from the angle sensor 241) (in the linked position).

  FIG. 17 is a diagram illustrating the linkage between the rotation operation of the ultrasonic probe 111 and the simple display image group 271.

  In the example of FIG. 17, simple display images 281a to 281l constituting the simple display image group 271 are shown. Here, the arrows P1 to P5 represent the rotation operation of the ultrasonic probe 111, and among the simple display images 281a to 281l, hatched images are generated in conjunction with the rotation operation of the ultrasonic probe 111. Is displayed. In the example of FIG. 17, only hatching is performed, but actually, when the simple display images 281a to 281l are displayed, the generated preview image is displayed.

  First, since the subject rotates the ultrasonic probe 111 to the position indicated by the arrow P1, the simplified display images 281a and 281b are generated and displayed at the linked positions in conjunction with the rotation operation. Next, since the subject rotates the ultrasonic probe 111 to the position indicated by the arrow P2, the simplified display images 281c and 281d are generated and displayed at the linked positions in conjunction with the rotation operation. .

  Further, since the subject rotates the ultrasonic probe 111 to the position indicated by the arrow P3, the simple display images 281e to 281g are generated and displayed at the linked positions in conjunction with the rotation operation. Next, since the subject rotates the ultrasonic probe 111 to the position indicated by the arrow P4, the simplified display images 281h to 281j are generated and displayed at the linked positions in conjunction with the rotation operation. .

  Finally, since the subject rotates the ultrasonic probe 111 to the position indicated by the arrow P5, the simplified display images 281k and 281l are generated and displayed at the linked positions in conjunction with the rotation operation. Is done.

  Thus, the simple display images 281a to 281l constituting the simple display image group 271 are generated and displayed in conjunction with the rotation operation of the ultrasonic probe 111.

  In the example of FIG. 17, an image that is not hatched (that is, an image that has not been generated) is also shown, but this may be hidden or displayed in advance. Good.

  Returning to FIG. 16, the interval of the simple display images 281 indicated by the angle θ between the simple display images 281 may be set in advance, but can also be linked to the rotation operation of the ultrasonic probe 111. . For example, when the angular velocity, which is the amount of change in angle detected from the angle sensor 241 of the ultrasonic probe 111, is small, the interval between the simple display images 281 can be narrowed, and when the angular velocity is large, the interval between the simple display images 281 can be widened. It is. The angular velocity information can be obtained from the angle information detected from the angle sensor 241 of the ultrasonic probe 111 and the time information, but may be obtained from the angular velocity sensor 242.

  As will be described with reference to FIG. 18, for example, in the range indicated by the arrow H, the angular velocity is small, that is, the ultrasonic probe 111 is slowly rotated. In this case, in the range indicated by the arrow H, the generation (display) interval of the simple display images 281 is narrowed as indicated by the angle θ1 between the simple display images 281.

  On the other hand, the range indicated by the arrow L has a high angular velocity, that is, the ultrasonic probe 111 is rapidly rotated. In this case, in the range indicated by the arrow L, the generation (display) interval of the simple display images 281 is widened as indicated by the angle θ2 between the simple display images 281.

  By doing in this way, according to the space | interval of the simple display image 281, it can be known whether the position is a position that is not focused by the subject. In particular, when the interval between the simple display images 281 is narrow (the angular velocity is small), it is understood that the photographed person is paying attention to the portion and is an important position.

  In the example of FIG. 18, the example in which the interval of the simple display image is changed according to the rotation operation (angular velocity) of the ultrasonic probe 111 has been described, but the simple display according to the rotation operation (angular velocity) of the ultrasonic probe 111. It is also possible to change the size of the image.

  For example, the simplified display image 281 in the range where the angular velocity indicated by the arrow H is small may be displayed large, while the simplified display image 281 in the range where the angular velocity indicated by the arrow L is large may be displayed small.

  Further, the size of the circle 282 surrounded by the simple display image 281 is displayed in conjunction with the size of the diagnosis target (that is, the length of the circumference of the joint or the like).

  By displaying as described above, the diagnosis target position can be easily and intuitively confirmed.

  In the examples of FIGS. 16 to 18, only the simple display image group is displayed. However, for example, an image corresponding to the diagnosis target and the simple display image group can be displayed in an overlapping manner. That is, in the example of FIG. 19, an image (for example, a finger image) corresponding to the diagnosis target is displayed at the position of the circle 282 shown in FIG. 16, and the simplified display image group 271 is displayed on the image. Has been. By displaying in this way, the position to be diagnosed can be confirmed more intuitively.

  For example, when one simple display image 281 is selected, an ultrasound image corresponding to the selected simple display image 281 is read from the recording device 213a and displayed on the same screen. In this case, information of the ultrasonic image corresponding to the selected simple display image 281 may be read from the recording device 213b and displayed at the same time.

  By doing in this way, follow-up observation etc. can be performed at the same angle as the angle browsed at the time of the last recording.

  In the example of FIG. 16, as shown in the arrangement image 272 of B of FIG. 16, the diagnosis object has a case where a cross section of a finger joint or the like is almost a perfect circle.

  For example, the cross section of the elbow or wrist becomes an ellipse as shown in the arrangement image 291 in FIG. In such a case, a simple display image group and a cross-sectional image are generated based on not only the angle detected from the angle sensor 241 of the ultrasonic probe but also the movement amount detected from the movement amount sensor 121.

  That is, it is possible to reproduce not only a perfect circle but also an accurate shape (for example, an elliptical shape) conforming to the human body structure.

  In the ultrasonic probe 111 described above, the example in which the simple display image group is generated using the angle sensor 241 and the movement amount sensor 121 has been described. However, although the accuracy is slightly reduced, the angular velocity is used instead of the angle sensor 241. A sensor 242 may be used. Further, by using the ultrasonic probe described below, the movement amount can be obtained without the movement amount sensor 121.

<Fourth embodiment>
[Other configuration examples of ultrasonic probe]
FIG. 21 is a diagram illustrating a configuration example of a probe to which the present technology is applied.

  The ultrasonic probe 301 shown in FIG. 21 is configured to include an A array transducer 311, a B array transducer 312, and a C array transducer 313. In the example of FIG. 21, only the array transducer constituting the ultrasonic probe 301 is shown, but these array transducers are basically similar to the ultrasonic probe 11 of FIG. Provided in the housing.

  The A array transducer 311 is, for example, a one-dimensional array transducer that is basically the same as the transducer included in the probe 21 of FIG. In the B array transducer 312 and the C array transducer 313, the arrangement direction of the transducers of the A array transducer 311 and the arrangement direction of the transducers of the B array transducer 312 and the C array transducer 313 are orthogonal to each other. Are connected to both ends on the short side of the A array transducer 311 (both left and right ends in the figure).

  That is, the transducers of the A array transducer 311 are arranged along the long side 301L of the ultrasonic probe 301 in the same manner as the transducers in the probe 21 of FIG. On the other hand, the transducers of the B array transducer 312 and the C array transducer 313 are arranged along the short side 301 </ b> S of the ultrasonic probe 301.

  As described above, the B array transducer 312 and the C array transducer 313 are provided so as to be oriented in the tangential direction of the rotation of the ultrasonic probe 301, so that the motion detection and the rotation detection described later can be easily performed.

  Here, the length of the long side 301L of the ultrasonic probe 301 is (the length of the long side of each transducer of the B array transducer 312) + (the length in the arrangement direction of the A array transducer 311) + (C The length includes the length of the long side of each transducer of the array transducer 313. The length of the short side 301S of the ultrasonic probe 301 is (the length of the long side of each transducer of the A array transducer 311) and the length in the arrangement direction of the B array transducer 312 and the C array transducer 313. ).

  Further, the length in the arrangement direction of the B array transducer 312 and the C array transducer 313 is shorter than the length in the arrangement direction of the A array transducer 311. The shape of the vibrators constituting each array vibrator is basically the same. That is, the number of transducers (n) arranged in the B array transducer 312 and the C array transducer 313 is smaller than the number of transducers (m) arranged in the A array transducer 311.

  As described above, the B array transducer 312 and the C array transducer 313 differ only in the number of arrangements and the direction in which they are installed on the ultrasonic probe 301, and the other configurations are basically the same as those of the A array transducer 311. It is.

  In the example of FIG. 21, a case where the number of transducers arranged in the B array transducer 312 and the C array transducer 313 is the same n is shown. However, the number of transducers arranged in the B array transducer 312 and the C array transducer 313 may be different as long as it is less than the number of A array transducers 311.

  Further, the physical configuration and characteristics such as the type, physical properties, and sealing material constituting the ultrasonic probe 301 are not limited.

  In the ultrasonic probe 301 configured as described above, images can be reconstructed on three scanning planes as shown in FIG.

[Example of image plane of array transducer]
FIG. 22 is a diagram illustrating an image plane of each array transducer.

  In the example of FIG. 22, the right direction is the positive direction of the x axis, the upward direction is the positive direction of the z axis, and the lower left front direction is the positive direction of the y axis. Then, the x-axis in the direction along the long side 301L of the ultrasonic probe 301 (arrangement direction of the A array transducer 311) and the direction along the short side 301S of the ultrasonic probe 301 (B array transducer 312 and C array). An A plane 321, a B plane 322, and a C plane 323 are shown perpendicular to the zx plane formed by the z axis in the arrangement direction of the vibrators 313.

  That is, the A plane 321 is located at the center of the long side of the transducers arranged in the A array transducer 311 and is a scanning plane parallel to the xy plane and perpendicular to the zx plane. It is a reconstructed image plane.

  The B plane 322 is located at the center of the long side of the transducers arranged in the B array transducer 312 and is a scan plane parallel to the yz plane and reconfigured to a scan plane perpendicular to the zx plane. This is the image plane.

  The C plane 323 is located at the center of the long side of the transducers arranged in the C array transducer 313 and is a scan plane parallel to the yz plane and reconfigured to a scan plane perpendicular to the xz plane. This is the image plane.

  That is, the B plane 322 and the C plane 323 are planes parallel to each other, and are each a plane perpendicular to the A plane 321.

  As described above, in the ultrasonic probe 301, the A array transducer 311 and the B array are arranged such that the B plane 322 and the C plane 323 are parallel to each other and are perpendicular to the A plane 321. A vibrator 312 and a C array vibrator 313 are provided.

  The ultrasonic probe 301 configured to have three scanning planes in this way is also referred to as a three-plane probe hereinafter.

[Acoustic lens in the probe]
FIG. 23 shows the internal structure of the ultrasonic probe 301 on the side contacting the subject of the A array transducer 311. In the example of FIG. 23, the upward direction in the figure is the positive direction of the y axis, and the ultrasonic probe 301 is on the side in contact with the subject. In the drawing, the right direction is the positive direction of the x axis, and the diagonal left direction is the positive direction of the z axis.

  An acoustic matching layer 351 is laminated on the upper side of the A array transducer 311 shown in FIG. 23, that is, the side in contact with the subject, and an acoustic lens 352 is laminated on the acoustic matching layer 351. A packing material 353 is provided below the A array transducer 311. That is, the A array transducer 311 is stacked on the packing material 353.

  The acoustic lens 352 has a lens shape that collects light along the short side 301 </ b> S of the ultrasonic probe 301. Due to the shape, the acoustic lens 352 has a short side 301 </ b> S of the ultrasonic probe 301 in the A array transducer 311. The beam focus in the direction along the z-axis is realized. The acoustic lens has the same lens shape as that of the x-axis with respect to the B array transducer 312 and the C array transducer 313 (dotted lines) provided at the left and right ends of the A array transducer 311 in the ultrasonic probe 301. It is formed to extend in the positive and negative directions.

  For example, at the center of the short side 301S of the ultrasonic probe 301 shown in FIG. 23, the shape of the acoustic lens 352 in a cross section cut from the top to the bottom (by the xy plane) in the drawing is flat as shown in FIG. It is represented by a rectangle.

  For this reason, in the beam forming in the x-axis direction of the A array transducer 311, the combined wavefront 361 A emitted from the A array transducer 311 is changed into the synthesized wavefront 361 B shown in FIG. Is output from. Therefore, in such a case, the effect of the acoustic lens 352 can be ignored.

  On the other hand, for example, at any position on the long side 301L of the ultrasonic probe 301 shown in FIG. 24, the acoustic lens 352 in a cross section cut from the top to the bottom (by the yz plane) in FIG. As shown in FIG. Therefore, in the beam forming in the z-axis direction of the B array transducer 312 and the C array transducer 313, the synthesized wavefront 363A emitted from the B array transducer 312 and the C array transducer 313 is the synthesized wavefront shown in FIG. Like 363B, it is affected by the acoustic lens 352. That is, the combined wavefront 363B changes so that R becomes tight due to the lens effect of the acoustic lens 352, and the focal point 364 is formed in the vicinity of the focal point 362 in the case of the synthetic wavefront 361B in FIG.

  Therefore, when transmitting beams from the B array transducer 312 and the C array transducer 313, it is necessary to calculate a delay amount for beam forming in consideration of the effect of the acoustic lens 352. However, it is only necessary to take this difference into account when calculating the delay amount for beam forming, and it does not lead to an increase in the processing amount of actual delay amount calculation or a reduction in processing speed.

  The ultrasonic probe 301 configured as described above is provided instead of the ultrasonic probe 111 in the image processing system 201 described above with reference to FIG. In this case, for example, an ultrasonic signal from the ultrasonic probe 301 is received by the ultrasonic receiver 232 and supplied not only to the ultrasonic image generation unit 252 but also to the sensor information acquisition unit 253. Then, it is assumed that the sensor information acquisition unit 253 performs the following movement amount calculation processing of the ultrasonic probe 301.

[Example of probe movement amount calculation processing]
In general, when coordinate conversion on a plane of a certain plane is considered, there are degrees of freedom of translation (x direction, y direction, scaling, and rotation (y axis center). Ultrasonic probe that moves on the surface of the human body. When the ground contact surface 301 and the surface of the human body are regarded as planes, there is no need to consider scaling, so in practice, if only translation (x direction, z direction) and rotation (y axis center) are known. Good.

  When calculating the parallel movement parameters, it is sufficient to know at least one movement (Δx, Δz), but when calculating the rotation angle, it is necessary to know at least two movements. As described above, the detection method based on two orthogonal planes described in Patent Document 2 can only determine the amount of movement of one corresponding point.

  On the other hand, in the ultrasonic probe 301, as shown in FIG. 26, the A plane 321, the B plane 322, and the C plane 323 have two intersections (intersection AB and intersection AC) on the body surface. Are arranged as follows.

  FIG. 26 illustrates an arrangement example when the A plane 321, the B plane 322, and the C plane 323 in FIG. 22 are viewed from the y-axis direction, for example. In the example of FIG. 26, the B plane 322 and the C plane 323 are orthogonal to the A plane 321, the intersection AB of the A plane 321 and the B plane 322, and the A plane 321 and the C plane 323 on the zx plane. Are arranged so that an intersection AC of

  Therefore, the sensor information acquisition unit 253 that has received the ultrasonic signal from the ultrasonic probe 301 can calculate the amount of movement of the intersection point AB and the intersection point AC on the zx plane, thereby calculating the rotation angle about the y-axis center. Can be calculated.

  In the example of FIG. 26, an example in which the A plane 321, the B plane 322, and the C plane 323 are orthogonal to each other is shown, but orthogonality is not essential, and the A plane 321 and B It suffices if the plane 322 and the C plane 323 intersect (not even in parallel). Moreover, although the B plane 322 and the C plane 323 are parallel, they may not be parallel.

  The sensor information acquisition unit 253 estimates the amount of movement of the ultrasonic probe 301 using an image (also referred to as a B-mode image) reconstructed on each scanning plane. The method for estimating the amount of movement of the ultrasonic probe 301 is basically the same as the image motion detection method. That is, at a certain time t and at the next frame t + Δt, between the reconstructed images, the intersection AB and the intersection AC on the image plane of the entire image plane using a technique such as feature point matching or block matching. Is calculated.

  The ultrasonic image includes physical features of the ultrasonic probe 301 (vibrator pitch, aperture diameter, etc.), ultrasonic physical features (frequency, speed of sound, etc.), and signal processing after reception (AD conversion frequency, etc.). Defined by Therefore, the moving amount (number of pixels) on the image can be easily converted into the actual moving amount in the body (distance unit such as mm).

  The reconstructed image is the xy plane in the case of the A plane 321, and the yz plane in the case of the B plane 322 and the C plane 323. Of the obtained movement amounts, the movement amount in the y direction is the subsequent coordinate conversion. It is not used in parameter calculation. That is, (xt, zbt) and (xt + Δt, zbt + Δt) are obtained for the intersection point AB shown in FIG. 26, and (xt, zct) and (xt + Δt, zct + Δt) are obtained for the intersection point AC.

By applying these relationships to the Helmat transform equation and developing the relationship, the movement amount (x0, z0) and the rotation angle θ of the ultrasonic probe 301 can be obtained. The Helmat transform equation is expressed by the following equation (1).

x '= x cosθ z sinθ + x0
z '= x sinθ + z cosθ + z0
... (1)

  It should be noted that the movement amount calculation method described above can also be applied to the case where a two-dimensional array probe comprising two-dimensionally arranged transducers as shown in FIG. 27 is used. Each lattice shown in FIG. 27 represents a vibrator.

  When applied to a two-dimensional array probe, a method of arranging the A plane 321, the B plane 322, and the C plane 323 as in the ultrasonic probe 301 having three scanning planes of the present disclosure may be used. A D plane 371 indicated by a dotted line may be added between the plane 322 and the C plane 323.

  The B plane 322, the C plane 323, and the D plane 371 are preferably orthogonal to the A plane 321 and the xz plane, respectively. However, if the plane A is not parallel to the A plane 321, the above-described method of calculating the movement amount is used. Can be applied. In the example of FIG. 27, the positional relationship among the B plane 322, the C plane 323, and the D plane 371 is an example, and need not be as shown in FIG. For example, it is desirable that the B plane 322 and the C plane 323 are at both ends of the detection range, but this is not essential.

  As described above, the movement (movement parameter) of the ultrasonic probe 301 can be calculated by the signal processing method for the ultrasonic probe 301 by the ultrasonic probe 301 and the sensor information acquisition unit 253. Therefore, in this case, it is not necessary to provide the movement amount sensor 121 of FIG.

  In the above description, the method of obtaining the movement amount by image matching after reconstructing the image has been described. However, the movement amount is obtained by signal processing of the RF signal at the stage of the RF signal before reconstructing the image. Based on this, it is possible to calculate the amount of movement of the ultrasonic probe 301 (in this case, the amount of phase change).

  By configuring the ultrasonic probe 301 as described above, the movement amount can be calculated, so that the movement of the ultrasonic probe 301 is detected with high accuracy. Thereby, the accuracy of applications such as position presentation and panorama can be improved.

  That is, one of the main purposes for accurately grasping the probe position information is panoramaization (wide viewing angle) by image switching and volume data conversion.

  In the conventional method using a one-dimensional probe, switching with respect to movement in the long axis direction (x direction) can be achieved with accuracy, but extension in the short axis direction (z direction) is difficult. In addition, a technique of tilting the probe ground plane for volume data creation about the axis has been put into practical use, but the angle at this time is fixed (instructions are given to how much to shake in a certain number of seconds) A special system with an angle sensor was used.

  With the method using the angle sensor, the volume can be reproduced with a certain degree of accuracy. However, since the contact surface of the probe does not move, a volume near the epidermis could not be created.

  Therefore, by using the ultrasonic probe 301, the movement of the probe can be detected with high accuracy, so that panorama conversion (wide viewing angle) and volume data conversion by image switching can be performed more accurately.

  Further, the ultrasonic probe of the present technology can be applied to the following image processing system. Although the ultrasonic probe having any configuration described above may be used, the ultrasonic probe 111 in FIG. 8 will be described as an example.

<Fifth embodiment>
[Image processing system configuration example]
FIG. 28 is a block diagram illustrating a configuration example of an image processing system 401 to which the present technology is applied.

  An image processing system 401 shown in FIG. 28 is an apparatus that captures and displays an image inside an object (that is, an ultrasonic image) using ultrasonic waves. The image processing system 401 is used, for example, for medical purposes for photographing the inside of a patient's body or a fetus, or for industrial purposes, for photographing a cross section of a product.

  The image processing system 401 is configured to include a probe unit 411 and a reception display device 412. For example, the probe unit 411 and the reception display device 412 transmit and receive data by wireless communication. Note that the wireless method is not particularly limited as long as a sufficient bandwidth for transmitting and receiving data can be secured. The communication method is not limited to wireless, but may be wired.

  The probe unit 411 includes, for example, the ultrasonic probe 111 and the signal processing block 422 in FIG. The ultrasonic probe 111 is a part that is pressed against the skin of a subject, and includes a plurality of transducers 421 called ultrasonic transducers therein. The ultrasonic probe 111 is composed of, for example, a vibrator 421 of 64ch or 128ch. The number of transducers 421 included in the ultrasonic probe 111 is not limited.

  The transducer 421 transmits an ultrasonic beam (hereinafter also referred to as a transmission wave) to the subject based on the signal from the signal processing block 422. The vibrator 421 receives a reflected wave from the subject (hereinafter also referred to as a received wave) and supplies the received signal to the signal processing block 422.

  The signal processing block 422 is a block that processes a signal from the vibrator 421 or a signal to the vibrator 421. The signal processing block 422 is configured to include a conversion unit 431, a front-end signal processing unit 432, and a wireless IF (interface) 433.

  The conversion unit 431 includes an AD (Analog / Digital) conversion unit 462 shown in FIG. 29 and a DA (Digital / Analog) conversion unit 482 shown in FIG. The conversion unit 431 converts the reflected wave from the transducer 421 into digital data, and supplies the converted digital data to the front end signal processing unit 432. The conversion unit 431 converts the digital data from the front end signal processing unit 432 into an analog signal, and supplies the converted analog signal to the vibrator 421.

  The front end signal processing unit 432 performs signal processing such as beam forming processing, signal compression processing, and error correction processing on the digital data from the conversion unit 431, and supplies the processed data to the wireless IF 433. The front-end signal processing unit 432 generates digital data that is the basis of the transmission wave transmitted by the transducer 421 and supplies the generated digital data to the conversion unit 431.

  The wireless IF 433 transmits the data from the front end signal processing unit 432 to the reception display device 412 by wireless communication.

  The reception display device 412 is configured to include a wireless IF 441, a back-end signal processing unit 442, and a display unit 443.

  The wireless IF 441 receives data from the probe unit 411 and supplies it to the back-end signal processing unit 442.

  The back end signal processing unit 442 decodes the compressed data transmitted from the wireless IF 441. The back-end signal processing unit 442 generates an ultrasonic image showing the inside of the subject based on the decoded data. The back-end signal processing unit 442 supplies the generated ultrasonic image to the display unit 443.

  The display unit 443 displays the ultrasonic image generated by the back end signal processing unit 442.

  In the example of FIG. 28, the configuration of the probe unit 411 is shown in a simplified manner, and a processing unit and a machine unit that are not related to the present technology are omitted.

[Configuration example of probe unit for receiving side processing]
FIG. 29 is a diagram illustrating a configuration example of a probe unit in the case of performing ultrasonic reception side processing.

  In the example of FIG. 29, the probe unit 411 includes a transducer 421, a signal processing block 422, an angle sensor 241 and a movement amount sensor 121 included in the ultrasonic probe 111, an input unit 451, a control unit 453, and a battery unit 454. It is configured to include.

  The signal processing block 422 in the case of performing ultrasonic reception-side processing is configured to include a switch unit 461, an AD conversion unit 462, a signal processing unit 463, a signal compression unit 464, and a transmission unit 465. Of the signal processing block 422 in FIG. 29, the signal processing unit 463, the signal compression unit 464, and the transmission unit 465 correspond to the front-end signal processing unit 432 in FIG.

  The vibrator 421 receives the reflected wave from the subject and supplies the received signal to the switch unit 461 of the signal processing block 422.

  The switch unit 461 selects which signal is read out of the signals received by each transducer of the transducer 421 under the control of the control unit 453. The vibrator 421 is composed of, for example, 128-ch vibrators, and among them, for example, when reading 32ch signals, the switch unit 461 selects which 32ch signals are read out of 128ch. The switch unit 461 reads the selected signal and supplies the read signal to the AD conversion unit 462.

  The AD conversion unit 462 performs AD conversion on the signal from the switch unit 461 under the control of the control unit 453. The AD conversion unit 462 supplies the digital data after AD conversion to the signal processing unit 463.

  The signal processing unit 463 performs beam forming processing on the digital data from the AD conversion unit 462 under the control of the control unit 453. The signal processing unit 463 also performs signal processing such as image enhancement and noise reduction on the data after beam forming (hereinafter also referred to as RF data) as necessary. The signal processing unit 463 supplies the processed data to the signal compression unit 464.

  The signal compression unit 464 compresses the digital data from the signal processing unit 463 in a predetermined compression format under the control of the control unit 453. The signal compression unit 464 supplies the compressed data to the transmission unit 465. The compression format is not limited.

  Under the control of the control unit 453, the transmission unit 465 adds a redundant error correction code for transmission error compensation to the data from the signal compression unit 464, and via the wireless IF 433 in FIG. , To the reception display device 412. The transmission unit 465 retransmits data to be transmitted in order to compensate for transmission errors.

  The angle sensor 241 and the movement amount sensor 121 are provided in the ultrasonic probe 111 as described above. The angle sensor 241 detects a rotation operation of the ultrasonic probe 111 by the user, and supplies a motion parameter, which is information indicating the characteristic of the motion, which is the detected rotation angle of the ultrasonic probe 111, to the control unit 453. The movement amount sensor 121 detects the amount of movement of the ultrasonic probe 111 by the user, and supplies the control unit 453 with a movement parameter that is information indicating the characteristic of movement, which is the detected movement amount of the ultrasonic probe 111.

  The input unit 451 inputs an instruction signal corresponding to a user operation to the control unit 453.

  The control unit 453 controls the operation of each unit constituting the signal processing block 422 according to the information detected by the angle sensor 241 and the movement amount sensor 121. As a result, power consumption stored in the battery unit 454 can be suppressed, or the image quality of the obtained image can be changed.

  For example, the control unit 453 controls the switch unit 461 to change the number of transducers 421 used for reception, for example. In general, information from a plurality of transducers 421 is used to increase the SN of the received signal. For example, by reducing the number of channels of the vibrator 421 used for the reception, it is possible to reduce the amount of calculation processing in the signal processing unit 463 in the subsequent stage. As a result, power consumption can be reduced.

  For example, the control unit 453 controls the AD conversion unit 462 to change the sampling frequency or the bit length of the digital data when converting the received analog signal of each channel into digital data.

  The image processing system 401 may be used in a medical image diagnosis support system CAD (Computer Aided Diagnosis). If the sampling frequency is increased, the amount of information of the obtained signal is increased, and more accurate beam forming can be performed, and as a result, the image quality is improved. Therefore, sampling at a high sampling frequency leads to an improvement in CAD diagnosis capability.

  However, a high frequency of AD conversion causes an increase in data and affects the amount of subsequent signal processing. When it is not used for CAD, that is, in the case of normal diagnosis, the image quality is not so high. Therefore, reducing the sampling frequency during normal diagnosis or the like can reduce the power consumption of the AD conversion unit 462 itself and reduce the amount of signal processing. As a result, power consumption can be reduced. In the AD converter 462, shortening the bit length of the digital data can provide the same effect as reducing the sampling frequency.

  Further, for example, when the position of the ultrasonic probe 111 is approaching a point on which details are desired to be seen on the abdomen or chest, the user tends to move the probe unit 411 small in a narrow range and slowly. That is, when the movement of the ultrasonic probe 111 is small, the speed is low, or the movement amount is small, there is a high possibility that the position of the ultrasonic probe 111 is close to a point where details are to be viewed. Therefore, in such a case, it is desired that the image quality is as high as possible.

  On the other hand, when searching for a point to see details in a wide range, the user tends to move the probe unit 411 large and fast in a wide range. That is, when the movement of the ultrasonic probe 111 is large, the speed is high, or the movement amount is large, there is a high possibility that the user is looking for a point to see details. Therefore, in such a case, the image quality may be lower than that described above.

  This is true for the joints as well as the abdomen and chest. Therefore, a portion where the ultrasonic probe 111 is slowly rotated around the joint is a portion to be carefully observed. Therefore, when the change in angle from the angle sensor 241 (that is, the angular velocity) is slow, the control unit 453 increases the sampling frequency so that the image quality of the ultrasonic image at that location is improved. On the other hand, in the periphery of the joint, the portion that is rotated quickly is a portion that is desired to be observed quickly. Therefore, when the change in angle from the angle sensor 241 (that is, the angular velocity) is fast, the control unit 453 does not need to make the image quality of the ultrasonic image at that point so high, so the sampling frequency is lowered.

  For example, the control unit 453 controls the signal processing unit 463 to change a reception focus score, a sampling frequency of RF data, or the like, which is a parameter related to power among parameters when performing beamforming.

  By reducing the number of reception focus points or reducing the sampling frequency of RF data, it is possible to reduce the processing itself and the amount of data in the subsequent stage, and as a result, it is possible to reduce power consumption.

  Note that ON / OFF of signal processing such as image enhancement and noise reduction in the signal processing unit 463 and control of algorithm complexity also affect power. The control unit 453 may also control these.

  For example, the control unit 453 controls the signal compression unit 464 to change the compression rate. By increasing the data compression rate, the amount of data to be transmitted from the probe unit 411 to the reception display device 412 is reduced, and as a result, transmission power can be suppressed.

  For example, the control unit 453 controls the transmission unit 465 to change the addition strength of the error correction code and the presence / absence thereof. By reducing the intensity of error correction or not using the error correction function itself, the amount of power required for transmission can be reduced. In addition, changing the acceptance of the data retransmission request generated by the cooperative operation with the reception display device 412 to the rejection to the transmission unit 465 also leads to a reduction in electric energy.

  The battery unit 454 is composed of a rechargeable battery or the like, and supplies power to each part of the probe unit 411.

[Configuration example of probe unit in case of sender processing]
FIG. 30 is a diagram illustrating a configuration example of a probe unit in the case of performing transmission-side processing.

  In the example of FIG. 30, the probe unit 411 includes the vibrator 421, the signal processing block 422, the angle sensor 241, the movement amount sensor 121, the input unit 451, the control unit 453, and the battery, similarly to the probe unit 411 of FIG. The unit 454 is configured to be included. Corresponding portions are denoted by corresponding reference numerals, and the description thereof will be omitted as appropriate because it will be repeated.

  Unlike the signal processing block 422 in FIG. 29, the signal processing block 422 in the case of performing ultrasonic transmission side processing is configured to include a switch unit 481, a DA conversion unit 482, and a signal processing unit 483. In the signal processing block 422 in FIG. 30, the signal processing unit 483 corresponds to the front end signal processing unit 432 in FIG.

  The switch unit 481 selects the vibrator 421 based on the analog signal from the DA conversion unit 482. That is, the switch unit 481 selects a combination of vibrators to be operated from among a plurality of vibrators constituting the vibrator 421. The switch unit 481 connects the selected vibrator 421 and transmits a signal to vibrate the selected vibrator 421. Thereby, an ultrasonic beam is transmitted from the vibrator 421 to the subject.

  The DA conversion unit 482 converts the digital data from the signal processing unit 483 into an analog signal and supplies the analog signal to the switch unit 481.

  The signal processing unit 483 generates digital data that is the basis of the ultrasonic beam that the transducer 421 transmits to the subject. The signal processing unit 483 supplies the generated digital data to the DA conversion unit 482.

  Also in the example of FIG. 30, the control unit 453 controls the operation of each unit constituting the signal processing block 422 according to the information detected by the angle sensor 241 and the movement amount sensor 121. As a result, power consumption stored in the battery unit 454 can be suppressed, or the image quality of the obtained image can be changed.

  However, unlike the case of the reception side processing of FIG. 29, in the case of the transmission side processing of FIG. 30, the switch unit 481, the DA conversion unit 482, and the signal processing unit 483 basically perform a cooperative operation.

  The digital data generated by the signal processing unit 483 uniquely determines the bit length, sampling frequency, and number of lines (number of transducers to be operated) of the digital data passing through the DA converter 482, and is connected (vibrated) by the switch unit 481. ) The combination of the vibrators 421 is also determined.

  In other words, the signal processing unit 483 uniquely determines the bit length of digital data passing through the DA conversion unit 482, the sampling frequency, the number of lines, and the combination of the vibrators 421 connected by the switch unit 481, and determines the determined parameters. In combination, digital data is generated.

  Therefore, in the case of transmission-side processing, the control unit 453 controls the signal processing unit 483, the bit length of digital data passing through the DA conversion unit 482, the sampling frequency, the number of lines, and the vibrator 421 connected by the switch unit 481. Change the combination.

  In the signal processing unit 483, DA conversion processing can be reduced by shortening the bit length of the digital data or changing the sampling frequency to be low. Further, by reducing the number of lines, it is possible to reduce the power related to ultrasonic transmission.

  On the other hand, in the signal processing unit 483, the image quality of the obtained image can be improved by increasing the bit length of the digital data, changing the sampling frequency higher, or increasing the number of lines.

  Further, the control unit 453 calculates a shooting angle when shooting around the joint based on the number of divisions input from the input unit 451, and when the angle from the angle sensor 241 becomes the calculated shooting angle. In addition, an ultrasonic beam is transmitted and received to generate an ultrasonic image.

  Thereby, since the transmission / reception of the ultrasonic beam more than necessary is not performed, the power related to the ultrasonic transmission can be reduced.

  As described above, the control unit 453 controls each signal processing unit included in the signal processing block 422 in both the ultrasonic transmission side processing and the ultrasonic reception side processing, so that the battery of the battery unit 454 can be controlled. Consumption can be suppressed and the image quality of the ultrasonic image can be improved.

[Flow of ultrasonic reception processing]
Next, the ultrasonic reception processing of the probe unit 411 will be described with reference to the flowchart of FIG.

  In step S111, the vibrator 421 receives a reflected wave from the subject. The vibrator 421 supplies the received signal to the switch unit 461 of the signal processing block 422.

  In step S112, the switch unit 461 selects a signal. That is, the switch unit 461 selects which signal is read out of the signals received by each transducer of the transducer 421. The number of reception transducers at this time is controlled by the control unit 453 according to the magnitude of the motion parameter from at least one of the angle sensor 241 and the movement amount sensor 121. The switch unit 461 reads out the selected signal and supplies it to the AD conversion unit 462.

  In step S113, the AD conversion unit 462 performs AD conversion on the signal from the switch unit 461 at a predetermined sampling rate. The AD (digital data) bit length and the AD sampling rate at this time are controlled by the control unit 453 according to the magnitude of the motion parameter from at least one of the angle sensor 241 and the movement amount sensor 121. The AD conversion unit 462 supplies the digital data after AD conversion to the signal processing unit 463.

  In step S <b> 114, the signal processing unit 463 performs beam forming processing on the digital data from the AD conversion unit 462. The signal processing unit 463 also performs signal processing such as image enhancement and noise reduction on the RF data under the control of the control unit 453.

  The frame rate and resolution at this time are controlled by the control unit 453 according to the magnitude of the motion parameter from at least one of the angle sensor 241 and the movement amount sensor 121. Processing such as image enhancement and noise reduction is also controlled by the control unit 453 according to the magnitude of the motion parameter from at least one of the angle sensor 241 and the movement amount sensor 121. The signal processing unit 463 supplies the processed data to the signal compression unit 464.

  In step S115, the signal compression unit 464 compresses the digital data from the signal processing unit 463 in a predetermined compression format. The bit rate at this time is controlled by the control unit 453 according to the magnitude of the motion parameter from at least one of the angle sensor 241 and the movement amount sensor 121. The signal compression unit 464 supplies the compressed data to the transmission unit 465.

  In step S116, the transmission unit 465 adds a redundant error correction code for transmission error compensation to the data from the signal compression unit 464, and transmits the data to the reception display device 412 via the wireless IF 433. Send. The addition of error correction at this time is controlled by the control unit 453 according to the magnitude of the motion parameter from at least one of the angle sensor 241 and the movement amount sensor 121.

  As described above, data obtained by performing signal processing on the received ultrasonic waves is transmitted from the probe unit 411 to the reception display device 412 via wireless communication.

[Receive display process flow]
Next, the reception display process of the reception display device 412 will be described with reference to the flowchart of FIG.

  In step S121, the wireless IF 441 receives the data transmitted in step S116 of FIG. 31 described above. The wireless IF 441 supplies the received data to the back-end signal processing unit 442.

  In step S122, the back-end signal processing unit 442 decodes the compressed data from the wireless IF 441 by a method corresponding to the compression of the signal compression unit 464, and generates an ultrasonic image indicating the inside of the subject. The back-end signal processing unit 442 supplies the generated ultrasonic image to the display unit 443.

  In step S123, the display unit 443 displays an ultrasonic image.

  As described above, the reception display device 412 displays an ultrasonic image corresponding to the data ultrasonically received by the probe unit 411.

[Flow of ultrasonic transmission processing]
Next, the ultrasonic transmission processing of the probe unit 411 will be described with reference to the flowchart of FIG.

  In step S131, the signal processing unit 483 generates digital data that is the basis of the ultrasonic beam that the transducer 421 transmits to the subject under the control of the control unit 453.

  That is, the signal processing unit 483 uniquely determines the bit length of digital data passing through the DA conversion unit 482, the sampling frequency, the number of lines, and the combination of the transducers 421 connected by the switch unit 481, and uses the determined combination of parameters. Generate digital data. Each processing parameter at this time is controlled by the control unit 453 according to the magnitude of the motion parameter from at least one of the angle sensor 241 and the movement amount sensor 121.

  The signal processing unit 483 supplies the generated digital data to the DA conversion unit 482.

  In step S132, the DA conversion unit 482 performs DA conversion. That is, the DA conversion unit 482 converts the digital data from the signal processing unit 483 into an analog signal and supplies it to the switch unit 481.

  In step S133, the transducer 421 transmits an ultrasonic beam to the subject. That is, the switch unit 481 selects the vibrator 421 based on the analog signal from the DA conversion unit 482. The switch unit 481 connects the selected vibrator 421 and transmits a signal to vibrate the selected vibrator 421. Thereby, an ultrasonic beam is transmitted from the vibrator 421 to the subject.

  As described above, the probe unit 411 transmits an ultrasonic beam to the subject.

  As described above, the image quality of the ultrasonic image required by the user can be determined by the movement of the probe unit 411 (ultrasonic probe 111) by the user. Accordingly, the probe unit 411 controls the processing of each part of the signal processing block 422 according to the movement of the ultrasonic probe 111. In particular, the probe unit 411 controls the image quality to be lowered and the signal processing performance to be lowered when the motion parameter indicating the motion characteristic of the ultrasonic probe 111 is large. On the contrary, the probe unit 411 performs control so that the image quality is improved and the signal processing performance is improved when the motion parameter indicating the motion characteristic of the ultrasonic probe 111 is small.

  Therefore, when the user uses the probe unit 411 and moves slowly or smallly, for example, to clearly see the place where the ultrasonic image is taken, the image quality is given priority over the suppression of power consumption. Can be improved.

  On the other hand, when the user is moving fast or greatly in order to search for a point from a rough place on the body using the probe unit 411, for example, the power consumption is suppressed in preference to the image quality. Can do. In this case, even when the probe unit 411 is used for diagnosis, the power consumption of the battery unit 454 in the probe unit 411 can be suppressed. As a result, the holding power of the battery unit 454 can be increased.

  In the processing of the probe unit 411 described above, the example using the angle sensor 241 has been described. However, instead of the angle sensor 241, an angular velocity sensor may be used.

[Flow of pre-shooting processing]
Further, imaging pre-processing in the probe unit 411 will be described with reference to the flowchart in FIG.

  For example, when photographing the periphery of the joint, the user inputs the number of divisions (number of photographing) around the joint via the input unit 451. In response to this, in step S201, the input unit 451 inputs the division number N to the control unit 453.

In step S202, the control unit 453 sets n to 0. In step S203, the control unit 453 uses the division number N from the input unit 451 to calculate each imaging angle of the joint to be diagnosed by the following equation (2).

θn = n * 360 / N (2)

  In step S204, the control unit 453 stores each shooting angle θn obtained in step S203 in a built-in memory or the like. The control unit 453 sets n to n + 1 in step S205, and determines whether n> N in step S206.

  If it is determined in step S206 that n> N is not satisfied, that is, n ≦ N, the process returns to step S203, and the subsequent processes are repeated.

  If it is determined in step S206 that n> N, the pre-shooting process is terminated.

[Flow of shooting process]
Next, imaging processing in the probe unit 411 will be described with reference to the flowchart in FIG. This process is started when, for example, a shooting start command is input via the input unit 451 of the image processing system 401.

  In step S231, the control unit 453 sets n to 0. In step S232, the control unit 453 acquires the shooting angle θ stored in the built-in memory. The photographing angle θ is obtained and stored by the preprocessing shown in FIG. 34, or is set as a default in advance.

  In step S233, the control unit 453 determines whether or not θ ≧ θn. If it is determined in step S233 that θ <θn, the process returns to step S232, and the subsequent processes are repeated.

  If it is determined in step S233 that θ ≧ θn, the process proceeds to step S234.

  In step S234, the control unit 453 transmits and receives an ultrasonic beam. That is, the control unit 453 controls the signal processing unit 483 so that the ultrasonic beam is transmitted and received when the angle information detected by the angle sensor 241 reaches the imaging angle θ acquired in step S232. Correspondingly, the ultrasonic transmission processing described above with reference to FIG. 33 is performed, the ultrasonic reception processing described above with reference to FIG. 32 is performed, and the reception display described above with reference to FIG. Step S121 of the process is performed.

  The back-end signal processing unit 442 generates an ultrasonic image In in step S235, and stores the generated ultrasonic image In in step S236.

  The control unit 453 sets n to n + 1 in step S237, and determines whether n> N in step S238.

  If it is determined in step S238 that n> N is not satisfied, that is, n ≦ N, the process returns to step S232, and the subsequent processes are repeated.

  If it is determined in step S238 that n> N, the pre-shooting process is terminated.

  As described above, in the image processing system 401, since the transmission timing of the ultrasonic beam is controlled according to the angle information, the ultrasonic beam is not transmitted and received more than necessary. Such electric power can be reduced.

  As described above, the ultrasonic probe of the present technology can be realized simply by attaching a jig to an existing probe.

  In this technology, a handle having a rotation axis with a built-in angle sensor is attached so as to be orthogonal to the probe, so that it is possible to detect an accurate angle of the probe that moves around the cylindrical subject. it can.

  Further, in the present technology, since the guide is attached, it is possible to always transmit and receive the ultrasonic beam from the vertical direction to the subject and to easily rotate around the subject.

  In the present technology, since the ball joint is provided separately from the rotating shaft, the operational feeling can be improved.

  That is, when an ultrasonic image is taken from the periphery of a joint part of a human body that can be approximated to a cylindrical shape, the present technology can be used to easily and accurately create a three-dimensional structure (volume data). The imaging angle and ultrasonic image data can be acquired. Thereby, since an imaging angle and an accurate three-dimensional structure (volume data) can be acquired, quantitative evaluation can be performed even in observation of a difference or follow-up after a preoperative operation.

  In addition, this technique can be used for both a medical use and a non-medical use. When used for non-medical purposes, for example, it is desirable to be able to appropriately adjust the frequency and intensity of ultrasonic waves so that internal organs and the like are not captured.

  Further, the present technology can be used not only for human beings but also for various scenes in which a cross section of a subject is photographed by ultrasonic waves such as animals, plants, and artificial objects.

  The series of processes described above can be executed by hardware or can be executed by software. When a series of processing is executed by software, a program constituting the software is installed in the computer. Here, the computer includes a computer incorporated in dedicated hardware, a general-purpose personal computer capable of executing various functions by installing various programs, and the like.

<Sixth Embodiment>
[Computer configuration example]
FIG. 36 is a block diagram illustrating a hardware configuration example of a computer that executes the above-described series of processing by a program.

  In a computer, a CPU (Central Processing Unit) 501, a ROM (Read Only Memory) 502, and a RAM (Random Access Memory) 503 are connected to each other by a bus 504.

  An input / output interface 505 is further connected to the bus 504. An input unit 506, an output unit 507, a storage unit 508, a communication unit 509, and a drive 510 are connected to the input / output interface 505.

  The input unit 506 includes a keyboard, a mouse, a microphone, and the like. The output unit 507 includes a display, a speaker, and the like. The storage unit 508 includes a hard disk, a nonvolatile memory, and the like. The communication unit 509 includes a network interface or the like. The drive 510 drives a removable medium 511 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory.

  In the computer configured as described above, the CPU 501 loads the program stored in the storage unit 508 to the RAM 503 via the input / output interface 505 and the bus 504 and executes the program, for example. Is done.

  The program executed by the computer (CPU 501) can be provided by being recorded on a removable medium 511 as a package medium or the like, for example. The program can be provided via a wired or wireless transmission medium such as a local area network, the Internet, or digital broadcasting.

  In the computer, the program can be installed in the storage unit 508 via the input / output interface 505 by attaching the removable medium 511 to the drive 510. Further, the program can be received by the communication unit 509 via a wired or wireless transmission medium and installed in the storage unit 508. In addition, the program can be installed in the ROM 502 or the storage unit 508 in advance.

  The program executed by the computer may be a program that is processed in time series in the order described in this specification, or in parallel or at a necessary timing such as when a call is made. It may be a program for processing.

  Further, in the present specification, the term “system” means an overall device configured by a plurality of devices, blocks, means, and the like.

  The embodiments in the present disclosure are not limited to the above-described embodiments, and various modifications can be made without departing from the gist of the present disclosure.

  The preferred embodiments of the present disclosure have been described in detail above with reference to the accompanying drawings, but the disclosure is not limited to such examples. It is clear that various changes and modifications can be conceived within the scope of the technical idea described in the claims if the person has ordinary knowledge in the technical field to which the present disclosure belongs, Of course, it is understood that these also belong to the technical scope of the present disclosure.

In addition, this technique can also take the following structures.
(1) an image generation unit that generates a plurality of simple display images corresponding to a plurality of ultrasonic images input from a probe;
An image processing apparatus comprising: a display control unit configured to display a plurality of simple display images generated by the image generation unit on a display screen by disposing the plurality of simple display images at positions interlocked with the rotation operation of the probe.
(2) The image processing apparatus according to (1), wherein the display control unit arranges and displays a plurality of simple display images generated by the image generation unit at a position surrounding a diagnosis target of ultrasonic diagnosis.
(3) The display control unit displays the circumference in which the plurality of simple display images generated by the image generation unit surround the diagnosis target of the ultrasonic diagnosis in conjunction with the size of the diagnosis target of the ultrasonic diagnosis. The image processing apparatus according to (1) or (2).
(4) The display control unit superimposes and displays an image to be diagnosed in the ultrasonic diagnosis and a plurality of simple display images arranged at positions interlocked with the rotation operation of the probe. The image processing apparatus according to any one of (3).
(5) The display control unit may display a plurality of simple display images generated by the image generation unit arranged and displayed at positions distorted three-dimensionally. Image processing apparatus.
(6) The image processing device according to any one of (1) to (5), wherein the image generation unit generates a plurality of simple display images based on a rotation angle or a rotation angular velocity from a sensor included in the probe. .
(7) The image processing apparatus according to (6), wherein the display control unit displays a wide interval between the plurality of simple display images when a rotation angle or a rotation angular velocity from a sensor included in the probe is large.
(8) The image processing device according to (6), wherein when the rotation angle or the rotation angular velocity from the sensor included in the probe is small, the display control unit displays the arrangement intervals of the plurality of simple display images narrowly.
(9) The probe includes a support portion provided at an angle orthogonal to the beam direction;
A rotation mechanism provided between the probe and the support;
The image processing apparatus according to any one of (1) to (8), further including a guide that supports a rotation operation of the probe by the rotation mechanism.
(10) The image processing apparatus is
A plurality of simple display images corresponding to a plurality of ultrasonic images input from the probe are generated, and the generated plurality of simple display images are arranged at positions linked with the rotation operation of the probe and displayed on the display screen. Image processing method.

  11 ultrasonic probe, 21 probe, 21a sensor surface, 22 pedestal, 23 rotating shaft, 24 handle, 25 guide, 26 joint, 51 ultrasonic probe, 61 joint, 81 ultrasonic probe, 91 guide, 111 ultrasonic Probe, 121 movement sensor, 151 handle, 152 ball joint, 153 auxiliary operation unit, 201 image processing system, 212 image processing device, 213a to 213d recording device, 214 display, 221 ultrasonic transmission / reception unit, 222 detection unit, 232 Ultrasonic receiver, 233 ultrasonic generator, 241 angle sensor, 242 angular velocity sensor, 251 ultrasonic control unit, 252 ultrasonic image generation unit, 253 sensor information acquisition unit, 254 probe state detection unit, 255 Surface image generation unit, 256 display control unit, 257 simple display image generation unit, 271 simple display image group, 272 arrangement image, 281 simple display image, 282 circle, 291 arrangement image, 301 ultrasonic probe, 311 A array transducer, 312 B array transducer, 313 C array transducer, 321 A plane, 322 B plane, 323 C plane, 371 D plane, 401 image processing system, 411 probe unit, 412 reception display device, 421 transducer, 422 signal processing block , 431 conversion unit, 432 front end signal processing unit, 433 wireless IF, 451 input unit, 453 control unit, 454 battery unit

Claims (10)

An image generation unit that generates a plurality of simple display images corresponding to a plurality of ultrasonic images input from a probe;
An image processing apparatus comprising: a display control unit configured to display a plurality of simple display images generated by the image generation unit on a display screen by disposing the plurality of simple display images at positions interlocked with the rotation operation of the probe. The image processing apparatus according to claim 1, wherein the display control unit arranges and displays a plurality of simple display images generated by the image generation unit at a position surrounding a diagnosis target of ultrasonic diagnosis. The display control unit causes a plurality of simple display images generated by the image generation unit to display a circumference surrounding a diagnosis target of the ultrasonic diagnosis in conjunction with a size of the diagnosis target of the ultrasonic diagnosis. 2. The image processing apparatus according to 2. The image processing according to claim 3, wherein the display control unit superimposes and displays an image to be diagnosed in the ultrasonic diagnosis and a plurality of simple display images arranged at positions interlocked with the rotation operation of the probe. apparatus. The image processing apparatus according to claim 2, wherein the display control unit arranges and displays a plurality of simple display images generated by the image generation unit at positions distorted three-dimensionally. The image processing apparatus according to claim 1, wherein the image generation unit generates a plurality of simple display images based on a rotation angle or a rotation angular velocity from a sensor included in the probe. The image processing apparatus according to claim 6, wherein the display control unit displays a wide interval between the plurality of simple display images when a rotation angle or a rotation angular velocity from a sensor included in the probe is large. The image processing apparatus according to claim 6, wherein the display control unit displays an arrangement interval of the plurality of simple display images narrowly when a rotation angle or a rotation angular velocity from a sensor included in the probe is small. The probe has a support portion provided at an angle orthogonal to the beam direction;
A rotation mechanism provided between the probe and the support;
The image processing apparatus according to claim 1, further comprising a guide that supports a rotation operation of the probe by the rotation mechanism. The image processing device
A plurality of simple display images corresponding to a plurality of ultrasonic images input from the probe are generated, and the generated plurality of simple display images are arranged at positions linked with the rotation operation of the probe and displayed on the display screen. Image processing method.

Images