WO2023074823A1

WO2023074823A1 - Heart sound acquisition device, heart sound acquisition system, heart sound acquisition method, and program

Info

Publication number: WO2023074823A1
Application number: PCT/JP2022/040255
Authority: WO
Inventors: 貴之内田; 知紀八田; 亮市川
Original assignee: テルモ株式会社
Priority date: 2021-10-28
Filing date: 2022-10-27
Publication date: 2023-05-04

Abstract

This heart sound acquisition device comprises an auscultation unit, an imaging unit, a storage unit, a display unit, and a control unit. The auscultation unit is configured to be able to acquire heart sounds of a user. The imaging unit captures an image of the user's body including feature points. The storage unit stores the relative position of where the auscultation unit is to be placed with respect to the feature points of the user. The display unit displays the image captured by the imaging unit. The control unit recognizes the positions of the feature points of the user from the image captured by the imaging unit, calculates the position where the auscultation unit is to be placed on the basis of the positions of the feature points and the relative position stored in the storage unit, and guides the auscultation unit to the position where the auscultation unit is to be placed on the image displayed on the display unit.

Description

Heart sound acquisition device, heart sound acquisition system, heart sound acquisition method and program

The present disclosure relates to a heart sound acquisition device, a heart sound acquisition system, a heart sound acquisition method, and a program.

In order to monitor a patient's health, it may be necessary to obtain heart sounds at home. For example, a method using an electrocardiogram, pulse wave, and heart sound has been developed as one of remote monitoring means for heart failure patients, and this method requires the patient to measure the heart sound every day at home. It is important that the heart sound sensor that measures heart sounds is worn at the same position every time because the output waveform differs depending on the position where the sensor is worn. In addition, since the heart sound sensor is to be measured at home, it is desirable that it be of a form that is easy for the patient to wear.

In telemedicine and home care, an electronic listening system has been proposed that includes a position acquisition unit that acquires position information indicating the position of a heart sound sensor abutted against the body to auscultate a patient's heart sounds (see, for example, Patent Reference 1). According to this system, the position information of the body in contact with the heart sound sensor is recorded at the time of examination by a doctor, and when the user obtains the heart sound from the next time onward, the heart sound is detected based on the recorded position information. Guidance information can be provided to the user to guide the sensor.

JP 2017-198

As a method of wearing the heart sound sensor in the same position every time, there is a method of fixing the heart sound sensor to clothes or a band. However, this method has problems such as that it is difficult to wash, the clothes or the band itself shifts depending on how it is worn, and the band is difficult to put on by oneself. Moreover, the above-mentioned Patent Document 1 proposes a method of using a laser, a projector, a head-mounted display, or the like to guide the heart sound sensor to the correct listening position. However, installing a large device in a patient's home may not be easy. In addition, these methods are based on the premise that a third party such as a doctor visually determines the position of the heart sound sensor. However, when the patient tries to perform this method by himself/herself, it is difficult for the patient to visually check the chest where the heart sound sensor is attached. It is difficult to Furthermore, it is necessary to guide and assist the patient in the measurement method so that the patient, who does not have specialized knowledge such as a doctor, can perform the measurement appropriately.

Accordingly, an object of the present disclosure, which focuses on these points, is to provide a heart sound acquisition device, a heart sound acquisition system, and a heart sound acquisition that enable a user to easily position an auscultation unit at a predetermined position to acquire heart sounds. The object is to provide a method and a program.

A heart sound acquisition device according to one aspect of the present disclosure includes an auscultation unit configured to acquire heart sounds of a user, an imaging unit that captures an image of the user's body including characteristic points, and a storage unit for storing the relative position of the position where the auscultation unit should be arranged; a display unit for displaying the image captured by the imaging unit; and the characteristic points of the user from the image captured by the imaging unit. and calculating the position where the auscultation unit should be placed based on the position of the feature point and the relative position stored in the storage unit, and displaying the image on the display unit and a control unit for guiding the auscultation unit to a position where the auscultation unit should be placed.

As one embodiment, the auscultation unit includes a marker, and the control unit recognizes the position of the auscultation unit on the image by detecting the marker from the image captured by the imaging unit.

As one embodiment, the control unit determines at least one of a distance between the imaging unit and the body of the user and an orientation of the body of the user based on the image captured by the imaging unit. configured to be recognizable.

As one embodiment, the heart sound acquisition device further includes a communication unit configured to be able to communicate with a server, and the control unit transmits the relative position stored in the storage unit to the Configured to retrieve from the server.

As one embodiment, the heart sound acquisition device further includes a speaker, and the control unit uses the sound from the speaker when guiding the auscultation unit to a position where the auscultation unit should be placed.

As one embodiment, when guiding the auscultation unit to a position where the auscultation unit should be placed, the control unit superimposes at least one of characters and graphics on the image captured by the image capture unit, and display on the display.

As one embodiment, the auscultation unit includes a pressure sensor that detects pressure between the body of the user and the control unit, based on the pressure detected by the pressure sensor, controls the pressure of the auscultation unit. A contact state of the user with respect to the body is determined.

As one embodiment, the heart sound acquisition device includes a notification unit that notifies that the auscultation unit is positioned at a position where the auscultation unit should be placed.

As one embodiment, the heart sound acquisition device includes a conversion unit that converts the heart sound acquired by the auscultation unit into an electrical signal.

As one embodiment, the control unit determines a contact state of the auscultation unit with respect to the body of the user based on the waveform of the electrical signal obtained by converting the heart sounds by the conversion unit.

As one embodiment, the control unit is configured to analyze the waveform of the electrical signal into which the heart sounds are converted by the conversion unit.

As one embodiment, the heart sound acquisition device further includes a physical information acquisition unit configured to be capable of acquiring physical information other than the heart sounds of the user, and the control unit controls the heart sounds converted by the conversion unit. The waveform of the electrical signal and the waveform of the physical information acquired by the physical information acquisition unit are temporally synchronized.

As one embodiment, the physical information includes at least one of an electrocardiogram and a pulse wave, and the control unit controls the heart sound of the user acquired from the auscultation unit and the heart sound acquired from the physical information acquisition unit. A hemodynamic parameter is calculated based on the physical information of the user.

A heart sound acquisition system according to one aspect of the present disclosure includes a server that stores a relative position where an auscultation unit configured to acquire a user's heart sound with respect to a feature point of the user should be placed; the auscultation unit; an imaging unit that captures an image of a body including characteristic points, a display unit that displays the image captured by the imaging unit, a communication unit that acquires the relative position from the server, and an image captured by the imaging unit recognizing the position of the feature point of the user from the image, calculating the position where the auscultation unit should be arranged based on the position of the feature point and the relative position obtained from the server, and displaying the display; a heart sound acquisition device including a control unit for guiding the auscultation unit to a position where the auscultation unit should be placed on the image displayed on the unit.

A heart sound acquisition method according to one aspect of the present disclosure is a heart sound acquisition method executed by a control unit to acquire heart sounds using an auscultation unit configured to be capable of acquiring heart sounds of a user. capturing an image of the user's body including the feature points; recognizing the positions of the feature points of the user from the image captured by the imaging unit; A position where the auscultation unit should be arranged is calculated based on the position where the auscultation unit should be arranged relative to the feature points, and the auscultation unit is displayed on the image displayed on the display unit. to the position where it should be placed.

A program as one aspect of the present disclosure is a program for acquiring heart sounds using an auscultation unit configured to be capable of acquiring heart sounds of a user, wherein an imaging unit captures an image of the user's body including characteristic points. a process of recognizing the position of the feature point of the user from the image captured by the imaging unit, the position of the feature point, and the auscultation of the feature point of the user stored in a storage unit a process of calculating a position where the auscultation unit should be arranged based on the relative position of the position where the auscultation unit should be arranged; A processor provided in the heart sound acquisition device executes a process of guiding to the desired position.

According to the present disclosure, the user can easily position the auscultation unit at a predetermined position to acquire heart sounds.

FIG. 1 is a schematic configuration diagram showing an example of a hemodynamic monitoring system including a heart sound acquisition device according to one embodiment. FIG. 2 is a diagram for explaining an example of usage of the hemodynamic monitoring system of FIG. FIG. 3 is a schematic configuration diagram showing an example of the heart sound acquisition device of FIG. 4 is a perspective view showing an example of the appearance of the main body shown in FIG. 3. FIG. FIG. 5 is a functional block diagram showing an example of a control unit in FIG. 3; 6 is a perspective view showing an example of the appearance of the auscultation unit of FIG. 3. FIG. FIG. 7 is a flow chart for determining the position of the auscultation unit in a medical institution. FIG. 8 is a diagram illustrating a method of determining the position of an auscultation unit in a medical institution. FIG. 9 is a flowchart for explaining the procedure for calculating hemodynamic parameters. FIG. 10 is a flow chart for explaining the process of placing the auscultation unit in FIG. FIG. 11 is a diagram illustrating an example of a method of arranging the auscultation unit when the user is at home.

Hereinafter, embodiments of the present disclosure will be described with reference to the drawings.

(Hemodynamic monitoring system)
In describing the heart sound acquisition device 15 of the present disclosure, the hemodynamic monitoring system 10 shown in FIG. 1 will be described as an example of a system including the heart sound acquisition device 15. FIG. The hemodynamic monitoring system 10 is a system for remotely monitoring the condition of a user who is a heart failure patient discharged from a medical institution. The hemodynamic monitoring system 10 acquires the user's electrocardiogram, pulse wave, and heart sound data, and analyzes the patient's hemodynamics. Hemodynamic parameters indicating hemodynamics include left ventricular pressure and pulmonary artery pressure. By monitoring changes in hemodynamics, it is possible to predict and prevent changes in cardiac conditions, recurrence of heart failure, and the like.

Hemodynamic monitoring is performed in cooperation with the medical institution and the user, for example, according to the procedure shown in Figure 2. First, a physician treating a heart failure patient decides to perform remote monitoring of a calm patient (user). A doctor at a medical institution specifies the position where the auscultation unit should be placed when acquiring the patient's heart sound, and stores it in the storage unit of the monitoring device or the server.

The user borrows home equipment for hemodynamic monitoring from a medical institution or rental company, and measures heart sounds, electrocardiograms, and pulse waves at home. Heart sounds, electrocardiograms and pulse waves are included in physical information. For example, the measurement is performed periodically at a time determined by a doctor every day. Heart sounds are measured by the user placing the auscultation unit at a position stored in the device or server by the doctor. The user sends the measurement results to the medical institution with the home device.

On the medical institution side, medical personnel such as doctors remotely monitor the data measured by the user. Based on the user's hemodynamic data, the doctor notifies the user of a change in prescription or the like if there is a sign that the user's symptoms are aggravating.

The user receives the prescription from the doctor and takes medicine based on the changed prescription. By doing so, the hemodynamic monitoring system 10 monitors changes in the state of the heart failure patient and, if necessary, quickly changes the prescription or the like, thereby reducing the possibility that the user's condition will worsen and the user will be re-hospitalized. can be reduced.

In order to perform the above-described processing, the hemodynamic monitoring system 10 includes a main unit 11 arranged on the user side, an auscultation unit 12 connected to the main unit 11, electrodes 13 and an arm band device 14, and a main unit. 11 and a medical institution system 16 that can communicate with each other via a communication network 17 . The medical institution system 16 is an information system within the medical institution and includes computers such as servers within the medical institution. The electrodes 13 and the arm band device 14 for acquiring physical information other than heart sounds, ie, electrocardiogram and pulse wave information, are included in the physical information acquiring section.

The main unit 11 is a computer that acquires and analyzes physical information. The auscultation unit 12 is a sensor that acquires the user's heart sounds, that is, heart sounds. The auscultation unit 12 can be rephrased as a heart sound sensor. The electrodes 13 are electrodes attached to a site such as a wrist and an ankle in order to obtain an electrocardiogram of the user. The electrodes 13 can detect minute electricity generated in the heart. The arm band device 14 wraps an arm band around the user's upper arm or the like and sends air into the arm band to compress blood vessels and measure pulse waves transmitted to the blood vessels due to heartbeats. Main unit 11, auscultation unit 12, electrodes 13, and arm band device 14 are connected by wire or wirelessly so that information can be transmitted and received.

The main unit 11 can analyze each waveform of heart sounds, electrocardiograms, and pulse waves. For example, the main unit 11 may store a machine-learned model having heart sounds, an electrocardiogram, and a pulse wave as input data and a patient's hemodynamic parameters as output data. Based on this learned model, the main body 11 may estimate hemodynamic parameters and changes in the patient's condition from the input heart sounds, electrocardiograms, and pulse waves.

The hemodynamic monitoring system 10 may be further connected with the server 18. The server 18 may store relative position information regarding the position where the auscultation unit 12 should be placed for each user determined by a doctor at the medical institution. The relative position information is information indicating the relative position of the position where the auscultation unit 12 should be placed with respect to the feature points of the user's body. The main unit 11 may read the relative position information of the auscultation unit 12 from the server 18 when acquiring the user's heart sounds. Server 18 may be located at a location separate from the medical institution or may be internal to the medical institution. Further, when the doctor determines the position where the user's auscultation unit 12 should be placed in the medical institution system 16, the relative position information is stored in the storage unit 22 (see FIG. 3) of the main unit 11 and passed to the user without using the server 18. can be registered directly.

(Heart sound acquisition device)
A heart sound acquisition device 15 of this embodiment includes a main unit 11 and an auscultation unit 12, which are part of the hemodynamic monitoring system 10 shown in FIG. The heart sound acquisition device 15 is not limited to the hemodynamic monitoring system 10, and can be used for other heart sound acquisition applications. A system including the heart sound acquisition device 15 and the server 18 is called a heart sound acquisition system. The hemodynamic monitoring system 10 of FIG. 1 is a heart sound acquisition system. A more detailed configuration of the main unit 11 and the auscultation unit 12 of the heart sound acquisition device 15 will be described below.

(Construction of main body)
As shown in FIG. 3 , main unit 11 includes imaging unit 21 , storage unit 22 , display unit 23 , imaging adjustment unit 24 , control unit 25 , communication unit 26 and speaker 27 . Further, as shown in FIG. 4, the main body portion 11 includes a stand 28 capable of adjusting the orientation of the display portion 23. As shown in FIG.

The imaging unit 21 is a camera capable of imaging the user's face and upper body. Therefore, the imaging unit 21 may be arranged above or below the display unit 23 . The imaging unit 21 may include a lens and an imaging element. The imaging device is, for example, a CCD image sensor (Charge-Coupled Device Image Sensor) or a CMOS image sensor (Complementary MOS Image Sensor).

The storage unit 22 is a memory that stores data required for processing performed in the main unit 11 and data generated in the main unit 11 . The storage unit 22 may store programs executed by the control unit 25, which will be described later. The storage unit 22 may include, for example, one or more of a semiconductor memory, a magnetic memory, an optical memory, and the like. Semiconductor memory may include volatile memory and non-volatile memory. The storage unit 22 may store information on the position where the auscultation unit 12 should be placed. The position where the auscultation unit 12 should be placed is stored as relative position information indicating the relative position with respect to the feature points of the user's body, as will be described later. When the control unit 25 estimates the user's hemodynamics by machine learning, the storage unit 22 may store a learned model.

The display unit 23 displays images under the control of the control unit 25 . The display unit 23 can display an image including at least partially the user's face and upper body imaged by the imaging unit 21, as shown in FIG. A commonly known display can be used for the display unit 23 . The display unit 23 can employ, for example, a liquid crystal display (LCD), an organic EL (Electro-Luminescence) display, an inorganic EL display, a plasma display (PDP: Plasma Display Panel), or the like.

The imaging adjustment unit 24 adjusts the imaging direction of the imaging unit 21 under the control of the control unit 25 . The imaging adjustment section 24 may include, for example, a driving section that is incorporated inside the main body section 11 and changes the orientation of the imaging section 21 . Further, the imaging adjustment section 24 may be incorporated in the stand 28 and adjust the orientation of the portion of the main body section 11 including the imaging section 21 .

The control section 25 controls each section of the main body section 11, and performs various arithmetic processing for positioning the auscultation section 12 and estimating the user's hemodynamic parameters. Control unit 25 includes one or more processors. The processor includes a general-purpose processor that loads a specific program and executes a specific function, and a dedicated processor that specializes in specific processing. Special purpose processors include Application Specific Integrated Circuits (ASICs) and Programmable Logic Devices (PLDs). PLDs include FPGAs (Field-Programmable Gate Arrays). The control unit 25 may be either SoC (System-on-a-Chip) or SiP (System In a Package) in which one or more processors cooperate. The control unit 25 may include memory built into the processor or memory independent of the processor. The control unit 25 can execute a program that defines control procedures. The control unit 25 may be configured to load and implement a program recorded on a non-transitory computer-readable medium into a memory. Processing performed by the control unit 25 will be further described below with reference to FIG. 5 and the like.

The communication unit 26 includes hardware and software for communicating with the medical institution system 16 and the server 18 via the communication network 17. The communication unit 26 corresponds to wired and/or wireless communication means. The communication unit 26 performs processing such as protocol processing related to transmission and reception of information, modulation of transmission signals and demodulation of reception signals.

Under the control of the control unit 25, the speaker 27 emits sounds for guiding the user. A known speaker can be used as the speaker 27 .

(Structure of stethoscope)
The auscultation unit 12 is a sensor that is brought into contact with the user's own chest to acquire heart sounds. The auscultation unit 12 includes a heart sound acquisition unit 31, a conversion unit 32, a pressure sensor 33, and a vibration unit 34, as shown in FIG. Each component of the auscultation unit 12 may be controlled by the control unit 25 of the main unit 11 . Further, the auscultation section 12 may have a control section (processor) for controlling each component of the auscultation section 12 in cooperation with the control section 25 of the main body section 11 . Note that the main unit 11 and the auscultation unit 12 may be connected by wire or wirelessly. The auscultation unit 12 also has a marker 36 on the surface opposite to the portion that contacts the user's chest, as shown in FIG. Further, the auscultation unit 12 has a handle 37 that the user uses when holding the auscultation unit 12 .

The heart sound acquisition unit 31 acquires the user's heart sound. The heart sound acquisition unit 31 can be provided on the side of the auscultation unit 12 that contacts the user's chest. The conversion unit 32 converts the heart sounds acquired by the heart sound acquisition unit 31 into electrical signals. The heart sound acquisition unit 31 and the conversion unit 32 may constitute a heart sound sensor that converts heart sounds into electrical signals. Heart sound sensors include, for example, MEMS (Micro Electro Mechanical Systems) heart sound sensors, heart sound sensors using piezoelectric elements, and heart sound sensors using accelerometers. Note that the heart sound acquisition device of the present disclosure is not limited to this embodiment. The heart sound acquisition device of the present disclosure also includes a form of auscultation unit 12 that does not convert heart sounds into electrical signals. The heart sounds are then transmitted to the physician or user as vibrations of air or objects.

The pressure sensor 33 is a sensor that detects the pressure with which the auscultation unit 12 is pressed against the user's chest and outputs it as an electrical signal. For the pressure sensor 33, for example, a pressure sensor 33 using a piezoelectric effect can be used. The pressure sensor 33 may have, for example, a ring shape along the outer circumference of the surface of the auscultation unit 12 that contacts the chest of the user. Alternatively, the pressure sensors 33 may be provided at a plurality of locations on the outer circumference of the surface of the auscultation unit 12 that contacts the user's chest.

The vibrating section 34 includes a vibrator that generates vibrations perceptible to humans. Vibrators include those using eccentric motors, those using linear vibrators, and those using piezoelectric elements.

The marker 36 is a mark used to specify the position of the auscultation unit 12 from the image captured by the imaging unit 21. It is preferable that the marker 36 can specify the position of the auscultation unit 12 as a point from the image of the auscultation unit 12 captured by the control unit 25 . The marker 36 is composed of, for example, a pattern including circles. In this case, the control unit 25 can determine that the center of the circle is the position of the auscultation unit 12 . The marker 36 is not limited to this. For example, the marker 36 can be a pattern in which two line segments intersect perpendicularly. In this case, it can be determined that the position of the auscultation unit 12 is the point where the two straight lines intersect.

The handle 37 is provided to prevent the user's hand from entering the imaging unit 21 side of the marker 36 and obstructing the imaging of the marker 36 when the user takes an image while holding the auscultation unit 12 . . It is also provided so that the patient can easily adjust the position by holding it with both hands. Various shapes can be adopted as the shape of the handle 37 .

(Processing executed by control unit)
As an example, as shown in FIG. 5, the control unit 25 of the main unit 11 includes an image recognition unit 25a, a camera adjustment unit 25b, a direction guide unit 25c, a contact determination unit 25d, a waveform processing unit 25e, a waveform analysis unit 25f, an estimation It includes functional blocks of the section 25g and the determination section 25h. The processing of each functional block may be executed by the same processor or by different processors. The processing of each functional block may be performed by a single software module or may be performed by multiple software modules. The processing of each functional block can be rearranged, separated, or combined. All the functions of each functional block can be regarded as functions of the control unit 25 .

The image recognition unit 25a recognizes the user's face, skeleton, etc. from the image captured by the imaging unit 21. The image recognition unit 25a can identify feature points from the recognized user's face, skeleton, and the like. Feature points include, for example, the user's left and right shoulder joints or acromion portions, as well as the eyes, ears, nose, mouth and chin of the face. The image recognition unit 25a can further recognize the marker 36 of the auscultation unit 12 from the image captured by the imaging unit 21 and specify the position of the auscultation unit 12 .

The image recognition unit 25a can further recognize at least one of the distance between the imaging unit 21 and the user's body and the direction of the user's body from the image captured by the imaging unit 21. The image recognition unit 25a can recognize when the orientation of the user's body deviates from the correct orientation, for example, the front. Further, when a predetermined range of the user's body is out of the field of view of the imaging section 21, the image recognition section 25a can recognize this.

If the image of the user's body recognized by the image recognition unit 25a is out of the correct orientation and position for heart sound measurement, the camera adjustment unit 25b controls the imaging adjustment unit 24 to change the imaging orientation of the imaging unit 21. can be adjusted.

The direction guide unit 25c identifies the position where the auscultation unit 12 should be placed based on the positions of the feature points recognized by the image recognition unit 25a and the relative position information stored in the storage unit 22. The relative position information is information indicating the relative position of the position where the auscultation unit 12 should be placed with respect to the feature point. When the position of the auscultation unit 12 specified from the image captured by the imaging unit 21 is deviated from the position where the auscultation unit 12 should be placed, the direction guide unit 25c displays the auscultation unit 12 on the image displayed on the display unit 23. The user is guided to move the unit 12 to the position where it should be arranged.

For example, the direction guide unit 25c displays the position where the auscultation unit 12 should be placed on the image displayed on the display unit 23. The position where the auscultation unit 12 should be placed may be displayed by displaying a circular or square mark in a specific color such as red, or by blinking the mark. The direction guide section 25c may further superimpose on the image displayed on the display section 23 to display the direction in which the auscultation section 12 should be moved by graphics such as straight lines and arrows, characters, and the like. The direction in which the auscultation unit 12 should move is the direction toward the position where the auscultation unit 12 should be arranged. The direction guide section 25c may guide the user in the direction in which the auscultation section 12 should be moved by voice from the speaker 27 . When the user is moving the auscultation unit 12 in a direction different from the direction in which the auscultation unit 12 should be moved, the direction guide unit 25c vibrates the vibrating unit 34 so that the user does not know that the auscultation unit 12 is moving in the wrong direction. You can let me know you are there. Conversely, the direction guide section 25c may cause the vibration section 34 to vibrate when the user moves the auscultation section 12 in the direction in which the auscultation section 12 should be moved.

When the position of the auscultation unit 12 matches the position where the auscultation unit 12 should be arranged, the direction guide unit 25c superimposes at least one of characters and graphics on the image displayed on the display unit 23, thereby And/or the user may be notified that the auscultation unit 12 is aligned with the correct measurement position by emitting sound using the speaker 27 and/or by vibrating the vibration unit 34 .

The contact determination unit 25d can determine the contact state of the auscultation unit 12 with the user's body based on the pressure detected by the pressure sensor 33. The contact determination unit 25d may consider the waveform of the heart sound when determining the contact state. For example, the contact determination unit 25d determines that the pressure detected by the pressure sensor 33 is equal to or less than a predetermined value, and/or the waveform analysis unit 25f, which will be described later, does not detect at least one of sound I and sound II from the waveform of the heart sound. At this time, it may be determined that the pressure of the auscultation unit 12 against the user's body is insufficient. When the contact determination unit 25d determines that the pressing pressure is insufficient, the contact determination unit 25d uses at least one of the image displayed on the display unit 23 and the sound from the speaker 27 to instruct the user to increase the pressing pressure of the auscultation unit 12. can guide you to do so. Also, the auscultation unit 12 may be provided with a lamp that indicates that the pressing force of the auscultation unit 12 is insufficient. The contact determination section 25 d may control lighting of this lamp according to the output of the pressure sensor 33 .

The waveform processing unit 25e removes noise from the waveform of the heart sound acquired by the auscultation unit 12 and converted into an electrical signal by the conversion unit 32. For example, the waveform processing unit 25e performs a filtering process on the electrical signal of heart sounds to remove noises not derived from heart beats, such as environmental sounds and breathing sounds. When the main unit 11 acquires electrocardiogram and pulse wave signals from the electrodes 13 and the arm band device 14, the waveform processing unit 25e may remove noise from the waveforms of these signals.

The waveform analysis unit 25f analyzes the cardiac waveform. The waveform analysis unit 25f extracts feature amounts from the waveform of the heart sound from which noise has been removed by the waveform processing unit 25e. The waveform analysis unit 25f, for example, extracts the timings of the first and second heart sounds. The waveform analysis unit 25f may analyze acceleration and fragmentation of the first sound and the second sound. The waveform analysis unit 25f further analyzes the waveforms of the electrocardiogram and pulse wave. The waveform analysis unit 25f, for example, extracts the timing, width, magnitude, etc. of the Q wave, R wave, and S wave from the waveform of the electrocardiogram. The waveform analysis unit 25f can extract, for example, the timing and duration of diastole and systole from the waveform of the pulse wave. The waveform analysis unit 25f can time-synchronize and analyze the waveform of the electrical signal obtained by converting the heart sounds, and the waveforms of the electrocardiogram and the pulse wave obtained by the electrodes 13 and the arm band device 14 .

The estimation unit 25g calculates hemodynamic parameters based on the feature amount extracted by the waveform analysis unit 25f. The hemodynamic parameters include at least one of left ventricular pressure and pulmonary artery pressure. The estimation unit 25g may estimate the hemodynamic parameters by machine learning as described above. The means by which the estimation unit 25g calculates the hemodynamic parameters is not limited to one using machine learning.

The determination unit 25h acquires hemodynamic parameters from the estimation unit 25g. The determination unit 25h continuously monitors hemodynamic parameters and determines changes and/or abnormalities in the heart condition.

Note that the control unit 25 does not have to execute at least part of the processing of the waveform processing unit 25e, the waveform analysis unit 25f, the estimation unit 25g, and the determination unit 25h. These processes may be performed by the medical institution system 16 instead of the control unit 25 . Also, these processes may be performed by the server 18 provided at a location different from the medical institution.

Next, the processing executed by the hemodynamic monitoring system 10 will be explained using a flowchart. In the following description, the processes executed by the image recognition unit 25a, the camera adjustment unit 25b, the direction guide unit 25c, the contact determination unit 25d, the waveform processing unit 25e, the waveform analysis unit 25f, the estimation unit 25g, and the determination unit 25h are 25 will be described as a process executed by .

(Processing for setting the position where the stethoscope should be placed)
Before the user uses the hemodynamic monitoring system 10 at home, it is necessary to specify and store the relative position of the auscultation unit 12 when acquiring the heart sound with respect to the characteristic points at the medical institution. A procedure for setting the position where the auscultation unit 12 acquires heart sounds will be described with reference to the flowchart of FIG.

First, a doctor at a medical institution uses the heart sound acquisition device 15 to bring the auscultation unit 12 into contact with the user's chest and searches for a position where heart sounds can be acquired satisfactorily (step S101). The heart sound acquisition device 15 used at this time may be the same as or different from the device used by the user at home.

The control unit 25 of the main unit 11 determines whether heart sounds can be acquired from the auscultation unit 12 (step S102). The control unit 25 can determine that the heart sounds have been acquired when the electrical signals of the heart sounds converted into electrical signals by the conversion unit 32 include predetermined sounds, such as the I sound and/or the II sound. The control unit 25 can determine that the heart sound could not be acquired when the predetermined sound signal in the electrical signal of the heart sound cannot be identified because it is buried in noise. If the heartbeat could be acquired (step S102: Yes), the controller 25 proceeds to the next step S104. If the heartbeat could not be acquired (step S102: No), the controller 25 proceeds to step S103.

In step S103, the control unit 25 uses the display unit 23 and/or the speaker 27 to guide the user to correct the position of the auscultation unit 12. The doctor changes the position of the auscultation unit 12 to acquire the heart sound according to the guidance. After step S103, the process returns to step S101, and the doctor again brings the auscultation unit 12 into contact with the user's chest. In this procedure, it has been described that the doctor determines the position of the auscultation unit 12 according to guidance from the control unit 25, but the present invention is not limited to this, and the doctor may determine the position for acquiring heart sounds by auscultation.

In step S102, when the auscultation unit 12 can acquire heart sounds, the doctor puts the auscultation unit 12 against the user's chest, and the control unit 25 of the main unit 11 controls the imaging unit 21 to detect the upper body of the user. Then, the auscultation unit 12 including the marker 36 is imaged (step S104).

When the imaging unit 21 images the user's upper body and the auscultation unit 12, the control unit 25 attempts to recognize the user's face and skeleton in the captured image (step S105). When the user's face and skeleton cannot be recognized in the captured image (step S105: No), the control unit 25 proceeds to step S106.

In step S106, the control unit 25 adjusts the orientation of the imaging unit 21 so that the user's image is included in the captured image. The control section 25 may control the imaging adjustment section 24 to automatically adjust the orientation of the imaging section 21 . If the orientation of the imaging unit 21 cannot be adjusted within the adjustable range of the imaging adjustment unit 24, the control unit 25 controls the display unit 23 and the speaker 27 to move the main unit 11 to adjust the orientation of the imaging unit 21. Advise the physician to change The doctor adjusts the orientation of the imaging unit 21 accordingly. After adjusting the orientation of the imaging unit 21, the control unit 25 returns to the process of step S104.

In step S105, if the user's face, skeleton, etc. can be recognized (step S105: No), the control unit 25 extracts the user's feature points from the image captured by the imaging unit 21 (step S107). The feature points include, for example, the left shoulder 41 and right shoulder 42, and the eyes, nose, mouth, ears, and chin included in the face 43, as shown in FIG. Left shoulder 41 and right shoulder 42 may be, for example, near the shoulder joint or acromion. These feature points can be distinguished to some extent even if the user is wearing clothes. Feature points may also include the user's clavicle and/or nipple. In order to recognize these feature points, the user needs to be imaged with the upper half of the body naked. In the following, feature points of the left shoulder 41, the right shoulder 42, and the face 43, which are recognizable to some extent even when the user is wearing clothes, are used.

After extracting the feature points in step S107, the control unit 25 determines two axes based on the feature points (step S108). The control unit 25 determines, for example, the median line and the horizontal line as two axes. The median line is a line connecting the center point of a set of symmetrically positioned feature points such as the eyes and the mouth or nose. The midline can be the y-axis. The horizontal line is, for example, a line connecting symmetrically positioned feature points such as the left shoulder 41 and the right shoulder 42 . The horizontal line can be the x-axis perpendicular to the y-axis.

When the two axes are determined, the control unit 25 determines the position of the auscultation unit 12 on the image captured by the imaging unit 21 (step S109). The position of the auscultatory portion 12 is determined by the position of the marker 36 on the image. The position of the auscultation unit 12 can be expressed as coordinates on the x-axis and y-axis determined in step S108. That is, the position of the auscultation unit 12 is expressed as relative position information indicating the relative position with respect to the feature point on the image.

When the position of the auscultation unit 12 is determined in step S109, the control unit 25 stores the relative position information of the auscultation unit 12 in the storage unit 22 and/or transmits it to the server 18 (step S110). When the user uses the heart sound acquisition device 15 at a medical institution to set the position of the auscultation unit 12 at home, the control unit 25 stores the relative position information of the auscultation unit 12 in the storage unit 22. You can remember. When the user uses at home a heart sound acquisition device 15 different from the heart sound acquisition device 15 used at a medical institution, the control unit 25 determines the position where the auscultation unit 12 should be placed so that the user can read from home. The relative position information is stored in server 18 . The relative position information of the position where the auscultation unit 12 should be placed stored in the server 18 is acquired from the server 18 by the control unit 25 via the communication unit 26 before the user uses the heart sound acquisition device 15 at home. It is stored in the storage unit 22 .

In the above explanation, it is assumed that a doctor uses a device similar to the heart sound acquisition device 15 used by a patient at home at a medical institution. However, in a medical institution, a dedicated device different from the device used by the patient at home may be used to specify the position where the auscultation unit 12 should be placed.

(Calculation processing of hemodynamic parameters at home)
Next, processing for calculating hemodynamic parameters performed by a user who is a patient at home will be described with reference to the flowcharts of FIGS. 9 and 10. FIG.

First, the user wears the electrodes 13 for acquiring an electrocardiogram and the arm band device 14 for acquiring a pulse wave (step S201).

Next, the user positions the auscultation unit 12 to acquire heart sounds (step S202). Details of the positioning of the auscultatory portion 12 are described below with reference to FIG. The procedure shown in the flowchart of FIG. 10 corresponds to the heart sound acquisition method of the present disclosure.

The user places the main unit 11 on a desk or the like so that the imaging unit 21 of the heart sound acquisition device 15 is positioned and oriented appropriately for imaging the user's upper body. When the imaging section 21 is independent of the main body section 11, the user arranges the imaging section 21 at an appropriate position. In this state, the user takes an image of his or her upper body using the imaging unit 21 (step S301). The control section 25 of the main body section 11 displays the image captured by the imaging section 21 on the display section 23 . The image displayed on the display unit 23 can be a left-right reversed image so that it looks like a mirror image to the user. The imaging section 21 of the main body section 11 may continuously capture images of the user. The following processing may be performed on successively captured images.

The control unit 25 of the main unit 11 attempts to recognize the user's face and skeleton in the image captured by the imaging unit 21 (step S302). When the user's face and skeleton can be recognized in the captured image (step S302: Yes), the control unit 25 proceeds to the next step S304. When the user's face and skeleton cannot be recognized in the captured image (step S302: No), it is conceivable that the range captured by the imaging unit 21 is not appropriate. In this case, the controller 25 proceeds to step S303.

In step S303, the control unit 25 adjusts the position and orientation of the imaging unit 21. The control section 25 may control the imaging adjustment section 24 to automatically adjust the orientation of the imaging section 21 . When the range captured by the imaging unit 21 cannot be adjusted to an appropriate range, the control unit 25 controls the display unit 23 and/or the speaker 27 to move the main unit 11 to change the direction of the imaging unit 21. to the user. The user adjusts the position and orientation of the imaging unit 21 accordingly. After step S303, the process of the flowchart returns to step S301.

In step S<b>304 , the control unit 25 attempts to display the position where the auscultation unit 12 should be placed so as to be superimposed on the image of the user's upper body displayed on the display unit 23 . Specifically, the control unit 25 extracts feature points included in the image captured by the imaging unit 21 from the user's face and skeleton recognized in step S302. Based on the feature points, the control unit 25 identifies the axes corresponding to the two axes identified in step S107. Based on the relative position information stored in the storage unit 22, the control unit 25 can specify the position where the auscultation unit 12 should be arranged as coordinates on these two axes.

For example, as shown in FIG. 11, the position where the auscultation unit 12 should be placed is displayed as the target position P on the image. The target position P may be displayed on the image with any mark. Since the image of the user's upper body displayed on the display unit 23 is a left-right reversed image, the target position P corresponding to the position where the auscultation unit 12 should be arranged, which is set in advance by the medical institution, is left-right reversed. is displayed at the position where the

If the target position P at which the auscultation unit 12 should be placed can be superimposed on the image displayed on the display unit 23 (step S304: Yes), the control unit 25 proceeds to the next step S305. If the target position P where the auscultation unit 12 should be placed cannot be superimposed on the image displayed on the display unit 23 (step S304: No), it is considered that the range captured by the imaging unit 21 is not appropriate. be done. In this case, the control unit 25 proceeds to step S303 described above and adjusts the position and orientation of the imaging unit 21 .

In step S304, the distance from the user's body to the imaging unit 21 and the orientation (inclination) of the user's body with respect to the imaging unit 21 may differ from the conditions when the image was captured at the medical institution. Based on the arrangement of the feature points included in the image captured by the imaging unit 21, the control unit 25 converts the coordinates of the auscultation unit 12 in the image captured by the user at the medical institution to the coordinates of the image captured at home. Coordinates can be transformed. As another method, the control unit 25 uses the display unit 23 and/or the speaker 27 so that the position, orientation, and size of the user's body substantially match those of the images taken at the medical institution. , may guide the user to change position and orientation.

In step S305, while viewing the image displayed on the display unit 23, the user brings the auscultation unit 12 into contact with the position of the user's own chest corresponding to the position where the target position P is displayed on the display unit 23 (step S305). S305). At this time, the control unit 25 uses the display on the display unit 23 and/or the sound from the speaker 27 to instruct the user to bring the auscultation unit 12 into contact with the target position P displayed on the display unit 23. can be encouraged.

When the user brings the auscultation unit 12 into contact with the chest in step S305, the control unit 25 detects the image of the marker 36 included in the image captured by the imaging unit 21 and attempts to recognize the position of the auscultation unit 12. (Step S306). If the auscultation unit 12 is recognized (step S306: Yes), the control unit 25 proceeds to the process of step S308. If the auscultation unit 12 cannot be recognized (step S306: No), the control unit 25 proceeds to step S307. In this case, it is considered that the reason why the auscultation unit 12 cannot be recognized is that it is not within the recognizable range on the image.

In step S307, the control unit 25 causes the display unit 23 to display characters and/or the speaker 27 to generate sound, thereby instructing the user to change the position of the auscultation unit 12 because it is not in the correct position. guide you to do so. According to this guidance, the user changes the position of the auscultation unit 12 placed on the chest. After step S307, the process of the flowchart returns to step S305.

In step S308, the control unit 25 guides the movement direction and movement distance of the auscultation unit 12 from the coordinates of the target position P and the coordinates of the current position of the auscultation unit 12 included in the image captured by the imaging unit 21. do. The control unit 25 guides the movement direction and distance by superimposing graphics and/or characters on the image displayed on the display unit 23, and also by sound from the speaker 27 and vibration from the vibration unit 34. can do. For example, the control unit 25 may display "3 cm downward" on the image displayed on the display unit 23. FIG. Also, the control unit 25 may utter the same content by voice. Further, for example, when the auscultation unit 12 approaches the target position P, the control unit 25 instructs the user to perform auscultation by blinking the color of the mark indicating the target position P, shortening the interval between sounds and vibrations, and the like. It is possible to guide the direction in which the unit 12 is moved.

When the difference between the coordinates of the target position P and the coordinates of the current position of the auscultation unit 12 is equal to or less than a predetermined value, the control unit 25 changes the current position of the auscultation unit 12 to the target position P (positioning the auscultation unit 12). position). When the control unit 25 determines that the current position of the auscultation unit 12 matches the target position P (step S309: Yes), the process proceeds to step S310. When the control unit 25 determines that the current position of the auscultation unit 12 does not match the target position P (step S309: No), the control unit 25 returns to step S308 and adjusts the position of the auscultation unit 12. FIG.

At step S310, the control unit 25 uses the display unit 23, the speaker 27 and/or the vibration unit 34 to notify the user that the auscultation unit 12 is at the target position P (step S310). For example, the control unit 25 changes the color of the mark indicated by the target position P displayed on the display unit 23, and/or generates a voice saying "correct position set" through the speaker 27, and / Or vibrate the vibrating section 34 . In this embodiment, the display unit 23, the speaker 27 and/or the vibration unit 34 are included in the notification unit that notifies that the auscultation unit 12 is positioned at the position where it should be placed.

When it is notified in step S310 that the auscultation unit 12 is at the target position P, the user fixes the auscultation unit 12 at that position (step S311). That is, the user does not move the hand holding the handle 37 of the auscultation unit 12 at this position.

When the position of the auscultation unit 12 is fixed, the control unit 25 acquires the electrical signal of the heart sound from the auscultation unit 12 . The control unit 25 determines whether the heart sounds are correctly acquired from the waveform of the electrical signal of the heart sounds (step S312). The control unit 25 can determine whether the contact state of the auscultation unit 12 with the user's body is good or bad based on the waveform of the heart sound converted into the electrical signal. The control unit 25 may acquire the pressure detected by the pressure sensor 33 in addition to the waveform of the heart sound converted into the electrical signal, and determine the contact state of the auscultation unit 12 with the user's body.

If the waveform of the heart sound is not correctly acquired (step 312: No), the control unit 25 causes the display unit 23 and the speaker 27 to finely adjust the pressing force of the auscultation unit 12 and/or the position of the auscultation unit 12. Guidance is provided (step S313). If the pressure of the auscultation unit 12 against the user's chest is insufficient, the control unit 25 may notify the user to increase the pressure. Also, if the pressure of the auscultation unit 12 against the chest of the user is sufficient, the auscultation unit 12 may be guided to move a minute distance from the current position. After step S313, the user fixes the auscultation unit 12 again (step S311), and the control unit 25 performs the process of step S312 again.

When the waveform of the heart sound is correctly acquired in step S312 (step S312: Yes), the control unit 25 returns to the processing of the flowchart in FIG.

In the flowchart of FIG. 9, when the auscultation unit 12, the electrodes 13, and the arm band device 14 are properly attached or fixed in steps S201 and S202, the control unit 25 starts automatic measurement of heart sounds, electrocardiograms, and pulse waves. (Step S203). A heart sound to be measured is a waveform converted into an electrical signal by the converter 32 .

The control unit 25 analyzes the heart sound, electrocardiogram, and pulse wave waveforms measured in step S203 (step S204). The control unit 25 can time-synchronize each waveform of the heart sound, the electrocardiogram, and the pulse wave. The control unit 25 can extract feature amounts from each waveform.

The control unit 25 calculates hemodynamic parameters based on the analysis results of each waveform in step S204. The control unit 25 may calculate the hemodynamic parameters by machine learning in which the feature values of each waveform are used as input parameters and the hemodynamic parameters are output as described above. Hemodynamic parameters include left ventricular pressure and pulmonary artery pressure.

The control unit 25 determines whether the hemodynamic parameters calculated in step S205 are normal (step S206). For example, a hemodynamic parameter is determined to be abnormal if the hemodynamic parameter contains abnormal values that cannot be measured by normal human measurements. Alternatively, when the hemodynamic parameters cannot be calculated, such as when each measured waveform is distorted and the feature amount cannot be calculated, it is determined to be abnormal. If it is determined that the hemodynamic parameter is not normal (step S206: No), the control unit 25 returns to step S203 and repeats the measurement. If the hemodynamic parameters are determined to be normal (step S206: Yes), the control unit 25 proceeds to the process of step S207.

In step S<b>207 , the control unit 25 displays the measurement results including the hemodynamic parameters on the display unit 23 and/or stores them in the storage unit 22 and/or transmits them to the medical institution system 16 . The control unit 25 may cause the display device 23 to display the waveform of the physical information and/or the time-series transition of the hemodynamic parameter as a graph. The control unit 23 may further cause the display device 23 to display the determination result as to whether the hemodynamic parameters are normal. From the display on the display device 23, the user can know information such as the change in the measurement result from the previous day, the trend, and whether or not the measurement result is within the reference value. The control unit 25 can further present a response such as "continue measurement" or "contact a doctor urgently" to the user. In a medical institution, doctors and/or nurses can continuously monitor the received hemodynamic parameters to check the status of the user, change prescriptions, and the like. After confirming the hemodynamic parameters, the doctor or nurse at the medical institution can send comments or instructions from the medical institution system 16 to the main unit 11 . The control unit 25 of the main unit 11 may display the comment or instruction from the doctor or nurse on the display unit 23 to notify the user.

　In the flowchart of FIG. 9, the processing from steps S204 to S207 is assumed to be executed by the control unit 25 of the main unit 11. However, the hemodynamic monitoring system 10 is configured such that the main unit 11 transmits the results measured in step S203 to the medical institution system 16, and the medical institution system 16 analyzes the measurement results and calculates hemodynamic parameters. may be

As described above, according to the heart sound acquisition device 15 of the present embodiment, the position where the auscultation unit 12 should be placed is determined in advance by the doctor, and this position is used as a relative position with respect to the feature points of the user's body. 22. Then, the control unit 25 arranges the auscultation unit 12 on the image displayed on the display unit 23 based on the user's feature points included in the captured image and the stored relative positions. Guide the user to the location. This allows the user to easily position the auscultation unit 12 and acquire heart sounds.

Further, according to the heart sound acquisition device 15 according to the present embodiment, the patient himself/herself can wear the auscultation unit 12 at the same position every time and acquire a stable heart sound waveform. This eliminates the need for support from doctors, family members, caregivers, etc., and makes it easier to continue measurement even when family members, caregivers, etc. are absent. Furthermore, since the cardiac waveform can be stably acquired, the accuracy of the algorithm for analyzing the cardiac waveform can be improved.

In addition, the heart sound acquisition device 15 of the present embodiment displays an image captured by the imaging unit 21 on the display unit 23, and guides the user on the position where the auscultation unit 12 should be placed on the image. Intuitive and easy to understand. Therefore, the heart sound acquisition device 15 is easy for the patient to use and to continue the measurement.

Furthermore, in this embodiment, as an example, the left shoulder, right shoulder, and face are used as feature points of the user's body used for positioning the auscultation unit 12 . This makes it possible to extract feature points and acquire heart sounds even when the user is wearing clothes.

In addition, in the hemodynamic monitoring system 10 of the present embodiment, the electrocardiogram and the pulse wave, as well as the cardiac sound waveform can be synchronously and stably acquired. It becomes possible to monitor in a non-invasive way. This makes it possible to grasp the state of a heart failure patient at home and to respond when the state changes.

In the above embodiment, a case has been described in which the heart sound acquisition device 15 is used to monitor the hemodynamic parameters of the user, who is a patient, at home. In another embodiment, the heart sound acquisition device can be installed in a medical facility to acquire heart sounds under the same conditions each time for each patient. In this case, the storage unit 22 stores relative position information regarding the position where the auscultation unit 12 should be placed for each patient. The control unit 25 reads the relative position information according to the patient from the storage unit 22 and positions the auscultation unit 12 with respect to each patient. Further, in this case, the display unit 23 may be configured to display an image that is not horizontally reversed toward the doctor side.

Also, in one embodiment, an external camera can be used as the imaging unit 21 instead of the camera built into the main unit 11 . For example, it is possible to connect a smartphone to the main unit 11 and use the camera of the smartphone as the imaging unit 21 . In this case, there are advantages such as no need for a camera in the main body 11, easy adjustment of the orientation of the camera since it is separated from the main body 11, and the ability to use two screens of the main body 11 and the smartphone.

Also, in one embodiment, a smartphone can be used as the main unit 11. In this case, a camera of a smart phone is used as the imaging unit 21 in FIG. A built-in memory of the smartphone is used as the storage unit 22 . A smartphone display is used as the display unit 23 . A processor of a smartphone is used as the control unit 25 . A wireless communication function of a smartphone is used as the communication unit 26 . A speaker built into the smartphone is used as the speaker 27 . Further, as the imaging adjustment unit 24, a dedicated stand capable of adjusting the orientation of the display of the smartphone may be prepared in cooperation with the smartphone. An external connection terminal of the smartphone can be used to connect the auscultation unit 12 and other sensors that acquire body information.

The smartphone stores in its memory an application for causing the processor of the smartphone to function as the control unit 25 of the heart sound acquisition device 15 of the present disclosure. When using a smart phone as the heart sound acquisition device 15, the user activates this application. The following operations can be performed in the same manner as the heart sound acquisition device 15 of the above embodiment. This makes it possible to acquire heart sounds without preparing special hardware.

Also, in one embodiment, the imaging unit 21 may include an infrared camera capable of imaging light with wavelengths in the infrared region. Infrared radiation can partially penetrate clothing. Since the imaging unit 21 includes an infrared camera, it is possible to capture an image of the user's body even if the user is wearing clothes, and extract feature points more accurately. As a result, even if the feature points used for positioning the auscultation unit 12 include a part that is hidden under the clothes, the user can position the auscultation unit 12 while wearing the clothes. become.

Although the embodiments according to the present disclosure have been described based on the drawings and examples, it should be noted that a person skilled in the art can easily make various modifications or modifications based on the present disclosure. Therefore, it should be noted that these variations or modifications are included within the scope of this disclosure. For example, functions included in each component or each step can be rearranged so as not to be logically inconsistent, and multiple components or steps can be combined into one or divided. is. Although the embodiments of the present disclosure have been described with a focus on the apparatus, the embodiments of the present disclosure may also be implemented as a method including steps performed by each component of the apparatus. Embodiments according to the present disclosure can also be implemented as a non-transitory computer-readable medium recording a method, program, or program executed by a processor provided in an apparatus. It should be understood that these are also included within the scope of the present disclosure.

10 hemodynamic monitoring system 11 main unit 12 auscultation unit 13 electrode (physical information acquisition unit)
14 arm band device (physical information acquisition unit)
15 heart sound acquisition device 16 medical institution system 17 communication network 18 server 21 imaging unit 22 storage unit 23 display unit 24 imaging adjustment unit 25 control unit 26 communication unit 27 speaker 28 stand 31 heart sound acquisition unit 32 conversion unit 33 pressure sensor 34 vibration unit 36 Marker 37 Handle 41 Left shoulder (feature point)
42 right shoulder (feature point)
43 face P target position

Claims

an auscultation unit configured to acquire a user's heart sound;
an imaging unit that captures an image of the user's body including feature points;
a storage unit that stores the relative position of the position where the auscultation unit should be arranged with respect to the feature point of the user;
a display unit that displays the image captured by the imaging unit;
The positions of the feature points of the user are recognized from the image captured by the imaging unit, and the auscultation unit is arranged based on the positions of the feature points and the relative positions stored in the storage unit. a control unit that calculates the position where the auscultation unit should be placed and guides the auscultation unit to the position where the auscultation unit should be placed on the image displayed on the display unit;
A heart sound acquisition device comprising:
2. The auscultation unit according to claim 1, wherein the auscultation unit includes a marker, and the control unit recognizes the position of the auscultation unit on the image by detecting the marker from the image captured by the imaging unit. Heart sound acquisition device.
The control unit is configured to be able to recognize at least one of a distance between the imaging unit and the body of the user and an orientation of the body of the user based on the image captured by the imaging unit. 3. The heart sound acquisition device according to claim 1 or 2.
further comprising a communication unit configured to be communicable with the server,
The heart sound acquisition according to any one of claims 1 to 3, wherein the control unit is configured to acquire the relative position stored in the storage unit from the server via the communication unit. Device.
Equipped with additional speakers,
5. The heart sound acquiring apparatus according to claim 1, wherein said control unit uses voice from said speaker when guiding said auscultation unit to a position where said auscultation unit should be placed.
When guiding the auscultation unit to a position where the auscultation unit should be placed, the control unit superimposes at least one of characters and graphics on the image captured by the imaging unit and causes the display unit to display the image. A heart sound acquisition device according to any one of claims 1 to 5.
The auscultation unit includes a pressure sensor that detects pressure between the auscultation unit and the body of the user, and the control unit controls the pressure of the auscultation unit against the body of the user based on the pressure detected by the pressure sensor. 7. The heart sound acquisition device according to any one of claims 1 to 6, which determines a contact state.
The heart sound acquisition device according to any one of claims 1 to 7, further comprising a notification unit that notifies that the auscultation unit is positioned at a position where the auscultation unit should be placed.
The heart sound acquisition device according to any one of claims 1 to 8, comprising a conversion unit that converts the heart sounds acquired by the auscultation unit into electrical signals.
The heart sound acquisition device according to claim 9, wherein the control unit determines a contact state of the auscultation unit with respect to the user's body based on the waveform of the electrical signal obtained by converting the heart sounds by the conversion unit.
The heart sound acquisition device according to claim 9 or 10, wherein the control unit is configured to analyze the waveform of the electrical signal into which the heart sounds have been converted by the conversion unit.
further comprising a physical information acquisition unit configured to be able to acquire physical information other than the heartbeat of the user;
12. The control unit according to any one of claims 9 to 11, wherein the waveform of the electrical signal converted by the conversion unit and the waveform of the physical information acquired by the physical information acquisition unit are temporally synchronized. 1. The heart sound acquisition device according to 1. or 1. above.
The physical information includes at least one of an electrocardiogram and a pulse wave, and the control unit controls the heart sound of the user acquired from the auscultation unit and the physical information of the user acquired from the physical information acquisition unit. 13. The heart sound acquisition device according to claim 12, wherein the hemodynamic parameter is calculated based on.
a server that stores the relative position of the position where the auscultation unit configured to be able to acquire the user's heart sound with respect to the user's characteristic point should be placed;
the auscultation unit, an imaging unit that captures an image of the user's body including characteristic points, a display unit that displays the image captured by the imaging unit, a communication unit that acquires the relative position from the server, and recognizing the position of the feature point of the user from the image captured by the imaging unit, and positioning the auscultation unit based on the position of the feature point and the relative position obtained from the server; , and guides the auscultation unit to a position where the auscultation unit should be placed on the image displayed on the display unit.
A heart sound acquisition method executed by a control unit to acquire heart sounds using an auscultation unit configured to acquire heart sounds of a user, comprising:
causing the imaging unit to capture an image of the user's body including the feature points;
recognizing the position of the feature point of the user from the image captured by the imaging unit;
calculating the position where the auscultation unit should be arranged based on the position of the feature point and the relative position of the position where the auscultation unit should be arranged with respect to the user's feature point stored in a storage unit;
guiding the auscultation unit to a position where the auscultation unit should be placed on the image displayed on the display unit;
Heart sound acquisition method.
A program for acquiring heart sounds using an auscultation unit configured to acquire heart sounds of a user,
a process of causing the imaging unit to capture an image of the user's body including feature points;
a process of recognizing the position of the feature point of the user from the image captured by the imaging unit;
a process of calculating the position where the auscultation unit should be arranged based on the position of the feature point and the relative position of the position where the auscultation unit should be arranged with respect to the user's feature point stored in a storage unit; ,
A program for causing a processor included in a heart sound acquisition device to execute a process of guiding the auscultation unit to a position where the auscultation unit should be placed on the image displayed on the display unit.