JP2022025395A - Biological sound acquisition support system - Google Patents

Abstract

To provide a system for acquiring biological sound by using a portable terminal such as a smartphone.SOLUTION: The present invention relates to a biological sound acquisition support system for supporting acquisition of biological sound by using a user terminal by a user. The biological sound acquisition support system includes: a sound data acquisition unit for acquiring sound data including the biological sound; a determination unit for determining whether a condition of acquiring the biological sound is satisfied; and a notification unit for outputting the result of the determination to the user terminal.SELECTED DRAWING: Figure 1

Description

The present invention relates to a biological sound acquisition support system.

A telemedicine support system has been proposed that enables communication such as a videophone between a doctor terminal and a patient terminal to perform telemedicine.

Patent Document 1 discloses an information processing device that provides a means for making a reservation for telemedicine and a means for video communication between a doctor and a patient.

Japanese Patent No. 6574391

In telemedicine, it is necessary for the patient to acquire various vital data necessary for the doctor's examination and provide it to the doctor's terminal. In addition, due to the growing health consciousness in recent years, the demand for daily health management is also increasing.

Therefore, one object of the present invention is to provide a system for acquiring a biological sound using a mobile terminal such as a smartphone.

According to the present invention, it is a biological sound acquisition support system that assists the user himself / herself to acquire a biological sound using a user terminal, and has a sound data acquisition unit that acquires sound data including the biological sound, and the acquired sound data acquisition unit. A sound evaluation unit that evaluates a specific sound component from sound data, a condition storage unit that stores conditions for determining whether or not biological sound can be acquired, and a determination unit that determines whether or not the conditions are met. , A biological sound acquisition support system including a notification unit that outputs the determination result to the user terminal can be obtained.

According to the present invention, a user can smoothly acquire a biological sound using a device such as a smartphone.

It is a figure which shows the structural example of the system by embodiment of this invention. It is a figure which shows the hardware configuration example of the user terminal by embodiment of this invention. It is a figure which shows the hardware configuration example of the server by embodiment of this invention. It is a figure which shows the software configuration example by embodiment of this invention. It is a figure which shows the structural example of the conditional storage part by embodiment of this invention. It is a figure which shows the structural example of the notification information storage part by embodiment of this invention. It is a figure which shows an example of the processing flow by embodiment of this invention. It is a figure which shows the software configuration example by embodiment of this invention. It is a figure explaining the embodiment of this invention. It is a figure which shows the structural example of the conditional storage part by embodiment of this invention. It is a figure explaining the embodiment of this invention. It is a figure which shows the structural example of the notification information storage part by embodiment of this invention. It is a figure which shows the software configuration example by embodiment of this invention. It is a figure explaining the embodiment of this invention. It is a figure which shows the structural example of the notification information storage part by embodiment of this invention. It is a figure which shows the structural example of the notification information storage part by embodiment of this invention. It is a figure which shows the software configuration example by embodiment of this invention. It is a figure which shows the software configuration example by embodiment of this invention. It is a figure explaining the embodiment of this invention. It is a figure which shows the software configuration example by embodiment of this invention. It is a figure which shows the structural example of the acquisition instruction information by embodiment of this invention.

The contents of the embodiments of the present invention will be described in a list. The present invention has the following configurations.
[Item 1]
It is a biological sound acquisition support system that assists the user himself to acquire biological sound using a user terminal.
A sound data acquisition unit that acquires sound data including biological sounds,
A determination unit for determining whether or not the condition for acquiring biological sound is satisfied, and a determination unit.
A biological sound acquisition support system including a notification unit that outputs the determination result to the user terminal.
[Item 2]
It is further equipped with a sound evaluation unit that evaluates a specific sound component from the acquired sound data.
The biological sound acquisition support system according to item 1, wherein the determination unit determines whether or not the biological sound acquisition condition is satisfied based on the evaluation result.
[Item 3]
The sound evaluation unit evaluates the volume of the target biological sound component and evaluates the volume.
The acquisition condition includes a condition relating to the volume of the biological sound component, and includes the condition.
The biological sound acquisition support system according to item 2, wherein the determination unit determines whether or not the volume of the biological sound component satisfies the acquisition condition.
[Item 4]
The sound evaluation unit evaluates the volume of the noise component and evaluates the volume.
The acquisition condition includes a condition relating to the volume of the noise component.
The biological sound acquisition support system according to item 2 or 3, wherein the determination unit determines whether or not the volume of the noise component satisfies the acquisition condition.
[Item 5]
Further, an angle data acquisition unit for acquiring the angle data of the user terminal is provided.
The acquisition condition includes an appropriate angle condition of the user terminal at the time of acquisition of the biological sound.
The biological sound acquisition support system according to any one of items 1 to 4, wherein the determination unit determines whether or not the angle data matches the angle condition.
[Item 6]
Further, a position calculation unit for calculating terminal position information accompanying the movement of the user terminal is provided.
The acquisition condition includes the acquisition position information of the biological sound, and includes the acquisition position information.
The biological sound acquisition support system according to any one of items 1 to 5, wherein the determination unit determines whether or not the terminal position information matches the acquisition position information.
[Item 7]
Further provided with an instruction information input unit that accepts input of instruction information regarding the acquisition position of the user's biological sound by an expert.
The biological sound acquisition support system according to item 6, wherein the acquisition position information is generated based on the instruction information.
[Item 8]
A body shape estimation unit that estimates the user's body shape based on the user's input information,
The biological sound acquisition support system according to item 6, further comprising an acquisition instruction information adjusting unit that adjusts the acquisition position information according to the body shape.

<Details of the embodiment>
Hereinafter, embodiments of the present invention will be described with reference to the drawings.

<Overview>
FIG. 1 is a diagram illustrating an outline of a biological sound acquisition system according to an embodiment of the present invention. As shown in the figure, the biological sound acquisition system includes a user terminal 2 and a server 1 to which the user terminal 2 is connected via an Internet network which is a network. Further, if necessary, a doctor terminal 3 connected to the network may be provided. The user terminal 2 and the doctor terminal 3 can be connected to the terminals of a plurality of users.

The biological sound acquisition system according to the embodiment of the present invention may provide the biological sound acquired by the user terminal 2 to the doctor terminal 3 by linking with the remote medical care system, or the biological sound for the user's own health management. Is used in the case of acquiring.

In the present embodiment, the user terminal 2 is a device that can be connected to an Internet network such as a smartphone, a computer, or a tablet terminal, which is a portable information terminal. In particular, the user terminal 2 is preferably a smartphone equipped with a microphone for acquiring voice and various sensors capable of measuring acceleration and terminal angle.

FIG. 2 is a diagram illustrating an outline of the hardware configuration of the user terminal 2. As shown in the figure, the user terminal 2 includes a control unit 10, a memory 11, a storage 12, a transmission / output unit 13, an input / output unit 14, a sensor 16, a microphone 17, and a display unit 18 as main configurations, and these have a bus 15 to each other. It is electrically connected via.

The control unit 10 is an arithmetic unit that controls the operation of the user terminal 2, controls the transmission / reception of data between each element, and performs processing necessary for executing an application program.

In the present embodiment, the control unit 10 is, for example, a CPU (Central Process).
It is a ssing Unit), and an application program stored in the storage 12 described later and expanded in the memory 11 is executed to perform each process.

The memory 11 includes a main storage device composed of a volatile storage device such as a DRAM (Dynamic Random Access Memory), and an auxiliary storage device composed of a non-volatile storage device such as a flash memory or an HDD (Hard Disk Drive). ..

While this memory 11 is used as a work area of the control unit 10, the BIOS (Basic Input / Output System) executed when the computer is started is used.
m) and various setting information are stored.

The storage 12 stores programs, data used for various processes, and the like.
Details of the programs and the like stored in the storage 12 will be described later. The storage 12 may be connected so as to be available using cloud computing technology.

The transmission / reception unit 13 connects the user terminal 2 to the Internet network. The transmission / reception unit 13 may be provided with a short-range communication interface such as Bluetooth (registered trademark) or BLE (Bluetooth Low Energy).

If necessary, an information input device such as a keyboard and a mouse and an output device such as a display are connected to the input / output unit 14.

The bus 15 transmits, for example, an address signal, a data signal, and various control signals between the connected control unit 10, memory 11, storage 12, transmission / reception unit 13, and input / output unit 14.

The sensor 16 may be various sensors such as an acceleration sensor, a tilt sensor (geomagnetic sensor or gyro sensor), and a proximity sensor.

The microphone 17 is a sound collecting device mounted on the user terminal 2. The microphone 17 converts the voice input to the terminal 2 into a voice signal. Further, a sound collector externally connected to the user terminal 2 may be used.

FIG. 3 is a diagram illustrating an outline of the hardware configuration of the server 1. As shown in the figure, the server 1 includes a control unit 20, a memory 21, a storage 22, a transmission / output unit 23, and an input / output unit 24 as a main configuration, and these are electrically connected to each other via a bus 25. Since the functions of each are the same as those described in the user terminal 2, they will be omitted.

<Embodiment 1>
In the first embodiment, it is determined whether or not the sound data acquired by using the microphone 17 is suitable for acquiring the biological sound. Specifically, by analyzing the acquired sound data, it is determined that the biological sound data included in the sound data can be acquired with sufficient quality.

FIG. 4 is a diagram showing a software configuration example of the user terminal 2 according to the first embodiment of the present invention. The user terminal 2 of the present invention has a condition storage unit 121 as a storage device, a notification information storage unit 122, a sound data storage unit 123, and as an arithmetic processing device, a data transmission / reception unit 111, a sound data acquisition unit 112, a sound evaluation unit 113, and a determination. A unit 114, a display control unit 115, and a notification unit 116 are provided. The data transmission / reception unit 111, the sound data acquisition unit 112, the sound evaluation unit 113, the determination unit 114, the display control unit 115, and the notification unit 116 read the program stored in the storage 12 by the control unit 10 into the memory 11. It is realized by execution, and the condition storage unit 121, the notification information storage unit 122, and the sound data storage unit 123 are realized as a part of the storage area provided by at least one of the memory 11 and the storage 12.

The condition storage unit 121 stores the conditions for determining whether or not it is suitable for acquiring the biological sound. FIG. 5 is a configuration example of condition information stored in the condition storage unit 121. In the first embodiment, the conditions regarding the quality of the biological sound included in the acquired sound data are defined. The quality is, for example, a volume, and may be a threshold value expressed by an index such as a decibel value, but the quality is not limited to this. All of these conditions include the type of biological sound you want to acquire (heart sounds, breath sounds, lung sounds, peristaltic sounds, swallowing sounds, etc.), the acquisition site (right chest, left chest, stomach, intestine, etc.), and the purpose of acquisition (to the doctor). Multiple settings can be made according to the provision of, self-health management), etc.

The notification information storage unit 122 stores the content of the notification output by the notification unit 116 according to the determination result. FIG. 6 is a configuration example of the notification information stored in the notification information storage unit 122. The notification information includes the setting of the notification content for each determination result. For example, the notification to be issued is set depending on whether the determination result is OK (suitable for acquiring biological sound) or NG (not suitable). The sound notifications 1 and 2 may be, for example, a voice message such as notification 1 "can acquire biological sound" and voice notification 2 such as sound notification 2 "cannot acquire biological sound". The notification by sound is not limited to voice, and may be a predetermined notification sound. The notification sound can be expressed by the pitch and pattern (piro-piro, boo-boo, etc.) of the sound. Further, the notification is not limited to the sound notification by voice or notification sound, but may be light, vibration, or the like, and may be a combination thereof. The light notification is output by the screen of the terminal 2 or the light emitting function, and can be expressed by the color of the light, the lighting / blinking pattern, the intensity of the light, and the like. Further, the vibration notification is output by the vibration function of the terminal, and can be expressed by the vibration pattern, strength, and the like. The notification information storage unit 122 can store the contents of these sound notifications, optical notifications, and vibration notifications according to each determination result. The notification information storage unit 122 may store the voice data itself or may store output instruction information of various notifications.

The sound data storage unit 123 stores the acquired sound data. The sound data can be stored together with information such as the acquired date and time, the type of acquired biological sound (heart sound, breath sound, lung sound, peristaltic sound, etc.), acquisition site (precordium, neck, back, etc.). The acquisition site may include a more detailed acquisition position, and may be position information represented by coordinate information from a reference position as described below, for example. Further, it may include information regarding the acquisition purpose such as for medical examination and self-management. Further, if necessary, it may include information indicating the state of the user when the sound data is acquired. For example, even if it includes subjective symptoms such as breathlessness and dullness, image data showing the user's face, body, and affected area, and vital data such as pulse, blood pressure, and body temperature acquired by various measuring devices such as wearable devices. good. Further, the sound data to be stored may include waveform data, or may be data processed by the sound evaluation unit 113 described below.

The data transmission / reception unit 111 uses the transmission / reception unit 13 to transmit various data to the server 1 or the doctor terminal 3.

The sound data acquisition unit 112 acquires sound data collected by the microphone 17 provided in the user terminal 2, stores it in the sound data storage unit 123, and transmits it to the sound evaluation unit 113.

The sound evaluation unit 113 evaluates the magnitude and quality of a specific sound component. The sound evaluation unit 113 can evaluate the magnitude and quality of the target biological sound component. If necessary, the sound evaluation unit 113 performs an extraction process of separating the biological sound component and the noise component other than the biological sound from the sound data by a predetermined process, or extracting the biological sound data by removing the noise portion. .. If the processing method and parameters differ depending on the type and acquisition position of the biological sound, the processing method and parameters are selected based on the input type and acquisition position of the biological sound, and the extraction process is performed. A known method can be adopted as the extraction method. For example, the noise component can be removed by applying a predetermined filter to the voice data. It is also possible to remove noise by converting sound data into a spectrogram and masking specific components. If necessary, the sound data may be subjected to a predetermined preprocessing before the spectrogram conversion. In addition, a learning model is created by using sound data including both the biological sound component and the noise component as input data and biological sound data including only the biological sound component or biological sound data including the noise component as teacher data. The biological sound data may be extracted by applying the sound data to. The input data of the training model may be a spectrogram obtained by converting sound data as long as it can be restored to waveform data. Similarly, the output data may be a spectrogram that can be restored to waveform data. Further, the noise component may be extracted from the sound data including the biological sound data and the noise component by the same method, and the other components may be extracted as the biological sound data.

The determination unit 114 determines whether the situation is suitable for acquiring the biological sound based on the evaluation result of the sound evaluation unit 113. That is, it is determined that the target biological sound is sufficiently acquired.

The notification unit 116 notifies the result of the determination. The notification unit 116 may, for example, emit any one of sound, light, and vibration, or a combination thereof, and commands a sound generating means, a light generating means, a vibration generating means, and the like mounted on the user terminal 2. Make a notification. According to the determination result of the determination unit 114, the notification information registered in the notification information storage unit 122 is read and the notification is output.

Further, the notification may be performed by displaying a predetermined message or image on the screen of the user terminal 2. In that case, the notification unit 116 can control the display control unit 115 so that a predetermined image or message is displayed on the screen based on the result of the determination unit 114, so that the image or message can be displayed on the user terminal 2. When the user terminal 2 is placed on the user's body, the user may not be able to see the terminal screen by himself / herself. Notification is preferred.

From the above, in the first embodiment, the biological sound data in the acquired sound data is evaluated, and it is determined whether or not sufficient biological sound data can be acquired. In the above, the biological sound data included in the sound data is evaluated, but the noise component may be evaluated and, for example, it may be determined that the magnitude of the noise component is not more than a predetermined value. In that case, the user acquires the sound by the microphone 17 of the user terminal 2 in the environment where the biological sound is to be acquired. At this time, the microphone 17 may be abutted against the body to acquire the sound as in the actual acquisition of the biological sound, but the sound may be acquired without necessarily abutting the body. The sound evaluation unit 113 evaluates the magnitude and quality of the noise component from the acquired sound, and the determination unit 114 determines whether or not the noise component satisfies a predetermined condition. A predetermined condition is, for example, volume. If the determination result is good (less noise), the notification unit 116 notifies that the biological sound can be acquired, while if the determination result is bad (more noise), that fact is notified. Notify them and encourage them to change the environment, such as moving to another location.

FIG. 7 is an operation flow in the first embodiment. The order of each step may be changed unless otherwise specified.

First, the user starts the application on his / her mobile terminal (user terminal 2) (S101), and in some cases, inputs the type of biological sound to be acquired and the acquisition site. The terminal 2 provides guidance by displaying a guidance screen for acquiring a biological sound on the screen of the terminal 2 or outputting it by voice (S102).

The user brings the microphone 17 of the user terminal 2 into contact with the acquisition portion and starts acquisition of sound data (S103). The sound evaluation unit 113 evaluates a target biological sound component from the acquired sound data. Further, the sound evaluation unit 113 may perform evaluation after extracting a biological sound component from the acquired sound data by a predetermined process. The sound evaluation unit 113 conveys the evaluation result to the determination unit 114. The determination unit 114 determines whether or not the acquired sound is suitable for examination by a doctor or analysis for estimation of a disease (S104). That is, it is determined whether the extracted biological sound data satisfies the criteria stored in the condition storage unit 121. Specifically, for example, it is determined whether the volume of the biological sound data is equal to or higher than a predetermined value.

When it is determined that the determination is NG (sufficient acquisition of biological sound is not possible) (S104 = No), the notification unit 116 notifies the user to re-acquire (S105). Notifications can be made, for example, by voice message or screen display such as "The biological sound could not be obtained. Please try again" or "Please move to a quiet place". When it is determined that the determination is OK (sufficient acquisition of the biological sound) (S104 = Yes), the notification unit 116 notifies that the biological sound has been acquired (S106). Further, the sound data (which may include analysis data such as waveform data) acquired as needed is transmitted to the server (S107).

In the first embodiment, it is premised that the user terminal 2 is provided with an arithmetic processing unit such as a sound evaluation unit 113 and a determination unit 114, and a storage unit such as a condition storage unit 121, a notification information storage unit 122, and a sound data storage unit 123. As described above, these may be provided in the server 1. In that case, the sound data acquired by the sound data acquisition unit 112 may be transmitted to the server 1, the server 1 may analyze and determine the sound, and the determination result may be transmitted to the user terminal 2.

<Embodiment 2>
In the second embodiment, it is determined whether or not the angle of the user terminal 2 is suitable for acquiring the biological sound. That is, when the microphone mounted on the user terminal 2 such as a smartphone is brought into contact with the user's body to acquire the biological sound, it may be easy to acquire the biological sound if the terminal angle with respect to the body is within a certain range. be. Further, in order to ensure the uniformity of the biological sound, it may be preferable to uniformly acquire the terminal angle. In this embodiment, in such a case, the actual angle of the user terminal 2 is grasped by the sensor data, and it is determined whether or not the angle is within the range of the terminal angle registered in advance.

FIG. 8 shows an example of software configuration of the user terminal 2 in the second embodiment, and further includes a sensor input control unit 117 in addition to the software configuration in the first embodiment. Further, the user terminal 2 includes a tilt sensor capable of detecting the angle of the terminal 2. The tilt sensor may be any as long as it can detect the tilt angle in each direction in the user terminal, and is a geomagnetic sensor, a gyro sensor, an acceleration sensor, or the like. As shown in FIG. 9, the tilt angles of the user terminals in each direction are as shown in FIG. 9, with the x-axis for the right side of the screen, the y-axis for the upward direction in the longitudinal direction, and the z-axis for the vertical upward direction on the screen. The x-axis rotation is represented by pitch, the y-axis rotation is represented by azimuth, the z-axis rotation is represented by rоll, and the like.

The condition storage unit 121 in the second embodiment is a site for acquiring a biological sound (right chest, left chest, stomach, intestine, etc.), a disease, or a type of biological sound (heart sound, respiratory sound, lung sound, peristaltic sound, swallowing sound, etc.). Etc.), the appropriate angle range of the user terminal 2 is registered in advance. FIG. 10 is a configuration example of condition information stored in the condition storage unit 121. As an example, when measuring heart sounds, as shown in FIG. 11, the lower surface (microphone portion) of the terminal 2 may be oriented toward the body and the screen orientation (Z axis) of the terminal 2 may be held so as to be horizontal. Suppose it is optimal. The condition in that case is "inclination angle 90 ° and azimuth angle 90 °". As the conditions of implementation, a predetermined allowable range may be appropriately set, and for example, "tilt angle 85 ° to 95 ° and azimuth angle 85 ° to 95 °" may be set. When making a judgment based on the terminal angle, the values and ranges of the tilt angle, the azimuth angle, and the rotation angle are conditions. All of these conditions are the type of biological sound you want to acquire (heart sounds, breath sounds, lung sounds, peristaltic sounds, swallowing sounds, etc.), disease, acquisition site (right chest, left chest, stomach, intestine, etc.), acquisition purpose ( Multiple settings can be made according to the provision to the doctor, self-health management), the number of acquisitions (first time, second time, etc.).

The notification information storage unit 122 of the second embodiment stores the content of the notification output by the notification unit 116 according to the determination result. FIG. 12 is a configuration example of the notification information stored in the notification information storage unit 122. The notification information includes the setting of the notification content for each determination result. The notification is not limited to voice, but may be a notification sound, light, vibration, display, or the like, and may be a combination thereof. When making a determination based on the angle of the user terminal 2, the notification when the determination is OK (the terminal angle is within the specified range) and the notification when the determination is NG are the same as in the first embodiment. You can register. Further, when the determination is NG, further notification may be set according to the magnitude relationship between the measured value of the terminal angle and the threshold value as shown in FIG. For example, if the tilt angle is smaller than the threshold value, it is conceivable to issue a voice message (sound notification 3) such as "Please tilt the terminal a little further". Further, when the angle of the user terminal 2 is within a predetermined range, predetermined sounds, lights, and vibrations may be emitted, and when the angle is out of the range, these outputs may be stopped. Alternatively, conversely, when the angle of the user terminal 2 is outside the predetermined range, predetermined sounds, lights, and vibrations may be emitted, and when the angle is within the range, these outputs may be stopped.

The sensor input control unit 117 transmits the angle data of the user terminal 2 acquired by the tilt sensor mounted on the user terminal 2 to the determination unit 114. The tilt sensor is, for example, a geomagnetic sensor, a gyro sensor, or an acceleration sensor, and the angle of the user terminal 2 can be represented by a tilt angle (Pitch), a rotation angle (rоll), and an azimuth angle (azimuth).

The determination unit 114 reads out the angle condition from the condition storage unit 121 according to the acquisition site and type of the biological sound to be acquired. If the acquisition site and type of the biological sound to be acquired are determined in advance, the corresponding one may be read out, and when the user inputs the acquisition site and type of the biological sound, the conditions are according to the input. Is read. Then, when the angle of the user terminal 2 input from the sensor input control unit 117 corresponds to the condition, it is determined that it is suitable for acquiring the biological sound, and when it does not correspond, it is determined that it is not suitable.

The notification unit 116 notifies the determination result. The notification content is read from the notification information storage unit 122 based on the determination result by the determination unit 114, and the notification is output.

<Embodiment 3>
In the third embodiment, it is determined whether or not the position of the user terminal 2 is suitable for acquiring the biological sound. For the acquisition of biological sound, the place where the microphone should be applied is determined according to the purpose, but when the user acquires the biological sound by himself / herself, it cannot be determined whether the microphone is applied to the appropriate acquisition position. There is a problem. Therefore, in the third embodiment, it is determined whether or not the position of the user terminal 2 is at a predetermined set position.

FIG. 13 shows an example of software configuration of the user terminal 2 in the third embodiment, and in addition to the software configuration in the first embodiment, further includes a sensor input control unit 117 and a position calculation unit 118.

The condition storage unit 121 of the third embodiment stores the conditions for determining whether or not the terminal 2 is arranged at an acquisition position suitable for acquiring the biological sound. For example, the condition is the distance to each measurement position with respect to the reference position and the moving direction. The type of biological sound you want to acquire (heart sounds, breath sounds, lung sounds, peristaltic sounds, swallowing sounds, etc.), disease, acquisition site (right chest, left chest, stomach, intestine, etc.), acquisition purpose (provided to a doctor, self) Multiple settings can be made according to health management). FIG. 14 is a configuration example of the condition information stored in the condition storage unit 121. The acquisition position condition may be image data in which the acquisition position and the order are plotted on the body as shown in FIG. 14A. Further, it may be coordinate information as shown in FIG. 14 (b). As an example, the coordinate information stores the relative coordinates of the second position with respect to the reference position (first position) on the user's body as a condition of the terminal position. As shown in FIG. 14 (a), for example, it is assumed that the first position is the epigastrium and the second position is the first position (on the right clavicle), which is the first biological sound acquisition position. In that case, as the condition information, the coordinate information of the second position with respect to the first position (for example, 〇 to 〇 in the X direction and 〇 to 〇 in the Y direction) can be defined in a predetermined range. Further, when the biological sound is continuously acquired at a plurality of positions, the next acquisition position (third position) can be determined by the coordinate information of the third position with respect to the above-mentioned second position. ..

The notification information storage unit 122 stores the content of the notification output by the notification unit 116 according to the determination result. FIG. 15 is a configuration example of the notification information stored in the notification information storage unit 122. When making a judgment based on the position of the user terminal 2, different sounds (notifications 9 and 10) are output depending on whether the judgment is within the coordinate range of the set acquisition position and the coordinate range of the acquisition position. Can be notified. For example, the terminal 2 may continuously emit a sound so that the sound changes when the coordinate range of the acquisition position is entered (notification 9 → notification 10), or a predetermined sound is specified only when the coordinate range is entered. A sound may be emitted (notification 10 only). Alternatively, different notifications may be issued when the coordinate range of the acquisition position is approaching and when the acquisition position is far away. By performing such a notification, it becomes easy for the user to adjust the position where the terminal 2 is arranged.

The sensor input control unit 117 transmits the information input from each sensor such as the acceleration sensor to the position calculation unit 118 and the determination unit 114.

The position calculation unit 118 calculates the position information of the terminal 2 by using the data obtained from the acceleration sensor mounted on the user terminal 2. The position calculation unit 118 calculates the movement direction and the movement distance from the acceleration data. The travel distance can be calculated by integrating the value of the accelerometer by a known method such as the trapezoidal integration method. As an example, after setting a reference position (first position) on the user's body and placing the user terminal 2 on the first position, the terminal 2 is placed at a second position different from the first position. When moved, the position calculation unit 118 can derive the coordinates of the second position (position relative to the first position) from the moving direction and the moving distance. The method of grasping the position of the terminal 2 is not limited to this, and a known method may be adopted.

The determination unit 114 determines whether or not the user terminal 2 is arranged on the user's body at a position suitable for acquiring the biological sound. As an example, a reference position (first position) in the body is determined. The first position is, for example, a position that is easy for the user to understand and is uniquely determined, and for example, an epigastrium or a clavicle can be specified. First, the user terminal 2 is instructed to be positioned at the first position, and the terminal 2 is once stopped for a predetermined time. Subsequently, the user terminal 2 is moved from the first position to the acquisition position (second position) of the biological sound, and the acceleration data during that time is acquired. The position calculation unit 118 calculates the movement direction and the movement distance of the terminal 2 from the acceleration data, and calculates the coordinates of the user terminal 2 with respect to the first position.

The notification unit 116 notifies the determination result. For example, when the position of the user terminal 2 is within the range of the acquisition position (second position) of the biological sound stored in the condition storage unit 121, a predetermined sound, light, vibration, or the like can be emitted, and the user can emit the predetermined sound, light, vibration, or the like. When the terminal 2 is moved out of the range of the acquisition position, the above-mentioned predetermined sound, light, and vibration may be stopped. Or vice versa, when the position of the terminal 2 is out of the range of the acquisition position, it emits a sound, light, or vibration to indicate that the position is inappropriate, and when it moves to an appropriate position, the sound is emitted. , Light and vibration may be stopped. Further, when the user terminal 2 is being moved, the first notification is issued when the position of the terminal 2 is approaching the range of the acquisition position of the biological sound, and conversely, the second notification is issued when the position of the terminal 2 is far away. You may issue a notification. The first notification is, for example, a notification prompting the user to continue moving, and the second notification is a notification prompting the user to correct the direction. The first notification and the second notification may be a voice message, or may be sound, light, or vibration. Further, only one of the first notification and the second notification may be given. For example, the second notification may be given only when it is necessary to prompt the correction of the moving direction without giving the first notification.

<Embodiment 4>
In the fourth embodiment, it is determined that the user terminal 2 is in a state where the loudest biological sound can be acquired. The state of the user terminal 2 is, for example, a state relating to an angle or a position with respect to the user's body. In the above embodiments 2 and 3, it is determined whether the angle or position of the user terminal 2 is within a predetermined range, but in the present embodiment 4, the angle or position is such that the biological sound can be satisfactorily acquired. Is relatively determined.

The software configuration example of the user terminal 2 in the fourth embodiment is the same as that of the fourth embodiment, and in addition to the software configuration in the first embodiment, the sensor input control unit 117 and the position calculation unit 118 are further provided.

In the fourth embodiment, since the state of the terminal 2 from which the biological sound can be acquired most is relatively determined, the condition for absolutely determining whether or not the terminal 2 is suitable for acquiring the biological sound is set. It does not have to be provided, but it can be provided as a supplement. For example, as in the first embodiment, it is possible to determine the conditions for determining that the loudness of the biological sound is equal to or larger than a predetermined value and that the noise is small. Further, the angle and the position of the terminal 2 may be set as the same as those of the second and third embodiments. In the fourth embodiment, on the premise that these conditions are satisfied, the optimum state may be further selected based on the loudness of the biological sound.

The notification information storage unit 122 of the fourth embodiment stores the content of the notification output by the notification unit 116 according to the determination result. FIG. 16 is a configuration example of the notification information stored in the notification information storage unit 122. Register a notification (notification 11) when the acquisition state of the biological sound is satisfactorily changing with the movement of the terminal 2 and a notification (notification 12) when the acquisition state of the biological sound is deteriorating. be able to. The notification may be sound, light, vibration, or a combination thereof. For example, the notification 11 and the notification 12 may register notification sounds having different tones and volumes, or may have different light colors and intensities. , Notifications with different vibration strength, tempo, and pattern can be registered. As will be described later, by changing the notification according to the result of the determination unit 114, it is possible to intuitively determine whether or not the direction in which the user is moving the terminal 2 at that time is correct.

The sound data acquisition unit 112 acquires sound data from the microphone 17 provided in the user terminal 2 for a predetermined time. At this time, after arranging the user terminal 2 so as to be in contact with the user's body in the same manner as when actually acquiring the biological sound, the sound data is continuously acquired while moving the angle of the user terminal 2 and the acquisition position on the body. do. Then, by comparing and analyzing the sound at a certain point in time with the sound at the previous point in time, the angle and the place on the body most suitable for acquiring the biological sound are specified.

Sounds may be acquired continuously or at predetermined time intervals. The user acquires the sound while slowly moving the terminal near the instructed acquisition part. To specify an appropriate terminal angle, rotate the terminal 2 in contact with the body to acquire sound while changing the angle. To specify an appropriate terminal position, move the terminal 2 on the body. Get the sound while.

The sound evaluation unit 113 evaluates the size of the biological sound data included in the sound data as in the first embodiment. If necessary, a process of extracting the target biological sound data from the sound data may be performed.

The determination unit 114 compares the biological sound data acquired at the time Tn with the biological sound data acquired at the time _{Tn + 1} after the time _Tn , and the biological sound data acquired at the newer time _{Tn + 1} is the time T. When it is larger than the biological sound data of _n , it is determined that the angle or position of the terminal 2 at time _{Tn + 1} is more suitable for acquiring the biological sound than the angle or position of the terminal 2 at time _Tn . Conversely, when the biological sound data acquired at the newer time _{Tn + 1} is smaller than the biological sound data at the time Tn, the angle or position of the terminal 2 at the time _{Tn + 1} is the angle or position of the terminal 2 at the time _Tn . It is judged that it is not suitable for acquiring biological sound. The time T _n and T _{n + 1} may be a predetermined interval, for example, every 0.1 second to 1 second. Further, the times T _n , T _{n + 1} , ... May have a predetermined time width, and in that case, the average of the sound data during that time can be compared. Further, the volume of the biological sound data may be compared, or the magnitude of the noise component may be compared.

The notification unit 116 gives a predetermined notification based on the comparison result between the sound data at time _{Tn + 1} and the sound data at time Tn. As an example, when the user terminal 2 continuously emits a notification sound and it is determined that the sound data at time _{Tn + 1} is more suitable for acquiring the biological sound than the sound data at time _Tn , the tone color of the notification sound or You can change the volume. Similarly, when light is continuously emitted from the user terminal 2 and it is determined that the sound data at time _{Tn + 1} is more suitable for acquiring the biological sound than the sound data at time _Tn , the color and intensity of the light are determined. Can be changed. Further, when vibration is continuously generated in the user terminal 2 and it is determined that the sound data at time _{Tn + 1} is more suitable for acquiring the biological sound than the sound data at time _Tn , the strength, tempo, and pattern of the vibration are determined. Can be changed. For example, changing the volume means gradually changing the notification content, for example, gradually increasing the volume when the determination result is good, and gradually decreasing the volume when the determination result is good. Further, when it is determined that the sound data at time T _{n + 1} is more suitable for acquiring the biological sound than the sound data at time T _n , notifications such as notification sound, light, and vibration are issued, and conversely, at time T _{n + 1} . It is also possible to say that the notification sound is not emitted when it is determined that the sound data is less suitable for acquiring the biological sound than the sound data at the time _Tn . Alternatively, when it is determined that the sound data at time _{Tn + 1} is less suitable for acquiring the biological sound than the sound data at time _Tn , notifications such as notification sound, light, and vibration are issued, and conversely, at time _{Tn + 1} It is also possible to say that the notification sound is not emitted when it is determined that the sound data is more suitable for acquiring the biological sound than the sound data at the time _Tn . By such a notification, the user should be able to move the terminal 2 to a state suitable for acquiring the biological sound while moving the terminal 2, or the trajectory should be corrected so as to be in a different direction or angle. You can know in real time.

Further, the determination unit 114 may further determine whether or not the sound data satisfies a predetermined condition. For example, on the condition that the volume of the biological sound included in the sound data is louder than a predetermined value, it is determined whether or not the condition is satisfied, and at the same time, as described above, the sound data at time _{Tn + 1} and the time _Tn The sound data is compared to determine whether the movement direction and movement accuracy are correct. Further, as in the second and third embodiments, it is determined that the terminal 2 is arranged in the predetermined angle range or position range of the terminal 2, and at the same time, the sound data at time _{Tn + 1} and the time T as described above. It is also possible to compare the sound data of _n and determine whether the movement direction and the movement accuracy are correct.

The notification unit 116 notifies the comparison result between the sound data at time _{Tn + 1} and the sound data at time Tn with the movement of the terminal 2 as described above, and when the sound data satisfies a predetermined condition. Further notification will be given. For example, when the user moves the terminal 2 in a certain direction toward an appropriate acquisition position, it is determined that the sound data at time _{Tn + 1} is more suitable for acquiring the biological sound than the sound data at time _Tn . , By issuing a predetermined notification such as gradually increasing the notification sound, the user is informed that the moving direction is correct. Then, when the sound data acquired at a certain recent time satisfies a predetermined condition (for example, when the biological sound is louder than a certain level), a notification to stop the terminal can be issued.

<Embodiment 5>
In the fifth embodiment, the doctor instructs the acquisition position of the biological sound. Depending on the age, disease, constitution, etc. of the user, the measurement method such as the acquisition position, order, number of times, etc. of the biological sound required for the medical examination by the doctor may differ. In addition, the acquisition position may be slightly different depending on the gender, body size, and body shape of the user. Therefore, the fifth embodiment relates to a method of providing image data including a user's body (including a biometric sound acquisition site) to a doctor terminal 3 and having a doctor input an instruction regarding a biometric sound acquisition method.

FIG. 17 shows an example of software configuration of the user terminal 2 in the fifth embodiment, and further includes a sensor input control unit 117, a position calculation unit 118, and an image data acquisition unit 119 in the software configuration in the first embodiment.

FIG. 18 shows a software configuration example of the doctor terminal 3 in the fifth embodiment, and includes a data transmission / reception unit 211, an instruction information input unit 212, an instruction image generation unit 213, and a position information generation unit 214 as arithmetic processing units. be able to.

In the fifth embodiment, first, an image of the body including the biological sound acquisition portion of the user is taken by a camera or the like mounted on the user terminal 2. The image may be captured by an image capturing means other than the camera mounted on the user terminal 2. The image data acquisition unit 119 of the user terminal 2 acquires the captured image data. Further, the image data may be a still image or a moving image.

The data transmission / reception unit 111 of the user terminal 2 transmits the image data to the doctor terminal 3, and the data transmission / reception unit 211 of the doctor terminal 3 acquires the image data. Alternatively, the data transmission / reception unit 111 of the user terminal 2 may transmit the image data to the server 1, and the server 1 may transmit the image data to the doctor terminal 3. Further, it is preferable to transmit the user's information (name, age, gender, disease, etc.) to the doctor terminal 3 together with the user's image data. In addition, if the user has been examined at the medical institution to which the doctor belongs, identification information such as a patient number may be included.

FIG. 19 is a diagram showing a state in which a doctor inputs instruction information. As shown in FIG. 19A, the user image 51 and the user information 52 are displayed on the screen display unit (display or the like) of the doctor terminal 3, and the doctor sees the user image 51 and the user information 52 acquired by the doctor terminal 3. , Determine the acquisition conditions for biological sound. As shown in FIG. 19B, the instruction information input unit 212 of the doctor terminal 3 receives input of an instruction regarding the acquisition condition of the biological sound from the doctor. For example, the input of the instruction may be performed by the doctor selecting the acquisition position on the image. That is, the acquisition position can be directly specified by moving the mouse cursor or touching the touch panel. In addition, information such as the number of acquisitions and the order of acquisition can be input according to the acquisition position.

The instruction image generation unit 213 generates an acquisition instruction image based on the information received by the instruction information input unit 212. As shown in FIG. 19C, the acquisition instruction image may include a superimposed image in which a display (mark or the like) 53 indicating an acquisition position is superimposed on an image 51 showing the user's body. Further, the superimposed image may further include information such as the order and number of acquisitions.

The position information generation unit 214 converts the position information including the coordinates representing the acquired position based on the instruction information input by the doctor. The acquisition position information is the same as in the third embodiment as shown in the table of FIG. 19C, and may include the relative coordinates of the acquisition position with respect to the reference position in the user's body.

The data transmission / reception unit 211 of the doctor terminal 3 transmits at least one of the acquisition instruction image 54 generated by the instruction image generation unit 213 and the acquisition position information generated by the position information generation unit 214 to the user terminal 2. Further, it may be transmitted to the user terminal 2 via the server 1.

The user terminal 2 can store the received acquisition instruction image 54 and the acquisition position information in a storage unit such as the condition storage unit 121, and can use the user terminal 2 when acquiring the biological sound next time. That is, the acquisition instruction image 54 is read out and displayed on the user terminal 2. Further, as in the third embodiment, when the determination is made based on the position of the terminal 2, the acquisition position information can be read out and the determination can be made.

In the present embodiment, the doctor terminal 3 is provided with the instruction image generation unit 213 and the position information generation unit 214, but a part of these may be provided in the server 1. In that case, the input information of the instruction information is transmitted from the doctor terminal 3 to the server 1, and the server 1 generates the instruction image and the position information. Then, the generated acquisition instruction image and acquisition position information can be transmitted to the user terminal 2.

<Embodiment 6>
In the sixth embodiment, the server 1 instructs the acquisition position of the biological sound according to the body shape of the user. In the sixth embodiment, the server 1 has acquisition instruction information such as the acquisition position, order, and number of times of biological sound in advance according to the disease, income purpose, and the like. However, it is necessary to modify the standard acquisition instruction information according to the user's body shape. In the sixth embodiment, standard acquisition instruction information is modified based on image data including a user's biological sound acquisition site.

The software configuration example of the user terminal 2 in the sixth embodiment is the same as that of the fifth embodiment, and the software configuration in the first embodiment further includes a sensor input control unit 117, a position calculation unit 118, and an image data acquisition unit 119.

Further, FIG. 20 shows an example of software configuration of the server 1 in the sixth embodiment. The server includes an acquisition instruction information storage unit 321, a data transmission / reception unit 311, an image analysis unit 312, and an acquisition instruction information adjustment unit 313.

A plurality of acquisition instruction information can be stored in the acquisition instruction information storage unit 321 of the server 1. FIG. 21 is a configuration example of acquisition instruction information. The acquisition instruction information stores the acquisition site, order, number of times, and the like. The acquisition site may be represented by the relative coordinates of the acquisition position with respect to the reference position. Further, the acquisition instruction information may include an instruction image or the like in which the acquisition position is plotted on the image of the body. A plurality of acquisition instruction information may be provided for each type of biological sound or disease. In addition, it is possible to have acquisition instruction information in which the acquisition site is adjusted according to a plurality of body types. For example, the body shape is divided into a plurality of types such as thin type A, slightly thin type B, standard C, slightly obese D, obesity E, and the like, and the acquisition position information is registered for each type.

In the sixth embodiment, first, an image of the body including the biological sound acquisition portion of the user is taken by a camera or the like mounted on the user terminal 2. The image may be captured by an image capturing means other than the camera mounted on the user terminal 2. The image data acquisition unit 119 of the user terminal 2 acquires the captured image data. Further, the image data may be a still image or a moving image.

The data transmission / reception unit 111 of the user terminal 2 transmits image data to the server 1, and the data transmission / reception unit 311 of the server 1 acquires the image data.

By analyzing the acquired image data, the image analysis unit 312 of the server 1 determines which of the body types registered in the condition storage unit 121 described above is applicable. The image analysis method is not particularly limited, and a known method can be used. For example, the contour line of the body may be extracted from the image and the type may be determined using the length and ratio of a predetermined portion, or the image may be classified using machine learning.

The acquisition instruction information adjustment unit 313 selects the acquisition instruction information of the corresponding body type from the acquisition instruction information storage unit 321 based on the body type type selected by the image analysis unit 312.

The server 1 transmits the selected acquisition instruction information to the user terminal 2. The user terminal 2 reads out the received acquisition instruction information when the user acquires the biological sound. When the acquisition instruction information includes the instruction image, the instruction image can be displayed on the user terminal 2. Further, based on the acquisition instruction information, it is possible to make a determination based on the position information of the terminal 2 in the third embodiment.

Further, the acquisition instruction information storage unit 321 may store only the acquisition instruction information of the standard body shape and correct it according to the user's body shape. In that case, the image analysis unit 312 calculates the difference between the user's body shape and the standard body shape from the user's image. Then, the acquisition instruction information adjustment unit 313 corrects the acquisition instruction information according to the difference. The method of calculating the difference between the user's body shape and the standard body shape is not particularly limited, and may be specified by, for example, comparing the lengths of predetermined portions such as shoulder width and chest width.

Further, in the above, the image analysis unit 312 determines the body type based on the image of the user, but the body type of the user may be estimated by another method. For example, the body shape can be estimated by measuring the distance of a predetermined part of the body using the acceleration sensor of the user terminal 2. Specifically, the length of the predetermined area can be calculated by moving the user terminal 2 between predetermined areas (for example, between the left and right shoulders) and measuring the moving distance of the terminal 2 with an acceleration sensor. .. The predetermined area is not limited to between the left and right shoulders, and may be any part of the body whose body shape can be estimated. Then, the body shape can be estimated according to the length of the predetermined portion. When the acquisition instruction information is prepared for each body type, the acquisition instruction information adjusting unit 313 determines the closest body type. When the acquisition instruction information is prepared only for the standard body shape, the acquisition instruction information adjusting unit 313 corrects the acquisition instruction information according to the difference between the standard body shape and the estimated user's body shape.

As described above, since the biological sound acquisition program of the present invention can determine whether or not it is suitable for biological sound acquisition by various methods, appropriate guidance is provided when the user wants to acquire biological sound from his / her own terminal 2. It is possible. The methods described in the above embodiments 1 to 6 may be carried out by one or a combination of two or more.

In any of the embodiments described above, a part of each software function unit is provided in any of the user terminal 2, the doctor terminal 3, and the server 1 as long as the purpose of this system can be realized. May be good.

The embodiments described above are merely examples for facilitating the understanding of the present invention, and are not intended to limit the interpretation of the present invention. It goes without saying that the present invention can be modified and improved without departing from the spirit thereof, and the present invention includes its equivalents.

1 server 2 user terminal 3 doctor terminal

Claims (8)

It is a biological sound acquisition support system that assists the user himself to acquire biological sound using a user terminal.
A sound data acquisition unit that acquires sound data including biological sounds,
A determination unit for determining whether or not the condition for acquiring biological sound is satisfied, and a determination unit.
A biological sound acquisition support system including a notification unit that outputs the determination result to the user terminal. It is further equipped with a sound evaluation unit that evaluates a specific sound component from the acquired sound data.
The biological sound acquisition support system according to claim 1, wherein the determination unit determines whether or not the biological sound acquisition condition is satisfied based on the evaluation result. The sound evaluation unit evaluates the volume of the target biological sound component and evaluates the volume.
The acquisition condition includes a condition relating to the volume of the biological sound component, and includes the condition.
The biological sound acquisition support system according to claim 2, wherein the determination unit determines whether or not the volume of the biological sound component satisfies the acquisition condition. The sound evaluation unit evaluates the volume of the noise component and evaluates the volume.
The acquisition condition includes a condition relating to the volume of the noise component.
The biological sound acquisition support system according to claim 2 or 3, wherein the determination unit determines whether or not the volume of the noise component satisfies the acquisition condition. Further, an angle data acquisition unit for acquiring the angle data of the user terminal is provided.
The acquisition condition includes an appropriate angle condition of the user terminal at the time of acquisition of the biological sound.
The biological sound acquisition support system according to any one of claims 1 to 4, wherein the determination unit determines whether or not the angle data matches the angle condition. Further, a position calculation unit for calculating terminal position information accompanying the movement of the user terminal is provided.
The acquisition condition includes the acquisition position information of the biological sound, and includes the acquisition position information.
The biological sound acquisition support system according to any one of claims 1 to 5, wherein the determination unit determines whether or not the terminal position information matches the acquisition position information. Further provided with an instruction information input unit that accepts input of instruction information regarding the acquisition position of the user's biological sound by an expert.
The biological sound acquisition support system according to claim 6, wherein the acquisition position information is generated based on the instruction information. A body shape estimation unit that estimates the user's body shape based on the user's input information,
The biological sound acquisition support system according to claim 6, further comprising an acquisition instruction information adjusting unit that adjusts the acquisition position information according to the body shape.

Images