JP7794407B2 - Body sound acquisition support system - Google Patents

Description

The present invention relates to a body sound acquisition support system.

A remote medical care support system has been proposed that enables remote medical care by communicating via videophone or the like between a doctor terminal and a patient terminal.

Patent Document 1 discloses an information processing device that provides a means for making reservations for remote medical treatment and a means for video calls between a doctor and a patient.

Patent No. 6574391

In remote medical care, patients need to collect various vital signs required for a doctor's examination and provide them to the doctor's device. In addition, with the rise in health consciousness in recent years, there is also an increasing demand for daily health management.

Therefore, an object of the present invention is to provide a system for acquiring body sounds using a mobile terminal such as a smartphone.

According to the present invention, a body sound acquisition support system for supporting a user in acquiring body sounds using a user terminal includes a sound data acquisition unit for acquiring sound data including body sounds, a sound evaluation unit for evaluating specific sound components from the acquired sound data, a condition storage unit for storing conditions for determining whether or not body sounds can be acquired, and a determination unit for determining whether or not the conditions are met.
A body sound acquisition support system is provided, which includes a notification unit that outputs the determination result to the user terminal.

According to the present invention, a user can smoothly acquire body sounds using a device such as a smartphone.

1 is a diagram illustrating an example of the configuration of a system according to an embodiment of the present invention. FIG. 2 is a diagram illustrating an example of a hardware configuration of a user terminal according to an embodiment of the present invention. FIG. 2 is a diagram illustrating an example of a hardware configuration of a server according to an embodiment of the present invention. FIG. 2 is a diagram illustrating an example of a software configuration according to an embodiment of the present invention. FIG. 3 is a diagram illustrating an example of a configuration of a condition storage unit according to the embodiment of the present invention. FIG. 2 is a diagram illustrating an example of a configuration of a notification information storage unit according to the embodiment of the present invention. FIG. 1 is a diagram showing an example of a processing flow according to an embodiment of the present invention. FIG. 2 is a diagram illustrating an example of a software configuration according to an embodiment of the present invention. 1 is a diagram illustrating an embodiment of the present invention. FIG. 3 is a diagram illustrating an example of a configuration of a condition storage unit according to the embodiment of the present invention. 1 is a diagram illustrating an embodiment of the present invention. FIG. 2 is a diagram illustrating an example of a configuration of a notification information storage unit according to the embodiment of the present invention. FIG. 2 is a diagram illustrating an example of a software configuration according to an embodiment of the present invention. 1 is a diagram illustrating an embodiment of the present invention. FIG. 2 is a diagram illustrating an example of a configuration of a notification information storage unit according to the embodiment of the present invention. FIG. 2 is a diagram illustrating an example of a configuration of a notification information storage unit according to the embodiment of the present invention. FIG. 2 is a diagram illustrating an example of a software configuration according to an embodiment of the present invention. FIG. 2 is a diagram illustrating an example of a software configuration according to an embodiment of the present invention. 1 is a diagram illustrating an embodiment of the present invention. FIG. 2 is a diagram illustrating an example of a software configuration according to an embodiment of the present invention. FIG. 10 is a diagram showing an example of the configuration of acquisition instruction information according to the embodiment of the present invention.

The present invention will be described below by listing the contents of the embodiments. The present invention has the following configuration.
[Item 1]
A body sound acquisition support system that supports a user in acquiring body sounds using a user terminal, comprising:
a sound data acquisition unit that acquires sound data including the body sounds;
a determination unit that determines whether or not the body sound acquisition condition is met;
a notification unit that outputs the determination result to the user terminal.
[Item 2]
a sound evaluation unit that evaluates a specific sound component from the acquired sound data;
2. The body sound acquisition support system according to claim 1, wherein the determination unit determines whether or not the body sound acquisition condition is met based on the evaluation result.
[Item 3]
the sound evaluation unit evaluates the volume of a target body sound component;
the acquisition conditions include conditions related to the volume of the body sound components,
3. The body sound acquisition support system according to claim 2, wherein the determining unit determines whether the volume of the body sound component satisfies the acquisition condition.
[Item 4]
the sound evaluation unit evaluates the volume of a noise component;
the acquisition conditions include a condition related to the volume of the noise component;
4. The body sound acquisition support system according to claim 2, wherein the determining unit determines whether the volume of the noise component satisfies the acquisition condition.
[Item 5]
further comprising an angle data acquisition unit that acquires angle data of the user terminal;
the acquisition conditions include appropriate angle conditions of the user terminal when acquiring the body sound,
5. The body sound acquisition support system according to any one of items 1 to 4, wherein the determining unit determines whether the angle data matches the angle condition.
[Item 6]
a position calculation unit that calculates terminal position information associated with movement of the user terminal;
the acquisition condition includes acquisition position information of the body sound,
6. The body sound acquisition support system according to any one of items 1 to 5, wherein the determination unit determines whether the terminal location information matches the acquired location information.
[Item 7]
an instruction information input unit that receives instruction information input by an expert regarding a position where the body sounds of the user are to be acquired;
7. The body sound acquisition support system according to claim 6, wherein the acquisition position information is generated based on the instruction information.
[Item 8]
a body type estimation unit that estimates a body type of the user based on input information of the user;
7. The body sound acquisition support system according to item 6, further comprising an acquisition instruction information adjustment unit that adjusts the acquisition position information in accordance with the body type.

<Details of the embodiment>
Hereinafter, embodiments of the present invention will be described with reference to the drawings.

<Overview>
1 is a diagram illustrating an outline of a body sound acquisition system according to an embodiment of the present invention. As shown in the figure, the body sound acquisition system includes a user terminal 2 and a server 1 to which the user terminal 2 is connected via a network, the Internet. If necessary, the system may also include a doctor terminal 3 connected to the network. The user terminal 2 and the doctor terminal 3 can be connected to terminals of multiple users.

The body sound acquisition system according to an embodiment of the present invention is used in cases where body sounds acquired by a user terminal 2 are provided to a doctor terminal 3 in cooperation with a remote medical care system, or in cases where a user acquires body sounds for their own health management.

In this embodiment, the user terminal 2 is a device that can be connected to an Internet network, such as a smartphone, a computer, or a tablet terminal, which is a portable information terminal. In particular, the user terminal 2 is preferably a smartphone equipped with a microphone for capturing voice and various sensors for measuring acceleration and terminal angle.

FIG. 2 is a diagram illustrating an outline of the hardware configuration of the user terminal 2. As shown in the figure,
The user terminal 2 includes a control unit 10, a memory 11, a storage 12, a transmission/reception unit 13, an input/output unit 1
4, the main components of which are a sensor 16, a microphone 17, and a display unit 18, which are electrically connected to each other via a bus 15.

The control unit 10 is a computing device that controls the operation of the user terminal 2, controls the transmission and reception of data between each element, and performs processing necessary for executing application programs.

In this embodiment, the control unit 10 is, for example, a CPU (Central Processor)
The CPU 11 is a processing unit (CPU), which executes application programs stored in the storage 12 (described later) and deployed in the memory 11 to perform various processes.

The memory 11 is a DRAM (Dynamic Random Access Memory).
) and other volatile storage devices such as flash memory and HDD (Har
The system is equipped with an auxiliary storage device that is configured as a nonvolatile storage device such as a 3D Disc Drive.

The memory 11 is used as a work area for the control unit 10, and also stores the BIOS (Basic Input/Output System) that is executed when the computer is started.
m), and various setting information, etc. are stored.

The storage 12 stores programs, data used in various processes, and the like.
Details of the programs and the like stored in the storage 12 will be described later. The storage 12 may be connected so as to be available using cloud computing technology.

The transmitting/receiving unit 13 connects the user terminal 2 to the Internet.
Bluetooth (registered trademark) and BLE (Bluetooth Low Energy)
It may also be equipped with a short-range communication interface.

The input/output unit 14 is connected to information input devices such as a keyboard and a mouse, and output devices such as a display, as needed.

The bus 15 transmits, for example, address signals, data signals, and various control signals between the connected control unit 10, memory 11, storage 12, transmission/reception unit 13, and input/output unit 14.

The sensor 16 may be any of various sensors, such as an acceleration sensor, a tilt sensor (geomagnetic sensor or gyro sensor), or a proximity sensor.

The microphone 17 is a sound collecting device mounted on the user terminal 2.
7 converts the voice input to the terminal 2 into a voice signal. Also, a sound collector externally connected to the user terminal 2 may be used.

FIG. 3 is a diagram illustrating an outline of the hardware configuration of the server 1. As shown in the figure, the server 1
The device includes a control unit 20, a memory 21, a storage 22, a transmission/reception unit 23, and an input/output unit 24 as its main components, which are electrically connected to each other via a bus 25. The functions of each unit are the same as those of the user terminal 2, so the explanation will be omitted.

<Embodiment 1>
In the first embodiment, it is determined whether or not the sound data acquired using the microphone 17 is suitable for acquiring body sounds. Specifically, the acquired sound data is analyzed to determine whether or not the body sound data included in the sound data has been acquired with sufficient quality.

4 is a diagram showing an example of the software configuration of the user terminal 2 according to the first embodiment of the present invention. The user terminal 2 of the present invention includes a condition storage unit 121 and a notification information storage unit 122 as storage devices.
, a sound data storage unit 123, and, as an arithmetic processing device, a data transmission/reception unit 111, a sound data acquisition unit 112, a sound evaluation unit 113, a determination unit 114, a display control unit 115, and a notification unit 116. The data transmission/reception unit 111, the sound data acquisition unit 112, the sound evaluation unit 113, the determination unit 114, the display control unit 115, and the notification unit 116 are realized by the control unit 10 reading out a program stored in the storage 12 into the memory 11 and executing the program, and the condition storage unit 121, the notification information storage unit 122, and the sound data storage unit 123 are realized as part of a storage area provided by at least one of the memory 11 and the storage 12.

The condition storage unit 121 stores conditions for determining whether or not a device is suitable for acquiring body sounds. FIG. 5 shows an example of the configuration of condition information stored in the condition storage unit 121. In the first embodiment, conditions are set regarding the quality of body sounds included in acquired sound data. The quality may be, for example, volume, and may be a threshold value expressed as an index such as a decibel value, but is not limited to this. A plurality of these conditions can be set depending on the type of body sound to be acquired (heart sounds, respiratory sounds, lung sounds, peristalsis sounds, swallowing sounds, etc.), the body part to be acquired (right chest, left chest, stomach, intestines, etc.), the purpose of acquisition (provision to a doctor, self-health management), etc.

The notification information storage unit 122 stores the content of the notification output by the notification unit 116 in accordance with the determination result. Fig. 6 shows an example of the configuration of the notification information stored in the notification information storage unit 122. The notification information is
This includes the setting of notification content for each judgment result. For example, depending on whether the judgment result is OK (suitable for acquiring biological sounds) or NG (not suitable), the notification to be issued is set. Sound notification 1, 2
For example, Notification 1 is a voice message such as "You can get biological sounds," and Sound Notification 2 is
The notification by sound may be a voice message such as "Bio-sound acquisition is not possible."
The notification may not be limited to audio, but may be a predetermined notification sound. The notification sound may be expressed by a pitch or pattern (beep beep, buzz, etc.). Furthermore, the notification may not be limited to audio or audio notification, but may be light, vibration, or a combination thereof. Optical notifications are output via the screen or light-emitting function of the terminal 2 and may be expressed by the color of the light, the lighting/flashing pattern, the intensity of the light, etc. Furthermore, vibration notifications are output via the vibration function of the terminal and may be expressed by the vibration pattern, intensity, etc. The notification information storage unit 122 can store the content of these audio notifications, optical notifications, and vibration notifications according to each determination result. The notification information storage unit 122 may store audio data itself, or may store output instruction information for various notifications.

The sound data storage unit 123 stores the acquired sound data. The sound data can be stored together with information such as the date and time of acquisition, the type of acquired biological sound (heart sounds, respiratory sounds, lung sounds, peristalsis sounds, etc.), and the acquisition site (anterior chest, neck, back, etc.). The acquisition site may include a more detailed acquisition location, such as location information expressed as coordinate information from a reference position as described below. Furthermore, the sound data may further include information regarding the purpose of acquisition, such as for medical examination or self-management. If necessary, the sound data may also include information describing the user's condition at the time the sound data was acquired. For example, subjective symptoms such as shortness of breath or fatigue, image data of the user's face, body, or affected area, and vital data such as pulse, blood pressure, and body temperature acquired by various measuring devices such as a wearable device may be included. The stored sound data may also include waveform data or data processed by the sound evaluation unit 113 described below.

The data transmission/reception unit 111 uses the transmission/reception unit 13 to transmit various data to the server 1 or the doctor terminal 3 .

The sound data acquisition unit 112 acquires sound data collected by the microphone 17 provided in the user terminal 2 , stores the sound data in the sound data storage unit 123 , and transmits the sound data to the sound evaluation unit 113 .

The sound evaluation unit 113 evaluates the loudness and quality of specific sound components. The sound evaluation unit 113 can evaluate the loudness and quality of target body sound components. If necessary, the sound evaluation unit 113 performs extraction processing to extract body sound data by separating body sound components from noise components other than body sounds using predetermined processing from the sound data, or by removing noise components. If the processing method and parameters differ depending on the type of body sound and the acquisition position, the processing method and parameters are selected based on the type of input body sound and the acquisition position, and the extraction processing is performed. A known extraction method can be used. For example, noise components can be removed by applying a predetermined filter to the audio data. Furthermore, noise can also be removed by converting the sound data into a spectrogram and masking specific components. If necessary, the sound data may be subjected to predetermined preprocessing before converting to a spectrogram. Furthermore, sound data containing both body sound components and noise components may be used as input data, and a learning model may be created using body sound data containing only body sound components or body sound data containing noise components as training data. Then, body sound data may be extracted by applying the created learning model to the sound data.
The input data of the learning model may be a spectrogram converted from sound data, as long as it can be restored to waveform data. Similarly, the output data may be a spectrogram that can be restored to waveform data. Furthermore, a similar method may be used to extract noise components from sound data containing both body sound data and noise components, and extract other components as body sound data.

The determination unit 114 determines whether the situation is suitable for acquiring body sounds based on the evaluation result of the sound evaluation unit 113. In other words, it determines whether the target body sounds are being acquired sufficiently.

The notification unit 116 notifies the result of the determination. The notification unit 116 may, for example, emit sound, light, vibration, or a combination of these, and issues the notification by issuing a command to a sound generating means, light generating means, vibration generating means, or the like mounted on the user terminal 2. Depending on the determination result of the determination unit 114, the notification unit 116 reads out the notification information registered in the notification information storage unit 122 and performs processing to output a notification.

The notification may also be performed by displaying a predetermined message or image on the screen of the user terminal 2. In this case, the notification unit 116 controls the display control unit 115 to display a predetermined image or message on the screen based on the result of the determination unit 114, thereby allowing the image or message to be displayed on the user terminal 2. When the user terminal 2 is held against the user's body, the user may not be able to see the terminal screen, so if this situation is anticipated, notification by sound, light, or vibration is preferable to notification by display.

As described above, in the first embodiment, the body sound data in the acquired sound data is evaluated,
It is determined whether sufficient body sound data has been acquired. In the above, body sound data included in the sound data is evaluated, but noise components may also be evaluated to determine whether the magnitude of the noise components is equal to or less than a predetermined value. In this case, the user uses the microphone 17 of the user terminal 2.
By doing this, sound is acquired in the environment where biological sounds are to be acquired.
The microphone 17 may be placed against the body as in actual body sound acquisition, but sound may also be acquired without being placed against the body. The sound evaluation unit 113 evaluates the volume and quality of noise components from the acquired sound, and the determination unit 114 determines whether the noise components satisfy a predetermined condition. The predetermined condition may be volume, for example. If the determination result is good (low noise), the notification unit 116 notifies the user that body sound acquisition is possible. On the other hand, if the determination result is bad (high noise), the notification unit 116 notifies the user of this fact and encourages the user to change their environment, such as by moving to a different location.

FIG. 7 shows the operation flow in the first embodiment. Unless otherwise specified, each step is as follows:
The order may be reversed.

First, the user starts the application on his/her mobile terminal (user terminal 2) (S101), and in some cases inputs the type of body sound to be acquired and the body part to be acquired. The terminal 2 provides guidance by displaying a guide screen for acquiring body sounds on the screen of the terminal 2 or by outputting audio (S102).

The user places the microphone 17 of the user terminal 2 against the sound acquisition site to start acquiring sound data (S103). The sound evaluation unit 113 evaluates the target body sound component from the acquired sound data. Furthermore, the sound evaluation unit 113 may extract body sound components from the acquired sound data by predetermined processing as needed, and then evaluate the extracted components. The sound evaluation unit 113 transmits the evaluation result to the determination unit 114. The determination unit 114 determines whether the acquired sounds are suitable for use in a doctor's examination or analysis for disease diagnosis (S104). That is, it determines whether the extracted body sound data meets the criteria stored in the condition storage unit 121. Specifically, for example, it determines whether the volume of the body sound data is equal to or greater than a predetermined value.

If the result is NG (body sounds are not acquired sufficiently) (S104=No
, the notification unit 116 notifies the user to re-acquire the information (S105).
This can be done by voice messages or screen displays such as "Vital sounds could not be obtained. Please try again" or "Please move to a quieter place."
If it is determined that the body sounds have been acquired sufficiently (S104 = Yes), the notification unit 116 notifies the user that the body sounds have been acquired (S106). Furthermore, the notification unit 116 transmits the acquired sound data (which may also include analysis data such as waveform data) to the server as necessary (S107).

In the first embodiment, the description has been given on the assumption that the user terminal 2 is provided with the arithmetic processing units such as the sound evaluation unit 113 and the determination unit 114, and the storage units such as the condition storage unit 121, the notification information storage unit 122, and the sound data storage unit 123, but these may also be provided in the server 1. In that case, the sound data acquired by the sound data acquisition unit 112 is transmitted to the server 1, where the server 1 analyzes and determines the sound, and transmits the determination result to the user terminal 2.

<Embodiment 2>
In the second embodiment, it is determined whether the angle of the user terminal 2 is suitable for acquiring body sounds. That is, when acquiring body sounds by placing a microphone mounted on the user terminal 2, such as a smartphone, against the user's body, body sounds may be easily acquired if the angle of the terminal relative to the body is within a certain range. Also, in order to ensure uniformity of body sounds, it may be preferable to acquire the body sounds at a uniform terminal angle. In this embodiment, in such cases, the actual angle of the user terminal 2 is determined from sensor data, and it is determined whether the angle is within a pre-registered range of terminal angles.

8 shows an example of the software configuration of the user terminal 2 in the second embodiment, which further includes a sensor input control unit 117 in addition to the software configuration in the first embodiment. The user terminal 2 also includes a tilt sensor that can detect the angle of the terminal 2. The tilt sensor may be any sensor that can detect the tilt angle in each direction of the user terminal, such as a geomagnetic sensor, a gyro sensor, or an acceleration sensor. Note that the tilt angle of the user terminal in each direction here is determined as follows: as shown in FIG. 9, the rightward direction of the screen is the x-axis, the longitudinal upward direction is the y-axis, and the vertical upward direction to the screen is the z-axis; as shown in FIG. 4(b), the x-axis rotation is pitch, the y-axis rotation is azimuth, and z
Axial rotation is expressed as roll, etc.

The condition storage unit 121 in the second embodiment stores the following conditions for each body sound acquisition site (right chest, left chest, stomach, intestines, etc.), disease, or type of body sound (heart sound, respiratory sound, lung sound, peristalsis sound, swallowing sound, etc.):
An appropriate angle range for the user terminal 2 is registered in advance. FIG. 10 shows an example of the configuration of condition information stored in the condition storage unit 121. As an example, when measuring heart sounds, it is optimal to hold the terminal 2 with the bottom (microphone part) facing the body and the orientation of the screen (Z axis) of the terminal 2 horizontal, as shown in FIG. 11 . In this case, the condition is "tilt angle 90° and azimuth angle 90°." The implementation condition may be set within a predetermined allowable range, such as "tilt angle 85° to 95° and azimuth angle 85° to 95°." When making a determination based on the terminal angle, the values and ranges of the tilt angle, azimuth angle, and rotation angle are the conditions. Multiple conditions can be set for each of these conditions depending on the type of biological sound to be acquired (heart sounds, respiratory sounds, lung sounds, peristalsis sounds, swallowing sounds, etc.), disease, acquisition site (right chest, left chest, stomach, intestines, etc.), acquisition purpose (provision to a doctor, self-health management), acquisition count (first time, second time, etc.), etc.

The notification information storage unit 122 of the second embodiment stores the content of the notification output by the notification unit 116 in accordance with the determination result. FIG. 12 shows an example of the configuration of notification information stored in the notification information storage unit 122. The notification information includes the setting of the notification content for each determination result. The notification is not limited to audio, but may be a notification sound, light, vibration, display, or a combination of these. When performing a determination based on the angle of the user terminal 2, a notification when the determination is OK (the terminal angle is within a specified range) and a notification when the determination is NG can be registered as in the first embodiment. Furthermore, when the determination is NG, a further notification may be set based on the magnitude relationship between the measured value of the terminal angle and a threshold value, as shown in FIG. 12. For example, if the tilt angle is smaller than the threshold value, a voice message (sound notification 3) such as "Please tilt the terminal a little further back" may be issued. Furthermore, when the angle of the user terminal 2 is within a predetermined range, a predetermined sound, light, or
Alternatively, a predetermined sound, light, or vibration may be emitted when the angle of the user terminal 2 is outside a predetermined range, and these outputs may be stopped when the angle is within the range.

The sensor input control unit 117 transmits angle data of the user terminal 2 acquired by a tilt sensor mounted on the user terminal 2 to the determination unit 114. The tilt sensor may be, for example, a geomagnetic sensor,
The sensor is a gyro sensor and an acceleration sensor, and can express the angle of the user terminal 2 as a tilt angle (pitch), a rotation angle (roll), and an azimuth angle.

The determination unit 114 reads out angle conditions from the condition storage unit 121 according to the body part and type of body sound to be acquired. If the body part and type of body sound to be acquired are predetermined, the determination unit 114 can read out the corresponding conditions. If the user inputs the body part and type of body sound to be acquired, the determination unit 114 reads out the conditions according to the input. If the angle of the user terminal 2 input from the sensor input control unit 117 meets the conditions, it is determined that the angle is suitable for acquiring body sounds. If the angle does not meet the conditions, it is determined that the angle is not suitable.

The notification unit 116 notifies the determination result. Based on the determination result by the determination unit 114, the notification content is read from the notification information storage unit 122 and the notification is output.

<Embodiment 3>
In the third embodiment, it is determined whether the position of the user terminal 2 is suitable for acquiring body sounds. To acquire body sounds, the position where the microphone should be placed is determined depending on the purpose.
When a user acquires body sounds by himself/herself, there is a problem in that it is not possible to determine whether the microphone is placed at an appropriate acquisition position. Therefore, in the third embodiment, it is determined whether the position of the user terminal 2 is at a predetermined setting position.

FIG. 13 shows an example of the software configuration of the user terminal 2 in the third embodiment, which further includes a sensor input control unit 117 and a position calculation unit 118 in addition to the software configuration in the first embodiment.

The condition storage unit 121 of the third embodiment stores conditions for determining whether the terminal 2 is located at an acquisition position suitable for acquiring body sounds. For example, the conditions include the distance and movement direction of each measurement position relative to a reference position. Multiple conditions can be set depending on the type of body sound to be acquired (heart sounds, respiratory sounds, lung sounds, peristalsis sounds, swallowing sounds, etc.), disease, acquisition site (right chest, left chest, stomach, intestines, etc.), acquisition purpose (provision to a doctor, self-health management), etc. FIG. 14 shows an example of the configuration of condition information stored in the condition storage unit 121. The acquisition position condition may be image data plotting the acquisition positions and order on the body as shown in FIG. 14(a). Alternatively, it may be coordinate information as shown in FIG. 14(b). As an example of the coordinate information, the relative coordinates of a second position relative to a reference position (first position) on the user's body are stored as the terminal position condition.
As shown in 4(a), for example, suppose the first position is the solar plexus and the second position is position 1 (above the right clavicle), which is the first body sound acquisition position. In this case, the condition information can be determined as coordinate information of the second position relative to the first position (for example, 0 to 0 in the X direction and 0 to 0 in the Y direction) within a predetermined range. Furthermore, when body sounds are acquired continuously at a plurality of positions, the next acquisition position (third position) can be determined by coordinate information of the third position relative to the second position described above.

The notification information storage unit 122 stores the content of the notification output by the notification unit 116 according to the determination result. FIG. 15 shows an example of the configuration of notification information stored in the notification information storage unit 122. When performing determination based on the position of the user terminal 2, notifications can be provided by outputting different sounds (notifications 9 and 10) when the terminal 2 is not within the coordinate range of the set acquisition position and when it is within the coordinate range of the acquisition position. For example, the terminal 2 may continuously emit a sound and change the sound when it enters the coordinate range of the acquisition position (notification 9 → notification 10), or a predetermined sound may be emitted only when it enters the coordinate range (notification 10 only). Alternatively, different notifications may be issued when the terminal 2 is approaching the coordinate range of the acquisition position and when it is moving away. Providing such notifications makes it easier for the user to adjust the location of the terminal 2.

The sensor input control unit 117 transmits information input from each sensor such as an acceleration sensor to the position calculation unit 118 and the determination unit 114 .

The position calculation unit 118 calculates the position information of the user terminal 2 using data obtained from an acceleration sensor mounted on the user terminal 2. The position calculation unit 118 calculates the movement direction and movement distance from the acceleration data. The movement distance can be calculated by integrating the acceleration sensor values using a known method such as trapezoidal integration. As an example, if a reference position (first position) is set on the user's body, the user terminal 2 is placed against the first position, and then the terminal 2 is moved to a second position different from the first position, the position calculation unit 118 can derive the coordinates of the second position (a position relative to the first position) from the movement direction and movement distance.
The method for determining the location of the terminal 2 is not limited to this, and any known method may be employed.

The determination unit 114 determines whether the user terminal 2 is placed in a position on the user's body that is suitable for acquiring body sounds. As an example, a reference position on the body (first position) is determined. The first position is, for example, a position that is clearly and uniquely determined for the user, such as the solar plexus or collarbone. First, the user is instructed to position the user terminal 2 at the first position, and the terminal 2 is stopped for a predetermined period of time. Next, the user terminal 2 is moved from the first position to a position (second position) where body sounds will be acquired, and acceleration data is acquired during that time. The position calculation unit 118 calculates the direction and distance of movement of the terminal 2 from the acceleration data, and calculates the coordinates of the user terminal 2 relative to the first position.

The notification unit 116 notifies the user of the result of the determination.
A predetermined sound, light, vibration, etc. may be emitted when the terminal 2 enters the range of the acquisition position (second position) of the body sound stored in the terminal 2, and the generation of the predetermined sound, light, or vibration may be stopped when the user moves the terminal 2 out of the range of the acquisition position. Alternatively, conversely, when the position of the terminal 2 is out of the range of the acquisition position, sound, light, or vibration may be emitted to indicate that the position is inappropriate, and the sound, light, or vibration may be stopped when the terminal 2 moves to an appropriate position. Furthermore, when the user is moving the user terminal 2, a first notification may be issued when the position of the terminal 2 is approaching the range of the acquisition position of the body sound, and a second notification may be issued when the terminal 2 is moving away. The first notification may be, for example, a notification encouraging the user to continue moving, and the second notification may be a notification encouraging the user to correct their direction. The first and second notifications may be voice messages, or may be sound, light, or vibration. Alternatively, only one of the first and second notifications may be issued. For example, the first notification may be omitted, and the second notification may be issued only when it is necessary to encourage the user to correct their moving direction.

<Fourth Embodiment>
In the fourth embodiment, it is determined whether the user terminal 2 is in a state where the loudest body sound can be acquired. The state of the user terminal 2 is, for example, a state related to the angle or position relative to the user's body. In the second and third embodiments, it was determined whether the angle or position of the user terminal 2 was within a predetermined range, but in the fourth embodiment, it is relatively determined whether the angle or position is such that body sound can be acquired satisfactorily.

The software configuration example of the user terminal 2 in the fourth embodiment is the same as that in the fourth embodiment.
In addition to the software configuration in the first embodiment, a sensor input control unit 117 and a position calculation unit 118 are further provided.

In the fourth embodiment, the state of the terminal 2 that allows the best acquisition of body sounds is relatively determined, and therefore conditions for absolutely determining whether or not the terminal 2 is suitable for acquiring body sounds do not need to be set, but such conditions can be set as an auxiliary condition. For example, as in the first embodiment, conditions for determining whether or not the volume of the body sounds is equal to or greater than a predetermined value and whether or not there is little noise can be set. Furthermore, as in the second and third embodiments, conditions can also be set for the angle and position of the terminal 2. In the fourth embodiment, assuming that these conditions are met, the optimal state can also be selected based on the volume of the body sounds.

The notification information storage unit 122 of the fourth embodiment stores the contents of the notification output by the notification unit 116 in accordance with the determination result. Fig. 16 shows an example of the configuration of notification information stored in the notification information storage unit 122. The notification (notification 11) when the state of acquisition of body sounds is improving as the terminal 2 moves is
), and a notification (notification 12) when the state of acquiring body sounds is deteriorating. The notification may be sound, light, vibration, or a combination thereof. For example, notification 1
It is possible to register notification sounds with different tones and volumes, light notifications with different light colors and intensities, and notifications with different vibration intensities, tempos, and patterns for notification 1 and notification 12. As will be described later, the notification changes depending on the result of determination unit 114, allowing the user to intuitively determine whether the direction in which terminal 2 is being moved at that time is correct.

The sound data acquisition unit 112 acquires sound data over a predetermined period of time from the microphone 17 provided in the user terminal 2. At this time, the user terminal 2 is placed in contact with the user's body, as in the case of acquiring actual body sounds, and sound data is continuously acquired while changing the angle of the user terminal 2 and the acquisition position on the body. Then, the sound at a certain point in time is compared and analyzed with the sound at the previous point in time to identify the angle and the position on the body that are most suitable for acquiring body sounds.

Sound may be acquired continuously or at predetermined time intervals. The user acquires sound while slowly moving the terminal 2 near the designated acquisition location. To identify an appropriate terminal angle, the user can acquire sound by rotating the terminal 2 while it is in contact with the body and changing the angle. To identify an appropriate terminal position, the user can acquire sound while moving the terminal 2 on the body.

The sound evaluation unit 113 evaluates the magnitude of body sound data included in the sound data in the same manner as in embodiment 1. If necessary, processing for extracting target body sound data from the sound data may be performed.

The determination unit 114 compares the body sound data acquired at time _Tn with _the body sound data acquired at time Tn ₊
The angle or position of the terminal 2 at time Tn ₊₁ is compared with the body sound data acquired at time _Tn , and if the body sound data acquired at more recent time Tn ₊₁ is greater than the body sound data at time _Tn , it is determined that the angle or position of the terminal 2 at time _{Tn+1 is} more appropriate for acquiring body sounds than the angle or position of the terminal 2 at time Tn. Conversely, if the body sound data acquired at more recent time Tn ₊ 1 is smaller than the body sound data at time _Tn , it is determined that the angle or position of the terminal 2 at time _{Tn +1 is} less appropriate for acquiring body sounds than the angle or position of the terminal 2 at time Tn.
The intervals may be predetermined, for example, every 0.1 to 1 second.
The times T _n , T _n+1 ... may be a predetermined time interval, in which case the average of the sound data during that time can be compared. Alternatively, the volume of the body sound data or the magnitude of the noise component may be compared.

The notification unit 116 performs a predetermined notification based on the comparison result between the sound data at time T _n+1 and the sound data at time T _n . As an example, the user terminal 2 continuously emits a notification sound, and when it is determined that the sound data at time T _n+1 is more suitable for acquiring body sounds than the sound data at time T _n , the tone and volume of the notification sound can be changed. Similarly, when the user terminal 2 continuously emits a notification sound, the tone and volume of the notification sound can be changed.
Light is emitted from the device 2, and when it is determined that the sound data at time _Tn+1 is more suitable for acquiring body sounds than the sound data at time _Tn , the color and intensity of the light can be changed. Furthermore, vibrations are continuously emitted from the user terminal 2, and when it is determined that the sound data at time Tn ₊₁ is more suitable for acquiring body sounds than the sound data at time _Tn , the intensity, tempo, and pattern of the vibrations can be changed. For example, changing the volume means gradually changing the notification content, such as gradually increasing the volume when the judgment result is good and gradually decreasing the volume when the judgment result is negative. Furthermore, when it is determined that the sound data at time Tn+1 is more suitable for acquiring body sounds than the sound data at time Tn, a notification such as a notification sound, light, or vibration is emitted, and conversely, when it is determined that the sound data at time _{Tn +1} is more suitable for acquiring body sounds than the sound data at time Tn, the intensity, tempo, and pattern of the vibrations can be changed.
Alternatively, if it is determined that the sound data at time _Tn+1 is less suitable for acquiring body sounds than the sound data at time _Tn , no notification sound is emitted. Alternatively, if it is determined that the sound data at time _Tn+1 is less suitable for acquiring body sounds than the sound data at time _Tn , a notification such as a notification sound, light, or vibration is emitted, and conversely, if it is determined that the sound data at time _Tn+1 is more suitable for acquiring body sounds than the sound data at time _Tn , no notification sound is emitted.
With such notification, the user can know in real time while moving the terminal 2 whether the terminal 2 is being moved to a state appropriate for acquiring biological sounds, or whether the trajectory should be corrected to a different direction or angle.

The determination unit 114 may further determine whether the sound data satisfies a predetermined condition. For example, the determination unit 114 may determine whether the sound data satisfies a predetermined condition, such as whether the volume of the body sound included in the sound data is greater than a predetermined value. At the same time, the determination unit 114 may compare the sound data at time T _n+1 with the sound data at time T _n as described above to determine whether the direction of movement and the accuracy of movement are correct.
, 3, it is possible to determine whether terminal 2 is located within a predetermined angle range or position range of terminal 2, and at the same time, it is possible to compare the sound data at time T _n+1 with the sound data at time T _n as described above to determine whether the direction of movement and the accuracy of movement are correct.

As described above, the notification unit 116 notifies the user of the comparison result between the sound data at time Tn ₊₁ and the sound data at time _Tn as the user moves the terminal 2, and further notifies the user if the sound data satisfies a predetermined condition. For example, if the user moves the terminal 2 in a certain direction toward an appropriate acquisition position, the sound data at time _Tn+1 is determined to be more suitable for acquiring body sounds than the sound data at time _Tn , and the notification unit 116 notifies the user that the movement direction is correct by issuing a predetermined notification, such as gradually increasing the volume of the notification sound. Then, if the sound data acquired recently satisfies a predetermined condition (for example, if the body sounds are louder than a certain level), a notification to stop the terminal can be issued.

<Embodiment 5>
In the fifth embodiment, the doctor instructs the location where the body sounds are to be acquired. The measurement method, such as the location, order, and number of times of acquiring body sounds, required by the doctor for the examination may differ depending on the user's age, illness, constitution, etc. Furthermore, the acquisition location may also differ slightly depending on the user's gender, body size, and body type. Therefore, the fifth embodiment relates to a method in which image data including the user's body (including the body sound acquisition site) is provided to the doctor terminal 3, and the doctor inputs instructions regarding the method of acquiring body sounds.

FIG. 17 shows an example of the software configuration of the user terminal 2 in the fifth embodiment. The software configuration in the first embodiment is further enhanced by a sensor input control unit 117, a position calculation unit 118,
and an image data acquisition unit 119 .

FIG. 18 shows an example of the software configuration of the doctor terminal 3 in the fifth embodiment. The arithmetic processing unit includes a data transmission/reception unit 211, an instruction information input unit 212, an instruction image generation unit 213,
A location information generating unit 214 may be provided.

In the fifth embodiment, first, an image of the user's body including the body part from which body sounds are to be acquired is captured by a camera or the like mounted on the user terminal 2. The image may be captured by an image capturing means other than the camera mounted on the user terminal 2. The image data acquisition unit 119 of the user terminal 2 acquires the captured image data. The image data may be a still image or a video image.

The data transmission/reception unit 111 of the user terminal 2 transmits the image data to the doctor terminal 3.
The data transmitter/receiver 211 of the user terminal 2 acquires the image data. Alternatively, the data transmitter/receiver 111 of the user terminal 2 may transmit the image data to the server 1, and the server 1 may transmit the image data to the doctor terminal 3. It is also preferable to transmit user information (such as name, age, sex, and disease) along with the user's image data to the doctor terminal 3. If the user has been examined at a medical institution to which the doctor belongs, identification information such as a patient number may also be included.

FIG. 19 is a diagram showing how a doctor inputs instruction information. As shown in FIG. 19( a), a user image 51 and user information 52 are displayed on the screen display unit (such as a display) of the doctor terminal 3, and the doctor determines the conditions for acquiring body sounds by looking at the user image 51 and user information 52 acquired on the doctor terminal 3. As shown in FIG. 19( b), the instruction information input unit 212 of the doctor terminal 3 accepts input of instructions regarding the conditions for acquiring body sounds from the doctor. For example, the doctor may input instructions by selecting an acquisition position on the image. That is, the acquisition position can be directly specified by a method such as placing a mouse cursor or touching a touch panel. Furthermore, information such as the number of acquisitions and the acquisition order can be input according to the acquisition position.

The instruction image generation unit 213 generates an acquisition instruction image based on the information received by the instruction information input unit 212. The acquisition instruction image may include a superimposed image in which an indication (such as a mark) 53 indicating the acquisition position is superimposed on an image 51 showing the user's body, as shown in FIG. 19(c).
The superimposed image may further include information such as the order and number of times of acquisition.

The position information generator 214 converts the instruction information input by the doctor into position information including coordinates representing the acquired position. The acquired position information is similar to that in the third embodiment, as shown in the table in Fig. 19(c), and may include relative coordinates of the acquired position with respect to a reference position on the user's body.

The data transmission/reception unit 211 of the doctor terminal 3 receives the acquisition instruction image 5 generated by the instruction image generation unit 213.
4 and at least one of the acquired position information generated by the position information generating unit 214 is transmitted to the user terminal 2
Alternatively, the information may be transmitted to the user terminal 2 via the server 1.

The user terminal 2 stores the received acquisition instruction image 54 and acquisition position information in a storage unit such as the condition storage unit 121, and can use them the next time body sounds are acquired. That is, the acquisition instruction image 54 is read out and displayed on the user terminal 2. Furthermore, when making a determination based on the position of the terminal 2, as in the third embodiment, the acquisition position information can be read out and used for the determination.

In this embodiment, the doctor terminal 3 includes an instruction image generating unit 213 and a position information generating unit 21
4, but some of these may be provided in the server 1. In that case,
The input information of the instruction information is transmitted to the server 1, and the server 1 generates an instruction image and position information. Then, the generated acquisition instruction image and acquired position information can be transmitted to the user terminal 2.

<Embodiment 6>
In the sixth embodiment, the server 1 instructs the body sound acquisition position according to the user's body type. In the sixth embodiment, the server 1 has acquisition instruction information such as the body sound acquisition position, order, and number of times according to the user's illness, purpose of income, etc. However, it is necessary to modify the standard acquisition instruction information according to the user's body type. In the sixth embodiment, the standard acquisition instruction information is modified based on image data including the user's body sound acquisition site.

The software configuration example of the user terminal 2 in the sixth embodiment is the same as that in the fifth embodiment.
The software configuration in the first embodiment is further configured to include a sensor input control unit 117 and a position calculation unit 11
8 and an image data acquisition unit 119.

20 shows an example of the software configuration of the server 1 in the present embodiment 6. The server includes an acquisition instruction information storage unit 321, a data transmission/reception unit 311, an image analysis unit 312, and an acquisition instruction information adjustment unit 313.

The acquisition instruction information storage unit 321 of the server 1 can store a plurality of pieces of acquisition instruction information.
FIG. 21 shows an example of the configuration of acquisition instruction information. The acquisition instruction information stores the acquisition site, order, number of times, etc. The acquisition site may be represented by relative coordinates of the acquisition position with respect to a reference position, etc. The acquisition instruction information may also include an instruction image in which the acquisition position is plotted on an image of the body. A plurality of pieces of acquisition instruction information may be provided for each type of body sound or disease. Furthermore, acquisition instruction information in which the acquisition site is adjusted according to a plurality of body type categories may be provided. For example, body types are divided into a plurality of types such as thin type A, slightly thin type B, normal type C, slightly obese type D, obese type E, etc., and acquisition position information is registered for each type.

In the sixth embodiment, first, an image of the user's body including the body part from which body sounds are to be acquired is captured by a camera or the like mounted on the user terminal 2. The image may be captured by an image capturing means other than the camera mounted on the user terminal 2. The image data acquisition unit 119 of the user terminal 2 acquires the captured image data. The image data may be a still image or a video image.

The data transmitter/receiver 111 of the user terminal 2 transmits the image data to the server 1, and the data transmitter/receiver 311 of the server 1 acquires the image data.

The image analysis unit 312 of the server 1 analyzes the acquired image data to determine which of the body shape types registered in the condition storage unit 121 described above applies. There are no particular limitations on the method of analyzing the image, and any known method can be used. For example, the body contour may be extracted from the image and the type may be determined using the lengths and proportions of specified parts, or the image may be classified using machine learning.

The acquisition instruction information adjustment unit 313 selects acquisition instruction information for the corresponding body type from the acquisition instruction information storage unit 321 based on the body type selected by the image analysis unit 312 .

The server 1 transmits the selected acquisition instruction information to the user terminal 2. When the user acquires body sounds, the user terminal 2 reads the received acquisition instruction information. If the acquisition instruction information includes an instruction image, the instruction image can be displayed on the user terminal 2. Furthermore, based on the acquisition instruction information, a determination based on the position information of the terminal 2 in the third embodiment can be made.

Alternatively, the acquisition instruction information storage unit 321 may store only acquisition instruction information for a standard body type, which may be corrected to match the user's body type. In this case, the image analysis unit 312 calculates the difference between the user's body type and the standard body type from the user's image. The acquisition instruction information adjustment unit 313 then corrects the acquisition instruction information according to this difference. There are no particular limitations on the method for calculating the difference between the user's body type and the standard body type, and the difference may be determined, for example, by comparing the lengths of specified parts such as shoulder width and chest width.

Although the image analysis unit 312 determines the body type based on the user's image in the above example, the user's body type may be estimated by other methods. For example, the body type may be estimated by measuring the distance between specific body parts using an acceleration sensor in the user terminal 2. Specifically, the length of the specific area can be calculated by moving the user terminal 2 between specific areas (e.g., between the left and right shoulders) and measuring the movement distance of the terminal 2 using the acceleration sensor. The specific area is not limited to between the left and right shoulders, but may be any part of the body for which body shape estimation is possible. The body type can then be estimated based on the length of the specific area. When acquisition instruction information is prepared for each body type, the acquisition instruction information adjustment unit 313 determines the closest body type. When acquisition instruction information is prepared only for a standard body type, the acquisition instruction information adjustment unit 313 corrects the acquisition instruction information according to the difference between the standard body type and the estimated user's body type.

As described above, the body sound acquisition program of the present invention can determine whether or not a device is suitable for acquiring body sounds using various methods, and therefore can provide appropriate guidance when a user wants to acquire body sounds using his or her own terminal 2. The methods described in the first to sixth embodiments may be implemented by combining one or more of them.

In any of the embodiments described above, some of the software functional units may be provided in either the user terminal 2, the doctor terminal 3, or the server 1, as long as the purpose of the system can be realized.

The above-described embodiment is merely an example for facilitating understanding of the present invention, and is not intended to limit the present invention. The present invention can be modified and improved without departing from the spirit thereof, and it goes without saying that the present invention includes equivalents thereof.

1 Server 2 User terminal 3 Doctor terminal

Claims (14)

A body sound acquisition support system that supports a user in acquiring body sounds using a user terminal, comprising:
a sound data acquisition unit that acquires sound data including the body sounds;
a determination unit that determines whether or not the body sound acquisition position information matches the body sound acquisition condition;
a transmitting unit that transmits the sound data to a doctor terminal of a doctor when the acquisition condition is satisfied;
A biological sound acquisition support system comprising: The body sound acquisition support system of claim 1, wherein the transmission unit converts the sound data into a spectrogram and transmits the spectrogram to the doctor terminal. a sound evaluation unit that evaluates a specific sound component from the acquired sound data;
The body sound acquisition support system according to claim 1 , wherein the determination unit determines whether or not the body sound acquisition conditions are met based on the evaluation result by the sound evaluation unit . the sound evaluation unit evaluates the volume of a target body sound component;
the acquisition conditions include conditions related to the volume of the body sound components,
The body sound acquisition support system according to claim 3 , wherein the determining unit determines whether or not the volume of the body sound component satisfies the acquisition condition. the sound evaluation unit evaluates the volume of a noise component;
the acquisition conditions include a condition related to the volume of the noise component;
The body sound acquisition support system according to claim 3 , wherein the determination unit determines whether or not the volume of the noise component satisfies the acquisition condition. a position calculation unit that calculates terminal position information associated with movement of the user terminal;
The body sound acquisition support system according to claim 1 , wherein the determination unit determines whether the terminal location information matches the acquired location information. the doctor terminal accepts input of instruction information regarding an acquisition position of the user's body sound by the doctor;
The body sound acquisition support system according to claim 1 , wherein the determination unit determines whether the acquisition position information matches the instruction information. The body sound acquisition support system of claim 7, characterized in that the instruction information includes the acquisition position information and at least one of the number of acquisitions or the acquisition order. The body sound acquisition support system of claim 7, wherein the doctor terminal allows the doctor to select the acquisition position on the displayed image. The body sound acquisition support system of claim 1, wherein the acquisition condition is an image in which acquisition positions are plotted on the body. The body sound acquisition support system of claim 10, wherein the acquisition condition is that the acquisition position and acquisition order are plotted on the image. The body sound acquisition support system of claim 1, wherein the acquisition conditions include a distance and a direction from a reference position, and the determination unit detects the distance and the direction of movement of the acquired position information from the reference position. The body sound acquisition support system of claim 1, wherein the acquisition conditions include a distance and a direction from a position, and the determination unit determines whether a second position to which the body sound is moved from a first position that is a reference position satisfies the acquisition conditions, and, after acquiring the body sounds at the second position, determines whether a third position to which the body sound is moved from the second position satisfies the acquisition conditions. The body sound acquisition support system of claim 1, wherein the acquisition conditions are set for each type of body sound, disease, acquisition site, and acquisition purpose.