JP7540700B2 - Body sound acquisition support system - Google Patents

Description

The present invention relates to a body sound acquisition support system.

A remote medical support system has been proposed that enables remote medical treatment by communicating via videophone between a doctor's terminal and a patient's terminal.

Patent document 1 discloses an information processing device that provides a means for remote medical consultation reservations and video calling between doctors and patients.

Patent No. 6574391

In remote medical care, patients themselves need to collect various vital signs data required for a doctor's examination and provide it to the doctor's device. In addition, with people becoming more health conscious in recent years, there is also a growing demand for daily health management.

Therefore, one object of the present invention is to provide a system for acquiring body sounds using a mobile terminal such as a smartphone.

According to the present invention, a body sound acquisition support system is obtained that supports a user in acquiring body sounds using a user terminal, the body sound acquisition support system comprising a sound data acquisition unit that acquires sound data including body sounds, a sound evaluation unit that evaluates specific sound components from the acquired sound data, a condition storage unit that stores conditions for determining whether or not body sounds can be acquired, a determination unit that determines whether or not the conditions are met, and a notification unit that outputs the determination result to the user terminal.

The present invention allows users to smoothly obtain body sounds using a device such as a smartphone.

FIG. 1 is a diagram illustrating an example of a configuration of a system according to an embodiment of the present invention. FIG. 2 is a diagram illustrating an example of a hardware configuration of a user terminal according to an embodiment of the present invention. FIG. 2 is a diagram illustrating an example of a hardware configuration of a server according to an embodiment of the present invention. FIG. 2 is a diagram illustrating an example of a software configuration according to an embodiment of the present invention. FIG. 4 is a diagram illustrating an example of a configuration of a condition storage unit according to the embodiment of the present invention. FIG. 4 is a diagram illustrating an example of a configuration of a notification information storage unit according to the embodiment of the present invention. FIG. 2 is a diagram showing an example of a process flow according to an embodiment of the present invention. FIG. 2 is a diagram illustrating an example of a software configuration according to an embodiment of the present invention. FIG. 1 is a diagram illustrating an embodiment of the present invention. FIG. 4 is a diagram illustrating an example of a configuration of a condition storage unit according to the embodiment of the present invention. FIG. 1 is a diagram illustrating an embodiment of the present invention. FIG. 4 is a diagram illustrating an example of a configuration of a notification information storage unit according to the embodiment of the present invention. FIG. 2 is a diagram illustrating an example of a software configuration according to an embodiment of the present invention. FIG. 1 is a diagram illustrating an embodiment of the present invention. FIG. 4 is a diagram illustrating an example of a configuration of a notification information storage unit according to the embodiment of the present invention. FIG. 4 is a diagram illustrating an example of a configuration of a notification information storage unit according to the embodiment of the present invention. FIG. 2 is a diagram illustrating an example of a software configuration according to an embodiment of the present invention. FIG. 2 is a diagram illustrating an example of a software configuration according to an embodiment of the present invention. FIG. 1 is a diagram illustrating an embodiment of the present invention. FIG. 2 is a diagram illustrating an example of a software configuration according to an embodiment of the present invention. FIG. 11 is a diagram showing an example of a configuration of acquisition instruction information according to an embodiment of the present invention.

The present invention will be described below with reference to the preferred embodiments.
[Item 1]
A body sound acquisition support system that supports a user to acquire a body sound by using a user terminal, comprising:
a sound data acquisition unit that acquires sound data including the body sound;
a determination unit that determines whether or not the body sound acquisition condition is met;
and a notification unit that outputs the determination result to the user terminal.
[Item 2]
A sound evaluation unit that evaluates a specific sound component from the acquired sound data,
2. The body sound acquisition support system according to claim 1, wherein the determination unit determines whether or not the body sound acquisition condition is met based on the evaluation result.
[Item 3]
The sound evaluation unit evaluates the volume of a target body sound component,
the acquisition conditions include conditions related to the volume of the body sound components,
3. The body sound acquisition support system according to claim 2, wherein the determining unit determines whether or not a volume of the body sound component satisfies the acquisition condition.
[Item 4]
The sound evaluation unit evaluates the volume of a noise component,
the acquisition condition includes a condition related to a volume of the noise component,
4. The body sound acquisition support system according to claim 2, wherein the determination unit determines whether or not a volume of the noise component satisfies the acquisition condition.
[Item 5]
An angle data acquisition unit that acquires angle data of the user terminal,
the acquisition condition includes an appropriate angle condition of the user terminal when acquiring the body sound,
5. The body sound acquisition support system according to any one of claims 1 to 4, wherein the determination unit determines whether the angle data matches the angle condition.
[Item 6]
A position calculation unit that calculates terminal position information associated with movement of the user terminal,
The acquisition condition includes acquisition position information of the body sound,
The body sound acquisition support system according to any one of claims 1 to 5, wherein the determination unit determines whether the terminal location information matches the acquired location information.
[Item 7]
An instruction information input unit that receives an input of instruction information related to an acquisition position of the body sound of the user by an expert,
7. The body sound acquisition support system according to claim 6, wherein the acquisition position information is generated based on the instruction information.
[Item 8]
a body type estimation unit that estimates a body type of the user based on input information of the user;
7. The body sound acquisition support system according to item 6, further comprising an acquisition instruction information adjustment unit that adjusts the acquisition position information in accordance with the body type.

<Details of the embodiment>
Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

＜Overview＞
Fig. 1 is a diagram for explaining an outline of a body sound acquisition system according to an embodiment of the present invention. As shown in the figure, the body sound acquisition system includes a user terminal 2 and a server 1 to which the user terminal 2 is connected via a network, the Internet. If necessary, the system may also include a doctor terminal 3 connected to the network. The user terminal 2 and the doctor terminal 3 can be connected to terminals of multiple users.

The body sound acquisition system according to the embodiment of the present invention is used in cases where body sounds acquired by a user terminal 2 are provided to a doctor terminal 3 by linking with a remote medical care system, and in cases where a user acquires body sounds for their own health management.

In this embodiment, the user terminal 2 is a device that can be connected to an Internet network, such as a smartphone, a computer, or a tablet terminal, which is a portable information terminal. In particular, it is preferable that the user terminal 2 is a smartphone equipped with a microphone for capturing voice and various sensors for measuring acceleration and terminal angle.

Figure 2 is a diagram outlining the hardware configuration of the user terminal 2. As shown in the figure, the user terminal 2 mainly comprises a control unit 10, memory 11, storage 12, a transmission/reception unit 13, an input/output unit 14, a sensor 16, a microphone 17, and a display unit 18, which are electrically connected to each other via a bus 15.

The control unit 10 is a computing device that controls the operation of the user terminal 2, controls the transmission and reception of data between each element, and performs processing necessary for executing application programs.

In this embodiment, the control unit 10 is, for example, a CPU (Central Processor
The processing unit 11 executes application programs stored in a storage 12 (to be described later) and deployed in a memory 11 to perform various processes.

Memory 11 includes a main memory device consisting of a volatile memory device such as a DRAM (Dynamic Random Access Memory), and an auxiliary memory device consisting of a non-volatile memory device such as a flash memory or a HDD (Hard Disc Drive).

The memory 11 is used as a working area for the control unit 10, and also stores the BIOS (Basic Input/Output System) that is executed when the computer is started.
m), and various setting information, etc. are stored.

The storage 12 stores programs, data used for various processes, and the like.
Details of the programs and the like stored in the storage 12 will be described later. The storage 12 may be connected so as to be available using cloud computing technology.

The transmission/reception unit 13 connects the user terminal 2 to the Internet network. This transmission/reception unit 13 may be equipped with a short-range communication interface such as Bluetooth (registered trademark) or BLE (Bluetooth Low Energy).

If necessary, information input devices such as a keyboard and a mouse and output devices such as a display are connected to the input/output unit 14.

The bus 15 transmits, for example, address signals, data signals, and various control signals between the connected control unit 10, memory 11, storage 12, transceiver unit 13, and input/output unit 14.

The sensor 16 may be any type of sensor, such as an acceleration sensor, a tilt sensor (geomagnetic sensor or gyro sensor), or a proximity sensor.

The microphone 17 is a sound collecting device mounted on the user terminal 2. The microphone 17 converts the sound input to the terminal 2 into an audio signal. Alternatively, a sound collector externally connected to the user terminal 2 may be used.

Figure 3 is a diagram outlining the hardware configuration of server 1. As shown in the figure, server 1 mainly comprises a control unit 20, memory 21, storage 22, transmission/reception unit 23, and input/output unit 24, which are electrically connected to each other via bus 25. The functions of each unit are the same as those of user terminal 2, so a detailed description is omitted.

<Embodiment 1>
In the first embodiment, it is determined whether or not the sound data acquired by the microphone 17 is suitable for acquiring body sounds. Specifically, the acquired sound data is analyzed to determine whether the body sound data included in the sound data has been acquired with sufficient quality.

Figure 4 is a diagram showing an example of the software configuration of the user terminal 2 in embodiment 1 of the present invention. The user terminal 2 of the present invention includes a condition storage unit 121, a notification information storage unit 122, and a sound data storage unit 123 as storage devices, and a data transmission/reception unit 111, a sound data acquisition unit 112, a sound evaluation unit 113, a judgment unit 114, a display control unit 115, and a notification unit 116 as a calculation processing unit. The data transmission/reception unit 111, the sound data acquisition unit 112, the sound evaluation unit 113, the judgment unit 114, the display control unit 115, and the notification unit 116 are realized by the control unit 10 reading out a program stored in the storage 12 into the memory 11 and executing it, and the condition storage unit 121, the notification information storage unit 122, and the sound data storage unit 123 are realized as part of the storage area provided by at least one of the memory 11 and the storage 12.

The condition storage unit 121 stores conditions for determining whether or not it is suitable for acquiring body sounds. FIG. 5 shows an example of the configuration of condition information stored in the condition storage unit 121. In the first embodiment, conditions are set regarding the quality of body sounds contained in the acquired sound data. Quality is, for example, volume, and may be a threshold value expressed as an index such as a decibel value, but is not limited to this. Multiple of these conditions can be set depending on the type of body sound to be acquired (heart sounds, respiratory sounds, lung sounds, peristalsis sounds, swallowing sounds, etc.), the acquisition site (right chest, left chest, stomach, intestines, etc.), the acquisition purpose (provision to a doctor, personal health management), etc.

The notification information storage unit 122 stores the contents of the notification output by the notification unit 116 according to the judgment result. FIG. 6 is a configuration example of the notification information stored in the notification information storage unit 122. The notification information includes the setting of the notification contents for each judgment result. For example, the notification to be issued is set depending on whether the judgment result is OK (suitable for acquiring body sounds) or NG (not suitable). The sound notifications 1 and 2 may be, for example, a voice message such as notification 1 "You can acquire body sounds" and sound notification 2 "You cannot acquire body sounds". The sound notification may not be limited to voice, but may be a predetermined notification sound. The notification sound may be expressed by the pitch or pattern of the sound (beep beep, buzz buzz, etc.). The notification may not be limited to sound notification by voice or notification sound, but may be light, vibration, or a combination of these. The light notification is output by the screen or light emitting function of the terminal 2, and may be expressed by the color of light, the pattern of lighting/flashing, the intensity of light, etc. Furthermore, the vibration notification is output by the vibration function of the terminal, and can be expressed by the vibration pattern, strength, etc. The notification information storage unit 122 can store the contents of these sound notifications, light notifications, and vibration notifications according to each determination result. The notification information storage unit 122 may store the voice data itself, or may store output instruction information for various notifications.

The sound data storage unit 123 stores the acquired sound data. The sound data can be stored together with information such as the date and time of acquisition, the type of acquired biological sound (heart sound, respiratory sound, lung sound, peristalsis sound, etc.), and the acquisition site (anterior chest, neck, back, etc.). The acquisition site may include a more detailed acquisition position, and may be position information expressed by coordinate information from a reference position as described below. In addition, it may further include information regarding the acquisition purpose such as for medical examination or self-management. Furthermore, if necessary, it may include information indicating the user's condition when the sound data was acquired. For example, it may include subjective symptoms such as shortness of breath and fatigue, image data showing the user's face, body, or affected area, and vital data such as pulse, blood pressure, and body temperature acquired by various measuring devices such as wearable devices. In addition, the stored sound data may include waveform data, or may be data processed by the sound evaluation unit 113 described below.

The data transmission/reception unit 111 uses the transmission/reception unit 13 to transmit various data to the server 1 or the doctor terminal 3.

The sound data acquisition unit 112 acquires sound data collected by the microphone 17 provided on the user terminal 2, stores it in the sound data storage unit 123, and transmits it to the sound evaluation unit 113.

The sound evaluation unit 113 evaluates the loudness and quality of a specific sound component. The sound evaluation unit 113 can evaluate the loudness and quality of a target body sound component. The sound evaluation unit 113 performs extraction processing to extract body sound data by separating body sound components and noise components other than body sounds by a predetermined process from the sound data as necessary, or by removing the noise portion. When the processing method and parameters differ depending on the type of body sound and the acquisition position, the processing method and parameters are selected based on the type of input body sound and the acquisition position, and the extraction process is performed. A known method can be used as the extraction method. For example, the noise components can be removed by applying a predetermined filter to the voice data. In addition, the sound data can be converted into a spectrogram and noise can be removed by masking a specific component. If necessary, the sound data can be subjected to a predetermined preprocessing before the spectrogram conversion. In addition, sound data including both body sound components and noise components can be used as input data, and a learning model can be created using body sound data including only body sound components or body sound data including noise components as teacher data, and body sound data can be extracted by applying the created learning model to the sound data. The input data for the learning model may be a spectrogram or the like obtained by converting sound data, so long as the data can be restored to waveform data. Similarly, the output data may be a spectrogram or the like that can be restored to waveform data. In addition, a similar method may be used to extract noise components from sound data that contains body sound data and noise components, and extract other components as body sound data.

The determination unit 114 determines whether the conditions are suitable for acquiring body sounds based on the evaluation result of the sound evaluation unit 113. In other words, it determines whether the target body sounds have been acquired sufficiently.

The notification unit 116 notifies the result of the determination. The notification unit 116 may, for example, emit sound, light, vibration, or a combination of these, and issues a notification by issuing a command to a sound generating means, light generating means, vibration generating means, etc., mounted on the user terminal 2. Depending on the determination result of the determination unit 114, the notification unit 116 reads out the notification information registered in the notification information storage unit 122 and performs processing to output a notification.

The notification may also be made by displaying a predetermined message or image on the screen of the user terminal 2. In this case, the notification unit 116 can display the predetermined image or message on the user terminal 2 by controlling the display control unit 115 to display the predetermined image or message on the screen based on the result of the determination unit 114. When the user terminal 2 is held up against the user's body, the user may not be able to see the terminal screen, so if this situation is anticipated, notification by sound, light, or vibration is preferable to notification by display.

As described above, in the first embodiment, the body sound data in the acquired sound data is evaluated, and it is determined whether sufficient body sound data has been acquired. In the above, the body sound data included in the sound data is evaluated, but the noise components may be evaluated and it may be determined that the noise component is equal to or smaller than a predetermined level. In this case, the user acquires sound using the microphone 17 of the user terminal 2 in the environment in which the body sound is about to be acquired. At this time, the microphone 17 may be brought into contact with the body as in the case of actually acquiring the body sound, but the sound may also be acquired without being brought into contact with the body. The sound evaluation unit 113 evaluates the level and quality of the noise components from the acquired sound, and the determination unit 114 determines whether the noise components satisfy a predetermined condition. The predetermined condition is, for example, the volume. If the determination result is good (low noise), the notification unit 116 notifies the user that the body sound can be acquired. On the other hand, if the determination result is poor (high noise), the notification unit 116 notifies the user of this fact and encourages the user to change the environment, such as by moving to a different location.

Figure 7 shows the operation flow in this embodiment 1. Unless otherwise specified, the order of each step may be changed.

First, the user starts the application on his/her mobile terminal (user terminal 2) (S101) and, in some cases, inputs the type of body sound to be acquired and the body part to be acquired. Terminal 2 provides guidance by displaying a guide screen for acquiring body sounds on the screen of terminal 2 or by outputting the guidance by voice (S102).

The user places the microphone 17 of the user terminal 2 against the acquisition site to start acquiring sound data (S103). The sound evaluation unit 113 evaluates the target body sound component from the acquired sound data. The sound evaluation unit 113 may also extract body sound components from the acquired sound data by a predetermined process as necessary and then evaluate them. The sound evaluation unit 113 communicates the evaluation result to the determination unit 114. The determination unit 114 determines whether the acquired sound is suitable for use in a doctor's examination or analysis for disease estimation (S104). That is, it determines whether the extracted body sound data meets the criteria stored in the condition storage unit 121. Specifically, for example, it determines whether the volume of the body sound data is equal to or greater than a predetermined value.

If the judgment is NG (body sounds have not been acquired sufficiently) (S104=No), the notification unit 116 notifies the user to urge them to acquire again (S105). The notification can be made, for example, by a voice message or a screen display such as "Body sounds could not be acquired. Try again" or "Please move to a quiet place". If the judgment is OK (body sounds have been acquired sufficiently) (S104=Yes), the notification unit 116 notifies the user that body sounds have been acquired (S106). In addition, the acquired sound data (which may also include analysis data such as waveform data) is transmitted to the server as necessary (S107).

In the first embodiment, the description has been given on the assumption that the user terminal 2 is provided with the calculation processing units such as the sound evaluation unit 113 and the judgment unit 114, and the storage units such as the condition storage unit 121, the notification information storage unit 122, and the sound data storage unit 123, but these may also be provided in the server 1. In that case, the sound data acquired by the sound data acquisition unit 112 is transmitted to the server 1, the server 1 analyzes and judges the sound, and the judgment result is transmitted to the user terminal 2.

<Embodiment 2>
In the second embodiment, it is determined whether or not the angle of the user terminal 2 is suitable for acquiring body sounds. That is, when acquiring body sounds by bringing a microphone mounted on the user terminal 2 such as a smartphone into contact with the user's body, it may be easier to acquire body sounds if the terminal angle with respect to the body is within a certain range. In addition, it may be preferable to acquire body sounds at a uniform certain terminal angle in order to ensure uniformity of the body sounds. In this embodiment, in such a case, the actual angle of the user terminal 2 is grasped from sensor data, and it is determined whether or not it is within a range of terminal angles registered in advance.

Figure 8 shows an example of the software configuration of the user terminal 2 in this embodiment 2, which further includes a sensor input control unit 117 in addition to the software configuration in embodiment 1. The user terminal 2 also includes a tilt sensor that can detect the angle of the terminal 2. The tilt sensor may be anything that can detect the tilt angle in each direction in the user terminal, such as a geomagnetic sensor, a gyro sensor, or an acceleration sensor. Note that here, the tilt angle in each direction of the user terminal is expressed as the x-axis toward the right of the screen, the y-axis toward the longitudinal direction upward, and the z-axis toward the vertical upward direction of the screen, as shown in Figure 9, and the x-axis rotation is expressed as pitch, the y-axis rotation as azimuth, and the z-axis rotation as roll, as shown in Figure 4(b).

In the second embodiment, the condition storage unit 121 preregisters an appropriate angle range of the user terminal 2 for each body sound acquisition site (right chest, left chest, stomach, intestines, etc.), disease, or type of body sound (heart sound, respiratory sound, lung sound, peristalsis sound, swallowing sound, etc.). FIG. 10 is an example of the configuration of condition information stored in the condition storage unit 121. As an example, when measuring heart sounds, it is optimal to hold the terminal 2 so that the bottom surface (microphone part) faces the body and the orientation of the screen of the terminal 2 (Z axis) is horizontal, as shown in FIG. 11. The condition in this case is "tilt angle 90° and azimuth angle 90°". The implementation condition may be set to a predetermined allowable range as appropriate, for example, "tilt angle 85° to 95° and azimuth angle 85° to 95°". When making a judgment based on the terminal angle, the values and ranges of the tilt angle, azimuth angle, and rotation angle are the conditions. All of these conditions can be set in multiple ways depending on the type of vital sound you want to acquire (heart sounds, breathing sounds, lung sounds, peristalsis sounds, swallowing sounds, etc.), disease, acquisition site (right chest, left chest, stomach, intestines, etc.), acquisition purpose (provision to a doctor, personal health management), number of acquisitions (first time, second time, etc.), etc.

The notification information storage unit 122 of the second embodiment stores the contents of the notification output by the notification unit 116 according to the judgment result. FIG. 12 is a configuration example of the notification information stored in the notification information storage unit 122. The notification information includes the setting of the notification contents for each judgment result. The notification is not limited to voice, but may be a notification sound, light, vibration, display, or a combination of these. When making a judgment based on the angle of the user terminal 2, a notification when the judgment is OK (the terminal angle is within a specified range) and a notification when the judgment is NG can be registered as in the first embodiment. In addition, when the judgment is NG, a further notification may be set according to the magnitude relationship between the measured value of the terminal angle and a threshold value as shown in FIG. 12. For example, if the tilt angle is smaller than the threshold value, a voice message (sound notification 3) such as "Please tilt the terminal a little further back" may be issued. In addition, when the angle of the user terminal 2 is within a specified range, a predetermined sound, light, and vibration may be emitted, and when it is outside the range, these outputs may be stopped. Conversely, a specific sound, light, or vibration may be emitted when the angle of the user terminal 2 is outside a predetermined range, and these outputs may be stopped when the angle is within the range.

The sensor input control unit 117 transmits angle data of the user terminal 2 acquired by a tilt sensor mounted on the user terminal 2 to the determination unit 114. The tilt sensor is, for example, a geomagnetic sensor, a gyro sensor, or an acceleration sensor, and can express the angle of the user terminal 2 as a tilt angle (pitch), a rotation angle (roll), and an azimuth angle (azimuth).

The determination unit 114 reads out angle conditions from the condition storage unit 121 according to the body part and type of body sound to be acquired. If the body part and type of body sound to be acquired are predetermined, the corresponding conditions may be read out. If the user inputs the body part and type of body sound to be acquired, the conditions are read out according to the input. Then, if the angle of the user terminal 2 input from the sensor input control unit 117 meets the conditions, it is determined that it is suitable for acquiring body sound, and if it does not meet the conditions, it is determined that it is not suitable.

The notification unit 116 notifies the judgment result. Based on the judgment result by the judgment unit 114, the notification content is read from the notification information storage unit 122 and the notification is output.

<Embodiment 3>
In the third embodiment, it is determined whether the position of the user terminal 2 is suitable for acquiring body sounds. In acquiring body sounds, the position where the microphone should be placed is determined according to the purpose, but when the user acquires body sounds by himself/herself, there is a problem that it is not possible to determine whether the microphone is placed at an appropriate acquisition position. Therefore, in the third embodiment, it is determined whether the position of the user terminal 2 is in a predetermined setting position.

Figure 13 shows an example of the software configuration of a user terminal 2 in this embodiment 3, which further includes a sensor input control unit 117 and a position calculation unit 118 in addition to the software configuration in embodiment 1.

The condition storage unit 121 of the third embodiment stores conditions for determining whether the terminal 2 is located at an acquisition position suitable for acquiring body sounds. For example, the conditions are the distance and the movement direction to each measurement position relative to the reference position. A plurality of conditions can be set according to the type of body sound to be acquired (heart sound, respiratory sound, lung sound, peristalsis sound, swallowing sound, etc.), disease, acquisition site (right chest, left chest, stomach, intestine, etc.), acquisition purpose (provision to a doctor, self-health management), etc. FIG. 14 is a configuration example of condition information stored in the condition storage unit 121. The acquisition position condition may be image data in which the acquisition position and order are plotted on the body as shown in FIG. 14(a). It may also be coordinate information as shown in FIG. 14(b). As an example of the coordinate information, the relative coordinates of the second position with respect to the reference position (first position) on the user's body are stored as the terminal position condition. As shown in FIG. 14(a), for example, it is assumed that the first position is the epigastrium and the second position is the first body sound acquisition position (right clavicle). In this case, the condition information can be determined as coordinate information of the second position relative to the first position within a predetermined range (for example, 0 to 0 in the X direction and 0 to 0 in the Y direction). In addition, when body sounds are continuously acquired at multiple positions, the next acquisition position (third position) can be determined by the coordinate information of the third position relative to the second position described above.

The notification information storage unit 122 stores the contents of the notification output by the notification unit 116 according to the determination result. FIG. 15 shows an example of the configuration of notification information stored in the notification information storage unit 122. When making a determination based on the position of the user terminal 2, a notification can be made by outputting different sounds (notifications 9 and 10) when the terminal 2 is not within the coordinate range of the set acquisition position and when the terminal 2 is within the coordinate range of the acquisition position. For example, the terminal 2 may continuously emit a sound and the sound may change when the terminal 2 enters the coordinate range of the acquisition position (notification 9 → notification 10), or a predetermined sound may be emitted only when the terminal 2 enters the coordinate range (only notification 10). Alternatively, different notifications may be issued when the terminal 2 is approaching the coordinate range of the acquisition position and when the terminal 2 is moving away. By providing such notifications, the user can easily adjust the position where the terminal 2 is placed.

The sensor input control unit 117 transmits information input from each sensor, such as an acceleration sensor, to the position calculation unit 118 and the determination unit 114.

The position calculation unit 118 calculates the position information of the user terminal 2 using data obtained from an acceleration sensor mounted on the user terminal 2. The position calculation unit 118 calculates the movement direction and movement distance from the acceleration data. The movement distance can be calculated by integrating the acceleration sensor value using a known method such as trapezoidal integration. As an example, when a reference position (first position) is set on the user's body, and the user terminal 2 is placed against the first position, and then the terminal 2 is moved to a second position different from the first position, the position calculation unit 118 can derive the coordinates of the second position (a position relative to the first position) from the movement direction and movement distance. The method of determining the position of the terminal 2 is not limited to this, and any known method may be adopted.

The determination unit 114 determines whether the user terminal 2 is placed in a position on the user's body suitable for acquiring body sounds. As an example, a reference position on the body (first position) is determined. The first position is, for example, a position that is easily understood by the user and can be uniquely determined, such as the solar plexus or clavicle. First, the user is instructed to position the user terminal 2 at the first position, and the terminal 2 is stopped once for a predetermined time. Next, the user terminal 2 is moved from the first position to a position (second position) where body sounds are acquired, and acceleration data during that time is acquired. The position calculation unit 118 calculates the moving direction and moving distance of the terminal 2 from the acceleration data, and calculates the coordinates of the user terminal 2 relative to the first position.

The notification unit 116 notifies the judgment result. For example, when the position of the user terminal 2 enters the range of the body sound acquisition position (second position) stored in the condition storage unit 121, a predetermined sound, light, vibration, etc. may be emitted, and when the user moves the terminal 2 out of the range of the acquisition position, the generation of the predetermined sound, light, and vibration may be stopped. Or, conversely, when the position of the terminal 2 is out of the range of the acquisition position, a sound, light, or vibration may be emitted to indicate that the position is inappropriate, and when the terminal 2 moves to an appropriate position, the sound, light, and vibration may be stopped. In addition, when the user terminal 2 is moving, if the position of the terminal 2 is approaching the range of the body sound acquisition position, a first notification may be issued, and conversely, if the terminal 2 is away from the range, a second notification may be issued. The first notification is, for example, a notification that prompts the user to continue moving, and the second notification is a notification that prompts the user to correct the direction. The first notification and the second notification may be voice messages, or may be sound, light, or vibration. In addition, only one of the first notification and the second notification may be issued. For example, the first notification may not be sent, and the second notification may be sent only when it is necessary to prompt the user to correct the direction of movement.

<Embodiment 4>
In the fourth embodiment, it is determined whether the user terminal 2 is in a state where the loudest body sound can be acquired. The state of the user terminal 2 is, for example, a state related to the angle or position with respect to the user's body. In the second and third embodiments, it is determined whether the angle or position of the user terminal 2 is within a predetermined range, but in the fourth embodiment, it is relatively determined whether the angle or position is such that the body sound can be acquired satisfactorily.

The software configuration example of the user terminal 2 in this embodiment 4 is the same as that in embodiment 4, and in addition to the software configuration in embodiment 1, it further includes a sensor input control unit 117 and a position calculation unit 118.

In this embodiment 4, the state of the terminal 2 in which the body sound can be best acquired is relatively determined, so conditions for absolutely judging whether or not it is suitable for acquiring body sound do not have to be set, but they can be set as an auxiliary condition. For example, as in the above embodiment 1, conditions can be set for judging that the volume of the body sound is equal to or greater than a predetermined value, or that there is little noise. In addition, as in embodiments 2 and 3, conditions can also be set for the angle and position of the terminal 2. In this embodiment 4, assuming that these conditions are met, the optimal state can also be selected based on the volume of the body sound.

The notification information storage unit 122 of the fourth embodiment stores the contents of the notification output by the notification unit 116 according to the judgment result. FIG. 16 shows an example of the configuration of the notification information stored in the notification information storage unit 122. A notification (notification 11) when the body sound acquisition state changes to good with the movement of the terminal 2, and a notification (notification 12) when the body sound acquisition state deteriorates can be registered. The notification may be sound, light, vibration, or a combination of these. For example, notification 11 and notification 12 may be registered as notification sounds with different tones or volumes, light notifications with different light colors or intensities, or notifications with different vibration intensities, tempos, or patterns. As described later, the notification changes according to the result of the judgment unit 114, allowing the user to intuitively judge whether the direction in which the terminal 2 is moving at that time is correct or not.

The sound data acquisition unit 112 acquires sound data from the microphone 17 provided on the user terminal 2 for a predetermined period of time. At this time, the user terminal 2 is placed in contact with the user's body, as when acquiring actual body sounds, and sound data is continuously acquired while changing the angle of the user terminal 2 and the acquisition position on the body. Then, the sound at a certain point in time is compared and analyzed with the sound at the previous point in time to identify the angle and location on the body that is most suitable for acquiring body sounds.

Sound may be acquired continuously or at a specified time interval. The user acquires sound while slowly moving the terminal near the designated acquisition area. To identify an appropriate terminal angle, the user may acquire sound by rotating the terminal 2 while it is in contact with the body and changing the angle, and to identify an appropriate terminal position, the user acquires sound while moving the terminal 2 on the body.

The sound evaluation unit 113 evaluates the volume of the body sound data included in the sound data in the same manner as in embodiment 1. If necessary, a process may be performed to extract the desired body sound data from the sound data.

The determination unit 114 compares the body sound data acquired at time _Tn with the body sound data acquired at time Tn ₊₁ after time _Tn , and when the body sound data acquired at the newer time Tn ₊₁ is larger than the body sound data at time _Tn , the determination unit 114 determines that the angle or position of the terminal 2 at time Tn ₊₁ is more suitable for acquiring body sound than the angle or position of the terminal 2 at time _Tn . Conversely, when the body sound data acquired at the newer time Tn ₊₁ is smaller than the body sound data at time _Tn , the determination unit 114 determines that the angle or position of the terminal 2 at time Tn ₊₁ is less suitable for acquiring body sound than the angle or position of the terminal 2 at time _Tn . The interval between times _Tn and Tn ₊₁ may be a predetermined interval, and may be, for example, 0.1 to 1 second. The times _Tn , Tn ₊₁ , ... may be a predetermined time width, in which case the average of the sound data during that time can be used for comparison. Also, the volume of the body sound data may be compared, or the magnitude of the noise components may be compared.

The notification unit 116 performs a predetermined notification based on the comparison result between the sound data at time T _n+1 and the sound data at time T _n . As an example, the user terminal 2 continuously emits a notification sound, and when it is determined that the sound data at time T _n+1 is more suitable for acquiring a body sound than the sound data at time T _n , the tone and volume of the notification sound can be changed. Similarly, when light is continuously emitted from the user terminal 2, and when it is determined that the sound data at time T n ₊₁ is more suitable for acquiring a body sound than the sound data at time T _n , the color and intensity of the light can be changed. In addition, when vibration is continuously emitted from the user terminal 2, and when it is determined that the sound data at time T _n+1 is more suitable for acquiring a body sound than the sound data at time T _n , the intensity, tempo, and pattern of the vibration can be changed. For example, changing the volume means gradually changing the notification content, for example, gradually increasing the volume when the judgment result is good, and gradually decreasing the volume in the opposite case. Also, when it is determined that the sound data at time Tn ₊₁ is more suitable for acquiring body sounds than the sound data at time _Tn , a notification such as a notification sound, light, or vibration is issued, and conversely, when it is determined that the sound data at time Tn ₊₁ is less suitable for acquiring body sounds than the sound data at time _Tn , a notification sound is not issued. Alternatively, when it is determined that the sound data at time Tn ₊₁ is less suitable for acquiring body sounds than the sound data at time _Tn , a notification such as a notification sound, light, or vibration is issued, and conversely, when it is determined that the sound data at time Tn ₊₁ is more suitable for acquiring body sounds than the sound data at time _Tn , a notification sound is not issued. With such notifications, the user can know in real time while moving the terminal 2 whether the terminal 2 has been moved to a state suitable for acquiring body sounds, or whether the trajectory should be corrected to a different direction or angle.

The determination unit 114 may further determine whether the sound data satisfies a predetermined condition. For example, the condition is that the volume of the body sound included in the sound data is greater than a predetermined value, and while determining whether the condition is satisfied, the determination unit 114 compares the sound data at time T _n+1 with the sound data at time T _n as described above to determine whether the moving direction and the moving accuracy are correct. Also, as in the second and third embodiments, while determining that the terminal 2 is placed within a predetermined angle range or position range of the terminal 2, the determination unit 114 may compare the sound data at time T _n+1 with the sound data at time T _n as described above to determine whether the moving direction and the moving accuracy are correct.

As described above, the notification unit 116 notifies the result of the comparison between the sound data at time Tn ₊₁ and the sound data at time _Tn as the terminal 2 moves, and further notifies the user when the sound data satisfies a predetermined condition. For example, when the user moves the terminal 2 in a certain direction toward an appropriate acquisition position, the sound data at time Tn ₊₁ is determined to be more suitable for acquiring body sounds than the sound data at time _Tn , so the notification unit 116 notifies the user that the moving direction is correct by issuing a predetermined notification such as gradually increasing the volume of the notification sound. Then, when the sound data acquired at a recent time satisfies a predetermined condition (for example, when the body sounds are louder than a certain volume), a notification to stop the terminal can be issued.

<Embodiment 5>
In the fifth embodiment, the doctor instructs the location where the body sounds are to be acquired. The measurement method, such as the location, order, and number of times of acquiring the body sounds required by the doctor for the examination, may differ depending on the user's age, illness, constitution, etc. In addition, the acquisition location may differ slightly depending on the user's gender, body size, and body type. Thus, the fifth embodiment relates to a method in which image data including the user's body (including the body sound acquisition site) is provided to the doctor terminal 3, and the doctor inputs instructions regarding the method of acquiring the body sounds.

Figure 17 shows an example of the software configuration of a user terminal 2 in this embodiment 5, which further includes a sensor input control unit 117, a position calculation unit 118, and an image data acquisition unit 119 in addition to the software configuration in embodiment 1.

Figure 18 shows an example of the software configuration of the doctor terminal 3 in this embodiment 5, and can include a data transmission/reception unit 211, an instruction information input unit 212, an instruction image generation unit 213, and a position information generation unit 214 as a calculation processing device.

In this embodiment 5, first, an image of the user's body including the body part from which body sounds are to be obtained is captured by a camera or the like mounted on the user terminal 2. The image may be captured by an image capturing means other than the camera mounted on the user terminal 2. The image data acquisition unit 119 of the user terminal 2 acquires the captured image data. The image data may be a still image or a video.

The data transmission/reception unit 111 of the user terminal 2 transmits the image data to the doctor terminal 3, and the data transmission/reception unit 211 of the doctor terminal 3 acquires the image data. Alternatively, the data transmission/reception unit 111 of the user terminal 2 may transmit the image data to the server 1, and the server 1 may transmit the image data to the doctor terminal 3. It is also preferable to transmit user information (such as name, age, sex, disease, etc.) to the doctor terminal 3 along with the user's image data. If the user has been examined at a medical institution to which the doctor belongs, identification information such as a patient number may also be included.

Figure 19 is a diagram showing how a doctor inputs instruction information. As shown in Figure 19(a), a user image 51 and user information 52 are displayed on the screen display unit (such as a display) of the doctor terminal 3, and the doctor determines the conditions for acquiring body sounds by looking at the user image 51 and user information 52 acquired by the doctor terminal 3. As shown in Figure 19(b), the instruction information input unit 212 of the doctor terminal 3 accepts input of instructions regarding the conditions for acquiring body sounds from the doctor. For example, the doctor may input instructions by selecting an acquisition position on the image. That is, the acquisition position can be directly specified by a method such as placing the mouse cursor or touching a touch panel. Also, information such as the number of acquisitions and the order of acquisition can be input according to the acquisition position.

The instruction image generating unit 213 generates an acquisition instruction image based on the information received by the instruction information input unit 212. The acquisition instruction image may include a superimposed image in which an indication (such as a mark) 53 indicating the acquisition position is superimposed on an image 51 showing the user's body, as shown in FIG. 19(c). The superimposed image may further include information such as the order and number of times of acquisition.

The position information generating unit 214 converts the instruction information input by the doctor into position information including coordinates representing the acquired position. The acquired position information is similar to that in the third embodiment, as shown in the table in FIG. 19(c), and may include relative coordinates of the acquired position with respect to a reference position on the user's body.

The data transmission/reception unit 211 of the doctor terminal 3 transmits at least one of the acquisition instruction image 54 generated by the instruction image generation unit 213 and the acquisition position information generated by the position information generation unit 214 to the user terminal 2. It may also be transmitted to the user terminal 2 via the server 1.

The user terminal 2 stores the received acquisition instruction image 54 and acquisition position information in a storage unit such as the condition storage unit 121, and can use them the next time body sounds are acquired. That is, the acquisition instruction image 54 is read out and displayed on the user terminal 2. Also, when making a judgment based on the position of the terminal 2 as in the above-mentioned third embodiment, the acquisition position information can be read out and used for judgment.

In this embodiment, the instruction image generating unit 213 and the position information generating unit 214 are provided in the doctor terminal 3, but some of these may be provided in the server 1. In that case, input information for the instruction information is sent from the doctor terminal 3 to the server 1, and the instruction image and position information are generated in the server 1. The generated acquired instruction image and acquired position information can then be sent to the user terminal 2.

<Embodiment 6>
In the sixth embodiment, the server 1 instructs the body sound acquisition position according to the user's body type. In the sixth embodiment, the server 1 has acquisition instruction information such as the body sound acquisition position, order, number of times, etc. according to the disease, purpose of income, etc. in advance. However, it is necessary to modify the standard acquisition instruction information according to the user's body type. In the sixth embodiment, the standard acquisition instruction information is modified based on image data including the body sound acquisition part of the user.

The software configuration example of the user terminal 2 in this embodiment 6 is the same as that in embodiment 5, and further includes a sensor input control unit 117, a position calculation unit 118, and an image data acquisition unit 119 in addition to the software configuration in embodiment 1.

Furthermore, FIG. 20 shows an example of the software configuration of the server 1 in this embodiment 6. The server includes an acquisition instruction information storage unit 321, a data transmission/reception unit 311, an image analysis unit 312, and an acquisition instruction information adjustment unit 313.

The acquisition instruction information storage unit 321 of the server 1 can store multiple pieces of acquisition instruction information. FIG. 21 shows an example of the configuration of the acquisition instruction information. The acquisition instruction information stores the acquisition part, order, number of times, etc. The acquisition part may be represented by relative coordinates of the acquisition position with respect to a reference position. The acquisition instruction information may also include an instruction image in which the acquisition position is plotted on an image of the body. Multiple pieces of acquisition instruction information may be provided for each type of body sound or disease. Furthermore, it is possible to have acquisition instruction information in which the acquisition part is adjusted according to multiple body type types. For example, body types are divided into multiple types such as thin type A, slightly thin type B, normal type C, slightly obese type D, obese type E, etc., and acquisition position information is registered for each type.

In this embodiment 6, first, an image of the user's body including the body part from which body sounds are to be obtained is captured by a camera or the like mounted on the user terminal 2. The image may be captured by an image capturing means other than the camera mounted on the user terminal 2. The image data acquisition unit 119 of the user terminal 2 acquires the captured image data. The image data may be a still image or a video.

The data transmission/reception unit 111 of the user terminal 2 transmits the image data to the server 1, and the data transmission/reception unit 311 of the server 1 acquires the image data.

The image analysis unit 312 of the server 1 analyzes the acquired image data to determine which of the body shape types registered in the condition storage unit 121 described above the image falls into. There are no particular limitations on the method of analyzing the image, and any known method can be used. For example, the body contour line may be extracted from the image, and the type may be determined using the length and ratio of specific parts, or machine learning may be used to classify the image.

The acquisition instruction information adjustment unit 313 selects acquisition instruction information for the relevant body type from the acquisition instruction information storage unit 321 based on the body type type selected by the image analysis unit 312.

The server 1 transmits the selected acquisition instruction information to the user terminal 2. When the user acquires body sounds, the user terminal 2 reads out the received acquisition instruction information. If the acquisition instruction information includes an instruction image, the instruction image can be displayed on the user terminal 2. In addition, a determination based on the position information of the terminal 2 in the above embodiment 3 can be made based on the acquisition instruction information.

Furthermore, the acquisition instruction information storage unit 321 may store only acquisition instruction information for a standard body type, and correct it to match the user's body type. In this case, the image analysis unit 312 calculates the difference between the user's body type and the standard body type from the user's image. Then, the acquisition instruction information adjustment unit 313 corrects the acquisition instruction information according to the difference. There are no particular limitations on the method of calculating the difference between the user's body type and the standard body type, and it may be determined, for example, by comparing the lengths of specified parts such as shoulder width and chest width.

In the above, the image analysis unit 312 determines the body type based on the image of the user, but the body type of the user may be estimated by other methods. For example, the body type can be estimated by measuring the distance of a specific part of the body using an acceleration sensor of the user terminal 2. Specifically, the user terminal 2 is moved between a specific area (for example, between the left and right shoulders) and the movement distance of the terminal 2 is measured by the acceleration sensor, so that the length of the specific area can be calculated. The specific area is not limited to between the left and right shoulders, and may be any part of the body that can be estimated as long as the body type can be estimated. Then, the body type can be estimated according to the length of the specific part. When the acquisition instruction information is prepared for each body type, the acquisition instruction information adjustment unit 313 determines the closest body type. When the acquisition instruction information is prepared only for a standard body type, the acquisition instruction information adjustment unit 313 corrects the acquisition instruction information according to the difference between the standard body type and the estimated body type of the user.

As described above, the body sound acquisition program of the present invention can determine whether or not a body sound is suitable for acquiring body sounds by various methods, and therefore can provide appropriate guidance when a user wants to acquire body sounds using his/her own terminal 2. The methods described in the above embodiments 1 to 6 may be implemented in one or a combination of two or more.

In any of the embodiments described above, a portion of each software function unit may be provided in either the user terminal 2, the doctor terminal 3, or the server 1, as long as the purpose of the system can be realized.

The above-described embodiment is merely an example to facilitate understanding of the present invention, and is not intended to limit the present invention. The present invention can be modified and improved without departing from the spirit of the invention, and it goes without saying that the present invention includes equivalents.

1 Server 2 User terminal 3 Doctor terminal

Claims (6)

A body sound acquisition support system that supports a user to acquire a body sound by using a user terminal, comprising:
a sound data acquisition unit that acquires sound data including the body sound;
An angle data acquisition unit that acquires angle data of the user terminal;
a determination unit that determines whether or not the body sound acquisition condition is met;
a notification unit that outputs the determination result to the user terminal ;
the acquisition condition includes an appropriate angle condition of the user terminal when acquiring the body sound,
the determination unit determines whether or not the angle data matches the angle condition;
A body sound acquisition support system characterized by the above . A sound evaluation unit is further provided for evaluating the volume of a target body sound component from the acquired sound data .
the acquisition conditions include conditions related to the volume of the body sound components,
The body sound acquisition support system according to claim 1 , wherein the determining unit determines whether or not a volume of the body sound component satisfies the acquisition condition. A sound evaluation unit that evaluates the volume of a noise component from the acquired sound data ,
the acquisition condition includes a condition related to a volume of the noise component,
The body sound acquisition support system according to claim 1 , wherein the determination unit determines whether or not a volume of the noise component satisfies the acquisition condition. A position calculation unit that calculates terminal position information associated with movement of the user terminal,
The acquisition condition includes acquisition position information of the body sound,
4. The body sound acquisition support system according to claim 1, wherein the determining unit determines whether or not the terminal location information matches the acquired location information. An instruction information input unit that receives an input of instruction information related to an acquisition position of the body sound of the user by an expert,
The body sound acquisition support system according to claim 4 , wherein the acquisition position information is generated based on the instruction information. a body type estimation unit that estimates a body type of the user based on input information of the user;
The body sound acquisition support system according to claim 4 , further comprising an acquisition instruction information adjustment unit that adjusts the acquisition position information in accordance with the body type.