JP7794728B2 - Sutra recitation support device, sutra recitation support method and program - Google Patents

Description

The present invention relates to a sutra recitation assistance device, a sutra recitation assistance method, and a program.

In recent years, chanting sutras, the recitation of Buddhist scriptures, has been gaining attention as a form of training to improve mental health. When chanting sutras, it is preferable that the sutras be recited at an appropriate pitch. Furthermore, it is preferable that sutra chanting as training is performed under stable emotional conditions.

Patent document 1 describes an audio information display device that displays audio waveform data of a speaker reading a string of characters and audio waveform data of a model on a display, allowing the speaker to understand the difference between their own voice and the model's voice.

JP 2016-156943 A

To support sutra chanting as training, it is necessary to evaluate the reader's emotions in real time in addition to the pitch of their voice.

The present invention has been made to solve the above-mentioned problems, and aims to provide a sutra recitation assistance device, sutra recitation assistance method, and program that can present an evaluation of the recitation voice and emotions of a user reciting sutras in real time.

A sutra recitation assistance device according to an embodiment of the present invention is characterized by having a presentation unit that presents, by image or sound, the speed at which a user should recite a sutra or the period during which the sutra should be recited; a pitch evaluation unit that evaluates the pitch of the user's recitation voice for each character segment of the sutra; an emotion evaluation unit that evaluates the user's emotion for each character segment based on successively acquired images of the user reciting the sutra; and an output unit that outputs an evaluation of the pitch and the evaluation of the emotion each time a character segment is recited.

The sutra chanting assistance device preferably further includes a reception unit that receives the user's selection of a sect, and the presentation unit preferably presents the speed at which the user should recite the sutra or the period during which the sutra should be recited, depending on the sect selection, by presenting the timing for the user to take a breath or by outputting the sound of a mokugyo at predetermined intervals.

It is also preferable that the output unit displays the pitch evaluation and the emotional evaluation in a single display area using different display formats.

It is also preferable that the output unit displays the pitch evaluation and the emotional evaluation using a single flame color and shape.

It is also preferable that the sutra chanting assistance device further includes an emotion determination unit that determines the emotion of the user reciting the sutra text, a skill evaluation unit that evaluates the user's skill in chanting based on a combination of pitch evaluation, emotional evaluation, and emotion determination, and a skill output unit that outputs the skill evaluation.

A sutra recitation assistance method according to an embodiment of the present invention is a sutra recitation assistance method executed by a sutra recitation assistance device, and is characterized by including the steps of: presenting, via image or audio, the speed at which a user should recite a sutra or the period during which the sutra should be recited; evaluating the pitch of the user's recitation voice for each character segment of the sutra; evaluating the user's emotion for each character segment based on successively acquired images of the user reciting the sutra; and outputting an evaluation of the pitch and the evaluation of the emotion each time a character segment is recited.

A program according to an embodiment of the present invention causes a computer to perform the following: present the speed at which a user recites sutras or the period during which the sutras should be recited using images or audio; evaluate the pitch of the audio when the user recites the sutra for each character segment of the sutra; evaluate the user's emotion for each character segment based on successively acquired images of the user reciting the sutra; and output an evaluation of the pitch and the emotion each time a character segment is recited.

The sutra recitation assistance device, sutra recitation assistance method, and program of the present invention make it possible to present an evaluation of the recitation voice and emotions of a user reciting sutras in real time.

FIG. 1 is a functional block diagram of a sutra recitation assistance device 1. FIG. 10 is a diagram showing the data structure of a sect table T1. FIG. 10 is a diagram showing a setting screen G1. FIG. 10 is a diagram showing a first sutra reading screen G2. 10A and 10B are schematic diagrams for explaining changes in an image displayed in a period display area G22. FIG. 10 is a diagram showing a second sutra reading screen G3. A figure showing the proficiency evaluation screen G4. FIG. 10 is a flowchart showing the flow of sutra recitation support processing. FIG. 10 is a flow chart showing the flow of the first sutra recitation process. FIG. 10 is a schematic diagram for explaining a method for evaluating a pitch. FIG. 1 is a schematic diagram for explaining a method for evaluating emotions. FIG. 10 is a flow chart showing the flow of the first sutra recitation process. FIG. 10 is a flowchart showing the flow of a proficiency evaluation process.

Various embodiments of the present invention will now be described with reference to the drawings. Please note that the technical scope of the present invention is not limited to these embodiments, but extends to the inventions set forth in the claims and their equivalents.

Figure 1 is a functional block diagram of a sutra recitation assistance device 1 according to an embodiment of the present invention. The sutra recitation assistance device 1 suggests the tempo or rhythm of recitation by presenting the speed at which the user should recite sutras (recitation speed) or the period during which the sutras should be recited (recitation period). The sutra recitation assistance device 1 also evaluates in real time the pitch of the user's recitation voice and the user's emotions as the user recites the sutras according to the suggested tempo or rhythm. By checking the output evaluation, the user can continue reciting at an appropriate pitch while maintaining appropriate emotions. The sutra recitation assistance device 1 also determines the emotions of the user reciting the sutras, and evaluates the user's level of chanting proficiency based on the evaluation of pitch, emotion, and emotion. By checking the evaluation of the user's level of chanting proficiency, the user can challenge themselves to chant sutras of an appropriate level of difficulty. The sutra recitation assistance device 1 is an information processing device such as a PC (Personal Computer), mobile phone, or smartphone, and includes a memory unit 11, an imaging unit 12, an audio input unit 13, an audio output unit 14, a display unit 15, an operation unit 16, a processing unit 17, etc.

The memory unit 11 is configured to store programs or data and includes, for example, a semiconductor memory device. The memory unit 11 stores operating system programs, driver programs, application programs, data, etc. used for processing by the processing unit 17. Programs are installed into the memory unit 11 from computer-readable, non-transitory, portable storage media such as CD-ROM (Compact Disc Read Only Memory) and DVD-ROM (Digital Versatile Disc Read Only Memory).

The imaging unit 12 is configured to capture an image of the user reciting sutras and includes a camera. The camera includes an imaging optical system for forming an image on a light-receiving surface, photoelectric conversion elements such as CCD (Charge Coupled Device) sensors that are arranged two-dimensionally on the light-receiving surface and output an electrical signal corresponding to the amount of incident light, and an image generation circuit that generates an image based on the output of the photoelectric conversion elements. The imaging unit 12 generates an image based on a control signal supplied from the processing unit 17 and supplies the generated image to the processing unit 17.

The audio input unit 13 is configured to capture the audio of the user reciting sutras, and is equipped with a microphone. The audio input unit 13 converts vibrations caused by the audio into an electrical signal. The audio input unit 13 converts the audio into an electrical signal based on a control signal supplied from the processing unit 17, and supplies the electrical signal to the processing unit 17.

The audio output unit 14 is configured to output audio and includes a speaker. The audio output unit 14 outputs audio by converting the electrical signal supplied from the processing unit 17 into mechanical vibrations.

The display unit 15 is configured to display images and includes, for example, a liquid crystal display or an organic EL (Electro Luminescence) display. The display unit 15 displays images based on display data supplied from the processing unit 17.

The operation unit 16 is configured to accept user operations and includes, for example, a keyboard, keypad, and mouse. The operation unit 16 may also include a touch panel and be integrated with the display unit 15. The operation unit 16 generates operation signals in response to user operations and supplies them to the processing unit 17.

The processing unit 17 is a device that controls the overall operation of the sutra recitation assistance device 1 and includes one or more processors and their peripheral circuits. The processing unit 17 includes, for example, a CPU (Central Processing Unit). The processing unit 17 may also include a GPU (Graphics Processing Unit), a DSP (Digital Signal Processor), an LSI (Large Scale Integration), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), etc. The processing unit 17 controls the operation of each component and executes various processes so that the various processes of the sutra recitation assistance device 1 are executed in the appropriate order based on the programs stored in the memory unit 11 and the operation signals from the operation unit 16.

The processing unit 17 has, as its functional blocks, a reception unit 171, a presentation unit 172, a pitch evaluation unit 173, an emotion evaluation unit 174, an output unit 175, a feeling determination unit 176, a proficiency evaluation unit 177, and a proficiency output unit 178. Each of these units is a functional module realized by a program executed by the processing unit 17. Each of these units may be implemented in the sutra chanting assistance device 1 as firmware.

Figure 2 shows the data structure of the sect table T1 stored in the memory unit 11. The sect table T1 stores sects, sutras, methods, and sutra categories in association with one another.

The sect and sutra are information identifying a Buddhist sect and a type of sutra, respectively. Because the sutra that a user should recite varies depending on the user's sect, a sutra is associated with each sect. Because some sutras are used in multiple sects, the same sutra may be associated with multiple sects. The method is information indicating the method for determining the tempo or rhythm of the chanting, and is either the breath method or the mokugyo method. The breath method is a method in which a recitation period during which the user should recite the sutra and a breathing period during which the user can take a breath are predetermined. The rhythm of the chanting is determined by the user reciting a passage of sutra during the recitation period and taking a breath during the breathing period. The mokugyo method is a method in which the sound of a mokugyo is output at a predetermined tempo while the user is reciting the sutra. By reciting the sutra while listening to the sound of the mokugyo, the user's chanting speed is guided to the tempo of the mokugyo sound, and the tempo of the chanting is determined. The sutra section is information that indicates each section of a sutra segmented based on the division positions that are predetermined for each sutra, and the identification information for each section is associated with the string of sutra text contained in each section.

Figure 3 shows an example of the settings screen G1 displayed on the display unit 15. The settings screen G1 is a screen for setting the user's sect and chanting speed. The settings screen G1 is displayed, for example, when the operation unit 16 receives an operation to execute a sutra chanting assistance application program. The settings screen G1 includes a sect selection area G11, a speed selection area G12, and a sutra chanting start object G13.

The sect selection area G11 is an area for selecting the user's sect.
The sutra text that the user should recite and the method of reciting the sutra vary depending on the user's sect. Therefore, before reciting the sutra, the user selects his/her own sect from among multiple sects. For example, the sect selection area G11 includes multiple objects corresponding to each of the multiple sects, and the user selects the object corresponding to his/her own sect from among the multiple objects. The sect selection area G11 displays the selected object in an emphasized manner. In the example shown in FIG. 3, the object corresponding to "Jodo Shinshu" selected by the user is displayed in an emphasized manner.

The speed selection area G12 is an area for selecting the chanting speed. For example, the speed selection area G12 includes radio buttons that allow the user to select one speed from a number of pre-set speeds. In the example shown in Figure 3, the speed selection area G12 allows the user to select one of "Excited," which corresponds to a fast speed, "Everyday," which corresponds to a medium speed, or "Calm," which corresponds to a slow speed. The speed selection area G12 may also include an object that allows the user to specify the chanting speed numerically.

If the timing of breathing when chanting is specified, the faster the chanting speed, the shorter the recitation period. If the timing of breathing when chanting is not specified, the faster the chanting speed, the shorter the interval between the mokugyo sounds output from the audio output unit 14. In the example shown in Figure 3, the user has selected "Excitement," which corresponds to a faster speed.

The sutra chanting start object G13 is an object for starting sutra chanting. After selecting the sect and sutra chanting speed, the user begins chanting by selecting the sutra chanting start object G13.

Figure 4 shows an example of the first sutra recitation screen G2 displayed on the display unit 15. The first sutra recitation screen G2 is a screen for reciting sutras using the breath-taking method. The first sutra recitation screen G2 is displayed when the start sutra recitation object G13 is selected while a sect using the breath-taking method is selected in the sect selection area G11 of the settings screen G1. The first sutra recitation screen G2 includes a sutra text display area G21, a period display area G22, and an evaluation display area G23.

The sutra display area G21 displays the sutra that the user is reciting. For example, the sutra display area G21 displays the sutra that the user is currently reciting by scrolling an image in which the sutra is displayed vertically to the right at a speed that corresponds to the recitation speed. In the example shown in Figure 4, the sutra begins on a new line at the end of each verse, so the sutra display area G21 scrolls the image of the sutra by one line between each breathing period and recitation period.

The period display area G22 displays the recitation period during which the user should recite the sutra displayed in the sutra display area G21, and the breathing period during which the user should take a breath. In the example shown in FIG. 2, the period display area G22 indicates whether the current period is a recitation period or a breathing period using the character string ST, and displays the progress of the recitation period and breathing period using the circular progress bar PB arranged to surround the character string ST.

Figure 5 is a schematic diagram illustrating changes in the image displayed in the period display area G22. First, the breathing period begins. Simultaneously with the start of the breathing period, the character string "Inhale" is displayed as the character string ST, as shown in the upper left of Figure 5. At this time, the length of the progress bar PB is zero. As time passes from this state, the progress bar PB extends clockwise from the 12 o'clock position, as shown in the upper center of Figure 5. Further time passes from this state, and when the end time of the breathing period arrives, the tip of the progress bar PB reaches the 12 o'clock position, as shown in the upper right of Figure 5. When the breathing period ends, the reading period begins. Simultaneously with the start of the reading period, the character string "Read" is displayed as the character string ST, as shown in the lower right of Figure 5. At this time, the length of the progress bar PB is zero. As time passes from this state, the progress bar PB again extends clockwise from the 12 o'clock position, as shown in the lower center of Figure 5. As time passes from this state and the end of the recitation period arrives, the tip of the progress bar PB reaches the 12 o'clock position, as shown in the lower left of Figure 5. When the recitation period ends, a breathing period begins again. The period display area G22 repeats the above display until the recitation ends.

The length of the recitation period varies depending on the recitation speed set on the settings screen G1. For example, the length of the recitation period is 2 seconds if "Excitement" is selected, 4 seconds if "Everyday" is selected, and 6 seconds if "Calm" is selected. The length of the breathing period is constant regardless of the recitation speed, for example, 2 seconds.

In this way, the sutra recitation assistance device 1 presents the recitation period and breathing period using images.

Returning to Figure 4, the evaluation display area G23 displays the pitch evaluation and the emotional evaluation using the color and shape of a flame FL lit on a candle. For example, the evaluation display area G23 displays the pitch evaluation using the shape of the flame FL, and the emotional evaluation using the color of the flame FL.

It is considered preferable to read sutras without any intonation. Therefore, the smaller the fluctuation in pitch during the recitation period, the better the pitch evaluation. The flame FL is displayed so that its shape periodically expands and contracts, and the smaller the fluctuation in pitch, the smaller the range of expansion and contraction. If the fluctuation in pitch is small, the flame FL may be displayed so that its shape remains constant regardless of the passage of time.

Emotions are evaluated based on the pulse rate of the user reciting the sutra. From the perspective of improving mental health, it is considered preferable for the pulse rate to fluctuate less, i.e., for emotions to be low. Therefore, the lower the emotion, the better the emotion evaluation. The Flame FL is displayed in a color that corresponds to the magnitude of the emotion. For example, the Flame FL is displayed closer to white for weaker emotions, and closer to red for stronger emotions.

Pitch and emotion evaluations are performed for each recitation period. During the recitation period, the user recites a passage of sutra displayed in the sutra display area G21. In other words, pitch and emotion evaluations are performed for each passage of sutra. Furthermore, each time a passage of sutra is recited, the evaluation display area G23 outputs the pitch evaluation and emotion evaluation by updating the shape and color of the flame FL based on the pitch evaluation and emotion evaluation. A sutra passage is an example of a character segment of a sutra.

In this way, the evaluation display area G23 displays the pitch evaluation and the emotional evaluation using a single flame color and shape. This allows the user to easily check both the pitch evaluation and the emotional evaluation.

Figure 6 is a diagram showing an example of the second sutra recitation screen G3 displayed on the display unit 15. The second sutra recitation screen G3 is a screen for performing sutra chanting using the Mokugyo method. The second sutra recitation screen G3 is displayed when the chanting start object G13 is selected with the Mokugyo method sect selected on the settings screen G1. The second sutra recitation screen G3 includes a sutra text display area G31, an animation display area G32, and an evaluation display area G33.

The sutra display area G31 displays the sutra to be recited by the user and indicates the speed at which the user recites the sutra. In the example shown in FIG. 5, the sutra display area G31 displays the sutra in white letters, like karaoke lyrics, and switches the sutra characters from white letters to solid black letters from front to back at a predetermined speed. The user recites the sutra in time with the speed at which the sutra display switches from white letters to solid black letters, allowing the sutra to be recited at the predetermined speed.

The animation display area G32 repeatedly displays an animation of a stick called a mokugyo bai striking a mokugyo. In mokugyo-style recitation, the sound of a mokugyo is output from the audio output unit 14 at predetermined intervals. The animation display area G32 displays the animation so that the timing at which the mokugyo sound is output is synchronized with the timing at which the stick strikes the mokugyo. The tempo at which the mokugyo sound is output varies depending on the chanting speed selected on the setting screen G1. For example, the tempo of the mokugyo sound is 100 bpm when "Exhilaration" is selected, 80 bpm when "Daily Life" is selected, and 65 bpm when "Calmness" is selected.

In this way, the sutra chanting assistance device 1 presents the chanting speed through images and audio.

Like the evaluation display area G23, the evaluation display area G33 displays a pitch evaluation and an emotional evaluation. The evaluation of pitch and emotional evaluation is performed for each evaluation period, which is separated by two consecutive mokugyo sounds. The user recites the portion of the sutra display area G31 where the white characters change to black characters while two consecutive mokugyo sounds are being output. In other words, the evaluation of pitch and emotional evaluation is performed for each evaluation period. Furthermore, the evaluation display area G33 outputs the pitch evaluation and emotional evaluation by updating the shape and color of the flame based on the pitch evaluation and emotional evaluation each time a portion of the sutra corresponding to the evaluation period is recited. The portion of the sutra corresponding to the evaluation period is an example of a character segment of the sutra.

Figure 7 shows the proficiency evaluation screen G4 displayed on the display unit 15. The proficiency evaluation screen G4 is a screen that displays an evaluation of the user's chanting proficiency. The proficiency evaluation screen G4 is displayed when chanting using the first chanting screen G2 or the second chanting screen G3 is completed. The proficiency evaluation screen G4 includes an item-specific evaluation display area G41 and a proficiency display area G42.

The item-specific evaluation display area G41 displays the pitch evaluation, emotion evaluation, and emotion judgment for the entire period from the start to the end of the chanting. The pitch evaluation for the entire period is the ratio of the period from the start to the end of the chanting during which the pitch fluctuation was judged to be small. The emotion evaluation for the entire period is the ratio of the period from the start to the end of the chanting during which the emotion was judged to be small. The emotion judgment for the entire period is the ratio of the period from the start to the end of the chanting during which the user was judged to have positive emotions.

The proficiency display area G42 displays an evaluation of the user's chanting proficiency. In the example shown in Figure 7, the user's chanting proficiency is indicated by text, indicating that the user is at a level of proficiency appropriate for the current chanting. The user's chanting proficiency is evaluated based on a combination of pitch evaluation, emotional evaluation, and emotion judgment.

Figure 8 is a flow diagram showing the flow of the sutra recitation assistance process executed by the sutra recitation assistance device 1. The sutra recitation assistance process is executed, for example, when the operation unit 16 receives an operation to launch a sutra recitation assistance application program. The sutra recitation assistance process is realized by the processing unit 17 working in cooperation with each component of the sutra recitation assistance device 1 based on the program stored in the memory unit 11.

First, the reception unit 171 receives the user's selection of a sect and a recitation speed (step S101). The reception unit 171 displays the setting screen G1 on the display unit 15 by supplying the display data for the setting screen G1, which has been stored in advance in the storage unit 11, to the display unit 15. The reception unit 171 receives an operation on the operation unit 16 to select a sect and an operation to select a chanting speed. Based on the received operation, the reception unit 171 changes the display of the sect selection area G11 and the speed selection area G12 on the setting screen G1. The reception unit 171 also receives an operation on the operation unit 16 to select a chanting start object G13. The reception unit 171 stores the sect and chanting speed selected at the time of the operation in the storage unit 11 as the sect and chanting speed selected by the user.

Next, the reception unit 171 references the sect table T1 and determines whether the chanting of the user's sect uses the breath-holding method (step S102).

If the sutra chanting of the user's sect is performed using the breath-taking method (step S102 - Yes), a first sutra chanting process is executed to support the breath-taking method of sutra chanting (step S103). Details of the first sutra chanting process will be described later.

If the chanting of the user's sect is not the breath-taking method (step S102 - No), i.e., if it is the mokugyo method, a second chanting process is executed to support the mokugyo method chanting (step S104). Details of the second chanting process will be described later.

After the first or second sutra recitation process, a proficiency evaluation process is executed to evaluate the user's chanting skill (step S105). Details of the proficiency evaluation process will be described later. This completes the sutra recitation assistance process.

Figure 9 is a flow diagram showing the flow of the first sutra recitation process. The first sutra recitation process is executed in step S103 of the sutra recitation support process.

First, the presentation unit 172 displays the first sutra recitation screen G2 on the display unit 15 by supplying the display data of the first sutra recitation screen G2, which has been stored in advance in the memory unit 11, to the display unit 15 (step S201). Thereafter, the presentation unit 172 updates the display of the sutra text displayed in the sutra text display area G21 of the first sutra recitation screen G2 as time passes.

Next, the presentation unit 172 controls the audio input unit 13 to begin acquiring the recitation audio of the user reciting the sutra (step S202). Thereafter, the presentation unit 172 generates recitation audio data that records the recitation audio in chronological order based on the electrical signals converted from the user's recitation audio supplied from the audio input unit 13, and stores the data in the memory unit 11.

The presentation unit 172 also controls the imaging unit 12 to begin capturing an image of the user reciting the sutra (step S203). Thereafter, the presentation unit 172 acquires moving image data of the user supplied from the imaging unit 12 and stores the data in the storage unit 11. Note that the moving image is an example of continuously obtained images.

Next, the presentation unit 172 presents an image of a breathing period during which the user can take a breath (step S204). The presentation unit 172 starts the breathing period. The presentation unit 172 displays the character string "Inhale" as the character string ST in the period presentation area G22. The presentation unit 172 also extends the progress bar PB in the period presentation area G22 so that it reaches the 12 o'clock position at the end of the breathing period. In this way, the presentation unit 172 presents the breathing period with an image displayed in the period presentation area G22.

When the breathing period ends, the presentation unit 172 presents, using an image, the recitation period during which the user should recite the sutra (step S205). The presentation unit 172 starts the recitation period. The presentation unit 172 displays "Read" as the character string ST in the period presentation area G22. The presentation unit 172 also extends the progress bar PB in the period presentation area G22 so that it reaches the 12 o'clock position at the end of the recitation period. In this way, the presentation unit 172 presents the recitation period using an image displayed in the period presentation area G22.

When the reading period ends, the pitch evaluation unit 173 evaluates the pitch of the user's reading voice (step S206). The pitch evaluation unit 173 obtains reading voice data for the most recently completed breath period and reading period from the storage unit 11. The pitch evaluation unit 173 calculates the amount of pitch fluctuation in the reading voice based on the reading voice data, and evaluates the pitch of the reading voice based on the amount of pitch fluctuation.

Figure 10 is a schematic diagram illustrating a method for evaluating pitch. The vertical axis of the graph in Figure 10 represents frequency f, and the horizontal axis represents time t. The pitch evaluation unit 173 applies an FFT (Fast Fourier Transform) to the reading voice data and performs spectral analysis of the reading voice data to calculate the change over time PT of the fundamental frequency F0 corresponding to the pitch of the reading voice. In Figure 9, the change over time PT of the fundamental frequency F0 during the reading period P1 and the breathing period P2 is shown as a curve. The pitch evaluation unit 173 extracts the maximum and minimum values of the fundamental frequency F0 during the reading period P1, and calculates the difference Δf between the maximum and minimum values as the amount of pitch fluctuation.

The pitch evaluation unit 173 evaluates the pitch of the reading voice by comparing the amount of pitch fluctuation with a predetermined threshold. For example, if the difference Δf is less than 20 Hz, the pitch evaluation unit 173 evaluates the amount of pitch fluctuation as small. If the difference Δf is equal to or greater than 20 Hz and less than 50 Hz, the pitch evaluation unit 173 evaluates the amount of pitch fluctuation as medium. If the difference Δf is 50 Hz or greater, the pitch evaluation unit 173 evaluates the amount of pitch fluctuation as large.

In this way, the pitch evaluation unit 173 evaluates the pitch of the recitation voice for each recitation period, i.e., for each passage of the sutra.

Returning to FIG. 9 , the emotion evaluation unit 174 evaluates the user's emotion (step S207). The emotion evaluation unit 174 acquires from the storage unit 11 video data of the user for the most recently completed breathing period and reading period. The emotion evaluation unit 174 estimates the user's pulse rate based on the video data of the user, and evaluates the user's emotion based on the number of fluctuations in the pulse rate. For example, the emotion evaluation unit 174 estimates the user's pulse rate using the method described in WO 2022/065446 or WO 2018/74371.

Figure 11 is a schematic diagram illustrating a method for evaluating emotions. The vertical axis of the graph in Figure 11(A) represents pulse wave amplitude A, and the horizontal axis represents time t. The vertical axis of the graph in Figure 11(B) represents pulse rate B, and the horizontal axis represents time t. The emotion evaluation unit 174 extracts multiple frame images from the user's video data. The emotion evaluation unit 174 applies a contour detection algorithm or a feature point extraction algorithm to the multiple frame images to identify an area in the frame images that corresponds to the user's forehead. The emotion evaluation unit 174 extracts pixels included in the area corresponding to the forehead in each frame image, and calculates the change in the G (green) pixel value over time by obtaining the G pixel value from the RGB pixel values of the extracted pixels. Capillaries are concentrated in the forehead, and the G pixel value reflects the user's blood flow. Therefore, the pulse wave is detected based on the change in the G pixel value over time. The emotion evaluation unit 174 extracts the pulse wave signal PW shown in Figure 11(A) by applying a bandpass filter with a passband of 0.5 Hz to 3 Hz, which corresponds to a human pulse wave, to the signal indicating the temporal change in the G pixel value. Note that the emotion evaluation unit 174 may also extract an area in the frame image that corresponds to an exposed part of the human skin other than the forehead.

The emotion evaluation unit 174 calculates the change over time in the user's pulse rate based on the extracted pulse wave signal PW. For example, the emotion evaluation unit 174 identifies the peak point P(n) of the pulse wave signal. The emotion evaluation unit 174 calculates the interval between two adjacent peak points (P(n), P(n+1)) as the pulse wave interval d(n). The emotion evaluation unit 174 calculates the change over time in the pulse rate PR shown in Figure 10 (B) by multiplying the reciprocal of the pulse wave interval by 60 to represent the pulse rate per minute.

The emotion evaluation unit 174 calculates the number of times the pulse rate fluctuates based on the time change PR of the pulse rate. The emotion evaluation unit 174 extracts the pulse rate b2 at the point where the time change PR of the pulse rate during the reading period P1 reaches its maximum value, and the pulse rate b1 a predetermined time before the point where the maximum value is reached. The emotion evaluation unit 174 determines that the pulse rate has fluctuated if the pulse rate b2 is greater than the average pulse rate bav of an average person and the difference Δb between the pulse rate b2 and the pulse rate b1 is equal to or greater than a predetermined value. The emotion evaluation unit 174 calculates the number of times the pulse rate fluctuates during the reading period P1 by making the above-mentioned determination for each point where the pulse rate reaches its maximum value during the reading period P1.

The emotion evaluation unit 174 evaluates emotion by comparing the number of pulse rate fluctuations during the reading period with a predetermined threshold. For example, the emotion evaluation unit 174 evaluates emotion as low if the number of pulse rate fluctuations during the reading period is 1 or less. The emotion evaluation unit 174 evaluates emotion as medium if the number of pulse rate fluctuations during the reading period is 2 to 4 or less. The emotion evaluation unit 174 evaluates emotion as high if the number of pulse rate fluctuations during the reading period is 5 or more.

In this way, the emotion evaluation unit 174 evaluates the user's emotions for each recitation period, i.e., for each passage of the sutra.

Returning to FIG. 9 , the output unit 175 outputs the pitch evaluation and the emotional evaluation using the color and shape of a single flame (step S208). The output unit 175 generates display data for a flame having a shape corresponding to the pitch evaluation and a color corresponding to the emotional evaluation. The output unit 175 displays the flame in the evaluation display area G23 based on the generated display data. Thereafter, the output unit 175 enlarges or reduces the shape of the flame displayed in the evaluation display area G23 as time passes. In this way, the output unit 175 outputs the pitch evaluation and the emotional evaluation each time the recitation period ends, that is, each time a passage of sutra is recited.

Next, the presentation unit 172 determines whether the sutra recitation has ended (step S209). For example, the presentation unit 172 determines that the sutra recitation has ended when a predetermined time has elapsed since the first sutra recitation screen G2 was displayed and all sutras have been displayed in the sutra display area G21.

If the sutra recitation has not finished (step S209—No), the first sutra recitation process returns to step S204, and the presentation unit 172 presents the breath period again. If the sutra recitation has finished (step S209—Yes), the first sutra recitation process ends.

Figure 12 is a flow diagram showing the flow of the second sutra recitation process. The second sutra recitation process is executed in step S104 of the sutra recitation support process.

First, the presentation unit 172 displays the second sutra recitation screen G3 on the display unit 15 based on display data previously stored in the memory unit 11 (step S301). Thereafter, the presentation unit 172 updates the display of the sutra text displayed in the sutra text display area G31 of the second sutra recitation screen G3 as time passes. The presentation unit 172 also updates the animation displayed in the animation display area G32 as time passes.

Next, the presentation unit 172 controls the audio input unit 13 to begin acquiring the audio of the user reciting the sutra (step S302).

The presentation unit 172 also controls the imaging unit 12 to begin capturing an image of the user reciting the sutra (step S303).

Next, the presentation unit 172 determines whether the output time for outputting the mokugyo sound has arrived (step S304). The output time is the time when a period of time determined based on the chanting speed has elapsed since the time when the second sutra recitation screen G3 was displayed or the time when the mokugyo sound was last output.

If the output time has not arrived (step S304—No), the second sutra recitation process returns to step S304. That is, the presentation unit 172 waits until the output time arrives.

If the output time has arrived (step S304—Yes), the presentation unit 172 controls the audio output unit 14 to output the sound of a mokugyo (step S305). The presentation unit 172 outputs the sound of a mokugyo by supplying audio data of the sound of a mokugyo that has been stored in advance in the storage unit 11 to the audio output unit 14. In this way, the presentation unit 172 presents the chanting speed using an image displayed in the animation display area G32 and audio output from the audio output unit 14.

Next, the pitch evaluation unit 173 evaluates the pitch of the user's recitation voice (step S306). The pitch evaluation unit 173 obtains recitation voice data for the period between the immediately preceding output time and the output time before that from the storage unit 11. As in step S206 of the first sutra recitation process, the pitch evaluation unit 173 performs spectral analysis of the recitation voice data to calculate the change in the fundamental frequency F0 over time. The pitch evaluation unit 173 also detects periods in the recitation voice data that contain the sound of a mokugyo, and extracts periods in the recitation voice data that do not contain the sound of a mokugyo as the evaluation period. The period that contains the sound of a mokugyo may be detected by comparing the audio waveform indicated by the recitation voice data with the audio waveform of a pre-stored mokugyo sound, or may be detected based on time.

The pitch evaluation unit 173 extracts the maximum and minimum values of the fundamental frequency during the evaluation period and calculates the difference between the maximum and minimum values as the amount of pitch fluctuation. By excluding the period that includes the sound of the mokugyo from the evaluation period, the change in fundamental frequency over time can be calculated appropriately without being affected by the sound of the mokugyo. The pitch evaluation unit 173 evaluates the pitch of the reading voice by comparing the amount of pitch fluctuation with a predetermined threshold.

In this way, the pitch evaluation unit 173 evaluates the pitch of the recited voice for each evaluation period, i.e., for each part of the sutra corresponding to the evaluation period.

The emotion evaluation unit 174 also evaluates the user's emotion (step S307). The emotion evaluation unit 174 acquires from the storage unit 11 video data of the user for the period between the most recent output time and the output time before that. The emotion evaluation unit 174 calculates the time change in the user's pulse rate based on the video data of the user, in the same manner as in step S207 of the first sutra chanting evaluation process. The emotion evaluation unit 174 calculates the number of times the pulse rate fluctuates during the evaluation period. The emotion evaluation unit 174 evaluates the user's emotion by comparing the number of times the pulse rate fluctuates with a predetermined threshold.

In this way, the emotion evaluation unit 174 evaluates the pitch of the recited voice for each evaluation period, i.e., for each part of the sutra corresponding to the evaluation period.

Next, the output unit 175 outputs the pitch evaluation and the emotional evaluation (step S308), similar to step S208 of the first sutra recitation process. The output unit 175 outputs the pitch evaluation and the emotional evaluation each time the evaluation period ends, i.e., each time a portion of the sutra corresponding to the evaluation period is recited.

Next, the presentation unit 172 determines whether the sutra recitation has finished (step S309). If the sutra recitation has not finished (step S309—No), the second sutra recitation process returns to step S304, and the presentation unit 172 waits again until the output time arrives. If the sutra recitation has finished (step S309—Yes), the second sutra recitation process ends.

Figure 13 is a flow diagram showing the flow of the proficiency evaluation process. The proficiency evaluation process is executed in step S105 of the sutra chanting support process.

First, the emotion determination unit 176 determines the emotion of the user (step S401). The emotion determination unit 176 determines the emotion of the user based on the pulse wave interval d(n) of the user calculated in step S207 of the first sutra recitation process or step S307 of the second sutra recitation process.

The emotion determination section 176 acquires the pulse wave interval d(n) included in each emotion determination period from the storage section 11. For each emotion determination period, the emotion determination section 176 calculates the maximum Lyapunov exponent λ, which is expressed by the following equation:
Here, M is a value obtained by subtracting 1 from the number of pulse wave interval data included in the emotion determination period.

The maximum Lyapunov exponent is an index that indicates the degree to which changes in pulse wave intervals over time contain complex system fluctuations; the higher the value, the greater the complex system fluctuations, and the smaller the value, the smaller the complex system fluctuations. A person's pulse wave intervals contain a certain degree of complex system fluctuations due to the function of the autonomic nervous system, but it is known that complex system fluctuations are lost when autonomic nervous function declines. Therefore, the smaller the value of the maximum Lyapunov exponent, the more the person's autonomic nervous function declines, and it is estimated that the person is experiencing stress factors, i.e., is experiencing negative emotions.

The emotion determination unit 176 determines the user's emotion during the emotion determination period by comparing the maximum Lyapunov exponent λ with a predetermined threshold. For example, if the maximum Lyapunov exponent λ is greater than +0.0150, the emotion determination unit 176 determines that the user has a high positive emotion that is appropriate for chanting, and if the maximum Lyapunov exponent λ is +0.0150 or less, the emotion determination unit 176 determines that the user has a low positive emotion or a negative emotion that is not appropriate for chanting.

Next, the skill evaluation unit 177 evaluates the user's chanting skill based on a combination of the pitch evaluation, emotion evaluation, and emotion determination (step S402). The skill evaluation unit 177 evaluates the pitch, emotion, and emotion determination for the entire period from the start to the end of the chanting.

The proficiency evaluation unit 177 calculates the ratio of the sum of the lengths of recitation periods or evaluation periods in which pitch fluctuations were evaluated to be small to the sum of the lengths of all recitation periods or evaluation periods as an evaluation of pitch for the entire period. Furthermore, the proficiency evaluation unit 177 calculates the ratio of the sum of the lengths of recitation periods or evaluation periods in which emotions were evaluated to be small to the sum of the lengths of all recitation periods or evaluation periods as an evaluation of emotion for the entire period. The proficiency evaluation unit 177 also calculates the ratio of the sum of the lengths of emotion determination periods in which the user was determined to have high positive emotions suitable for chanting to the sum of the lengths of all emotion determination periods as an evaluation of emotion for the entire period.

The proficiency evaluation unit 177 evaluates the user's proficiency in chanting based on a combination of the results of comparing the pitch evaluation, emotion evaluation, and emotion judgment for the entire period with predetermined thresholds. For example, if the pitch evaluation, emotion evaluation, and emotion judgment are all 50% or higher, the proficiency evaluation unit 177 evaluates that the user's proficiency is appropriate for the current settings. If the pitch evaluation, emotion evaluation, and emotion judgment are all less than 50%, the proficiency evaluation unit 177 evaluates that the user's proficiency is low for the current settings. If the emotion evaluation is 50% or higher and the pitch evaluation and emotion judgment are less than 50%, the proficiency evaluation unit 177 evaluates that the user's proficiency is very low for the current settings. This is because it is thought that the user has given up on matching the pitch because they are unable to recite sutras appropriately, making it difficult for them to express emotions. If the pitch evaluation is 50% or higher and the emotion judgment is less than 50%, the proficiency evaluation unit 177 evaluates that the user's proficiency is too high for the current settings. This is because it is believed that the user is able to recite at an appropriate pitch without concentrating on chanting.

Next, the proficiency output unit 178 outputs the proficiency evaluation (step S403). The proficiency output unit 178 generates display data for a proficiency evaluation screen G4 that includes the pitch evaluation, emotional evaluation, and emotion judgment for the entire period calculated in step S401 in the item-specific evaluation display area G41, and the proficiency evaluated in step S401 in the proficiency display area G42. The proficiency output unit 178 supplies the generated display data to the display unit 15. This completes the proficiency evaluation process.

As explained above, the sutra recitation assistance device 1 evaluates the pitch of the recitation voice and the user's emotion for each character segment of the sutra, and outputs a pitch evaluation and an emotion evaluation each time the recitation of a character segment is completed. This enables the sutra recitation assistance device 1 to present an evaluation of the recitation voice and emotion of the user reciting sutras in real time.

The sutra chanting assistance device 1 also accepts the user's selection of a sect and, depending on the sect selected, indicates the chanting speed or the duration of the sutra recitation by indicating when the user should take a breath or by outputting the sound of a mokugyo at predetermined intervals. This allows the sutra chanting assistance device 1 to assist the user in chanting sutras regardless of the user's sect.

In addition, the sutra chanting assistance device 1 displays the pitch evaluation and the emotional evaluation using a single flame color and shape. This allows the user chanting sutras to easily grasp both the evaluation of the recited voice and the evaluation of the emotional evaluation.

The sutra chanting assistance device 1 also assesses the user's emotions and evaluates the user's level of chanting proficiency based on a combination of pitch evaluation, emotional evaluation, and emotion assessment. This allows the sutra chanting assistance device 1 to enable the user to select a sutra text and recitation speed appropriate for their level of proficiency.

In the above explanation, the reception unit 171 receives the selection of a sect in step S101 of the sutra chanting support process, but this is not limited to this example. For example, the reception unit 171 does not have to receive the selection of a sect. Furthermore, the reception unit 171 may receive the selection of any type of sutra instead of the selection of a sect. When chanting as training to improve mental health, it is also conceivable that a sutra from a different sect will be recited. By having the reception unit 171 receive the selection of the type of sutra, it is possible to meet the needs of users who wish to recite a variety of sutras.

In the above description, the presentation unit 172 begins acquiring moving image data of the user in step S203 of the first sutra recitation process and step S303 of the second sutra recitation process, but this is not limited to this example. The presentation unit 172 may also acquire data of multiple still images captured continuously of the user instead of moving images. Multiple still images captured continuously of the user are another example of continuously obtained images.

In the above explanation, in step S206 of the first recitation process and step S306 of the second recitation process, the pitch evaluation unit 173 determines the amount of pitch fluctuation based on the difference between the maximum and minimum values of the fundamental frequency during the recitation period, but this is not limited to this example. For example, the peak frequency of the spectral envelope of the audio signal may be used instead of the fundamental frequency. Furthermore, the standard deviation may be used instead of the difference between the maximum and minimum values.

In the above description, in step S208 of the first sutra recitation process and step S308 of the second sutra recitation process, the output unit 175 displays the pitch and emotional evaluations using the color and shape of a flame. However, this is not a limited example. For example, the output unit 175 may display an arbitrary object other than a flame in the evaluation display area G23 or G33 and display the pitch and emotional evaluations using the color and shape of the object. The output unit 175 may also display the pitch and emotional evaluations using a combination of two different display modes other than color and shape. For example, the output unit 175 may display the pitch and emotional evaluations using a combination of the basic shape of a flame and changes in the shape over time. The output unit 175 may display the pitch and emotional evaluations using the hue and brightness of the color of the flame. Alternatively, the output unit 175 may display the pitch and emotional evaluations using two different display modes in a single display area. The output unit 175 may also output the pitch and emotional evaluations using audio.

In the above explanation, the emotion determination unit 176 determines the user's emotion in step S401 of the proficiency evaluation process after the chanting is completed, but this is not limited to this example. The emotion determination unit 176 may determine the user's emotion during each emotion determination period that has elapsed since the chanting began. In this case, the output unit 175 may output the emotion determination by displaying it on the first chanting assistance screen each time the user's emotion is determined. This allows the user to grasp the emotion determination in real time.

Each function of the sutra recitation assistance device 1 may be realized by multiple devices. For example, the processing performed by the processing unit 17 may be executed by a server that communicates with the sutra recitation assistance device 1, and the sutra recitation assistance device 1 may receive the processing results from the server. Alternatively, the user may be photographed by an external imaging device such as a surveillance camera or IP (Internet Protocol) camera, and the sutra recitation assistance device 1 may acquire moving or still images of the user by communicating with the external imaging device. In this case, the sutra recitation assistance device 1 may not have an imaging unit 12.

Those skilled in the art will understand that various changes, substitutions, and modifications can be made to the present invention without departing from the scope of the present invention. For example, the above-described embodiments and variations may be implemented in any suitable combination within the scope of the present invention.

REFERENCE SIGNS 1 sutra recitation assistance device 171 reception unit 172 presentation unit 173 pitch evaluation unit 174 emotion evaluation unit 175 output unit 176 emotion determination unit 177 skill level evaluation unit 178 skill level output unit

Claims (7)

an imaging unit that images a user;
a presentation unit that presents the speed at which the user should recite the sutra or the period during which the user should recite the sutra by means of an image or sound;
a pitch evaluation unit that evaluates the pitch of the recitation voice when the user recites the sutra for each character segment of the sutra during the presentation of the speed or the period ;
an emotion evaluation unit that evaluates the emotion of the user who is reciting the sutra for each character segment while the speed or the period is being presented , based on images including areas corresponding to exposed skin of the human body of the user who is reciting the sutra, which are continuously obtained by the imaging unit;
an output unit that outputs the pitch evaluation and the emotion evaluation each time the character segment is read aloud;
A sutra recitation assistance device comprising: The system further includes a reception unit that receives a user's selection of a denomination,
the presenting unit presents the speed at which the user should recite the sutra or the period during which the sutra should be recited by the user, by presenting the timing for taking a breath or by outputting the sound of a mokugyo at predetermined intervals, in accordance with the selection of the sect;
The sutra recitation assistance device according to claim 1. the output unit displays the pitch evaluation and the emotion evaluation in a single display area using different display modes.
The sutra recitation assistance device according to claim 1. the output unit displays the pitch evaluation and the emotion evaluation using a single color and shape of a flame.
The sutra recitation assistance device according to claim 3. an emotion determination unit that determines an emotion based on a maximum Lyapunov exponent indicating a degree of fluctuation in pulse wave intervals of the user who is reciting the sutra;
a skill evaluation unit that evaluates the skill of the user's sutra chanting based on a combination of the evaluation of the pitch, the evaluation of the emotion, and the determination of the feeling;
A skill output unit that outputs the skill evaluation,
The sutra recitation assistance device according to claim 1. A sutra recitation assistance method executed by a sutra recitation assistance device,
The speed at which the user should recite the sutra or the period during which the user should recite the sutra is displayed by image or sound;
During the presentation of the speed or the period, an evaluation is made of the pitch of the recitation voice when the user recites the sutra for each character segment of the sutra;
During the presentation of the speed or the period, the emotion of the user is evaluated for each character segment based on images including areas corresponding to exposed skin of the user reciting the sutra, the images being continuously obtained by an imaging unit ;
outputting the pitch evaluation and the emotion evaluation each time the character segment is read aloud;
A sutra recitation support method comprising: The speed at which the user should recite the sutra or the period during which the user should recite the sutra is displayed by image or sound;
During the presentation of the speed or the period, an evaluation is made of the pitch of the recitation voice when the user recites the sutra for each character segment of the sutra;
During the presentation of the speed or the period, the emotion of the user is evaluated for each character segment based on images including areas corresponding to exposed skin of the user reciting the sutra, the images being continuously obtained by an imaging unit ;
outputting the pitch evaluation and the emotion evaluation each time the character segment is read aloud;
A program that causes a computer to execute the following: