JP7611277B2 - Subtitle generation device, video transmission device, subtitle generation method, and program - Google Patents

Description

This disclosure relates to a subtitle generation device, a video transmission device, a subtitle generation method, and a program.

Services that deliver video and audio in real time (streaming) using communication lines such as the Internet are becoming more common. At the same time, services that subtitle and display audio are also on the rise. In some cases, video distributors provide viewers with a service that uses voice recognition technology to automatically subtitle audio and display it.

Patent Documents 1 and 2 describe technologies related to subtitles on television. Patent Document 1 discloses a technology for easily correcting subtitle text. Patent Document 2 discloses a technology for constructing a language model that corresponds to variations in expression during speech recognition.

Patent Documents 3 and 4 describe technologies related to subtitles in streaming. Patent Document 3 discloses a technology that suppresses delays in subtitles relative to program content. Patent Document 4 discloses a technology that generates subtitle text based on audio reference information created by analyzing the audio state.

JP 2022-073807 A JP 2016-024325 A JP 2020-027984 A JP 2019-213198 A

In a subtitle service, subtitles are generally displayed in sequence following the audio spoken by a person. In the case of Japanese, a mixture of kanji, hiragana, katakana, etc. is displayed. When adding subtitles to video data, it is necessary to display subtitles that are appropriate for the content of the video.

However, sometimes subtitles that are not appropriate for the content of the video are displayed. For example, when converting to kanji, the text may be mistakenly converted to kanji with the same pronunciation but a different meaning.

This disclosure has been made in consideration of the above problems, and aims to provide a technology for generating appropriate subtitles from the audio of video data.

In order to achieve the above object, one aspect of the present disclosure is a subtitle generation device having a division unit that divides video data including audio into files of a predetermined length, a detection unit that detects speech segments in each divided file using segments of audio extracted from the file, a data generation unit that generates the audio of the speech segments as recognition data, and a voice recognition unit that converts the audio of the recognition data into text data as subtitles, and when the speech segment spans multiple files, the data generation unit generates the recognition data by combining the audio corresponding to the speech segment in the multiple files.

One aspect of the present disclosure is a subtitle generation method performed by a computer, which divides video data including audio into files of a predetermined length, detects audio segments in each divided file using audio segments extracted from the file, generates the audio of the audio segments as recognition data, converts the audio of the recognition data into text data as subtitles, and, in generating the recognition data, if the audio segment spans multiple files, combines the audio corresponding to the audio segment from the multiple files to generate the recognition data.

One aspect of the present disclosure is a program that causes a computer to function as the above-mentioned subtitle generation device.

This disclosure provides a technology that generates appropriate subtitles from the audio of video data.

FIG. 1 is a conceptual diagram showing an overview of video transmission according to this embodiment. FIG. 2 is a diagram showing an example of the configuration of a video transmission device according to the present embodiment. FIG. 3 is an image diagram of data output from each unit of the video transmission device. FIG. 4 is a flowchart showing the subtitle generation process of the video transmission device. FIG. 5 is a diagram showing a hardware configuration.

Embodiments of the present disclosure will be described below with reference to the drawings. The same reference symbols are used for the same parts in the drawings, and descriptions will not be repeated.

Figure 1 is an image diagram of video transmission using a video transmission device (subtitle generation device) according to this embodiment. Here, we will explain an overview of the procedure by which the video transmission device generates subtitles for video data (e.g., video content, etc.).

Video data including video and audio is supplied to the video transmission device 1 from a device such as a camera or a server or a storage medium. Note that the video data supplied to the video transmission device 1 does not include subtitles.

Here, sound includes human speech and non-speech sounds (such as music). In this embodiment, sound means human speech (voice), and does not include other sounds. In this specification, the term "utterance" is synonymous with voice.

The video transmission device 1 generates a subtitle setting file and a playlist based on the supplied video data. Then, the video transmission device 1 uploads the generated subtitle setting file and playlist together with the input video data to a distribution server 2 or the like.

In response to a request from a user (viewer), the distribution server 2 distributes video data, a subtitle setting file, and a playlist to a playback device 3 such as the user's PC or smartphone. The subtitle setting file is a file that specifies which subtitles are to be displayed at what time in the video data (e.g., the time from the start of playback). In this embodiment, the distribution server 2 performs streaming distribution to the playback device 3. The user uses a browser on the playback device 3 to play the streamed video data together with the subtitles.

The video transmission device 1 can be implemented as a physical device, or can be configured as software on the cloud.

Next, we will explain the outline of the operation of the video transmission device 1 shown in Figure 1.

The video transmission device 1 sequentially divides the video data input as stream data into files of a specified length and generates the divided files (S1).

Video data can be divided, for example, by using a video distribution service based on HLS (HTTP Live Streaming). HLS is a streaming network distribution service that complies with the HTTP protocol and is widely used. Data division using HLS is performed by a device such as a transcoder that has a division function. In this embodiment, data that complies with HLS is used as the video data.

In HLS, video data in TS (Transport Stream) format is divided into multiple segment files (hereafter referred to as TS files) and transmitted. TS files are also called MPEG2-TS segment files, segments, or TS chunk files.

The length of time into which video data is divided can be specified as desired. Specifically, the time width of each TS file (division width) can be any value. As will be described later, testing has shown that a time width of TS files of around 10 seconds provides the best results in improved subtitle accuracy.

The video transmission device 1 generates a playlist in which the playback order of the TS files, playback time, content variations, etc. are set. The playlist is also called an index file or m3u8 file. The playlist is transmitted to the distribution server 2 together with the TS files.

In this embodiment, the case where the HLS method is used will be described as an example, but the present disclosure is not limited to the HLS method. Other methods may be used as long as the input video data can be divided into files of a predetermined length. For example, instead of TS data, MPEG-DASH or CMAF (Common Media Application Format) methods for dividing mp4 data may be used.

The video transmission device 1 sequentially generates recognition data for audio recognition for each divided TS file (S2). In particular, when a series of sounds spans multiple TS files, related sounds are combined to generate recognition data for a single chunk of audio.

The video transmission device 1 performs voice recognition on the generated recognition data and generates subtitles (subtitle display) to be displayed together with the video (S3). For the voice recognition, it is preferable to use a voice recognition AI engine.

The video transmission device 1 generates a subtitle setting file for displaying (adding) the subtitles (text data) generated by voice recognition to the video (S4). The subtitle setting file is a file that specifies which subtitles are to be displayed at what time or time period in the video. Note that the video transmission device 1 also updates the playlist when generating the subtitle setting file.

Through the above process, the video transmission device 1 outputs the files into which the input video data has been divided, as well as the generated subtitle setting file and playlist, to the distribution server 2.

Figure 2 is a block diagram showing the configuration of the video transmission device 1 shown in Figure 1. Figure 3 is a diagram showing an example of data output by each unit of the video transmission device 1.

The illustrated video transmission device 1 includes a division unit 11, an audio extraction unit 12, a detection unit 13, a data generation unit 14, an audio recognition unit 15, a subtitle setting unit 16, a playlist creation unit 17, and a playlist update unit 18. Each of the units 11 to 18 may be implemented as software such as a program, or may be implemented as hardware such as a circuit.

The splitter 11 splits the video data including audio into files of a predetermined length. Specifically, the splitter 11 splits the TS data (video data) input to the video transmission device 1 sequentially into data of a predetermined length to generate TS files (TS file 1, TS file 2, ...). The predetermined length refers to the amount of data corresponding to the time width for playing the video data, and specifically refers to the time width (division width) when the data of the TS file is played.

Note that it is preferable that the splitting unit 11 sequentially splits the TS data into TS files with a length of about 10 seconds. A length of about 10 seconds is, for example, 10 seconds - n seconds or more, or 10 seconds + n seconds. For example, n = 5, and the predetermined length may be 5 seconds or more and 15 seconds or less.

Figure 3(a) shows TS data input to the splitting unit 11 from a camera, stored data, etc. Figure 3(b) shows a TS file output from the splitting unit 11. The splitting unit 11 can not only split the input TS data in real time to generate TS files, but can also temporarily store the input TS data in a buffer memory and then split it at high speed. Although not shown in Figure 3, in actual processing, some processing delays occur in each of the units 11 to 18. For the sake of explanation, processing delays are not shown in Figure 3.

The audio extraction unit 12 extracts audio from the sound (audio) contained in each TS file.

The detection unit 13 detects a voice interval (speech interval) in each divided TS file using the breaks in the voice (utterance) extracted from the TS file. Specifically, the detection unit 13 detects the start time and end time of the voice interval. The detection unit 13 detects a break in the voice interval. The break in the voice can be detected based on the presence or absence of a silence in the extracted voice. The conditions for the silence can be set arbitrarily in terms of duration, volume, etc. Furthermore, the silence can be detected using a general-purpose application.

Figure 3(c) shows an example of the output of the detection unit 13. In the example shown, the detection unit 13 first detects the voice section of TS file 1. The detection unit 13 detects voice section 1, voice section 2, and voice section 3 using the voice division X (silent portion) of TS file 1.

No audio separation X is detected at the end of the final voice interval 3 in TS file 1. Therefore, it is clear that voice interval 3 continues into the next TS file 2 (straddles the next TS file 2). In this case, the detection unit 13 determines that a discontinuity Y of the voice interval or a separation Y of the voice interval has been detected.

An audio division X indicates silent parts before and after an audio section. An audio section break Y indicates that an audio section that should be continuous has been split into multiple TS files. Note that an audio division X is sometimes referred to as an audio section break, and an audio section break Y is sometimes referred to as an audio break.

Next, the detection unit 13 detects the voice section in TS file 2. The detection unit 13 detects voice section 3 and voice section 4 using the voice division X. Here, the detection unit 13 determines that the voice section 3 in TS file 1 has been divided, and therefore determines that the voice section 3 first detected in TS file 2 is the voice section that continues to the last voice section 3 in the immediately preceding TS file 1. Therefore, the detection unit 13 associates the voice section first detected in TS file 2 with the last voice section 3 in TS file 1. For example, the detection unit 13 assigns the same identifier as the last voice section 3 in TS file 1 to the voice section first detected in TS file 2.

The last detected audio section 4 in TS file 2 has an audio break X detected at its end. Therefore, the detection unit 13 determines that audio section 4 is complete in TS file 2 without being interrupted.

Then, the detection unit 13 detects the voice section of the TS file 3. Because the last voice section 4 is completed in the immediately preceding TS file 2, the detection unit 13 determines that the first voice section 5 is the voice section that started in that TS file 3. Then, because no voice break X is detected at the end of the first and last voice section 5, the detection unit 13 determines that a voice break Y has occurred and that the voice section 5 has been divided.

Then, the detection unit 13 detects the voice section in TS file 4. Because it has been determined that the voice section 5 in the immediately preceding TS file 3 has been divided, the detection unit 13 determines that the first voice section 5 detected in TS file 4 is a voice section that continues to the voice section 5 in TS file 3.

The detection unit 13 similarly detects the voice section for subsequent TS files. The detection unit 13 outputs information about the voice section detected in each TS file to the data generation unit 14. The information about the voice section is, for example, information such as the time or time period when the voice break X was detected, the time period such as the start time and end time of the voice section, and whether or not it is a divided voice section that spans the next voice section.

Next, the data generation unit 14 generates the audio of the audio section detected by the detection unit 13 as recognition data. When an audio section spans multiple TS files, the data generation unit 14 generates recognition data by combining audio corresponding to the audio section in the multiple TS files. In other words, the data generation unit 14 generates recognition data that is easy to process by voice recognition. When no audio break X is detected at the end of a TS file, the data generation unit 14 may determine that the audio section detected at the end of the TS file is divided and spans the next file.

Figure 3(d) shows an image of the recognition data output by the data generation unit 14. In the example shown, the data generation unit 14 treats the speech of speech section 1 and speech section 2 detected in TS file 1 as recognition data 1 and 2 as they are, respectively, because they are complete within TS file 1.

The final audio section 3 of TS file 1 is split and spans the next TS file 2, so the data generator 14 combines the audio of audio section 3 of TS file 1 with the audio of the first audio section 3 detected in the next TS file 2 to generate recognition data 3.

In the voice section 4 of the TS file 2, the voice boundary X has been detected within the TS file 2, so the data generator 14 uses the voice in the voice section 4 as recognition data 4 as is.

Because the voice section 5 of TS file 3 is divided and spans the next TS file 4, the data generation unit 14 combines the voice of the voice section 5 of TS file 3 with the voice of the voice section 5 that is first detected in the next TS file 4 to generate recognition data 5. The data generation unit 14 outputs the generated recognition data sequentially to the voice recognition unit 15.

The speech recognition unit 15 recognizes the speech of the recognition data generated by the data generation unit 14 and converts it into text data as subtitles. In this embodiment, speech intervals are detected using speech breaks X, and when a speech interval is interrupted and spans multiple TS files, recognition data is generated by combining the interrupted speech intervals, and speech recognition is performed using the recognition data. In this way, the recognition data is speech equivalent to a series of speech intervals without speech breaks X, and is therefore data that is easy for the speech recognition unit 15 to recognize. Therefore, in this embodiment, erroneous speech recognition conversion due to speech breaks can be prevented, and the accuracy of subtitles can be further improved.

The voice recognition unit 15 may recognize voice using a voice recognition engine provided within the voice recognition unit 15, or may recognize voice using an external voice recognition engine 4 (for example, a voice recognition AI engine from Google or the like). It is preferable that the voice recognition unit 15 uses a voice recognition AI engine as the voice recognition engine to convert the voice of the recognition data into text data. By using AI (artificial intelligence), it is possible to convert into subtitles with higher accuracy than with conventional voice recognition.

Figure 3(e) shows an image of the subtitles converted by the speech recognition unit 15. In Figure 3(e), the results of speech recognition of the speech of recognition data 1 to 5, respectively, are written as subtitles 1 to 5.

The subtitle setting unit 16 generates a WebVTT file (subtitle setting file) for each TS file in order to display the text data converted by the voice recognition unit 15 as subtitles. The WebVTT file includes the subtitles to be displayed and the display time of the subtitles. When the data generation unit 14 combines the voices of multiple TS files to generate recognition data, the subtitle setting unit 16 may split the text data converted from the recognition data to generate subtitle setting files for each of the multiple TS files.

Figure 3(f) shows an image of a WebVTT file generated by the subtitle setting unit 16. For TS file 1, the subtitle setting unit 16 generates WebVTT file 1 so that subtitle 1, subtitle 2, and subtitle 3-1 are displayed corresponding to speech section 1, speech section 2, and speech section 3 of TS file 1, respectively. Subtitle 3-1 is the subtitle for the part of subtitle 3 converted from recognition data 3 that corresponds to speech section 3 in TS file 1.

For TS file 2, the subtitle setting unit 16 generates a WebVTT file 2 so that subtitle 3-2 and subtitle 4 are displayed corresponding to speech section 3 and speech section 4, respectively, in the TS file 2. Subtitle 3-2 is the subtitle for the portion of subtitle 3 converted from recognition data 3 that corresponds to speech section 3 in the TS file 2.

For TS file 3, the subtitle setting unit 16 generates a WebVTT file 3 so that subtitle 5-1 is displayed corresponding to audio section 5 of TS file 3. Subtitle 5-1 is a subtitle within subtitle 5 that corresponds to audio section 5 of TS file 3.

For TS file 4, the subtitle setting unit 16 generates a WebVTT file 4 so that subtitle 5-2 is displayed corresponding to the audio section 5 of the TS file 4. Subtitle 5-2 is the subtitle of the part of the subtitle 5 converted from the recognition data 5 that corresponds to the audio section 5 of the TS file 4.

The playlist generation unit 17 creates a playlist when the division unit 11 divides the TS data into multiple TS files. The playlist sets the playback order of the divided TS files, content variations, and the like. When the subtitle setting unit 16 generates a WebVTT file, the playlist update unit 18 sets information about subtitles included in the WebVTT file in the playlist and updates the playlist.

In the above-mentioned video transmission device 1, the operations from the audio extraction unit 12 to the subtitle setting unit 16 are actually performed for each TS file. That is, when TS file 1 is input, the detection unit 13 detects speech segments 1 to 3, and the data generation unit 14 generates recognition data 1 from speech segment 1 and generates recognition data 2 from speech segment 2. The data generation unit 14 carries over the recognition data for speech segment 3 to the processing of the next TS file 2.

Then, the voice recognition unit 15 generates subtitle 1 from the recognition data 1, and generates subtitle 2 from the recognition data 2. The subtitle setting unit 16 generates WebVTT file 1 using information about subtitles 1 and 2.

For TS file 2, the same processing is carried out as for TS file 1. In particular, the voice section 3 carried over from TS file 1 is combined with the voice section 3 first detected in TS file 2 to generate recognition data 3. The recognition data 3 is converted into subtitles 3 by the voice recognition unit 15, and information related to subtitles 3 is divided and set in WebVTT file 1 and WebVTT file 2.

At this point, WebVTT file 1 is complete and is output from the video transmission device 1 together with TS file 1. Note that TS file 1 is output from the splitting unit 11, and WebVTT file 1 is output from the subtitle setting unit 16.

In this way, the video transmission device 1 sequentially outputs each TS file and the corresponding WebVTT file. Therefore, every time a TS file is output from the video transmission device 1, a playlist indicating the existence of that TS file is also output from the playlist update unit 18 together with the WebVTT file.

The division unit 11, the subtitle setting unit 16, and the playlist update unit 18 work together to synchronize the output of the TS file with the output of the WebVTT file and the playlist.

Figure 4 is a flowchart showing the processing of the video transmission device 1 of this embodiment. This shows the processing up to the point where the video transmission device 1 generates a WebVTT file (subtitle setting file).

The division unit 11 divides the TS data (video data) including audio into pieces of a predetermined length, and sequentially outputs the TS files of the divided data (S11). The video transmission device 1 performs the subsequent processes (S12 to S23) for each TS file. Here, the TS file to be processed is referred to as TS file n.

The audio extraction unit 12 extracts audio from the divided TS file n (S12).

The detection unit 13 detects a voice section from the voice extracted by the voice extraction unit 12 using the voice division X (S13). Specifically, the detection unit 13 detects silent parts in the voice extracted in S12 to detect the voice division X and identify the voice section. Here, if the voice division X is not detected at the rear end of the voice section, the detection unit 13 determines that the voice section is a divided voice section that spans the next TS file n+1. In other words, the detection unit 13 determines that a break Y has occurred in the voice section. The detection unit 13 sends information about each detected voice section to the data generation unit 14.

The data generation unit 14 uses the information about the voice section to determine whether or not there is a voice section that has been interrupted (carried over) in the immediately preceding TS file n-1 (S14). If there is no interrupted voice section (S14: NO), the data generation unit 14 generates the voice of the voice section that is followed by voice division X in the TS file n as recognition data, and outputs it sequentially to the voice recognition unit 15 (S15).

If there is no segmented audio section that is not followed by audio division X (S16: NO), proceed to S19. On the other hand, if there is a segmented audio section (S16: YES), the data generator 14 stores the audio of the segmented audio section in a storage unit (not shown) to carry it over to the next TS file n+1 (S17).

Also, if there is a segmented (carried over) audio section in the immediately preceding TS file n-1 (S14: YES), the data generator 14 combines the audio of the first audio section in the TS file n with the audio of the audio section carried over from the immediately preceding TS file n-1 (S18). The data generator 14 then proceeds to S15. S15 to S17 are as described above.

The voice recognition unit 15 performs voice recognition on the voice of each recognition data output by the data generation unit 14 and converts each recognition data into text data (S19).

The subtitle setting unit 16 generates a WebVTT file (subtitle setting file) for displaying the converted text data as subtitles (S20). Here, the generated WebVTT file is a WebVTT file for TS file 1. The subtitle setting unit 16 sets the subtitles (text data) to be displayed in the WebVTT file and the display time for displaying the subtitles. The display time can be obtained (calculated) from information about the speech section detected by the detection unit 13, etc.

When the data generating unit 14 generates the recognition data by combining the audio sections of multiple TS files (S18), the subtitle setting unit 16 extracts the text data converted from the recognition data that corresponds to the audio section of the TS file n, and generates a WebVTT file.

Then, the subtitle setting unit 16 outputs the generated WebVTT file together with the corresponding TS file n.

If there is a next TS file n+1 (S21: YES), the video transmission device 1 returns to S12 and repeats the subsequent processes. If there is no next TS file n+1 (S21: NO), the video transmission device 1 ends the process shown in FIG. 4. In this embodiment, the processes S12 to S20 in FIG. 4 are performed sequentially for each TS file, so that the processing delay of the video transmission device 1 can be reduced.

As described above, the video transmission device 1 (subtitle generation device) of this embodiment has a division unit 11 that divides a TS file (video data) containing audio into TS files of a predetermined length, a detection unit 13 that detects audio sections in each of the divided TS files using audio segments extracted from the TS file, a data generation unit 14 that generates the audio of the audio section as recognition data, and a voice recognition unit 15 that converts the audio of the recognition data into text data as subtitles. When the audio section spans multiple TS files, the data generation unit 14 combines the audio corresponding to the audio section of the multiple TS files to generate recognition data.

In this manner, in this embodiment, video data is divided into multiple TS files, and when a segmented speech section that spans multiple TS files is detected in each TS file, the segmented speech sections are joined to generate recognition data that is easy to use for speech recognition. This makes it possible to prevent erroneous conversion due to speech segmentation, and improve the accuracy of subtitles.

By using an AI engine for voice recognition, it is possible to convert the speech into subtitles with higher accuracy.

The time unit for dividing the video data was set to about 10 seconds (for example, between 5 and 15 seconds). If it was longer than about 10 seconds, the delay until the video was distributed increased, and if it was shorter than about 10 seconds, the accuracy of the subtitles decreased. As a result of the verification, it was found that a time unit between 5 and 15 seconds was preferable.

The video transmission device 1 described above can use, for example, a general-purpose computer system as shown in FIG. 5. The computer system shown in the figure includes a CPU (Central Processing Unit, processor) 901, a memory 902, a storage 903 (HDD: Hard Disk Drive, SSD: Solid State Drive), a communication device 904, an input device 905, and an output device 906. The memory 902 and the storage 903 are storage devices. In this computer system, the functions of the video transmission device 1 are realized by the CPU 901 executing a predetermined program loaded on the memory 902.

The video transmission device 1 may be implemented in one computer, or in multiple computers. The video transmission device 1 may also be a virtual machine implemented in a computer. The program of the video transmission device 1 may be stored in a computer-readable recording medium such as a HDD, SSD, USB (Universal Serial Bus) memory, CD (Compact Disc), or DVD (Digital Versatile Disc), or may be distributed via a network. The computer-readable recording medium is, for example, a non-transitory recording medium.

Note that the present disclosure is not limited to the above embodiment, and various modifications are possible within the scope of the gist. In the above embodiment, the video transmission device 1 is described as performing the subtitle generation process, but the present disclosure is not limited to this. Any device capable of performing the subtitle generation process of the present disclosure is not limited to the video transmission device 1.

1: Video transmission device (subtitle generation device)
11: Division unit 12: Audio extraction unit 13: Detection unit 14: Data generation unit 15: Audio recognition unit 16: Subtitle setting unit 17: Playlist generation unit 18: Playlist update unit 2: Distribution server 3: Playback device 4: Audio recognition AI engine

Claims (10)

a division unit for dividing the video data including audio into files of a predetermined length;
a detection unit for detecting a voice section in each of the divided files by using a segment of the voice extracted from the file;
a data generating unit that generates speech in the speech section as recognition data;
a voice recognition unit that converts the voice of the recognition data into text data as subtitles,
When the speech section spans a plurality of files, the data generation unit generates recognition data by combining speech corresponding to the speech section of the plurality of files. The subtitle generation device according to claim 1 , wherein, when the break is not detected at the end of a file, the data generation unit determines that the audio section detected at the end of the file is broken and spans over to a next file. The subtitle generating device according to claim 1 , further comprising a subtitle setting unit configured to generate, for each of the files, a subtitle setting file for displaying the text data as subtitles. The subtitle generation device according to claim 3 , wherein when the data generation unit generates the recognition data by combining audio from a plurality of files, the subtitle setting unit divides text data converted from the recognition data to generate subtitle setting files for each of the plurality of files. The subtitle generating device according to claim 1 , wherein the division unit divides the video data into files each having a length of 5 seconds or more and 15 seconds or less. The subtitle generation device according to claim 1 , wherein the video data is data conforming to HLS (HTTP Live Streaming). The subtitle generation device according to claim 1 , wherein the speech recognition unit converts the speech of the recognition data into text data using a speech recognition AI engine. A subtitle generating device according to claim 3 or 4,
A video transmission device that outputs each of the divided files as video data together with the subtitle setting file. 1. A computer-implemented method for generating subtitles, comprising the steps of:
Divide the video data including audio into files of a specified length,
Detecting a voice section in each divided file using the voice segment extracted from the file;
generating speech in the speech section as recognition data;
converting the speech of the recognition data into text data as subtitles;
a subtitle generating method for generating the data for recognition, wherein, when the speech section spans a plurality of files, speech corresponding to the speech section of the plurality of files is combined to generate the data for recognition. A program that causes a computer to function as a subtitle generation device according to any one of claims 1 to 7.

Images