JP7517366B2 - Voice recording management system, voice recording management device, voice recording management method and program - Google Patents

Description

The present invention relates to an audio recording management system, an audio recording management device, an audio recording management method, and a program.

Technology has been known for some time now for automatically generating meeting minutes by converting audio information into text information.

For example, a technique is known in which conference text information, which is the result of speech recognition of audio data recorded at a conference, is divided into multiple sections, and summary information is generated for each section of the divided text information (see, for example, Patent Document 1).

However, with conventional technology, when editing a voice recording generated based on voice information, it was necessary to separately edit the text data contained in the voice recording and the image data and voice data corresponding to that text data, resulting in a problem of low convenience.

In order to solve the above-mentioned problem, the invention according to claim 1 is a voice recording management system including a voice recording management device that manages voice recording information obtained based on voice information, and one or more communication terminals that can display the voice recording information by communicating with the voice recording management device, wherein the voice recording management device includes an acquisition means for acquiring voice data representing the voice information transmitted by a first communication terminal among the one or more communication terminals, and screen data representing a screen displayed on the first communication terminal, and a predetermined text data representing a predetermined text obtained based on the acquired voice data, an image included on the screen related to the acquired screen data, and predetermined image data representing a predetermined image associated with the predetermined text, and a predetermined voice data indicated by the predetermined text, and a predetermined voice data, which is displayed by the predetermined text, and a predetermined voice data, which is displayed by the predetermined text, are transmitted to the first communication terminal. and a transmission means for transmitting the edit request transmitted by the first communication terminal to the one or more communication terminals including the first communication terminal, the transmission means transmits edited screen data including edited text data obtained by editing the specified text data and edited image data obtained by editing the specified image data, and edited audio data obtained by editing the specified audio data, to a second communication terminal different from the first communication terminal in response to the edit request transmitted by the first communication terminal for the specified text or the specified image, the edit request being transmitted by the first communication terminal, and the second communication terminal has a display control means for displaying an edited screen related to the edited screen data transmitted by the voice recording management device on a display means, and an audio playback means for playing back the edited audio related to the edited audio data transmitted by the voice recording management device.

As described above, according to the present invention, when editing an audio recording generated based on audio information, it is sufficient to edit the text data contained in the audio recording or the image data corresponding to that text data, thereby achieving the effect of improving the convenience of editing the audio recording.

FIG. 1 illustrates an example of an overall configuration of a communication system. FIG. 2 is a diagram illustrating an example of a hardware configuration of a communication terminal and a voice recognition server. FIG. 2 is a diagram illustrating an example of a hardware configuration of a voice recording management device. FIG. 1 illustrates an example of a functional configuration of a communication system. FIG. 13 is a conceptual diagram illustrating an example of a login management table. FIG. 13 is a conceptual diagram showing an example of a record bibliographic information management table. FIG. 13 is a conceptual diagram illustrating an example of a text information management table. FIG. 13 is a conceptual diagram illustrating an example of a capture image management table. FIG. 13 is a conceptual diagram illustrating an example of a capture image acquisition interval table. FIG. 13 is a conceptual diagram illustrating an example of a private voice management table. 11 is a sequence diagram showing an example of application startup, authentication processing, and session establishment processing. FIG. 13 is an example of a screen display when an application is started on a communication terminal. FIG. 11 is a sequence diagram illustrating an example of a recording start process. 13 is an example of a screen display of a recording start instruction on a communication terminal. FIG. 11 is a sequence diagram showing an example of a process for registering record bibliographic information. FIG. 11 is a sequence diagram illustrating an example of a voice recognition process. 13 is an example of a screen display during recording on the communication terminal. FIG. 11 is a sequence diagram illustrating an example of a screen capture process. FIG. 11 is a sequence diagram showing an example of a recording end process. 13 is an example of a screen display when recording ends on the communication terminal. 13 is an example of a screen display when selecting recording on the communication terminal. 13 is an example of a shared information input dialogue screen displayed on the communication terminal. FIG. 13 is a sequence diagram showing an example of a process for generating a record viewing and editing screen. 13 is a flowchart showing an example of a record viewing and editing screen generation process for the creator's communication terminal or for the first viewing. 13 is a flowchart showing an example of a record viewing screen generation process for a communication terminal of a user other than the creator. 13 is a flowchart showing an example of a record viewing screen generation process for a communication terminal of a user other than the creator. 13 is a screen display example of a record viewing and editing screen on the creator's communication terminal. 13 is an example of a screen display for viewing a record on a communication terminal of a user other than the creator. FIG. 11 is a sequence diagram showing an example of a record viewing and editing process. 11 is a flowchart showing an example of a process branched by various button operations. 11 is a flowchart showing an example of a process branched by various button operations. 13 is a flowchart showing an example of a process performed when a private button is operated on text. 13 is an example of a screen display displayed on a communication terminal of a user other than the creator when a private button for text is operated. 13 is a flowchart showing an example of a process performed when a publish button is operated on text. 13 is a screen display example when a creator operates a publish button on the text on the communication terminal. 13 is a flowchart showing an example of a process performed when a delete button is operated on text. 13 is an example of a screen display displayed on a communication terminal of a user other than the creator when a delete button for text is operated. 13 is a flowchart showing an example of a process performed when a private button is operated on a captured image. 13 is a diagram illustrating an example of a screen display displayed on a communication terminal of a user other than the creator when a private button for a captured image is operated. 13 is a flowchart showing an example of a process performed when a publish button is operated on a captured image. 13 is a flowchart showing an example of a process performed when a delete button is operated on a captured image. 13 is an example of a screen display displayed on a communication terminal of a user other than the creator when a delete button for a captured image is operated. FIG. 13 is a conceptual diagram illustrating an example of a text information management table according to the second embodiment. FIG. 11 is a conceptual diagram illustrating an example of a summary information management table according to the second embodiment. 13 is a flowchart showing an example of a process branched by various button operations according to the second embodiment. 13 is a flowchart showing an example of a process branched by various button operations according to the second embodiment. 13 is a flowchart illustrating an example of a process performed when a private button is operated on a text group according to the second embodiment. 13 is an example of a screen display when a summarization button is operated for a text group in the communication terminal according to the second embodiment. 13 is a diagram illustrating an example of a screen display of a summary information input dialogue in a communication terminal according to a second embodiment. 13 is an example of a screen display including a summary display field in a communication terminal according to a second embodiment. 13 is a flowchart showing an example of a process performed when a publish button is operated on a text group according to the second embodiment. 13 is a flowchart illustrating an example of a process performed when a delete button is operated on a text group according to the second embodiment. 13 is a flowchart illustrating an example of a process performed when a private button is operated on a captured image according to the second embodiment. 13 is a flowchart illustrating an example of a process performed when a publish button is operated on a captured image according to the second embodiment. 13 is a flowchart showing an example of a process performed when a delete button is operated on a captured image according to the second embodiment. FIG. 11 is a diagram illustrating an example of a functional configuration of a communication system according to a third embodiment. FIG. 13 is a conceptual diagram illustrating an example of a record bibliographic information management table according to the third embodiment. FIG. 13 is a conceptual diagram illustrating an example of a text information management table according to the third embodiment. FIG. 13 is a conceptual diagram illustrating an example of a capture image management table according to the third embodiment. 13 is an example of a screen display when selecting recording in the communication terminal according to the third embodiment. FIG. 13 is a sequence diagram showing an example of a record viewing and editing process according to the third embodiment. 13 is a screen display example of a record viewing and editing screen on a creator's communication terminal according to the third embodiment. 13 is another example of a screen display of the record viewing and editing screen of the creator's communication terminal according to the third embodiment. 13 is another example of a screen display of the record viewing and editing screen of the creator's communication terminal according to the third embodiment. 13 is a flowchart showing an example of a division process of a record viewing and editing screen according to the third embodiment. A conceptual diagram showing an example of a text information management table after splitting processing of the record viewing and editing screen in the third embodiment, where (a) is a conceptual diagram of the text information management table constituting the first split audio recording screen, and (b) is a conceptual diagram of the text information management table constituting the second split audio recording screen. 13 is a conceptual diagram showing an example of a capture image management table after splitting processing of a record viewing and editing screen relating to the third embodiment, where (a) is a conceptual diagram of a capture image management table constituting the first split audio recording screen, and (b) is a conceptual diagram of a capture image management table constituting the second split audio recording screen. This is a conceptual diagram showing an example of a recorded bibliographic information management table after the division processing of the record viewing and editing screen related to the third embodiment, and is a conceptual diagram of the recorded bibliographic information management table that constitutes the first divided voice recording screen. This is a conceptual diagram showing an example of a recorded bibliographic information management table after splitting processing of the record viewing and editing screen relating to the third embodiment, and is a conceptual diagram of the recorded bibliographic information management table constituting the second split voice recording screen. 13 is an example of a screen display when selecting a record after division processing of the record viewing and editing screen on the communication terminal according to the third embodiment. 13 is a screen display example of a first divided voice recording screen displayed on a communication terminal of a user other than the creator according to the third embodiment. 13 is a diagram illustrating another example of a screen display when selecting a record after division processing of the record viewing and editing processing screen in the communication terminal according to the third embodiment. 13 is a screen display example of a second divided voice recording screen displayed on a communication terminal of a user other than the creator according to the third embodiment. 13 is a flowchart showing an example of a division process of a record viewing and editing screen according to a modified example of the third embodiment. 13 is a conceptual diagram showing an example of a text information management table after splitting processing of a record viewing and editing screen relating to a modified example of the third embodiment, where (a) is a conceptual diagram of a text information management table constituting a first split audio recording screen, and (b) is a conceptual diagram of a text information management table constituting a second split audio recording screen. This is a conceptual diagram showing an example of a recorded bibliographic information management table after splitting processing of a record viewing and editing screen related to a modified example of the third embodiment, and is a conceptual diagram of the recorded bibliographic information management table that constitutes the first split voice recording screen. A conceptual diagram showing an example of a recorded bibliographic information management table after splitting processing of a record viewing and editing screen relating to a modified example of the third embodiment, which is a conceptual diagram of a recorded bibliographic information management table constituting a second split voice recording screen.

Below, we will explain the form for implementing the invention using the drawings. Note that in the explanation of the drawings, the same elements are given the same reference numerals, and if there are overlapping parts, their explanation will be omitted.

First Embodiment
The first embodiment will be described with reference to FIGS. 1 to 39. FIG.

[Overall configuration of communication system]
<System configuration example>
FIG. 1 is a diagram showing an example of the overall configuration of a communication system. As shown in FIG. 1, the communication system 1 has devices including one or more communication terminals 3, a voice recording management device 5, and a voice recognition server 7. The communication terminals 3, the voice recording management device 5, and the voice recognition server 7 are connected to each other via a communication network 100. Here, the communication network 100 is a communication network in which an unspecified number of communications are performed, and is constructed by the Internet, an intranet, a LAN (Local Area Network), or the like. Note that the communication network 100 may include not only wired communication, but also communication networks using wireless communication such as 3G (3rd Generation), 4G (4th Generation), 5G (5th Generation), WiMAX (Worldwide Interoperability for Microwave Access), and LTE (Long Term Evolution). Furthermore, the communication system 1 includes a voice recording management system 2 constructed by the communication terminals 3 and the voice recording management device 5. In addition, the communication terminal 3 and the voice recording management device 5 may be connected to each other via a dedicated in-house network or the like, or may be connected to each other inside the communication network 100 via a firewall.

<Communication terminal>
The communication terminal 3 is realized by one or more information processing devices (computer systems) for communication equipped with a general OS, etc. The communication terminal 3 is capable of communicating with the voice recording management device 5 via a communication network 100. As shown in Fig. 1, the communication terminal 3 is composed of one or more communication terminals including a communication terminal 3(A), a communication terminal 3(B), and a communication terminal 3(C).

The communication terminal 3(A) has installed thereon, for example, a browser application for communicating with the audio recording management device 5 and a record management application for creating minutes of meetings and the like based on text information sent by the audio recording management device 5. Furthermore, the communication terminal 3(A) is a communication terminal capable of participating in a specific event such as a conversation and creating minutes between the communication terminal 3(B) and the communication terminal 3(A) by using a general-purpose tool (herein referred to as a "conversation tool") for remote work, video conferencing, instant messaging, group chat, and the like. In this way, the communication terminal 3(A) functions, for example, as a minutes-creation terminal.

Communication terminal 3(B) uses communication terminal 3(A) and the general-purpose tool described above to participate in a specific event together with a user (e.g., user A) using communication terminal 3(A) who is participating in the specific event. In this way, communication terminal 3(B) functions, for example, as an event participation terminal.

The communication terminal 3(A) and the communication terminal 3(B) can hear each other's voice by speaking at a specific event using the general-purpose tool as described above. Therefore, the communication terminal 3(A) as a minutes-creation terminal can acquire not only the voice of user A who uses the communication terminal 3(A), but also the voice of a user (e.g., user B) who uses the communication terminal 3(B) as an event participation terminal communicating with the communication terminal 3(A).

The communication terminal 3(C) is a terminal for viewing minutes created based on a specific event in which the communication terminals 3(A) and 3(B) participated. In this case, the communication terminal 3(C) is capable of viewing the minutes of the specific event via a browser even if a record management app is not installed. In this way, the communication terminal 3(C) functions as, for example, a minutes viewing terminal.

In this embodiment, unless otherwise specified, it will simply be referred to as "communication terminal 3". Note that the communication terminal 3 may be a communication terminal having a communication function, such as a commonly used PC (Personal Computer), a portable notebook PC, a mobile phone, a smartphone, a tablet terminal, or a wearable terminal (sunglasses type, wristwatch type, etc.). The communication terminal 3 may further be a communication device or communication terminal capable of operating software such as browser software.

<Audio Recording Management Device>
The voice recording management device 5 is realized by one or more information processing devices (computer systems) equipped with a general server OS or the like. The voice recording management device 5 has a function of a cloud server that executes a dedicated application program and manages voice recording information obtained based on voice information transmitted by the communication terminal 3 via the communication network 100. Note that the voice recording management device 5 may manage various voice recording information based on not only minutes of a meeting as an example of an event, but also voice memos for any individual or group activity, voice records of operators answering the phone, work records at a specific location such as a factory, etc., as the voice recording information.

The voice recording management device 5 further manages users who use the communication terminal 3 with creator identification information. In the communication system according to this embodiment, users can log in to the voice recording management device 5 using the creator identification information. Therefore, the creator identification information has a function that allows the voice recording management device 5 to uniquely identify the user. The creator identification information includes information that can uniquely identify the user, such as an email address, ID, or telephone number.

The audio recording management device 5 may be constructed by a single computer, or may be constructed by multiple computers to which each section (function or means) such as storage is divided and arbitrarily assigned. In addition, all or part of the functions of the audio recording management device 5 may be a server computer that exists in a cloud environment, or a server computer that exists in an on-premise environment.

<Speech recognition server>
The voice recognition server 7 is realized by one or more information processing devices (computer systems) equipped with a general server OS, etc. When the voice recognition server 7 receives voice information (data) transmitted by the voice recording management device 5, the voice recognition server 7 has a function of activating a voice recognition engine to convert the voice information (data) into text data, and returning (transmitting) the converted text data to the voice recording management device 5. That is, in the communication system according to this embodiment, the voice recognition server 7 has a cloud service function of converting voice information into text information.

Specifically, the voice recognition server may use a third-party service that enables voice recognition, and may be provided, for example, by a general-purpose voice recognition engine service.

●About terminology●
In this embodiment, a user refers to a person who falls under any of the following categories. For example, users include participants who speak at a specific event, machines equipped with AI capable of generating language spoken by humans, humanoid robots, etc. In this embodiment, the term "user" is used for the sake of convenience of explanation.

Furthermore, in this embodiment, an event refers to various occasions and activities. For example, events include meetings, conferences, lectures, speeches, competitions, etc.

Furthermore, in this embodiment, text refers to various information that is converted from audio information related to the voice spoken by the user into words, phrases, numbers, symbols, sentences, etc., expressed in visible characters based on the results of recognition using a specified dictionary, etc.

Furthermore, in this embodiment, an audio recording refers to a record consisting of text information (data), images, minutes including spoken voice, etc. obtained based on audio information (data) related to spoken voices by participants (users) at one or more events at a specific event such as a conference. And, audio recording information refers to information related to the audio recording described above.

[Hardware configuration]
Next, the hardware configuration of the device or terminal constituting the communication system according to the embodiment will be described with reference to Figures 2 and 3. Note that components may be added or deleted from the hardware configuration of the device or terminal shown in Figures 2 and 3 as necessary.

<Hardware configuration of communication terminal and voice recognition server>
Fig. 2 is a diagram showing an example of the hardware configuration of the communication terminal and the voice recognition server. As shown in Fig. 2, the communication terminal 3 is constructed by, for example, a computer. The communication terminal 3 includes a CPU 301, a ROM 302, a RAM 303, an EEPROM 304, a CMOS (Complementary Metal Oxide Semiconductor) sensor 305, an image sensor I/F (Interface) 306, a media I/F 309, a bus line 310, a network I/F 312, an antenna 312a of the network I/F 312, a microphone 315, a speaker 316, a sound input/output I/F 317, a display 318, an external device connection I/F 319, a short-range communication circuit 320, an antenna 320a of the short-range communication circuit 320, and a touch panel 321.

Of these, the CPU 301 controls the overall operation of the communication terminal 3. The ROM 302 stores programs used in the processing of the CPU 301. The RAM 303 is used as a work area for the CPU 301. The EEPROM 304 reads or writes various data such as applications under the control of the CPU 301. The CMOS sensor 305 is a type of built-in imaging means that captures an image of a subject under the control of the CPU 301 to obtain image data or video data. The imaging means may be an imaging means configured with a CCD (Charge Coupled Device) sensor or the like instead of a CMOS sensor. The imaging element I/F 306 is a circuit that controls the driving of the CMOS sensor 305. The media I/F 309 controls the reading or writing (storing) of data from or to a recording medium 308 such as a flash memory. The bus line 310 is an address bus, a data bus, or the like for electrically connecting each component such as the CPU 301.

The network I/F 312 is a communication interface for communicating various data (information) with other devices via the communication network 100. At this time, the network I/F 312 may communicate using the antenna 312a of the network I/F 312. The microphone 315 is a built-in circuit that converts sound into an electrical signal, and acquires voice or sound waves emitted from an external speaker or the like to acquire information using the electrical signal. The speaker 316 is a built-in circuit that converts electrical signals into physical vibrations to generate sounds such as music and voice. The sound input/output I/F 317 is a circuit that processes input and output of sound signals between the microphone 315 and the speaker 316 under the control of the CPU 301. The display 318 is a type of display means such as liquid crystal or organic EL (Electro Luminescence) that displays images of subjects, characters, various icons, etc. The external device connection I/F 319 is an interface for connecting various external devices. In this case, the external device is, for example, a USB (Universal Serial Bus) memory or the like. The short-range communication circuit 320 is a communication circuit for performing short-range wireless communication with a communication device or communication terminal equipped with a wireless communication interface such as NFC (Near Field Communication), Bluetooth (registered trademark, omitted below), millimeter wave wireless communication, Wi-Fi (registered trademark, omitted below), QR code (registered trademark, omitted below), visible light, environmental sound, or ultrasonic wave. The short-range communication circuit 320 also includes an antenna 320a for the short-range communication circuit 320. The touch panel 321 is a type of input means for operating the communication terminal 3 by the user pressing, clicking, tapping, or other operations on a specific button, icon, or the like arranged on the display 318.

The communication terminal 3 may be a communication device or a communication terminal capable of running a program such as browser software.

The voice recognition server 7 includes a CPU 701, a ROM 702, a RAM 703, an EEPROM 704, a CMOS (Complementary Metal Oxide Semiconductor) sensor 705, an image sensor I/F 706, a media I/F 709, a bus line 710, a network I/F 712, an antenna 712a of the network I/F 712, a microphone 715, a speaker 716, an audio input/output I/F 717, a display 718, an external device connection I/F 719, a short-range communication circuit 720, an antenna 720a of the short-range communication circuit 720, and a touch panel 721. These hardware resources are similar to the hardware resources of the communication terminal 3, including the CPU 301, ROM 302, RAM 303, EEPROM 304, CMOS (Complementary Metal Oxide Semiconductor) sensor 305, image sensor I/F 306, media I/F 309, bus line 310, network I/F 312, antenna 312a of network I/F 312, microphone 315, speaker 316, sound input/output I/F 317, display 318, external device connection I/F 319, short-range communication circuit 320, antenna 320a of short-range communication circuit 320, and touch panel 321, and therefore will not be described.

<Hardware configuration of the voice recording management device>
Fig. 3 is a diagram showing an example of the hardware configuration of the voice recording management device. As shown in Fig. 3, the voice recording management device 5 is constructed by, for example, a computer, and includes a CPU 501, a ROM 502, a RAM 503, an EEPROM 504, a HD 505, a HDD (Hard Disk Drive) controller 506, a display 507, a short-range communication I/F 508, a CMOS sensor 509, an image sensor I/F 510, a network I/F 511, a keyboard 512, a pointing device 513, a media I/F 515, an external device connection I/F 516, an audio input/output I/F 517, a microphone 518, a speaker 519, and a bus line 520.

Of these, the CPU 501 controls the operation of the entire voice recording management device 5. The ROM 502 stores programs used to drive the CPU 501. The RAM 503 is used as a work area for the CPU 501. The EEPROM 504 reads or writes various data such as apps under the control of the CPU 501. The HD 505 stores various data such as programs. The HDD controller 506 controls the reading or writing of various data from the HD 505 under the control of the CPU 501. The display 507 displays various information such as a cursor, menu, window, text, or image. The short-range communication I/F 508 is a communication circuit for performing data communication with a communication device or a communication terminal equipped with a wireless communication interface such as NFC (Near Field Communication), Bluetooth (registered trademark, omitted below), or Wi-Fi (registered trademark, omitted below), etc. The CMOS sensor 509 is a type of built-in imaging means that captures an image of a subject under the control of the CPU 501 to obtain image data or video data. Note that the imaging means may be an imaging means configured with a CCD (Charge Coupled Device) sensor or the like instead of a CMOS sensor. The imaging element I/F 510 is a circuit that controls the driving of the CMOS sensor 509.

The network I/F 511 is an interface for data communication using the communication network 100. The keyboard 512 is a type of input means having multiple keys for inputting characters, numbers, various instructions, etc. The pointing device 513 is a type of input means for selecting and executing various instructions, selecting a processing target, moving a cursor, etc. The media I/F 515 controls the reading or writing (storing) of data to a recording medium 514 such as a flash memory. The external device connection I/F 516 is an interface for connecting various external devices. In this case, the external device is, for example, a USB (Universal Serial Bus) memory, etc. The sound input/output I/F 517 is a circuit that processes the input and output of sound signals between the microphone 518 and the speaker 519 according to the control of the CPU 501. The microphone 518 is a built-in circuit that converts sound into an electrical signal, and obtains voice and sound waves emitted from an external speaker, etc., and obtains information using the electrical signal. The speaker 519 is a built-in circuit that converts electrical signals into physical vibrations to produce sounds such as music and voice. The bus line 520 is an address bus, data bus, etc., for electrically connecting each component such as the CPU 501.

The voice recording management device 5 may also notify (send) data (information) to the communication terminal 3 by push notification (transmission). In this case, the voice recording management device 5 can realize this by using, for example, FCM (Firebase Cloud Messaging), which is an example of a push notification server, to send a push notification. The voice recording management device 5 may be a commonly used PC (Personal Computer). The voice recording management device 5 may further be a communication device or communication terminal capable of running software such as browser software.

Furthermore, the above program may be recorded in a computer-readable recording medium as an installable or executable file, or may be distributed by downloading via a network. Examples of recording media include CD-Rs (Compact Disc Recordable), DVDs (Digital Versatile Disks), Blu-ray Discs, SD cards, and USB memories. The recording media may also be provided domestically or internationally as a program product. For example, the voice recording management device 5 realizes the voice recording management method according to the present invention by executing the program according to the present invention.

[Functional configuration of communication system]
Next, the functional configuration of this embodiment will be described with reference to Fig. 4 to Fig. 9. Fig. 4 is a diagram showing an example of the functional configuration of a communication system.

<Functional configuration of communication terminal>
As shown in Fig. 4, the communication terminal 3 has a transmission/reception unit 31, an operation reception unit 32, a sound/image acquisition unit 33, a display control unit 34, an audio playback unit 36, an application launch unit 38, and a memory readout unit 39. Each of these functional units is a function or means realized by operating any of the hardware resources shown in Fig. 2 by an instruction from the CPU 301 according to a program for the communication terminal 3 expanded from at least one of the ROM 302 and the EEPROM 304 to the RAM 303. The communication terminal 3 also has a storage unit 3000 constructed by at least one of the ROM 302 and the EEPROM 304 shown in Fig. 2. Furthermore, the storage unit 3000 stores a communication program (communication application) for communicating with the voice recording management device 5 via the communication network 100, and a browser application, a record management application, etc. for generating minutes, etc. based on voice information.

<<Functional configuration of communication terminal>>
Next, a detailed description will be given of each functional configuration of the communication terminal 3. The transmission/reception unit 31 of the communication terminal 3 shown in Fig. 4 is mainly realized by the processing of the CPU 301 on the network I/F 312 and the short-range communication circuit 320, and transmits and receives various data (or information) to and from the voice recording management device 5 via the communication network 100. In this embodiment, the transmission/reception unit 31 functions as an example of at least one of a transmitting means and a receiving means.

The operation reception unit 32 is realized mainly by the CPU 301 processing signals generated by various operations received by the touch panel 321. Note that the operation reception unit 32 may use input means such as a keyboard or a pointing device instead of the touch panel 321. In this embodiment, the operation reception unit 32 functions as an example of a reception means.

The sound and image acquisition unit 33 is mainly realized by the processing of the CPU 301 on the microphone 315, the sound input/output I/F 317, the CMOS sensor 313, and the image sensor I/F 314, and collects voice (sound) related to the speech of the user using the communication terminal 3 to acquire voice information (voice data) or sound information (sound data). The sound and image acquisition unit 33 further captures an image related to an image of the user's face, etc. to acquire image information (image data). The sound and image acquisition unit 33 also acquires screen data displayed on the display 318 at a predetermined time interval. Note that the sound information includes speech voice information indicating speech voice uttered by a human, and artificial voice information which is an artificial voice generated by a machine equipped with AI, a humanoid robot, etc. In this embodiment, the sound and image acquisition unit 33 functions as an example of an acquisition means.

The display control unit 34 is mainly realized by the processing of the CPU 301 on the display 318, and controls the display of various screens and information (data) in the communication terminal 3. The display control unit 34 also uses, for example, a browser to display a display screen created by HTML or the like on the display 318. The display control unit 34 also displays an edited screen related to the edited screen data transmitted by the voice recording management device 5 on the display 318. The display control unit 34 also displays a text hide operation unit operated to generate a text hide request near a specified text displayed on the display 318. The display control unit 34 also displays an image hide operation unit operated to generate an image hide request near a specified image displayed on the display 318. Here, the text hide request is a request sent to the voice recording management device 5 to hide the specified text. The text hide operation unit includes a "private" button (icon) 3542 and a "delete" button (icon) 3544, which will be described later. Furthermore, the image non-display request is a request sent to the audio recording management device 5 to hide a specified image. The image non-display operation unit includes a "private" button (icon) 3545 and a "delete" button (icon) 3546, which will be described later. Furthermore, the first non-displayed text data, second non-displayed text data, first non-displayed image data, and second non-displayed image data that are hidden by the text non-display request and image non-display request are each confidential data that includes performance information, sales information, profit information, and personal information of the specified business.

The display control unit 34 also performs the following processing when there is specific text data obtained based on audio data spanning the acquisition time when the screen data of the screen to be edited is acquired by the acquisition unit 52. Specifically, when an image non-display request is received for a specific image acquired at the acquisition time, the display control unit 34 displays on the display 318, in addition to the second non-display text data, a third non-display screen related to third non-display screen data obtained by non-display processing of the specific text data. The display control unit 34 further displays on the display 318 a screen in which specific text related to the specific text data is maintained in a non-display state when an image display request to display the specific image again is received after an image non-display request for the specific image is received.

When a text group including a plurality of texts displayed on the display 318 is selected, the display control unit 34 further displays a summary input operation unit adjacent to the text group for inputting a summary representing the text group. Here, the summary input operation unit includes a summary input dialog 3181 described below. The display control unit 34 further displays a text hide operation unit adjacent to the text group together with the summary input operation unit. Here, the text hide operation unit includes a "Private" button (icon) 3542 or a "Delete" button (icon) 3544 described below. In this embodiment, the display control unit 34 functions as an example of a display control means.

The audio playback unit 36 is mainly realized by the processing of the CPU 301 on the speaker 316 and the audio input/output I/F 317, and plays audio information (audio data) or sound information (sound data) for the user using the communication terminal 3. The audio playback unit 36 also plays edited audio related to edited audio data sent by the audio recording management device 5, in which editing such as making a specific text private, making it public, or deleting it has been performed. The audio playback unit 36 also plays audio in which audio data indicated by each text related to the second non-display text data and the specific text data has been muted. The audio playback unit 36 further plays audio in which a specific audio related to specific audio data indicated by specific text has been maintained in a muted state. In this embodiment, the audio playback unit 36 functions as an example of an audio playback means.

The application launch unit 38 is mainly realized by the processing of the CPU 301, and launches various applications between the application launch unit 38 and the voice recording management device 5 via the communication network 100. The application launch unit 38 also operates a recording management application and a browser application for editing and managing various text information managed by the voice recording management device 5 in a predetermined working area of the RAM 303. In this embodiment, the application launch unit 38 functions as an example of a launching means.

The memory/read unit 39 is realized mainly by the processing of the CPU 301 on at least one of the ROM 302 and the EEPROM 304 shown in FIG. 2, and stores various data (or information) in the memory unit 3000 and reads various data (or information) from the memory unit 3000. In this embodiment, the memory/read unit 39 functions as an example of a memory/read means.

<Functional configuration of the voice recording management device>
As shown in Fig. 4, the voice recording management device 5 has a transmission/reception unit 51, an acquisition unit 52, a calculation specification unit 53, a display control unit 54, a judgment unit 55, an authentication unit 56, a generation/processing unit 57, a setting registration unit 58, and a storage/reading unit 59. Each of these functional units is a function or means realized by operating any of the hardware resources shown in Fig. 3 by an instruction from the CPU 501 according to a program for the voice recording management device 5 expanded from at least one of the ROM 502 and the HD 505 to the RAM 503. The voice recording management device 5 also has a storage unit 5000 constructed by at least one of the ROM 502 and the HD 505 shown in Fig. 3. Furthermore, the storage unit 5000 stores a communication program (communication application) for communicating with the communication terminal 3 and the voice recognition server 7 via the communication network 100, and a browser application, a record management application, etc. executed between the communication terminal 3 and the voice recognition server 7.

●Login management table●
Fig. 5 is a conceptual diagram showing an example of a login management table. A login management DB 5001 configured by a login management table as shown in Fig. 5 is constructed in the memory unit 5000. In the login management table, terminal identification information, IP addresses of participating communication terminals, creator identification information (user ID), passwords, and user names are stored and managed in association with each session ID. Of these, the session ID is information for identifying a session established by communication between the voice recording management device 5 and one or more communication terminals 3, and is given, for example, SE0001, SE0002, etc.

The terminal identification information is information for identifying the communication terminal 3, and is given, for example, as TM0001, T0002, etc. The participating communication terminal IP address indicates the IP address of the communication terminal that participated in a specific event, and is unique information given in correspondence with the version such as IPv4, IPv6, etc. In this embodiment, it is given, for example, as information such as 1.2.1.3, 1.2.2.4, etc. The creator identification information is information for identifying the user, and in this embodiment, an email address such as "taroh.r@ricoh.ex.com" is given as the user ID of the user. The password is identification information associated with the creator identification information at the time of login when using the communication system 1, and is set by the user at the time of initial startup (login), etc. The password is information of multiple characters (digits) that randomly include any character string, numbers, and symbols. The user name represents the name of the user indicated by the creator identification information, and is given, for example, as "Rikotaro", "Hanako Magome", "Jiro Ebina", etc.

●Record bibliographic information management table●
Fig. 6 is a conceptual diagram showing an example of a record bibliographic information management table. A record bibliographic information management DB 5002 configured by the record bibliographic information management table shown in Fig. 6 is constructed in the storage unit 5000. In the record bibliographic information management table, the record identification information is treated as tabs, and the record name, start date and time, end date and time, audio data path, creator identification information (user ID), event URL, passcode, and each item of the association process are associated and stored and managed by each tab. The record identification information as a tab is given, for example, as "R5006", "R5007", etc.

Of these, the recording name is the name of the recorded content recorded at a specific event, and may be given a name such as "Healthcare Business Performance Reporting Meeting." The start date and time and end date and time are given as the date and time when the specific event started and ended, and are, for example, information such as "2021/03/31 11:00:00" and "2021/03/31 12:00:00." The audio data path indicates the location where the audio recording data (all data) recorded at a specific event is stored and managed, and is given as path information including a file name, such as ".../00005006/record.mp3." The audio data path may be URL information representing a dedicated audio server device located in the communication system 1.

The creator identification information (user ID) is the identification information of the creator who creates the record view, such as the minutes of an event such as a conference, and is given, for example, as "taro.r@ricoh.ex.com". The event URL indicates the location for editing the record view of an event such as a conference, and the user can access the "record view editing screen" described later by accessing this event URL. The event URL may be URL information representing a dedicated image server device arranged in the communication system 1. The passcode is code information for other users other than the creator who creates (edits) the record view such as the minutes to view the record view of the minutes. Specifically, the creator copies the event URL and passcode, and sends the event URL and passcode to other users using e-mail, chat, etc. The other users receive the event URL and passcode, and enter the passcode along with the event URL. This allows other users to view the record view created by the creator.

The matching process is an item that indicates whether matching process has been performed on image identification information that represents an image that corresponds to a specific text obtained based on audio data when generating a record view based on audio information (audio data) recorded at an event such as a conference. If the matching process is "unprocessed", it indicates that the above-mentioned matching process has not yet been performed, and if the matching process is "processed", it indicates that the above-mentioned matching process has been performed. Image identification information will be explained in detail in the text information management table below.

Of the items managed in the record bibliographic information management table in this embodiment, the end date and time and audio data path items are items that are edited or added by the user or editor when the event ends.

●Text information management table●
Fig. 7 is a conceptual diagram showing an example of a text information management table. A text information management DB 5003 configured by a text information management table as shown in Fig. 7 is constructed in the storage unit 5000. In the text information management table, the record identification information is treated as a tab, and the text identification information, start time, end time, text, publication flag, deletion flag, and image identification information separated by each tab are stored and managed in association with each other. The record identification information as a tab is given, for example, as "R5006", "R5007", etc.

Among these, the text identification information is information for identifying a specific speech content or speech sentence as a unit when it is treated as one text (or text information), and is given, for example, as "TX0005", "TX0006", ... "TX0009". The start time and end time are information for managing the time when a specific speech starts and ends in minutes and seconds. For example, if the start date and time when a specific event starts is "2021/03/31 11:00:00" managed in the record bibliographic information management DB 5002 (see FIG. 6), the start time is managed as the time elapsed from the start date and time. In other words, if the start time is "1 minute 53 seconds", the date and time when the speech managed by the text identification information "TX0005" started is "2021/03/31 11:01:53". The end time is managed as the time elapsed from the start date and time, like the start time. In other words, if the end time is "1 minute 58 seconds", this indicates that the date and time when the utterance managed by the text identification information "TX0005" ended was "2021/03/31 11:01:58". Therefore, in this case, the time required for the utterance managed by the text information "TX0005" is 58 seconds - 53 seconds = 5 seconds.

As described above, the text is the content of the speech spoken by the user between the start time and the end time replaced with text information. For example, the text is managed as content such as "Today's meeting will be recorded," or "Today's agenda is the performance of the healthcare business."

The public flag is a status flag that indicates whether or not a specific text managed in the text information management table is displayed (hereinafter also referred to as public) on the record viewing and editing screen. When a specific text is publicly displayed on the record viewing and editing screen, the public flag is managed as "True." On the other hand, when a specific text is hidden or deleted (hereinafter also referred to as private) on the record viewing and editing screen, the public flag is managed as "False." Note that the initial setting of the public flag may be, for example, "True," which means that the text is public (displayed).

The deletion flag is a status flag that indicates whether or not a specific text managed in the text information management table has been deleted from the record viewing/editing screen. If the specific text has been deleted from the record viewing/editing screen, the deletion flag is managed as "True." On the other hand, if the specific text has not been deleted from the record viewing/editing screen, the deletion flag is managed as "False." The initial setting of the deletion flag may be, for example, "False," which means that the text is not deleted.

The image identification information is identification information that identifies a specific capture image displayed on the record viewing and editing screen, and is given, for example, "IM0004", "IM0005", etc.

●Capture image management table●
Fig. 8A is a conceptual diagram showing an example of a capture image management table. A capture image management DB 5004 configured by the capture image management table shown in Fig. 8A is constructed in the storage unit 5000. In this embodiment, the display control unit 34 of the communication terminal 3 has a function of capturing a viewing and editing screen displayed and edited during execution of a predetermined event at a predetermined time interval (e.g., every 30 seconds). The capture image management table is a table for storing and managing information related to the screen capture function, and in the capture image table, record identification information is treated as a tab, and image identification information, acquisition time, public flag, deletion flag, and image data path, which are separated by each tab, are stored and managed in association with each other.

Of these, image identification information is information for identifying the captured image, and is given, for example, as "IM0001", ... "IM0003", etc. The acquisition time indicates the elapsed time of a specific event when the captured image was acquired, and is given, for example, as "1 minute 30 seconds", "2 minutes 0 seconds", etc.

The public flag is a status flag that indicates whether or not a specific captured image managed in the captured image management table is displayed (hereinafter also referred to as public) on the record viewing and editing screen. When a specific captured image is publicly displayed on the record viewing and editing screen, the public flag is managed as "True." On the other hand, when a specific captured image is hidden or deleted (hereinafter also referred to as private) on the record viewing and editing screen, the public flag is managed as "False." Note that the initial setting of the public flag may be, for example, "True," which means public (displayed).

The deletion flag is a status flag that indicates whether or not a specific captured image managed in the captured image management table has been deleted from the record viewing/editing screen. If a specific captured image has been deleted from the record viewing/editing screen, the deletion flag is managed as "True." On the other hand, if a specific captured image has not been deleted from the record viewing/editing screen, the deletion flag is managed as "False." Note that the initial setting of the deletion flag may be, for example, "False," which means that the image has not been deleted.

The image data path indicates the location where the captured image data captured at a specific event is stored and managed, and is given as path information such as ".../00005006/0003.jpg". The image data path may be URL information indicating a dedicated image server device located in the communication system 1.

●Capture image acquisition interval table●
8B is a conceptual diagram showing an example of a capture image acquisition interval table. In the storage unit 5000, a capture image acquisition interval DB 5005 configured by the capture image acquisition interval table shown in FIG. 8B is constructed. In the capture image acquisition interval table, the capture image acquisition interval for acquiring a capture image is set and managed as, for example, "30 seconds,""60 seconds (1 minute)," etc. Note that the predetermined time interval for acquiring a capture image is not limited to 30 seconds, and may be set arbitrarily, such as every 10 seconds or every minute. Furthermore, the capture image acquisition management table may be managed by the communication terminal 3 side that actually acquires a capture image of its own screen, instead of being managed by the voice recording management device 5.

●Private audio management table●
Fig. 9 is a conceptual diagram showing an example of a private voice management table. A private voice management DB 5006 configured by the private voice management table shown in Fig. 9 is constructed in the storage unit 5000. In the private voice management table, the record identification information is treated as a tab, and start times and end times separated by each tab are stored and managed in association with each other. The start times and end times are managed for each acquired voice recording.

<<Functional configuration of the voice recording management device>>
Next, each functional configuration of the voice recording management device 5 will be described in detail. The transmission/reception unit 51 of the voice recording management device 5 shown in FIG. 4 is mainly realized by the processing of the CPU 501 for the short-range communication I/F 508 and the network I/F 511, and transmits and receives various data (or information) to and from the communication terminal 3 via the communication network 100. The transmission/reception unit 51 receives voice information related to speech voices uttered by one or more users who use one or more communication terminals. In addition, the transmission/reception unit 51 transmits predetermined text data representing a predetermined text obtained based on the acquired voice data, predetermined image data representing a predetermined image that is included in a screen related to the acquired screen data and is associated with the predetermined text, and predetermined voice data indicated by the predetermined text to one or more communication terminals including the communication terminal 3 (A) (an example of a first communication terminal).

The transmitter/receiver 51 also transmits edited screen data including edited text data obtained by editing the specified text data and edited image data obtained by editing the specified image data, and edited audio data obtained by editing the specified audio data, to the communication terminal 3(B) (an example of a second communication terminal) in response to an editing request for a specified text or a specified image, which is an editing request transmitted by the communication terminal 3(A) (a first communication terminal).

The transmitter/receiver unit 51 also receives, as an editing request, either a text hide request for hiding specified text or an image hide request for hiding a specified image associated with the specified text, sent by the communication terminal 3 (A).

In addition, when the editing request sent by the communication terminal 3 is a text non-display request for hiding text, the transmission/reception unit 51 transmits, as an editing process, to the communication terminal 3 (B) first non-display text data that has been subjected to non-display processing of the specified text data.

In addition, the transmission/reception unit 51 transmits to the communication terminal 3 (B) first non-display screen data including first non-display image data obtained by non-display processing of specific image data, and first muted audio data obtained by muting specific audio data as an editing process.

When the editing request transmitted by the communication terminal 3 is a request to not display an image, the transmission/reception unit 51 transmits, as an editing process, to the communication terminal 3 (B) second hidden screen data including second hidden image data obtained by non-display processing of specific image data and second hidden text data obtained by non-display processing of one or more pieces of text associated with the specific image. The transmission/reception unit 51 also transmits, as an editing process, second muted audio data obtained by muting one or more pieces of specific audio data associated with the one or more pieces of text to the communication terminal 3 (B). In this embodiment, the transmission/reception unit 51 functions as an example of at least one of a transmitting means and a receiving means.

The acquisition unit 52 is mainly realized by the processing of the CPU 501, and performs the acquisition of text data based on text information transmitted by the voice recognition server 7, the acquisition of capture information on the viewing and editing screen, etc. The acquisition unit 52 also acquires voice data representing voice information transmitted by a communication terminal 3(A) (first communication terminal) used by a creator (editor) who edits the voice recording, among one or more communication terminals 3, and screen data representing a screen displayed on the communication terminal 3(A). In this embodiment, the acquisition unit 52 functions as an example of an acquisition means.

The calculation/identification unit 53 is mainly realized by the processing of the CPU 501, and calculates the start date and time when the user starts speaking in a specified event. This calculation may use clock information generated using a clock signal from the CPU 501. In this embodiment, the calculation/identification unit 53 functions as an example of a calculation means.

The display control unit 54 is mainly realized by the processing of the CPU 501 on the display 507, and controls the display of various screens and information (data) in the voice recording management device 5. The display control unit 54 can also use a browser to display a display screen created using HTML or the like on the display 318 of the communication terminal 3 via the communication network 100. In this embodiment, the display control unit 54 functions as an example of a display control means.

The judgment unit 55 is mainly realized by the processing of the CPU 501, and performs various judgments in the voice recording management device 5. The judgment unit 55 also judges whether the "certainty level" (described later) transmitted by the voice recognition server exceeds a predetermined threshold. In this embodiment, the judgment unit 55 functions as an example of a judgment means for judging whether a given predetermined condition is satisfied.

The authentication unit 56 is mainly realized by the processing of the CPU 501, and performs authentication processing of the communication terminal 3, for example, based on an authentication request made from the communication terminal 3. In this embodiment, the authentication unit 56 functions as an example of an authentication means.

The generation/processing unit 57 is mainly realized by the processing of the CPU 501, and generates voice recording data based on the text information transmitted by the voice recognition server. The generation/processing unit 57 also generates screen data for the record viewing/editing screen displayed on the communication terminal 3. The generation/processing unit 57 also generates first hidden screen data and first muted voice data in response to a text non-display request transmitted by the communication terminal 3 (A), and generates second hidden screen data and second muted voice data in response to an image non-display request. The generation/processing unit 57 further performs a process of deleting the specified text data and the specified image data when the text non-display request and the image non-display request are requests to delete the specified text data and the specified image data among requests to delete or make the specified text data and the specified image data private. The generation/processing unit 57 further performs a process of making the specified text data and the specified image data private without deleting them, on the first hidden screen represented by the first hidden screen data or the second hidden screen represented by the second hidden screen data. In this embodiment, the generation/processing unit 57 functions as an example of a generating means. Additionally, the generation/processing unit 57 functions as an example of a processing means.

The setting registration unit 58 is mainly realized by the processing of the CPU 501, and for example, registers the authentication information of the user who uses the communication terminal 3, which is made to the voice recording management device 5, in the storage unit 5000. The setting registration unit 58 further registers correction text, bookmarks, etc., and deletes bookmarks. In this embodiment, the setting registration unit 58 functions as an example of a setting means. Also, in this embodiment, the setting registration unit 58 functions as an example of a registration means.

The memory/read unit 59 is mainly realized by the processing of the CPU 501 on at least one of the ROM 502, the EEPROM 504, and the HD 505, and stores various data (or information) in the memory unit 5000 and reads various data (or information) from the memory unit 5000. In this embodiment, the memory/read unit 59 functions as an example of a memory/read means.

<Functional configuration of the voice recognition server>
As shown in Fig. 4, the voice recognition server 7 has a transmission/reception unit 71, a voice recognition unit 76, and a storage/readout unit 79. Each of these functional units is a function or means realized by any of the hardware resources shown in Fig. 2 operating in response to an instruction from the CPU 701 in accordance with a program for the voice recognition server 7 expanded from at least one of the ROM 702 and the EEPROM 704 to the RAM 703. The voice recognition server 7 also has a storage unit 7000 constructed from at least one of the ROM 702 and the EEPROM 704 shown in Fig. 2. Furthermore, the storage unit 7000 stores a communication program (communication application) for communicating with the voice recording management device 5 via the communication network 100, etc.

<<Functional configuration of the speech recognition server>>
Next, a detailed description will be given of each functional configuration of the voice recognition server 7. The transmission/reception unit 71 of the voice recognition server 7 shown in Fig. 4 is mainly realized by the processing of the CPU 701 on the network I/F 712 and the short-range communication circuit 720, and transmits and receives various data (or information) to and from the voice recording management device 5 via the communication network 100. In this embodiment, the transmission/reception unit 71 functions as an example of at least one of the transmitting means and the receiving means.

The voice recognition unit 76 is mainly realized by the processing of the CPU 701 on the microphone 715 and the sound input/output I/F 717, and recognizes the voice data or sound data (voice information) transmitted by the voice recording management device 5 and converts it into text data (text information). In this embodiment, the voice recognition unit 76 functions as an example of a voice recognition means.

The memory readout unit 79 is mainly realized by the processing of the CPU 701 on at least one of the ROM 702 and the EEPROM 704, and stores various data (or information) in the memory unit 7000 and reads various data (or information) from the memory unit 7000. In this embodiment, the memory readout unit 79 functions as an example of a memory readout means.

The communication system according to this embodiment includes a voice recognition server 9 in addition to the voice recognition server 7 described above, but the functional configurations of the voice recognition server 9 are similar to those of the voice recognition server 7, so a description of them will be omitted.

[Processing or Operation of the Embodiment]
Next, each process or operation in the voice recording management system according to the first embodiment will be described with reference to Fig. 10 to Fig. 39. Fig. 10 is a sequence diagram showing an example of application startup, authentication processing, and session establishment processing.

<<App launch and authentication process>>
First, the user of the communication terminal 3 performs an operation to start the record management application and the browser application running on the communication terminal 3. As a result, the operation acceptance unit 32 of the communication terminal 3 accepts the operation to start the communication application and the browser application by the user (step S11). In this embodiment, it is sufficient that the communication terminal 3 (A), which is an example of the minutes creation terminal described above, starts the record management application, and the communication terminal 3 (B), which participates in a predetermined event together with the communication terminal 3 (A), does not need to start the record management application. If the communication terminal 3 (B) starts the above-mentioned conversation tool instead of starting the record management application, voice communication with the communication terminal 3 (A) becomes possible, and as a result, each other's voice can be recognized. Furthermore, in this embodiment, it is assumed that the communication application used between the communication terminal 3 and other devices is in a state in which communication with other devices is possible by a predetermined protocol or the like.

Next, the application launch unit 38 launches a communication application and a browser application for communicating with the voice recording management device 5 that have been pre-installed in the storage unit 3000 (step S12). After that, the display control unit 34 displays an authentication screen (such as a sign-in screen) on the display 318 and waits for the user to perform an authentication operation (step S13).

Next, the user performs authentication processing (signs in) to the voice recording management device 5. As a result, the operation reception unit 32 accepts the authentication information input by the user (step S21).

Then, the transmission/reception unit 31 transmits a request for authentication processing to the voice recording management device 5 based on the received authentication information (step S22). As a result, the transmission/reception unit 41 of the voice recording management device 5 receives the request for authentication processing transmitted by the communication terminal 3. At this time, the request for authentication processing includes a session ID for establishing a session with the voice recording management device 5, terminal identification information of the communication terminal 3, creator identification information for identifying the user, and a password.

Next, the authentication unit 46 of the voice recording management device 5 performs login authentication processing by comparing the received session ID, terminal identification information, creator identification information, and password with the session ID, terminal identification information, and password corresponding to the creator identification information read from the login management DB 5001 (see FIG. 5) by the memory readout unit 59 (step S23). Here, it is assumed that the login authentication processing by the user to the voice recording management device 5 has been successful. In the processing of step S23, the setting registration unit 58 may register the IP address of the communication terminal 3 that has performed the login authentication processing in the login management DB 5001 (see FIG. 5).

Next, when the login authentication process is successful in step S23 and a communication session with the communication terminal 3 is established, the transmission/reception unit 51 transmits a response to the authentication process and a response to the participation process to the communication terminal 3 (step S24). As a result, the transmission/reception unit 31 of the communication terminal 3 receives the response to the authentication process and the response to the participation process transmitted by the voice recording management device 5. At this time, the response to the authentication process and the response to the participation process include the session ID and a participation process result that permits the participation process to the communication session with the voice recording management device 5.

Then, in order to simplify the login process and communication session establishment from the next time onwards, the application launch unit 38 of the communication terminal 3 cooperates with the memory readout unit 39 to store and register the creator identification information, password, and user name as a set in a predetermined area of the memory unit 3000 (step S25). Note that the process of step S25 may be omitted.

The above-mentioned example of the processing sequence has been explained on the assumption that a browser application is installed in the communication terminal 3, but a web service using a browser may be provided, such as in the communication terminal 3 (C) used as an example of a minutes viewing terminal. In the case of a web service, the voice recording management device 5 has a web server function, and may transmit screen data such as a viewing and editing screen (data for displaying the entire screen or part of the screen) to the communication terminal 3 for display.

In the voice recording management system according to this embodiment, for example, when the processing of steps S22 and S24 described above is executed, other devices may be present between the communication terminal 3 and the voice recording management device 5. In other words, each piece of information (data) transmitted and received between the communication terminal 3 and the voice recording management device 5 may be configured to be transmitted and received once via another device. The above-described configuration is applicable even if other processing steps exist between the communication terminal 3 and the voice recording management device 5.

●Screen display example●
11 is an example of a screen display when an application is started in a communication terminal. An application start-up screen 3101 is displayed on the display 318 of the communication terminal 3 by the display control unit 34. The application start-up screen 3101 displays, for example, a mark representing the application (a picture of a microphone) and version information of the application.

<<Recording start process>>
Next, the recording start process will be described. Fig. 12 is a sequence diagram showing an example of the recording start process. As shown in Fig. 12, the display control unit 34 of the communication terminal 3 displays the application start screen 3101 as shown in Fig. 11, and then displays a recording start instruction screen on the display 318 (step S31).

●Screen display example●
13 is an example of a screen display of a recording start instruction in a communication terminal. When the process of step S32 described above is executed, a recording start instruction screen 3111 is displayed by the display control unit 34 on the display 318 of the communication terminal 3. The recording start instruction screen 3111 includes input fields for, for example, "Summary (topic),""Participants,""Meetingnotes,""Recordingmicrophone," and "Screen to record."

In the "Summary (agenda)" field, for example, the agenda of the event such as a meeting to be held is input. In the "Participants" field, for example, the participants of the event are input. In the "Meeting notes" field, for example, the main agenda of the event is input. The main agenda is, for example, an overview, decisions, action items, etc. The items input into each of the input fields "Summary (agenda)", "Participants", and "Meeting notes" may be input in advance by participants of the event such as a meeting, or may be added and edited after the event ends. In the input fields "Recording microphone" and "Recording screen", the microphone arrangement and the number of the screen to be recorded are displayed. These microphone arrangements and the number of the screen to be recorded may be set in advance by the user in the voice recording management device 5, or may be set by the voice recording management device 5 based on arbitrary conditions, timing, etc. In addition, the recording start instruction screen 3111 may be provided with a "bookmark" field in which the user can input particularly important contents of the event such as a meeting.

Furthermore, a start recording button 3511 is displayed on the start recording instruction screen 3111 by the display control unit 34. A user of the communication terminal 3 can start recording the content of speech spoken at an event such as a conference by operating (pressing or tapping, etc.) the start recording button 3511.

Returning to FIG. 12, the operation reception unit 32 receives a recording start instruction from a user (step S32). The recording start instruction is received by receiving an operation from the user on the above-mentioned recording start button 3501.

Next, the transmission/reception unit 31 of the communication terminal 3 transmits a recording start request to the voice recording management device 5 (step S33). As a result, the transmission/reception unit 51 of the voice recording management device 5 receives the recording start request transmitted by the communication terminal 3.

Next, the transmission/reception unit 51 transmits a voice data transmission start notification to the voice recognition server 7 (step S34). As a result, the transmission/reception unit 71 of the voice recognition server 7 receives the voice data transmission start notification transmitted by the voice recording management device 5.

Next, the transmission/reception unit 71 of the voice recognition server 7 transmits a permission response to start transmission to the voice recording management device 5 (step S35). As a result, the transmission/reception unit 51 of the voice recording management device 5 receives the permission response to start transmission transmitted by the voice recognition server 7.

Next, the transmission/reception unit 51 of the voice recording management device 5 transmits a request to display the recording screen to the communication terminal 3 (step S36). As a result, the transmission/reception unit 31 of the communication terminal 3 receives the request to display the recording screen transmitted by the voice recording management device 5.

Next, the display control unit 34 of the communication terminal 3 displays a recording screen on the display 318, and the operation reception unit 32 receives the operation of the "Stop Recording button" operated by the user (step S37).

<<Registration process for bibliographic information>>
14 is a sequence diagram showing an example of a process for registering bibliographic information. The transceiver 51 of the voice recording management device 5 further transmits a voice data transmission start request to the communication terminal 3 (step S41). As a result, the transceiver 31 of the communication terminal 3 receives the voice data transmission start request transmitted by the voice recording management device 5.

Next, the transmission/reception unit 31 of the communication terminal 3 transmits the voice data and the record bibliographic information to the voice recording management device 5 (step S42). As a result, the transmission/reception unit 51 of the voice recording management device 5 receives the voice data and the record bibliographic information transmitted by the communication terminal 3. The record bibliographic information transmitted and received at this time includes information corresponding to each item of the record name, start date and time, creator identification information (user ID), event URL, passcode, and association process managed in the record bibliographic information management DB 5002 (see FIG. 6).

Next, the transmission/reception unit 51 of the voice recording management device 5 transmits a voice recognition request to the API (Application Programming Interface) of the voice recognition server 7 (step S43). As a result, the transmission/reception unit 71 of the voice recognition server 7 receives the voice recognition request transmitted by the voice recording management device 5. At this time, the voice recognition request includes voice data to be converted into text by the voice recognition server. By performing the processes of steps S42 and S43, the transmission/reception unit 51 of the voice recording management device 5 continuously transmits the voice data transmitted by the communication terminal 3 (voice data by voice streaming) to the voice recognition server 7. In this case, even if the user's speech is not detected, the transmission/reception unit 51 may continue to transmit the voice streaming to the voice recognition server 7 continuously. However, the voice recording management device 5 may transmit the voice data individually based on the specifications of the API of the voice recognition server 7 and request voice recognition.

Next, the setting registration unit 58 of the voice recording management device 5 registers the record bibliographic information received in step S42 in the record bibliographic information management DB 5002 (see FIG. 6) (step S44). The record bibliographic information registered at this time includes the record name "Healthcare Business Performance Report Meeting", "Start Date and Time", "Creator Identification Information (User ID)", "Event URL", passcode, and association process, as well as new information indicating the "End Date and Time" and "Voice Data Path".

<<Speech recognition processing>>
15 is a sequence diagram showing an example of the voice recognition process. First, the sound/image acquisition unit 33 of the communication terminal 3(A), which is an example of a minutes creation terminal among the one or more communication terminals 3, collects speech or sounds uttered by a user using the communication terminal 3(A) via the microphone 315 and acquires voice information (including voice data or sound data; hereinafter, simply referred to as "voice information") (step S51).

Then, the transmission/reception unit 31 transmits the acquired voice information to the voice recording management device 5 (step S52). As a result, the transmission/reception unit 51 of the voice recording management device 5 receives the voice information related to the voice spoken by the user of the communication terminal 3(A) transmitted by the transmission/reception unit 31 of the communication terminal 3(A). In the following explanation, this will simply be referred to as the communication terminal 3.

Next, the transmission/reception unit 51 of the voice recording management device 5 transmits a voice recognition request to the voice recognition server 7 (step S53). As a result, the transmission/reception unit 71 of the voice recognition server 7 receives the voice recognition request transmitted by the voice recording management device 5. At this time, the voice recognition request includes the voice information (voice data, sound data) transmitted by the communication terminal 3. In other words, the voice recording management device 5 also serves as an intermediary device between the communication terminal 3 and the voice recognition server 7.

Next, the voice recognition unit 76 of the voice recognition server 7 performs voice recognition processing on the received voice information and converts the voice information into text information (text data) (step S54). For example, the voice recognition unit 76 recognizes the received voice data as sound data, and then converts it into predetermined text data corresponding to the recognized sound data. The voice recognition unit 76 then converts the converted predetermined text data into optimal text data using one or more voice recognition engines (dictionaries) provided in the voice recognition server 7. Note that the above-mentioned method of converting voice data into text data is not limited to this. Therefore, the communication system 1 may be configured as a system that obtains the desired text data using a generally known voice recognition engine.

Then, the transmission/reception unit 71 transmits the voice recognition result to the voice recording management device 5 (step S55). As a result, the transmission/reception unit 51 of the voice recording management device 5 receives the voice recognition result transmitted by the voice recognition server 7. At this time, the voice recognition result includes the converted text data, the start time, and the end time. Specifically, the voice recognition server 7 transmits, for example, the content "This development will use the gyro method" managed in the text information management DB 5005A (see FIG. 9A) and the start time (1 minute 59 seconds) when this content was spoken to the voice recording management device 5. Note that the start time is managed as the time elapsed from the start date and time, as explained in the text information management table. The same concept is applied to the end time.

Next, the setting registration unit 58 registers the received text information for one record in the text information management DB 5003 (see FIG. 7) (step S56). In this case, the text information for one record is information for "text identification information," "start time," "end time," "text," public flag, "end flag," and "image identification information."

Then, the acquisition unit 52 acquires the corresponding text data by searching the text information management DB 5005A (see FIG. 9A), for example, using the received start time as a search key (step S57).

Then, the calculation specification unit 53 calculates the start date and time (step S58). The calculation of the start date and time is performed based on the following formula. In other words, the start date and time is the sum of the start date and time managed in the record bibliographic information management DB 5002 (see FIG. 6) and the start date and time managed in the text information management DB 5005A (see FIG. 9A). Specifically, 2021/03/31 11:00:00 + 00:01:53 = 2021/03/31 11:01:53 is the start date and time calculated in step S58.

Then, the transmission/reception unit 51 transmits a recording screen update request to the communication terminal 3 (step S59). As a result, the transmission/reception unit 31 of the communication terminal 3 receives the recording screen update request transmitted by the voice recording management device 5. At this time, the recording screen update request includes the text information (text data) recognized by the voice recognition server 7 and the start date and time information indicating the start date and time calculated in step S58.

Next, the display control unit 34 of the communication terminal 3 displays the voice recording indicated by the corrected text information transmitted by the voice recording management device 5 on the display 318 (step S60). Specifically, the display control unit 34 displays the voice recording indicated by the first text information included in the recording screen as shown in FIG. 18.

In addition, in the communication system according to this embodiment, the generation/processing unit 57 may be provided in the voice recording management device 5, or may be provided in another device that can communicate between the voice recording management device 5 and the communication terminal 3 via the communication network 100.

In the voice recording management system according to this embodiment, for example, when the processes of steps S53 and S55 described above are executed, other devices may be present between the voice recording management device 5 and the voice recognition server 7. In other words, each piece of information (data) transmitted and received between the voice recording management device 5 and the voice recognition server 7 may be configured to be transmitted and received once via another device. The above-mentioned configuration is applicable even if other processing steps are present between the voice recording management device 5 and the voice recognition server 7.

●Screen display example●
FIG. 16 is an example of a screen display during recording in the communication terminal. By executing the process of step S60 described above, a recording screen 3121 is displayed by the display control unit 34 on the display 318 of the communication terminal 3. In addition to the contents of the recording start instruction screen 3111 shown in FIG. 13, the recording screen 3121 displays at least one text display field. In this text display field, for each user who has spoken, a face photo or image of the user, the speech date and time, and the speech content are displayed in chronological order as one unit. Here, in the text display field, for example, text information such as "Today's meeting will be recorded,""Today's agenda is the performance of the healthcare business," and "Please look at the shared screen first" transmitted by the voice recognition server 7 is displayed. Note that the contents displayed in the text display field are changed after the voice recognition engine change screen described later, but the voice recognition may be changed in the recording screen 3121 and recognized again.

The recording screen 3121 further displays "screen 3" as a captured image captured by the display control unit 34 on the communication terminal 3. The content of this "screen 3" may be a thumbnail image, an image icon, or the like representing "screen 3."

The recording screen 3121 further displays a "pause" button (icon) 3521 for instructing the user to pause recording by the display control unit 34, and an "end recording" button (icon) 3522 for instructing the user to end recording. A user can pause or end the recording of speech content spoken at an event such as a conference by operating (pressing or tapping, etc.) the "pause" button (icon) 3521 or the "end recording" button (icon) 3522. Then, when the "end recording" button 3522 is operated, the communication terminal 3 starts transmitting audio data to the audio recording management device 5.

<<Screen capture processing>>
Next, the screen capture process will be described. Fig. 17 is a sequence diagram showing an example of the screen capture process. As shown in Fig. 17, the memory readout unit 59 of the voice recording management device 5 reads out information on the capture image acquisition interval (e.g., 30 seconds) managed in the capture image acquisition interval DB 5005 (see Fig. 8B) (step S71).

Then, the transmission/reception unit 51 transmits the read information on the capture image acquisition interval to the communication terminal 3 (step S72). As a result, the transmission/reception unit 31 of the communication terminal 3 receives the information on the capture image acquisition interval transmitted by the voice recording management device 5. At this time, the information on the capture image acquisition interval is time information of 30 seconds.

Next, the sound/image acquisition unit 33 of the communication terminal 3 captures the screen displayed on the display 318 of the communication terminal 3 itself every 30 seconds to acquire a captured image (step S73). The screen capture process may be, for example, a method equivalent to the commonly known screen capture process using a print screen key. In this case, the voice recording management device 5 realizes this by automatically executing the screen capture process at the above-mentioned predetermined time intervals, without the user operating the print screen key.

Then, the transmission/reception unit 31 transmits capture image data indicating the acquired capture image to the voice recording management device 5 (step S74). As a result, the transmission/reception unit 51 of the voice recording management device 5 receives the capture image data transmitted by the communication terminal 3. At this time, the information transmitted by the communication terminal 3 includes the image identification information and the capture image data corresponding to the image identification information.

Next, the setting registration unit 58 of the audio recording management device 5 registers information including the captured image data as the captured image information corresponding to the image identification information, the acquisition time, and the image data path indicating the storage destination of the captured image in the captured image management DB 5004 (see FIG. 8A) (step S75).

Next, the transmission/reception unit 51 transmits a recording screen update request to the communication terminal 3 (step S76). As a result, the transmission/reception unit 31 of the communication terminal 3 receives the recording screen update request transmitted by the voice recording management device 5.

Next, the display control unit 34 of the communication terminal 3 updates the recording screen (step S77) and displays the updated screen on the display 318 (step S78). The recording screen that is updated at this time will be described in detail later with reference to FIG. 24.

In this embodiment, the audio recording management system 2 repeatedly executes the above-mentioned processes from steps S71 to S78 until the specified event ends, that is, the audio recording management system 2 repeatedly executes the above-mentioned processes from steps S71 to S78 until, for example, audio recording of a conference ends.

<<End of recording process>>
Next, the recording end process will be described. Fig. 18 is a sequence diagram showing an example of the recording end process. First, the operation reception unit 32 of the communication terminal 3 receives a recording end instruction by a user's operation on the "recording end" button (icon) 3522 (step S81). This recording end instruction is an instruction to end the recording of the contents of speech spoken at an event such as a conference, and is issued, for example, by the user operating (pressing or tapping, etc.) the "recording end" button (icon) 3522 displayed on the recording screen 3121 shown in Fig. 16.

Next, the transmission/reception unit 31 transmits a request to end recording to the voice recording management device 5 (step S82). As a result, the transmission/reception unit 51 of the voice recording management device 5 receives the request to end recording transmitted by the communication terminal 3.

Next, the transmission/reception unit 51 of the voice recording management device 5 transmits a voice data transmission end notification to the voice recognition server 7 (step S83). As a result, the transmission/reception unit 71 of the voice recognition server 7 receives the voice data transmission end notification transmitted by the voice recording management device 5.

Then, the transmission/reception unit 71 of the voice recognition server 7 transmits a receipt of the voice data transmission end notification to the voice recording management device 5 (step S84). As a result, the transmission/reception unit 51 of the voice recording management device 5 receives the receipt of the voice data transmission end notification transmitted by the voice recognition server 7.

Next, the transmission/reception unit 51 of the voice recording management device 5 transmits a recording end screen display request to the communication terminal 3 (step S85). As a result, the transmission/reception unit 31 of the communication terminal 3 receives the recording end screen display request transmitted by the voice recording management device 5.

Next, the display control unit 34 of the communication terminal 3 displays a recording end screen as shown in FIG. 19 on the display 318 (step S86). Note that the process of step S86 may be omitted.

Meanwhile, the generation/processing unit 57 of the audio recording management device 5 that sent the recording end screen display request in step S85 generates audio recording data for the specified event for which the recording end operation was performed (step S87). Specifically, the generation/processing unit 57 generates an audio data path (such as ".../00005006/record.mp3") in which the audio recording data for the specified event for which the recording end operation was performed is stored and managed.

Then, the setting registration unit 58 registers the record bibliographic information including the audio data path generated in the processing of step S87 in the record bibliographic information management DB 5002 (see FIG. 6), which is the storage destination of the audio data path (step S88). At this time, the record bibliographic information registered by the setting registration unit 58 includes the end date and time and the audio data path (".../00005006/record.mp3").

Meanwhile, the operation acceptance unit 32 of the communication terminal 3 accepts the selection of the shared minutes operated by the user while the record selection screen described below is displayed (step S89). Specifically, the operation acceptance unit 32 accepts the selection of the "Share minutes" button, described below, which is associated with a specific event title operated by the user.

Then, the transmitting/receiving unit 31 of the communication terminal 3 transmits a URL request to the audio recording management device 5 (step S90). As a result, the transmitting/receiving unit 51 of the audio recording management device 5 receives the URL request transmitted by the communication terminal 3. At this time, the URL request includes the recording identification information associated with the "Share minutes" button that is arranged in combination with the specified event title selected in step S89.

Next, the memory readout unit 59 of the voice recording management device 5 searches the record bibliographic information management DB 5002 (see FIG. 6) using the record identification information received in the processing of step S90 as a search key to search (read) the event URL of the corresponding event (step S91).

Next, the transmitting/receiving unit 51 transmits the dialog screen information to the communication terminal 3 (step S92). As a result, the transmitting/receiving unit 31 of the communication terminal 3 receives the dialog screen information transmitted by the voice recording management device 5. At this time, the dialog screen information includes the event URL and the passcode.

Next, the display control unit 34 of the communication terminal 3 displays a shared information input dialog, which will be described later, on the display 318 (step S93). This allows, for example, a creator who creates a record viewing and editing screen to copy the event URL and passcode displayed in the shared information input dialog and provide the event URL and passcode information to other users.

Note that the processes of steps S87 and S88 and steps S86 and S89 are performed asynchronously, so either one can be performed first.

●Screen display example●
19 is an example of a screen display at the end of recording in the communication terminal. When the process of step S86 described above is executed, a recording end screen 3131 is displayed by the display control unit 34 on the display 318 of the communication terminal 3. On the recording end screen 3131, for example, a save end mark indicating the end of saving, a "conference record" button 3531 for recording a new conference, and a "log check" button 3532 for viewing the log are displayed. By operating the "conference record" button 3531, the user can transition to a screen for recording a new conference. In addition, by operating the "log check" button 3532, the user can check various information (data) including the history of the contents spoken in a specific event, various buttons operated by the user, and a processing log.

●Screen display example●
FIG. 20 is an example of a screen display when selecting a record in a communication terminal. By executing the process of step S89 described above, a record selection screen 3141 is displayed by the display control unit 34 on the display 318 of the communication terminal 3. The record selection screen 3141 displays, for example, a date indicating the recorded contents (minutes, etc.) of an event such as a meeting, an event title, and a minutes share button (icon) 3535 in a selectable display form as a set. This allows a user to operate and select the minutes share button (icon) 3535 associated with a specific event represented by an arbitrary date and event title. In this embodiment, a case is shown in which the minutes share button (icon) 3535 corresponding to the event "Healthcare Business Performance Report Meeting" given date information of "2021/3/31 11:01:28-12:00:00" is selected by the user. Specifically, when the user of communication terminal 3(A) moves mouse pointer (cursor) 3701 over a specific event title on record selection screen 3141 by mouse over operation, minutes share button (icon) 3535 corresponding to the event title overlaid by mouse pointer (cursor) 3701 is displayed. Then, the user of communication terminal 3(A) can access a dialogue including a specific URL and a passcode by operating minutes share button (icon) 3535. The user of communication terminal 3(A) can access a voice recognition engine change screen, which will be described later, by inputting a specific URL and passcode into this dialogue.

●Screen display example●
Fig. 21 is a screen display example of a shared information input dialogue in a communication terminal. By operating a "Minutes Share" button 3535 arranged in association with any event title shown in Fig. 20 on the display 318 of the communication terminal 3, the display control unit 34 displays a shared information input dialogue 3142 as shown in Fig. 21. The shared information input dialogue 3142 includes an event URL display, a passcode input field, and a "URL passcode copy" button 3551. A creator who creates and edits an audio recording can check the displayed event URL, input a predetermined passcode in the passcode input field, and operate the "URL passcode copy" button 3551. The creator can then provide the copied URL (e.g., "https://.../00005006") and passcode to other users by email, chat, or the like.

<<Record viewing and editing screen generation process>>
Next, the process of generating the record viewing and editing screen by the voice recording management device 5 will be described. Fig. 22 is a sequence diagram showing an example of the process of generating the record viewing and editing screen. First, the operation acceptance unit 32 of the communication terminal 3 accepts the event URL and passcode entered by the user (step S101). As a specific example of the process, the user enters the event URL into the address input field of the browser, and if it is necessary to enter a password, a password input field is displayed and the password (passcode) is entered there.

Next, the transmission/reception unit 31 transmits a record viewing/editing screen request to the voice recording management device 5 (step S102). As a result, the transmission/reception unit 51 of the voice recording management device 5 receives the record viewing/editing screen request transmitted by the communication terminal 3. At this time, the record viewing/editing screen request includes the recording identification information in which the voice recording of the specified event indicated by the event URL is recorded.

Next, the acquisition unit 52 of the voice recording management device 5 acquires the record bibliographic information, text information, and capture information (step S103). Specifically, the acquisition unit 52 acquires the corresponding record bibliographic information by searching the record bibliographic information management DB 5002 (see FIG. 6) using the record identification information received in step S102 as a search key. Here, the record bibliographic information includes the start date and time, the end date and time registered in step S88, and the voice data path (".../00005006/record.mp3"), "event URL", and "passcode". Furthermore, the acquisition unit 52 acquires the corresponding text information by searching the text information management DB 5003 (see FIG. 7) using the record identification information received in step S102 as a search key. Here, the text information includes "text identification information", "start time", "end time", "text", a public flag, a deletion flag, and "image identification information". Furthermore, the acquisition unit 52 searches the capture image management DB 5004 (see FIG. 8A) using the record identification information received in step S102 as a search key to acquire corresponding capture information. Here, the capture information includes "image identification information," "acquisition time," a public flag, a deletion flag, and an "image data path."

Next, the generation/processing unit 57 generates a record viewing/editing screen using the record bibliographic information management DB 5002 (see FIG. 6), the text information management DB 5003 (see FIG. 7), the capture image management DB 5004 (see FIG. 8A), and the capture image acquisition interval DB 5005 (see FIG. 8B), and stores and manages it in a predetermined area of the storage unit 5000 (step S104). More specifically, the generation/processing unit 57 cooperates with the storage/reading unit 59 to read screen configuration data (template data for the screen) that constitutes the record viewing/editing screen, for example, from a predetermined area of the storage unit 5000. After that, the generation/processing unit 57 incorporates the record bibliographic information, including the "record name", "start date and time", "creator identification information (user ID)", and "event URL" managed in the record bibliographic information management DB 5002 (see FIG. 6), text information, and capture images into the screen configuration data, and generates record viewing/editing screen data.

<<Details of the process of generating the record viewing and editing screen: 1st time>>
Next, the process of generating the record viewing and editing screen will be described in detail. Fig. 23 is a flowchart showing an example of the process of generating the record viewing and editing screen for the creator's communication terminal or the first viewing. This flowchart shows the process of first associating a plurality of texts obtained based on the voice data with image data related to images associated with each text, and voice data corresponding to each of the plurality of texts, after a voice recording of a specific event such as a conference is recorded. Furthermore, this flowchart shows the process executed when a creator (editor) who can edit the voice recording opens the record viewing and editing screen.

First, the acquisition unit 52 of the voice recording management device 5 searches the record bibliographic information management DB 5002 (see Figure 6) using the recording identification information received in step S102 as a search key to acquire the item value of the corresponding "matching processing" item (step S104-1-1).

Then, the determination unit 55 determines whether the item value of the "Matching Process" acquired by the acquisition unit 52 is "Processed" (step S104-1-2). If the item value of the acquired "Matching Process" is "Processed" (step S104-1-2: YES), the generation/processing unit 57 groups the text by captured image (step S104-1-3).

Next, the generation/processing unit 57 generates a screen based on the grouping results, and the storage/reading unit 59 stores screen data relating to the generated screen in a specified area of the storage unit 5000, and then exits this flow (step S104-1-4).

On the other hand, if the acquired item value of "Matching Process" is not "Processed", i.e., if it is "Unprocessed" (step S104-1-2: NO), the voice recording management device 5 repeatedly executes the following processes from step S104-1-5 to step S104-10. Specifically, the voice recording management device 5 executes the following processes for each text registered in the text information management DB 5003 (see FIG. 7) (step S104-1-5).

First, the acquisition unit 52 acquires each piece of time information of the start time and end time corresponding to the text (step S104-1-6). Specifically, the acquisition unit 52 acquires each piece of time information of the corresponding start time and end time by searching the text information management DB 5003 (see FIG. 7) using each piece of text identification information as a search key.

Next, the calculation determination unit 53 determines a first interval between the acquired start time and end time (step S104-7). Specifically, the calculation determination unit 53 determines the following intervals as the first intervals:
1. Five seconds from 1 minute 53 seconds to 1 minute 58 seconds (Nth time of step S104-1-5)
2. Three seconds from 1 minute 59 seconds to 2 minutes 2 seconds (N+1 times of step S104-1-5)
3. 8 seconds from 2 minutes 5 seconds to 2 minutes 12 seconds (N+2 times of step S104-1-5)
4. Five seconds from 2 minutes 18 seconds to 2 minutes 23 seconds (step S104-1-5 N+3)
5. Three seconds from 2 minutes 25 seconds to 2 minutes 28 seconds (step S104-1-5 N+4)
6. ...
Next, the calculation identification unit 53 identifies captured images in which the second interval between the acquisition time and the time after the "captured image acquisition interval" overlaps with the first interval (step S104-8). Specifically, the calculation identification unit 53 searches the captured image acquisition interval DB 5005 (see FIG. 8B) and reads out the set captured image acquisition interval: 30 seconds. Next, the calculation identification unit 53 performs the following identification using the text information management DB 5003 (see FIG. 7) and the captured image management DB 5004 (see FIG. 8A). In this case, the second interval between the acquisition time and the time after the "captured image acquisition interval" is
11. The 30 seconds from 1 minute 00 seconds to 1 minute 30 seconds 12. The 30 seconds from 1 minute 30 seconds to 2 minutes 00 seconds 13. The 30 seconds from 2 minutes 00 seconds to 2 minutes 30 seconds 14. ...
From these sections, the calculation specification unit 53 determines that the second section overlaps with the first section when
2. The 3 seconds from 1 minute 59 seconds to 2 minutes 2 seconds, and 13. the 30 seconds from 2 minutes 00 seconds to 2 minutes 30 seconds. The calculation and identification unit 53 then identifies the captured image (image data path: ".../00005006/0005.jpg") associated with the image identification information "IM0005" as a captured image in which the second interval overlaps with the first interval.

Next, the setting registration unit 58 registers the image identification information of the identified capture image in the "image identification information" field of the target text (step S104-1-9). Specifically, the setting registration unit 58 registers "IM0005" in the image identification information field corresponding to the text identification information "TX0006" in the text information management DB 5003 (see FIG. 7).

In step S104-1-10 after the processing of step S104-9 is executed, the voice recording management device 5 returns to step S104-1-5 to perform loop processing, and repeats the processing from step S104-5 to step S104-1-10 described above.

After repeating the processes from step S104-5 to step S104-1-10, the setting registration unit 58 sets the item value of the "Matching Process" item in the recorded bibliographic information from "Unprocessed" to "Processed" (step S104-1-11).

Then, the voice recording management device 5 executes the processes of steps S104-1-3 and S104-1-4 described above, and then exits this flow.

<<Details of the process for generating the record viewing and editing screen: second time onwards>>
Next, another case of the generation process of the record viewing and editing screen will be described in detail. Fig. 24 is a flowchart showing an example of the generation process of the record viewing screen for a communication terminal of a user other than the creator. This flowchart shows the process of associating a plurality of texts obtained based on the voice data with image data related to the images associated with each text, and the voice data corresponding to each of the plurality of texts, which is performed for the second or subsequent times after an audio recording of a specific event such as a conference is recorded. Furthermore, this flowchart shows the process executed when a user other than the creator (editor) who can edit the audio recording opens the record viewing screen.

When the contents of "display of text or captured image" and "display of each operation button" corresponding to the user, deletion flag, and public flag are compiled in the process of generating the record viewing and editing screen, the combinations shown in Table 1 are obtained. In this embodiment, the flowchart in Figure 24 will be explained based on the combinations summarized in Table 1 below.

Table 1 shows an example of a combination of users, various flags, text or image displays, and various operation button displays.

First, the acquisition unit 52 searches the login management DB 5001 (see FIG. 5) to obtain the corresponding creator identification information by searching the record bibliographic information management DB 5002 (see FIG. 6) using the login user ID (user identification information) of the login user who performed the login process and the record identification information received in step S102 as search keys (step S104-2-1).

Then, the voice recording management device 5 repeatedly executes the following processes from step S104-2-2 to step S104-2-9. First, the voice recording management device 5 executes the following processes (step S104-2-2) for each "text" registered in the text information management DB 5003 (see FIG. 7) and each captured image ("image identification information") registered in the captured image management DB 5004 (see FIG. 8A).

First, the acquisition unit 52 searches the text information management DB 5003 (see FIG. 7) using the record identification information as a search key to acquire the publication flag and deletion flag corresponding to each piece of text identification information. The acquisition unit 52 further searches the capture image management DB 5004 (see FIG. 8A) using the record identification information as a search key to acquire the publication flag and deletion flag corresponding to each piece of image identification information (step S104-2-3).

Then, the judgment unit 55 judges whether the login user ID (user identification information) and the creator identification information match (step S104-2-4). If it is judged that the login user ID (user identification information) and the creator identification information match (step S104-2-4: YES), the judgment unit further judges whether the deletion flag acquired in step S104-2-3 is "True" (step S104-2-5).

If it is determined that the deletion flag is "True" (step S104-2-5: YES), the judgment unit 55 further determines whether the publication flag is "True" (step S104-2-6).

If it is determined that the public flag is "True" (step S104-2-6: YES), the determination unit 55 determines that the "text" or the capture image ("image identification information"), the "Delete" button, and the "Private" button should be "displayed" on the communication terminal 3 (step S104-2-7), and the process proceeds to step S104-2-9.

On the other hand, if it is determined that the public flag is not "True" (step S104-2-6: NO), the judgment unit 55 determines that the "text" or the capture image ("image identification information"), the "Delete" button, and the "Public" button are to be "displayed" on the communication terminal 3 (step S104-2-8), and proceeds to the processing of step S104-2-9, where the subsequent processing of circled 1 is executed.

If it is determined in the processing of step S104-2-4 that the "login user ID (user identification information)" and the "creator identification information (user ID)" match, the judgment unit 55 further determines whether the publication flag is "True" and the deletion flag is "False" (step S104-2-10).

If it is determined that the publication flag is "True" and the deletion flag is "False" (step S104-2-10: YES), the determination unit 55 determines that each edit button is "not displayed" on the communication terminal 3 and that the text or capture image is "displayed" on the communication terminal 3 (step S104-2-11), and proceeds to the processing of step S104-2-9.

On the other hand, if it is determined that the publication flag is "True" and the deletion flag is not "False" (step S104-2-10: NO), the determination unit 55 determines that each edit button and the text or capture image are "not to be displayed" on the communication terminal 3 (step S104-2-12), and the process proceeds to step S104-2-9.

After each of the above processes has been performed and step S104-2-9 has been performed, the voice recording management device 5 returns to step S104-2-1 and performs the same process on all of the target texts.

After repeating the process from step S104-2-1 to step S104-2-9, the generation/processing unit 57 groups the text in units of captured images as the subsequent process of circle 1 (step S104-2-13).

Next, the generation/processing unit 57 generates a screen based on the grouping results, and the storage/reading unit 59 stores screen data relating to the generated screen in a specified area of the storage unit 5000, and then exits this flow (step S104-2-14).

Returning to FIG. 22, the transmission/reception unit 51 transmits a record viewing/editing screen response to the communication terminal 3 as a response to the record viewing/editing screen request (step S105). As a result, the transmission/reception unit 31 of the communication terminal 3 receives the record viewing/editing screen response transmitted by the voice recording management device 5. At this time, the record viewing/editing screen response includes screen data that constitutes the record viewing/editing screen.

Next, the display control unit 34 of the communication terminal 3 displays a record viewing and editing screen as shown in FIG. 25 on the display 318 (step S106).

●Screen display example●
FIG. 25 is a screen display example of the record viewing and editing screen of the creator's communication terminal. By executing the process of step S106 described above, the display control unit 34 displays the record viewing and editing screen 3151 on the display 318 of the communication terminal 3. The record viewing and editing screen 3151 displays, for example, each input field of "Summary (topic)", "Participants", and "Meeting notes" in the same manner as the recording screen 3121 shown in FIG. 16. The record viewing and editing screen 3151 further displays at least one text display field. The record viewing and editing screen 3151 further displays screens 3, 4, and 5 captured by the screen capture process in association with each text displayed in each text display field for each capture time. The user of the communication terminal 3 on which the record viewing and editing screen 3151 is displayed is, for example, "Rikotaro".

On the record viewing and editing screen 3151, for example, when the mouse 3701 is placed near each text display field (mouse-over operation), the display control unit 34 displays the "Share minutes" button (icon) 3541, "Private" button (icon) 3542, "Play audio" button (icon) 3543, and "Delete" button (icon) 3544 near the area where the mouse was over.

Of these buttons (icons), the "Share minutes" button (icon) 3541 is a button (icon) for allowing a record viewing screen displaying the minutes edited by the creator who creates an audio record (e.g., minutes) of a specified event to be shared with a communication terminal 3 used by a user other than the creator. This "Share minutes" button (icon) 3541 has the same function as the "Share minutes" button (icon) 3535 described in FIG. 20. Therefore, if the record selection screen 3141 shown in FIG. 20 is not displayed, the "Share minutes" button (icon) 3541 displayed in the record viewing and editing screen 3151 may be operated to share the minutes, etc. This allows the creator to share the minutes, etc. with other users by operating this "Share minutes" button (icon) 3541.

The "Private" button (icon) 3542 is a button (icon) for making the text near the mouse over private (hidden) on the record viewing screen of the communication terminal 3 of a user other than the creator, which will be described later. When the user operates the "Private" button (icon) 3542, the voice recording management device 5 can make the text near the mouse over private (hidden) on the record viewing screen of the communication terminal 3 of a user other than the creator, which will be described later. Note that after the "Private" button (icon) is displayed and the private process is executed, if the same text is again moused over, the record viewing and editing screen 3151 may display a "Play" button (icon) near the target text. When the user operates the "Publish" button (icon), the voice recording management device 5 can make the text near the mouse over public (redisplay) on the record viewing screen of the communication terminal 3 of a user other than the creator, which will be described later. In this way, the private process is an example of a function that does not delete the target text, but makes it temporarily private (hidden).

The "Play Audio" button (icon) 3543 is a button that allows the user to play back a specific audio. By operating the "Play Audio" button (icon) 3543, the user can play back the audio associated with a specific audio recording that the user has selected.

The "Delete" button (icon) 3544 is a button for deleting specific text. When the user operates the "Delete" button (icon) 3544, the voice recording management device 5 can delete the text near the area where the mouse is hovered over on the record viewing and editing screen 3151 and the record viewing screen on the communication terminal 3 of a user other than the creator, which will be described later.

Furthermore, on the record viewing and editing screen 3151, when the mouse is hovered over the vicinity of each screen, the display control unit 34 displays a "Private" button (icon) 3545 and a "Delete" button (icon) 3546.

Of these buttons (icons), the "Private" button (icon) 3545 is a button (icon) for making private (hidden) an image in a nearby screen that is moused over on the record viewing screen of a communication terminal 3 of a user other than the creator, which will be described later. When the user operates the "Private" button (icon) 3545, the audio recording management device 5 can make private (hidden) an image in a nearby screen that is moused over on the record viewing screen of a communication terminal 3 of a user other than the creator, which will be described later.

The "Delete" button (icon) 3546 is a button for deleting a specific image. When the user operates the "Delete" button (icon) 3546, the voice recording management device 5 can delete images in the screen near the area where the mouse is hovered on the record viewing and editing screen 3151 and the record viewing screen on the communication terminal 3 of a user other than the creator, which will be described later.

Furthermore, on the record viewing and editing screen 3151, an audio playback display section 3601 linked to the operation of the above-mentioned "audio playback" button (icon) 3543 is displayed by the display control section 34. The ● mark in the audio playback display section 3601 moves corresponding to the date and time information (time information) at which each text is displayed. For example, when the user operates the "audio playback" button (icon) 3543 to seek the ● mark to the position of "2021/03/31 11:02:18", the audio playback section 36 of the audio recording management device 5 plays the audio "Sales for fiscal year 2020 are XX."

The above-mentioned "Private" button (icon) 3542, "Play audio" button (icon) 3543, "Delete" button (icon) 3544, "Private" button (icon) 3545, and "Delete" button (icon) 3546 are displayed near the respective text and screen by hovering the mouse over them, etc. In the example shown in FIG. 25, for convenience, a symbol (number) is given to only one of the buttons (icons), and the other buttons (icons) are displayed with dotted lines. However, the same symbol (number) is given to buttons of the same type.

●Screen display example●
Fig. 26 is an example of a screen display for viewing a record on a communication terminal of a user other than the creator. On the display 318 of the communication terminal 3 of the user other than the creator, a record viewing screen 3161 is displayed by the display control unit 34 in correspondence with the record viewing and editing screen 3151 shown in Fig. 25. The record viewing screen 3161 displays the same content as the record viewing and editing screen 3151, but the "Private" button (icon) 3545, the "Delete" button (icon) 3546, the "Private" button (icon) 3545, and the "Delete" button (icon) 3546 are not displayed. This is because the creator of the record viewing and editing screen is "Rikotaro" and users other than the creator are not given editing authority for the record viewing and editing screen.

The user other than the creator for whom the record viewing and editing screen 3151 is displayed is, for example, "Ebina Jiro."

<<Record viewing and editing process>>
Next, the record viewing and editing process will be described. Fig. 27 is a sequence diagram showing an example of the record viewing and editing process. As shown in Fig. 27, the display control unit 34 of the communication terminal 3 displays the record viewing and editing screen generated in step S106 on the display 318, and the operation reception unit 32 receives an editing operation instruction from the user (creator) (step S111).

Then, the transmission/reception unit 31 transmits an editing operation instruction request to the audio recording management device 5 (step S112). As a result, the transmission/reception unit 51 of the audio recording management device 5 receives the editing operation instruction request transmitted by the communication terminal 3. At this time, the editing operation instruction request includes at least one of the following three pieces of information. One is text identification information that identifies the target text and various operation button information associated with the target text. The other is image identification information that identifies the target image and various operation button information associated with the target image. The third is a text group ID that identifies a text group, which will be described later, and various operation button information associated with the target text group.

Next, the voice recording management device 5 performs a record viewing and editing process (step S113). In this record viewing and editing process, the record bibliographic information management DB 5002 (see FIG. 6), the text information management DB 5003 (see FIG. 7), the capture image management DB 5004 (see FIG. 7A), the capture image acquisition interval DB 5005 (see FIG. 7B), and the private voice management DB 5006 (see FIG. 8) are used.

<<Distribution of record viewing and editing processes>>
Next, the allocation of the record viewing and editing process will be described. Fig. 28A is a flow chart showing an example of a process branched by various button operations. First, the acquisition unit 52 acquires the various button information received in step S112 (step S113-1).

Then, the judgment unit 55 judges whether the acquired button information is a "Private" button (icon) in the text (step S113-1-1). If the acquired button information is a "Private" button (icon) in the text (step S113-1-1: YES), the voice recording management device 5 transitions to the process of circle A described later.

If the acquired button information is not a "Private" button (icon) of the text (step S113-1-1: NO), the judgment unit 55 further judges whether the acquired button information is a "Public" button (icon) of the text (step S113-1-2). If the acquired button information is a "Public" button (icon) of the text (step S113-1-2: YES), the voice recording management device 5 transitions to the process of circle B described below.

If the acquired button information is not a "Publish" button (icon) in the text (step S113-1-2: NO), the judgment unit 55 further judges whether the acquired button information is a "Delete" button (icon) in the text (step S113-1-3). If the acquired button information is a "Delete" button (icon) in the text (step S113-1-3: YES), the voice recording management device 5 transitions to the process of circle C described below.

If the acquired button information is not a text "Delete" button (icon) (step S113-1-3: NO), the voice recording management device 5 transitions to process 2 of acquiring various button information (step S113-1-4).

Figure 28B is a flowchart showing an example of a process branched by various button operations. In Figure 28B, the acquisition unit 52 acquires the various button information received in step S112 (step S113-2).

Then, the judgment unit 55 judges whether the acquired button information is a "Private" button (icon) of the captured image (step S113-2-1). If the acquired button information is a "Private" button (icon) of the captured image (step S113-2-1: YES), the voice recording management device 5 transitions to the process of circle D described below.

If the acquired button information is not the "Private" button (icon) of the captured image (step S113-2-1: NO), the judgment unit 55 further judges whether the acquired button information is the "Public" button (icon) of the captured image (step S113-2-2). If the acquired button information is the "Public" button (icon) of the captured image (step S113-2-2: YES), the voice recording management device 5 transitions to the process of circle E described below.

If the acquired button information is not the "Publish" button (icon) of the captured image (step S113-2-2: NO), the judgment unit 55 further judges whether the acquired button information is the "Delete" button (icon) of the captured image (step S113-2-3). If the acquired button information is the "Delete" button (icon) of the captured image (step S113-2-3: YES), the voice recording management device 5 transitions to the process of circle F described below.

If the acquired button information is not a text "Delete" button (icon) (step S113-2-3: NO), the voice recording management device 5 transitions to process 3 of acquiring various button information (step S113-2-4).

<<Processing when the private button is pressed on text>>
Next, the process when the non-disclosure button is operated on the text will be described. Fig. 29 is a flowchart showing an example of the process when the non-disclosure button is operated on the text. In this flowchart, the process of circle A as the transition destination determined in Fig. 28A (steps S113-1-101 to S113-1-105) is executed.

First, the setting registration unit 58 of the voice recording management device 5 sets the public flag corresponding to the text identification information received in step S112, which is managed in the text information management DB 5003 (see Figure 7), to "False" (step S113-1-101).

Next, the acquisition unit 52 searches the text information management DB 5003 (see FIG. 7) using the text identification information as a search key to acquire image identification information corresponding to the target text (step S113-1-102). The image identification information at this time is information for identifying a capture image obtained by capturing a screen including the target text.

Next, the setting registration unit 58 sets the public flag that is managed in the capture image management DB 5004 (see FIG. 8A) and corresponds to the image identification information that identifies each capture image to "False" (step S113-2-201). (step S113-1-103).

Next, the acquisition unit 52 searches the text information management DB 5003 (see FIG. 7) using the text identification information as a search key to acquire the time information of the corresponding start time and end time (step S113-1-104).

Next, the setting registration unit 58 registers the acquired start time and end time information in the recorded bibliographic information management DB 5002 (see Figure 6), mutes the audio between each piece of time information managed in the private audio management DB 5006 (see Figure 9), and exits this flow (step S113-1-105).

●Screen display example●
FIG. 30 is an example of a screen display displayed on a communication terminal of a user other than the creator when the private button for the text is operated. On the display 318 of the communication terminal 3 of the user other than the creator, the display control unit 34 displays the record viewing screen 3161 shown in FIG. 30 based on the process shown in FIG. 24 and the process of circle A described above. In this case, the record viewing screen 3161 displays a state in which the specific text ("Sales for the second half of fiscal year 2020 are XX.") has been hidden from the text display field based on the hide operation for the specific text specified in FIG. 25. At this time, the text display field may be in a state in which the text cannot be deciphered by a process such as gray hatching or mesh pattern processing. Furthermore, the text display field may be processed so that the hidden text is not displayed. In the record viewing screen 3161, the image of the screen 5 corresponding to the specific text is also hidden. In this case, the image of the screen 5 may be a blank image. Also, the image of the screen 5 may be replaced with another image. Furthermore, the audio data corresponding to the specific text is also muted on the record viewing screen 3161. In this case, the uttered voice corresponding to the specific text is muted at the time when the uttered voice is to be played. Instead of being muted, the uttered voice may be converted into another signal sound or the like.

<<Processing when the publish button is used on text>>
Next, the process when the publish button is operated on the text will be described. Fig. 31 is a flowchart showing an example of the process when the publish button is operated on the text. In this flowchart, the process of circle B as the transition destination determined in Fig. 28A (steps S113-1-201 to S113-1-207) is executed.

First, the setting registration unit 58 of the voice recording management device 5 sets the public flag corresponding to the text identification information received in step S112, which is managed in the text information management DB 5003 (see Figure 7), to "True" (step S113-1-201).

Next, the acquisition unit 52 searches the text information management DB 5003 (see FIG. 7) using the text identification information as a search key to acquire the time information of the corresponding start time and end time (step S113-1-202).

Then, the acquisition unit 52 searches the text information management DB 5003 (see FIG. 7) using the text identification information as a search key to acquire image identification information corresponding to the target text (step S113-1-203). The image identification information at this time is information for identifying a capture image obtained by capturing a screen including the target text.

Next, the acquisition unit 52 searches the text information management DB 5003 (see FIG. 7) using the acquired image identification information as a search key to acquire the public flag of each text corresponding to the capture image indicated by the acquired image identification information (step S113-1-204).

Next, the judgment unit 55 judges whether all of the acquired public flags are "True" (step S113-1-205). If all of the acquired public flags are "True" (step S113-1-205: YES), the setting registration unit 58 sets the public flag of the capture image, which is managed in the capture image management DB 5004 (see FIG. 8A) and indicated by the acquired image identification information, to "True" (step S113-1-206).

Then, the setting registration unit 58 deletes the time information for the start time and end time managed in the private audio management DB 5006 (see Figure 9) and exits this flow (step S113-1-207).

On the other hand, if all of the retrieved public flags are not "True," i.e., if at least one public flag is "False" (step S113-1-205: NO), the process of step S113-1-207 described above is executed and the flow is exited.

●Screen display example●
FIG. 32 is an example of a screen display when the public button is operated for text on the creator's communication terminal. On the display 318 of the creator's communication terminal 3, a record viewing and editing screen 3151 is displayed by the display control unit 34 based on the process shown in FIG. 24 and the process of circle B described above. The record viewing and editing screen 3151 shows a state in which the "Private" button (icon) 3542 is displayed on the record viewing and editing screen 3151 shown in FIG. 25, and then the "Private" button (icon) 3547 is displayed instead of the "Private" button (icon) 3542. In other words, the "Private" button (icon) 3542 and the "Public" button (icon) 3547 are displayed on the record viewing and editing screen 3151 in pairs for the same text. The "Public" button (icon) 3547 is a button or icon that is operated when a specific text is made private (hidden) on the record viewing screen 3161 of the communication terminal 3 of a user other than the creator and then a specific text is made to be displayed again on the record viewing screen 3161. In the record viewing and editing screen 3151, when the creator performs the "private" process on a specific text, for example, a unique mark indicating that the specific text has been set to "private" may be displayed by the display control unit 34 next to the specific text. This unique mark is, for example, a "private" mark 3201 with the character "private" in a square. This allows the creator to easily distinguish which text has been made private. Note that the content of the "private" mark 3201 is not limited to the content with the character "private" in a square as described above, but may be a combination of the symbol "private" and the character. Furthermore, the display position of the "private" mark 3201 is not limited to the side of the specific text described above, and may be any position in the vicinity of the text that has been made private.

When the mouse is again placed over text that has been made private by the creator, the display control unit 34 displays a "Public" button (icon) 3547 near the text that has been made private. Accordingly, the "Private" mark 3201 is hidden. In other words, on the record viewing and editing screen 3151, the "Private" button (icon) 3542 and the "Public" button (icon) 3547 for a certain text are displayed in a toggle manner. When this "Public" button (icon) 3547 is operated on the communication terminal 3 of the creator, the specific text that was made private on the communication terminal 3 of a user other than the creator is redisplayed, and the image on screen 5 is also redisplayed. Furthermore, the audio data corresponding to the specific text is restored to its original playable state, making it possible to listen to the audio corresponding to the specific text.

<<Processing when the delete button is used on text>>
Next, the process when the delete button is operated on the text will be described. Fig. 33 is a flowchart showing an example of the process when the delete button is operated on the text. In this flowchart, the process of circle C as the transition destination determined in Fig. 28A (steps S113-1-301 to S113-1-307) is executed.

First, the setting registration unit 58 of the voice recording management device 5 sets the text managed in the text information management DB 5003 (see FIG. 7) that corresponds to the text identification information received in step S112 to blank, and sets the deletion flag to "True" (step S113-1-301). In other words, the contents of the target text are deleted at this point.

Next, the acquisition unit 52 searches the text information management DB 5003 (see FIG. 7) using the text identification information as a search key to acquire image identification information corresponding to the target text (step S113-1-302). The image identification information at this time is information for identifying a capture image obtained by capturing a screen including the target text.

Next, the acquisition unit 52 searches the text information management DB 5003 (see FIG. 7) using the acquired image identification information as a search key to acquire the public flag of each text corresponding to the capture image indicated by the acquired image identification information (step S113-1-303).

Next, the judgment unit 55 judges whether all of the acquired deletion flags are "True" (step S113-1-304). If all of the acquired deletion flags are "True" (step S113-1-304: YES), the setting registration unit 58 replaces the image data of the capture image managed in the capture image management DB 5004 (see FIG. 8A) and corresponding to the acquired image identification information with a specified image, and sets the disclosure flag to "True" (step S113-1-305).

Next, the acquisition unit 52 searches the text information management DB 5003 (see FIG. 7) using the text identification information as a search key to acquire the time information of the corresponding start time and end time (step S113-1-306).

Next, the setting registration unit 58 deletes (silences) the voice data between the acquired start time and end time, which is managed in the record bibliographic information management DB 5002 (see Figure 6), and exits this flow (step S113-1-307).

On the other hand, if all of the acquired deletion flags are not "True", that is, if at least one deletion flag is "False" (step S113-1-304: NO), the setting registration unit 58 sets the public flag of each capture image that is managed in the capture image management DB 5004 (see FIG. 8A) and corresponds to the acquired image identification information to "False" (step S113-1-308), and thereafter executes the processing of steps S113-1-306 and S113-1-307 to exit this flow.

●Screen display example●
FIG. 34 is an example of a screen display displayed on a communication terminal of a user other than the creator when the delete button for the text is operated. On the display 318 of the communication terminal 3 of the user other than the creator, a record viewing screen 3161 is displayed by the display control unit 34 based on the process shown in FIG. 24 and the process of circle C described above. In this case, the record viewing screen 3161 displays a state in which the other specific text ("Today's agenda is the performance of the healthcare business.") has been deleted from the text display field based on the delete operation for the specific text specified in FIG. 25. Furthermore, since the image of the screen 4 corresponding to the other specific text is also subject to deletion, the image of the screen 4 is also deleted. As a result, for example, the text display field in which the other specific text was displayed is blacked out, and the screen 4 displays a blank image or an image replaced with another image. Furthermore, when the delete process is executed, the text to be deleted and the image corresponding to the text cannot be made public (restored) thereafter. Furthermore, on the record viewing screen 3161, the audio data corresponding to the other specific text is also muted. In this case, the speech corresponding to the other specific text is silenced at the time when the speech is to be reproduced. Note that instead of the silence process, a conversion process to another signal sound or the like may be performed.

<<Processing when pressing the private button on a captured image>>
Next, the process when the private button is operated on a captured image will be described. Fig. 35 is a flowchart showing an example of the process when the private button is operated on a captured image. In this flowchart, the process of circle D as the transition destination determined in Fig. 28B (steps S113-2-101 to S113-2-105) is executed.

First, the setting registration unit 58 of the audio recording management device 5 sets the public flag associated with the image identification information of the target capture image managed in the capture image management DB 5004 (see FIG. 8A) and received in step S112 to "False" (step S113-2-101).

Next, the acquisition unit 52 searches the text information management DB 5003 (see FIG. 7) using the image identification information as a search key to acquire text identification information for each text corresponding to the target captured image (step S113-2-102).

Next, the setting registration unit 58 sets the public flag of each text that is managed in the text information management DB 5003 (see FIG. 7) and corresponds to each acquired text identification information to "False" (step S113-2-103).

Next, the acquisition unit 52 searches the text information management DB 5003 (see FIG. 7) using each acquired text identification information as a search key to acquire time information for the start time and end time of each corresponding text (step S113-2-104).

Next, the setting and registration unit 58 registers the earliest start time and the latest end time of the acquired time information in the private audio management DB 5006 (see FIG. 9) (step S113-2-105). Note that the above-mentioned processes from step S113-2-103 to step S113-2-105 are defined as the circle X process, and are executed again in the subsequent processes.

●Screen display example●
Fig. 36 is an example of a screen display displayed on a communication terminal of a user other than the creator when the private button for a captured image is operated. On the display 318 of the communication terminal 3 of the user other than the creator, a record viewing screen 3161 is displayed by the display control unit 34 based on the process shown in Fig. 24 and the process of circle D described above. In this case, the record viewing screen 3161 displays a state in which the specific image (the captured image displayed on the screen 5) and one or more texts associated with the specific image have been hidden based on the private operation for the specific image specified in Fig. 25. The one or more pieces of text include three pieces of text ("11:02:05", "11:02:18", and "11:02:25") uttered after the time ("11:02:00") when the capture image of screen 5 was acquired, as well as a piece of text ("11:02:05", "11:02:18", and "11:02:25") uttered after the time ("11:01:30") when the capture image of screen 4 was acquired and at a time ("11:01:59") that straddles "11:02:00". In other words, when specific text data exists that straddles the acquisition time ("11:02:00") when the screen data representing screen 5 was acquired, the screen data of screen 5 acquired at the acquisition time, the text data of the three pieces of text associated with the screen data of screen 5, and the specific text data are subject to non-disclosure processing. This is based on the premise that the content of the text uttered at a time ("11:01:59") that straddles "11:02:00" is determined to have a high probability of being related to the content of the image on screen 5, and therefore needs to be hidden like the three pieces of text associated with the image on screen 5. The hiding process may be the same as the method shown in FIG.

As explained above, the spoken voice (audio data) associated with the text to be hidden is also muted in the same way. Note that instead of muting, conversion to other signal sounds, etc. may be performed.

<<Processing when the publish button is operated on a captured image>>
Next, the process when the publish button is operated for a captured image will be described. Fig. 37 is a flowchart showing an example of the process when the publish button is operated for a captured image. In this flowchart, the process of circle E as the transition destination determined in Fig. 28B (steps S113-2-201 to S113-2-211) is executed.

First, the setting registration unit 58 of the audio recording management device 5 sets the public flag associated with the image identification information of the target capture image managed in the capture image management DB 5004 (see FIG. 8A) and received in step S112 to "True" (step S113-2-201).

Next, the acquisition unit 52 searches the text information management DB 5003 (see FIG. 7) using the image identification information as a search key to acquire text identification information for each text corresponding to the target captured image (step S113-2-202).

Next, the acquisition unit 52 searches the text information management DB 5003 (see FIG. 7) using each acquired text identification information as a search key to acquire time information for the start time and end time of each text corresponding to each text identification information (step S113-2-203).

Next, based on the acquired time information, the setting registration unit 58 deletes the time information of the earliest start time and the latest end time from the time information managed in the private audio management DB 5006 (see Figure 9) (step S113-2-204).

Then, the voice recording management device 5 repeatedly executes the processes from step S113-2-205 to step S113-2-209 below. Specifically, the voice recording management device 5 executes the following processes for each identified text (step S113-2-205).

First, the acquisition unit 52 searches the capture image management DB 5004 (see FIG. 8A) using the image identification information of each capture image corresponding to the target text as a search key to acquire the corresponding public flag (step S113-2-206).

Next, the judgment unit 55 judges whether the public flags for each acquired capture image are all "True" (step S113-2-207).

If the public flags for all of the acquired captured images are "True" (step S113-2-207: YES), the setting and registration unit 58 sets the public flag of the target text managed in the text information management DB 5003 (see FIG. 7) to "True" (step S113-2-208), and returns to step S113-2-205 via the next step S113-2-209. The processes from step S113-2-205 to step S113-3-209 are repeated for each text.

On the other hand, if all of the public flags for each of the acquired capture images are not "True", i.e., if at least one public flag is "False" (step S113-2-207: NO), the acquisition unit 52 acquires the corresponding start time and end time by searching the text information management DB 5003 (see FIG. 7) using the text identification information corresponding to the public flag as a search key (step S113-2-210).

Then, the setting registration unit 58 registers the acquired start time and end time information in the private audio management DB 5006 (see FIG. 9) (step S113-2-211), transitions to step S113-2-209, and returns to the processing of step S113-2-205. Then, when the repeated processing from step S113-2-205 to step S113-2-209 is completed, the audio recording management device 5 exits this flow. Note that the processing from step S113-2-203 to step S113-2-211 described above is defined as processing marked with a circle Y, and will be executed again in the subsequent processing.

<<Processing when the delete button is used on a captured image>>
Next, the process when the delete button is operated on a captured image will be described. Fig. 38 is a flowchart showing an example of the process when the delete button is operated on a captured image. In this flowchart, the process of circle F as the transition destination determined in Fig. 28B (steps S113-2-301 to S113-2-312) is executed.

First, the setting registration unit 58 of the audio recording management device 5 replaces (rewrites) the image data of the target capture image managed by the capture image management DB 5004 (see FIG. 8A) and corresponding to the image identification information received in step S112 with specified image data, and sets the deletion flag corresponding to the image identification information to "True" (step S113-2-301).

Next, the acquisition unit 52 searches the text information management DB 5003 (see FIG. 7) using the image identification information as a search key to acquire text identification information for each text corresponding to the target captured image (step S113-2-302).

Then, the voice recording management device 5 repeatedly executes the processes from step S113-2-303 to step S113-2-312 below. Specifically, the voice recording management device 5 executes the following processes for each identified text (step S113-2-303).

First, the acquisition unit 52 searches the capture image management DB 5004 (see FIG. 8A) using the image identification information of each capture image corresponding to the target text as a search key to acquire the corresponding deletion flag (step S113-2-304).

Next, the judgment unit 55 judges whether the deletion flags for each acquired capture image are all "True" (step S113-2-305).

If all of the deletion flags for each acquired capture image are "True" (step S113-2-305: YES), the setting registration unit 58 sets the target text managed in the text information management DB 5003 (see FIG. 7) and associated with the image identification information to blank (deletes), and sets the deletion flag corresponding to the target text to "True" (step S113-2-306).

Next, the acquisition unit 52 acquires the time information of the start time and end time corresponding to the target text, which are managed in the text information management DB 5003 (see FIG. 7) (step S113-2-307), and returns to step S113-2-303 via the next step S113-2-308. The process from step S113-2-303 to step S113-3-308 is repeated for each text.

On the other hand, if all of the deletion flags for each acquired capture image are not "True", i.e., if at least one deletion flag is "False" (step S113-2-305: NO), the setting registration unit 58 sets the public flag of the target text managed in the text information management DB 5003 (see FIG. 7) to "False" (step S113-2-309).

Then, the acquisition unit 52 searches the text information management DB 5003 (see FIG. 7) using the text identification information of the target text as a search key to acquire the time information of the corresponding start time and end time (step S113-2-310).

Next, the setting registration unit 58 registers the acquired start time and end time information in the private audio management DB 5006 (see FIG. 9) (step S113-2-311), and returns to step S113-2-303 via step S113-2-308.

When the repeated processing up to step S113-2-308 is completed, the setting registration unit 58 deletes (silences) a part of the voice data managed in the record bibliographic information management DB 5002 (see FIG. 6) (step S113-2-312) and exits this flow. Specifically, the setting registration unit 58 deletes (silences) the voice data between the earliest start time and the latest end time among the voice data managed in the record bibliographic information management DB 002 (see FIG. 6), which is voice data corresponding to text managed in the text information management DB 5003 (see FIG. 7) and whose deletion flag is set to "True." Note that the processing from step S113-2-303 to step S113-2-312 described above is defined as the processing of circle Z, and is executed again in the subsequent processing.

●Screen display example●
FIG. 39 is an example of a screen display displayed on a communication terminal of a user other than the creator when the delete button for a captured image is operated. On the display 318 of the communication terminal 3 of the user other than the creator, a record viewing screen 3161 is displayed by the display control unit 34 based on the process shown in FIG. 24 and the process of circle F described above. In this case, the record viewing screen 3161 displays a state in which a specific image (the captured image displayed on the screen 5) and one or more texts associated with the specific image have been hidden based on the delete operation for the specific image specified in FIG. 25. The one or more texts include three texts ("11:02:05", "11:02:18", "11:02:25") uttered after the time ("11:02:00") when the captured image of the screen 5 was acquired, as well as text uttered after the time ("11:01:30") when the captured image of the screen 4 was acquired and at a time ("11:01:59") that straddles "11:02:00". That is, when specific text data exists across the acquisition time ("11:02:00") when the screen data representing the screen 5 is acquired, the screen data of the screen 5 acquired at the acquisition time and the text data of the three texts associated with the screen data of the screen 5 are deleted. Furthermore, the specific text data is subject to non-disclosure processing. That is, the difference from FIG. 36 is that the three texts ("11:02:05", "11:02:18", "11:02:25") uttered after the time ("11:02:00") when the capture image of the screen 5 is acquired are deleted instead of being made private. The text to be deleted is, for example, blacked out. Note that the screen data related to the screen 4 associated with the specific text data is not subject to non-disclosure processing. This is because there is another text data in addition to the specific text data associated with the screen data of the screen 4. The other text data is, for example, the "2021/03/31 11:01:53" text data. Since the other text data is not associated with the screen data of screen 5 that is associated with the specific text data, the screen data of screen 4 is not subject to the non-display process.

As explained above, the spoken voice (audio data) associated with the text to be hidden is also muted in the same way. Note that instead of muting, conversion to other signal sounds, etc. may be performed.

<<Update screen generation process>>
Next, the process of generating the updated screen will be described. Returning to Fig. 27, the generation/processing unit 57 generates an updated screen for the communication terminal 3, and stores screen data (template data) of the generated updated screen in a predetermined area of the storage unit 5000 (step S114). Note that the specific content of the process of generating the updated screen is the process of step S104 described above that is executed depending on each case.

Next, the transmission/reception unit 51 of the audio recording management device 5 transmits an editing operation instruction response to the communication terminal 3 (step S115). As a result, the transmission/reception unit 31 of the communication terminal 3 receives the editing operation instruction response. At this time, the editing operation instruction response includes the screen data (text data, screen data) of the updated record viewing/editing screen, and the audio data.

Next, the display control unit 34 of the communication terminal 3 displays the updated record viewing and editing screen in response to the various editing operation instructions on the display 318 (step S116).

[Major Effects of the First Embodiment]
As described above, according to this embodiment, the voice recording management device 5 transmits edited screen data including edited text data obtained by editing the specified text data and edited image data obtained by editing the specified image data, and edited audio data obtained by editing the specified audio data, to the communication terminal 3 in response to an editing request for a specified text or a specified image transmitted from the communication terminal 3. The communication terminal 3 displays an edited screen related to the edited screen data transmitted by the voice recording management device 5 on the display means of the display 318, and plays back edited audio related to the edited audio data transmitted by the voice recording management device 5. This provides the effect of improving convenience in editing voice recordings, since when editing a voice recording generated based on audio information, it is only necessary to edit the text data included in the voice recording or the image data corresponding to the text data.

Furthermore, according to this embodiment, a UI (User Interface) is provided that allows the user to selectively edit text or images in an audio recording, enabling the user to perform various processes, including non-disclosure, disclosure, deletion, and playback, on the desired text or image, thereby allowing the user to control the scope of disclosure in a limited manner. This makes it possible to improve confidentiality for audio recordings that contain confidential information.

Second Embodiment
Next, the second embodiment will be described with reference to Fig. 40 to Fig. 51. The difference between the first embodiment and the second embodiment is that when there is an editing request for a specific text (for example, a request for non-disclosure, disclosure, or deletion), the text to be edited is treated as a text group, and a specific summary is given to the text group before editing. In other words, the hardware resources constituting the communication system 1 and the functional configuration of each hardware resource are the same as those in the first embodiment, with the structure of the data table being partially changed. Therefore, the sequence diagram for the record viewing and editing process (steps S111-S116) shown in Fig. 27 is also used in the second embodiment.

●Text information management table●
FIG. 40 is a conceptual diagram showing an example of a text information management table according to the second embodiment. In the storage unit 5000, a text information management DB 5007 configured by a text information management table as shown in FIG. 40 is constructed. In the text information management DB 5007, an item of a text group ID is added to the text information management table constituting the text information management DB 5003 shown in FIG. 7. This text group ID is identification information that is given when the creator generates one or more texts to be edited as one text group by mouse operation or the like on the record viewing and editing screen. The text group ID is given, for example, as "TG0003", "TG0004", etc. In the text information management DB 5007, this text group ID is managed for each record identification information.

●Summary information management table●
Fig. 41 is a conceptual diagram showing an example of a summary information management table according to the second embodiment. A summary information management DB 5008 configured by a summary information management table as shown in Fig. 41 is constructed in the storage unit 5000. In the summary information management DB 5008, the record identification information is treated as a tab, and text group IDs and summary contents separated by each tab are stored and managed in association with each other. The text group IDs and summary contents are managed for each acquired voice recording.

<<Distribution of record viewing and editing processes>>
Next, the allocation of the record viewing and editing process will be described. Fig. 42A is a flowchart showing an example of a process branched by various button operations according to the second embodiment. First, the acquisition unit 52 continues the process of step S113-2-4 in Fig. 28B and acquires various button information received in step S112 (step S113-3).

Then, the judgment unit 55 judges whether the acquired button information is a "Private" button (icon) of a text group (step S113-3-1). If the acquired button information is a "Private" button (icon) of a text group (step S113-3-1: YES), the voice recording management device 5 transitions to the process of circle G described below.

If the acquired button information is not a "Private" button (icon) of a text group (step S113-3-1: NO), the judgment unit 55 further judges whether the acquired button information is a "Public" button (icon) of a text group (step S113-3-2). If the acquired button information is a "Public" button (icon) of a text group (step S113-3-2: YES), the voice recording management device 5 transitions to the process of circle H described below.

If the acquired button information is not the "Publish" button (icon) of the text group (step S113-3-2: NO), the judgment unit 55 further judges whether the acquired button information is the "Delete" button (icon) of the text group (step S113-3-3). If the acquired button information is the "Delete" button (icon) of the text group (step S113-3-3: YES), the voice recording management device 5 transitions to the process of circle I described below.

If the acquired button information is not the "Delete" button (icon) of the text group (step S113-3-3: NO), the voice recording management device 5 transitions to process 4 of acquiring various button information (step S113-3-4).

Fig. 42B is a flowchart showing an example of a process branched by various button operations according to the second embodiment. In Fig. 42B, the acquisition unit 52 acquires the various button information received in step S112 (step S113-4).

Then, the judgment unit 55 judges whether the acquired button information is a "Private" button (icon) of the captured image (step S113-4-1). If the acquired button information is a "Private" button (icon) of the captured image (step S113-4-1: YES), the voice recording management device 5 transitions to the process of circle J, which will be described later.

If the acquired button information is not the "Private" button (icon) of the captured image (step S113-4-1: NO), the judgment unit 55 further judges whether the acquired button information is the "Public" button (icon) of the captured image (step S113-4-2). If the acquired button information is the "Public" button (icon) of the captured image (step S113-4-2: YES), the voice recording management device 5 transitions to the process of circle K described below.

If the acquired button information is not the "Publish" button (icon) of the captured image (step S113-4-2: NO), the judgment unit 55 further judges whether the acquired button information is the "Delete" button (icon) of the captured image (step S113-4-3). If the acquired button information is the "Delete" button (icon) of the captured image (step S113-4-3: YES), the voice recording management device 5 transitions to the process of circle L described below.

If the acquired button information is not a text "Delete" button (icon) (step S113-4-3: NO), the voice recording management device 5 transitions to the process of acquiring various button information 1 (step S113-4-4). In other words, the voice recording management device 5 returns to the process of FIG. 28A.

<<Processing when the private button is operated on a text group>>
Next, a process when the private button is operated on a text group will be described. Fig. 43 is a flowchart showing an example of a process when the private button is operated on a text group according to the second embodiment. In this flowchart, the process of circle G as the transition destination determined in Fig. 42A (steps S113-3-101 to S113-3-105) is executed.

First, the storage/reading unit 59 of the voice recording management device 5 searches the summary information management DB 5008 (see FIG. 41) using the summary content received in step S112 as a search key to read out the corresponding text group ID. Next, the setting/registration unit 58 sets the public flag of each text corresponding to the text group ID of the text group to be processed to "False." Specifically, the setting/registration unit 58 sets the public flag of each text corresponding to the read out text group ID, which is managed in the text information management DB 5007 (see FIG. 40), to "False" (step S113-3-101).

Next, the acquisition unit 52 acquires image identification information corresponding to each text, which corresponds to the text group ID. Specifically, the acquisition unit 52 acquires image identification information associated with each corresponding text by searching the text information management DB 5007 (see FIG. 40) using the text group ID of the text group as a search key (step S113-3-102).

Next, the setting registration unit 58 sets the public flag of the capture image managed in the capture image management DB 5004 (see FIG. 8A) and indicated by the acquired image identification information to "False" (step S113-3-103).

Next, the acquisition unit 52 searches the text information management DB 5007 (see FIG. 40) using the text group ID as a search key to acquire the start time and end time of each corresponding text (step S113-3-104).

Next, the setting registration unit 58 registers the earliest start time and the latest end time among the acquired time information in the private audio management DB 5006 (see FIG. 9) (step S113-3-105), and then exits this flow.

●Screen display example●
Fig. 44 is an example of a screen display when a summary input button is operated for a text group in a communication terminal according to the second embodiment. A record viewing and editing screen 3171 is displayed by the table control unit 34 on the display 318 of the creator's communication terminal 3. In Fig. 44, a group designation area 3172 and a "summary input" button (icon) 3561 displayed near the group designation area 3172 are displayed by the display control unit 34 for the creator's record viewing and editing screen 3151 shown in Fig. 25. The group designation area 3172 may be designated and displayed by, for example, a drag-and-drop operation of the mouse 3701 by the creator. For example, the user drags the mouse 3701 from an arbitrary start point A on the record viewing and editing screen 3171 and drops the mouse 3701 at an arbitrary end point B to designate an arbitrary area. As a result, the group designation area 3172 as shown in Fig. 44 is displayed on the record viewing and editing screen 3171. In addition, calculation/identification unit 53 and determination unit 55 compare the coordinate values of each vertex of group designation area 3172 with the coordinate values of each vertex of each text display field, and calculate and identify which text display fields are present within group designation area 3172. In this way, calculation/identification unit 53 and determination unit 55 can identify which text display fields are included in group designation area 3172.

The "summary input" button (icon) 3561 is also displayed by the display control unit 34 in response to a mouseover operation performed near the designated group designation area 3172.

●Screen display example●
Fig. 45 is an example of a screen display of a summary information input dialogue in a communication terminal according to the second embodiment. When an operation is performed on the "summary input" button (icon) 3561 shown in Fig. 44 on the display 318 of the creator's communication terminal 3, a summary input dialogue 3181 is displayed by the display control unit 34. The "summary input" dialogue 3181 is a dialogue for inputting an arbitrary summary (sentence) representing a group designation area 3172 arbitrarily designated by the creator. When the creator inputs various information such as an arbitrary word, sentence, link destination, etc. into this dialogue and operates the "register" button (icon) 3571, the various input information is confirmed and transmitted to the voice recording management device 5.

●Screen display example●
Fig. 46 is an example of a screen display including a summary display field in a communication terminal according to the second embodiment. A record viewing and editing screen 3171 is displayed by the display control unit 34 on the display 318 of the creator's communication terminal 3. In Fig. 46, for example, when a mouse-over operation is performed near the group designation area 3172, a balloon-shaped summary display field 3191 is displayed by the display control unit 34. In this summary display field 3191, the summary content (summary information) input in the "summary input" dialogue 3181 shown in Fig. 45 is displayed. Then, when the "edit" button (icon) 3581 is operated, an editing screen (dialog screen) for editing the input summary information is displayed. The editing screen (dialog screen) at this time may be the same as the summary input dialogue 3181 shown in Fig. 45. Then, the summary is re-edited to an arbitrary summary content in the summary input dialogue 3181, and when the "register" button (icon) 3571 is operated, the re-edited content is transmitted to the voice recording management device 5.

In addition, in response to a mouse-over operation performed near the group designation area 3172 shown in FIG. 46, the display control unit 34 displays a "private" button (icon) 3591. In the communication terminal 3, similar to the "private" button (icon) 3591, a "public" button (icon) and a "delete" button (icon) are displayed by the display control unit 34 according to their respective states.

<<Processing when the publish button is used on a text group>>
Next, the process when the disclosure button is operated on a text group will be described. Fig. 47 is a flowchart showing an example of the process when the disclosure button is operated on a text group according to the second embodiment. In this flowchart, the process of the circle H as the transition destination determined in Fig. 42A (steps S113-3-201 to S113-3-207) is executed.

First, the storage/reading unit 59 of the voice recording management device 5 searches the summary information management DB 5008 (see FIG. 41) using the summary content received in step S112 as a search key to read out the corresponding text group ID. Next, the setting/registration unit 58 sets the public flag of each text corresponding to the text group ID of the text group to be processed to "True." Specifically, the setting/registration unit 58 sets the public flag of each text corresponding to the read out text group ID, which is managed in the text information management DB 5007 (see FIG. 40), to "True" (step S113-3-201).

Next, the acquisition unit 52 acquires image identification information corresponding to each text, which corresponds to the text group ID. Specifically, the acquisition unit 52 acquires the corresponding image identification information by searching the text information management DB 5007 (see FIG. 40) using the text group ID as a search key (step S113-3-202).

Next, the acquisition unit 52 acquires the public flag of each text that is managed in the text information management DB 5007 (see FIG. 40) and corresponds to the captured image indicated by the acquired image identification information (step S113-3-203).

Then, the judgment unit 55 judges whether all of the acquired public flags are "True" (step S113-3-204). If all of the acquired public flags are "True" (step S113-3-204: YES), the setting registration unit 58 sets the public flag corresponding to the image identification information of the capture image indicated by the acquired image identification information, which is managed in the capture image management DB 5004 (see FIG. 8A), to "True" (step S113-3-205).

Then, the acquisition unit 52 acquires time information on the start time and end time of each text corresponding to the text group ID. Specifically, the acquisition unit 52 acquires the start time and end time of each corresponding text by searching the text information management DB 5007 (see FIG. 40) using the text group ID as a search key (step S113-3-206).

Then, the setting registration unit 58 deletes the earliest start time and the latest end time from the acquired time information from the private audio management DB 5006 (see FIG. 9) (step S113-3-207), and exits this flow.

On the other hand, if all of the obtained public flags are not "True", that is, if at least one of the obtained public flags is "False" (step S113-3-204: NO), the voice recording management device 5 executes the processing of the above-mentioned steps S113-2-206 and S113-3-207 and exits this flow.

<<Processing when the Delete button is used on a text group>>
Next, the process when the delete button is operated on a text group will be described. Fig. 48 is a flowchart showing an example of the process when the delete button is operated on a text group according to the second embodiment. In this flowchart, the process of circle I as the transition destination determined in Fig. 42A (steps S113-3-301 to S113-3-307) is executed.

First, the storage/reading unit 59 of the voice recording management device 5 searches the summary information management DB 5008 (see FIG. 41) using the summary content received in step S112 as a search key to read out the corresponding text group ID. Next, the setting/registration unit 58 sets all text corresponding to the target text group ID to blank, and sets the deletion flag to "True." Specifically, the setting/registration unit 58 sets all text corresponding to the read-out text group ID, which is managed in the text information management DB 5007 (see FIG. 40), to blank, and sets the corresponding deletion flag to "True" (step S113-3-301).

Next, the acquisition unit 52 acquires image identification information corresponding to each text, which corresponds to the target text group ID. Specifically, the acquisition unit 52 acquires the corresponding image identification information by searching the text information management DB 5007 (see FIG. 40) using the text group ID as a search key (step S113-3-302).

Next, the acquisition unit 52 acquires the deletion flag of each text that is managed in the text information management DB 5007 (see FIG. 40) and corresponds to the captured image indicated by the acquired image identification information (step S113-3-303).

Then, the judgment unit 55 judges whether all of the acquired deletion flags are "True" (step S113-3-304). If all of the acquired deletion flags are "True" (step S113-3-304: YES), the setting registration unit 58 replaces (rewrites) the image data of the capture image indicated by the acquired image identification information, which is managed in the capture image management DB 5004 (see FIG. 8A), with specified image data, and sets the corresponding deletion flag to "True" (step S113-3-305).

Then, the acquisition unit 52 acquires time information of the start time and end time corresponding to each text, which corresponds to the target text group ID. Specifically, the acquisition unit 52 acquires the start time and end time of each text corresponding to the text group to be processed by searching the text information management DB 5007 (see FIG. 40) using the text group ID as a search key (step S113-3-306).

Then, the setting registration unit 58 deletes (silences) the audio data between the earliest start time and the latest end time among the audio data managed in the record bibliographic information management DB 5002 (see FIG. 6) that corresponds to text managed in the text information management DB 5007 (see FIG. 40) and whose deletion flag is set to "True" (step S113-3-307), and exits this flow.

On the other hand, if all of the acquired deletion flags are not "True", that is, if at least one of the acquired deletion flags is "False" (step S113-3-304: NO), the setting registration unit 58 sets the deletion flag managed in the capture image management DB 5004 (see FIG. 8A) corresponding to the captured image indicated by the acquired image identification information to "False" (step S113-3-308), and executes the processes of steps S113-3-306 and S113-3-307 described above before exiting this flow.

<<Processing when pressing the private button on a captured image>>
Next, a process when the private button is operated on a captured image will be described. Fig. 49 is a flowchart showing an example of a process when the private button is operated on a captured image according to the second embodiment. In this flowchart, the process of the circle J as the transition destination determined in Fig. 42B (steps S113-4-101 to S113-4-107) is executed.

First, the setting registration unit 58 sets the public flag corresponding to the image identification information indicating the target capture image to "False." Specifically, the setting registration unit 58 sets the public flag corresponding to the image identification information indicating the target capture image, which is managed in the capture image management DB 5004 (see FIG. 8A) and for which the private button (icon) has been operated, to "False" (step S113-4-101).

Next, the acquisition unit 52 acquires each piece of text corresponding to the image identification information indicating the target capture image. Specifically, the acquisition unit 52 acquires each piece of corresponding text by searching the text information management DB 5007 (see FIG. 40) using the image identification information indicating the capture image to be processed as a search key (step S113-4-102).

Next, the calculation/identification unit 53 searches the text information management DB 5007 (see FIG. 40) using each acquired text as a search key to identify the corresponding text group (text group ID) (step S113-4-103).

Next, the judgment unit 55 judges whether the text group (text group ID) has been identified (step S113-4-104). If the text group (text group ID) has been identified (step S113-4-104: YES), the setting registration unit 58 sets the public flag of all texts that are managed in the text information management DB 5007 (see FIG. 40) and that correspond to the identified text group (text group ID) to "False" (step S113-4-105).

Next, the acquisition unit 52 searches the text information management DB 5007 (see FIG. 40) using the identified text group (text group ID) as a search key to acquire time information for the start and end times of all corresponding texts (step S113-4-106).

Then, the voice recording management device 5 executes process X (steps S113-2-103 to S113-2-105) defined in FIG. 35 (step S113-4-107) and exits this flow.

On the other hand, if the text group (text group ID) cannot be determined (step S113-4-104: NO), the voice recording management device 5 executes process X (steps S113-2-103 to S113-2-105) defined in FIG. 35 (step S113-4-107) and exits this flow.

<<Processing when the publish button is operated on a captured image>>
Next, a process when the public button is operated on a captured image will be described. Fig. 50 is a flowchart showing an example of a process when the public button is operated on a captured image according to the second embodiment. In this flowchart, the process of the circle K as the transition destination determined in Fig. 42B (steps S113-4-201 to S113-4-208) is executed.

First, the setting registration unit 58 sets the public flag corresponding to the image identification information indicating the target capture image to "True." Specifically, the setting registration unit 58 sets the public flag corresponding to the image identification information indicating the target capture image, which is managed in the capture image management DB 5004 (see FIG. 8A) and for which the public button (icon) has been operated, to "True" (step S113-4-201).

Next, the acquisition unit 52 acquires each piece of text corresponding to the image identification information indicating the target capture image. Specifically, the acquisition unit 52 acquires each piece of corresponding text by searching the text information management DB 5007 (see FIG. 40) using the image identification information indicating the target capture image as a search key (step S113-4-202).

Next, the calculation/identification unit 53 searches the text information management DB 5007 (see FIG. 40) using each acquired text as a search key to identify the corresponding text group (text group ID) (step S113-4-203).

Next, the judgment unit 55 judges whether the text group (text group ID) has been identified (step S113-4-204). If the text group (text group ID) has been identified (step S113-4-204: YES), the setting registration unit 58 sets the public flag of all texts that are managed in the text information management DB 5007 (see FIG. 40) and that correspond to the identified text group (text group ID) to "True" (step S113-4-205).

Then, the acquisition unit 52 acquires time information on the start time and end time of each text corresponding to the identified text group (text group ID). Specifically, the acquisition unit 52 acquires time information on the start time and end time of each corresponding text by searching the text information management DB 5007 (see FIG. 40) using the identified text group (text group ID) as a search key (step S113-4-206).

Next, the setting registration unit 58 deletes the earliest start time and the latest end time from the acquired time information from the private audio management DB 5006 (see FIG. 9) (step S113-4-207).

Then, the voice recording management device 5 executes process Y defined in FIG. 37 (steps S113-2-203 to S113-2-211) (step S113-4-208) and exits this flow.

On the other hand, if the text group (text group ID) cannot be determined (step S113-4-204: NO), the voice recording management device 5 executes process Y (steps S113-2-203 to S113-2-211) defined in FIG. 37 (step S113-4-208) and exits this flow.

<<Processing when the delete button is used on a captured image>>
Next, a process when the delete button is operated on a captured image will be described. Fig. 51 is a flowchart showing an example of a process when the delete button is operated on a captured image according to the second embodiment. In this flowchart, the process of the circle L as the transition destination determined in Fig. 42B (steps S113-4-301 to S113-4-308) is executed.

First, the setting registration unit 58 replaces the image data of the target capture image with a specified image, and sets the deletion flag corresponding to the image identification information indicating the target capture image to "True." Specifically, the setting registration unit 58 sets to "True" the public flag corresponding to the image identification information indicating the capture image that is managed in the capture image management DB 5004 (see FIG. 8A) and is the target of the process for which the private button (icon) was operated (step S113-4-301).

Next, the acquisition unit 52 acquires each piece of text corresponding to the image identification information indicating the capture image to be processed. Specifically, the acquisition unit 52 acquires each piece of corresponding text by searching the text information management DB 5007 (see FIG. 40) using the image identification information indicating the target capture image as a search key (step S113-4-302).

Next, the calculation/identification unit 53 searches the text information management DB 5007 (see FIG. 40) using each acquired text as a search key to identify the corresponding text group (text group ID) (step S113-4-303).

Next, the judgment unit 55 judges whether the text group (text group ID) has been identified (step S113-4-304). If the text group (text group ID) has been identified (step S113-4-304: YES), the setting registration unit 58 sets all text managed in the text information management DB 5007 (see FIG. 40) and corresponding to the identified text group (text group ID) to blank, and sets the deletion flag to "True" (step S113-4-305).

Then, the acquisition unit 52 acquires time information on the start time and end time of each text corresponding to the identified text group (text group ID). Specifically, the acquisition unit 52 acquires time information on the start time and end time of each corresponding text by searching the text information management DB 5007 (see FIG. 40) using the identified text group (text group ID) as a search key (step S113-4-306).

Next, the setting registration unit 58 deletes (silences) the audio data between the earliest start time and the latest end time of each acquired piece of time information from the recorded bibliographic information management DB 5002 (see Figure 6) (step S113-4-307).

Then, the voice recording management device 5 executes process Z (steps S113-2-303 to S113-2-312) defined in FIG. 38 (step S113-4-308) and exits this flow.

On the other hand, if the text group (text group ID) cannot be determined (step S113-4-304: NO), the voice recording management device 5 executes process Z (steps S113-2-303 to S113-2-312) defined in FIG. 38 (step S113-4-308) and exits this flow.

[Major Effects of the Second Embodiment]
As described above, according to this embodiment, the voice recording management system collectively sets text data to be edited, including non-disclosure, disclosure, and deletion. This provides the effect of the first embodiment, as well as the effect of making it possible to collectively select one or more pieces of text data to be edited and perform editing, thereby further improving the operability of editing voice recordings.

Furthermore, according to this embodiment, since summary content is set for a text group that contains one or more pieces of text to be edited, it is expected that the user will be able to easily select the editing content according to the summary content. In other words, by referring to the summary content that combines one or more pieces of text, the user will be able to easily decide whether to make public or private the part of the audio recording that is the subject of the editing process, and even whether to delete the audio recording of the summarized part itself.

Third Embodiment
Next, the third embodiment will be described with reference to Figs. 52 to 68. In the third embodiment, a record viewing and editing process (record viewing and editing screen division process) is performed on a post-edit screen shown by post-edit screen data including post-edit text data obtained by editing predetermined text data and post-edit image data obtained by editing predetermined image data, in which the record viewing and editing process is divided at a predetermined date and time. Each hardware resource constituting the communication system 1 according to the third embodiment is the same as that of the first embodiment. Here, Fig. 52 is a diagram showing an example of the functional configuration of the communication system according to the third embodiment. In the communication system according to the third embodiment, the functional configuration of each function that operates in cooperation with each hardware resource is the same as that of the first embodiment. Therefore, the sequence diagram related to the generation process (steps S101-S106) of the record viewing and editing screen shown in Fig. 27 is also applied to the third embodiment.

However, in the third embodiment, the contents of each data table described below differ from those in the first embodiment, so the differences will be explained below.

●Record bibliographic information management table●
Fig. 53 is a conceptual diagram showing an example of a record bibliographic information management table. In the storage unit 5000, a record bibliographic information management DB 5302 configured by the record bibliographic information management table shown in Fig. 53 is constructed. The difference from the record bibliographic information management DB 5002 in the first embodiment is that the items "Participants" and "Meeting Notes" are added to each tab of each record identification information. Note that the contents displayed in the summary (topic) column in the record viewing and editing screen shown in Fig. 58 etc. described later correspond to the contents of the "Record Name" item managed in the record bibliographic information management table.

●Text information management table●
FIG. 54 is a conceptual diagram showing an example of a text information management table according to the third embodiment. In the storage unit 5000, a text information management DB 5303 configured by a text information management table as shown in FIG. 54 is constructed. The difference from the text information management DB 5003 in the first embodiment is that a bookmark item is added to each tab of each record identification information. This bookmark is a flag associated by a user operating a bookmark button (icon) displayed near each text on the record viewing and editing screen. For example, this bookmark is normally managed with a value (flag) of "False", and is managed with a value (flag) of "True" by a user operating a bookmark button (icon) for a specific text as described above.

●Capture image management table●
Fig. 55 is a conceptual diagram showing an example of a capture image management table according to the third embodiment. A capture image management DB 5304 configured by the capture image management table as shown in Fig. 54 is constructed in the storage unit 5000. The difference from the capture image management DB 5004 in the first embodiment is that the disclosure flag and deletion flag items have been deleted for each tab of each record identification information.

<<Functional configuration of communication terminal>>
In the third embodiment, details regarding the following functions are added to the communication terminal 3, and these functions will be described in detail.

The display control unit 34 of the communication terminal 3 is mainly realized by the processing of the CPU 301 on the display 318, and controls the display of various screens and information (data) on the communication terminal 3. The display control unit 34 also causes the display 318 to display a display screen created by HTML or the like using a browser, for example. The display control unit 34 also displays a split edited screen related to the split edited screen data transmitted by the voice recording management device 5 on the display 318. The display control unit 34 also displays the split edited screen on the display 318, including bookmark information contained in the split edited screen data transmitted by the voice recording management device 5. In this embodiment, the display control unit 34 functions as an example of a display control means.

The audio playback unit 36 is mainly realized by the processing of the CPU 301 on the speaker 316 and the audio input/output I/F 317, and plays audio information (audio data) or sound information (sound data) for the user using the communication terminal 3. The audio playback unit 36 also plays the divided and edited audio related to the divided and edited audio data transmitted by the audio recording management device 5. When the end date and time of an utterance that started at a date and time before the division date and time at which the specified text data was divided and closest to the division date and time straddles the division date and time, the audio playback unit 36 plays the divided and edited audio related to the divided and edited audio data transmitted by the audio recording management device 5, with the end date and time of the utterance as the division date and time. In this embodiment, the audio playback unit 38 functions as an example of an audio playback means.

<<Functional configuration of the voice recording management device>>
In the third embodiment, details regarding the following functions are added to the voice recording management device 5, and these functions will be described in detail.

The transmission/reception unit 51 of the voice recording management device 5 is mainly realized by the processing of the CPU 501 for the short-range communication I/F 508 and the network I/F 511, and transmits and receives various data (or information) to and from the communication terminal 3 via the communication network 100. The transmission/reception unit 51 also receives a split operation instruction request for splitting a predetermined area displayed on the post-edit screen as an editing request transmitted by the communication terminal 3. In response to the split operation instruction request, the transmission/reception unit 51 also transmits to the communication terminal 3 split-edited screen data including split-edited text data obtained by splitting the predetermined text data and split-edited image data obtained by splitting the predetermined image data. In response to the area division request, the transmission/reception unit 51 also transmits to the communication terminal 3 predetermined bookmark information including the predetermined text data contained in the split-edited screen data. The transmission/reception unit 51 further transmits to the communication terminal 3 split-edited audio data obtained by splitting the predetermined audio data. In this embodiment, the transmission/reception unit 51 functions as an example of at least one of the transmission means and the reception means.

<Record (event) selection process>
Next, a record (event) selection process according to the third embodiment will be described. As described above, in the third embodiment, the process for generating a record viewing and editing screen (steps S101-S106) that is applied in the same manner as in the first embodiment is assumed to have been executed in advance, and therefore a description thereof will be omitted here.

●Screen display example●
FIG. 56 is an example of a screen display when selecting a record in a communication terminal according to the third embodiment. The display control unit 34 displays a record selection screen 3341 on the display 318 of the communication terminal 3 by executing the process of step S89 described above. The record selection screen 3341 displays, for example, a date indicating the recorded contents (minutes, etc.) of an event such as a meeting, an event title, and a minutes share button (icon) 3535 in a selectable display form as a set. As a result, for example, a user of the communication terminal 3 (A) that creates minutes can operate and select the minutes share button (icon) 3535 associated with a specific event represented by an arbitrary date and event title. In this embodiment, a case is shown in which the minutes share button (icon) 3535 corresponding to the event "Healthcare Business Performance Report Meeting" given date information of "2021/3/31 11:00:00-12:00:00" is selected by the user. Specifically, when the user of the communication terminal 3(A) places the mouse pointer (cursor) 3701 over a specific event title in the record selection screen 3341 by mouse over operation, a minutes share button (icon) 3535 corresponding to the event title overlaid by the mouse pointer (cursor) 3701 is displayed. The user of the communication terminal 3(A) can then access a dialogue including a specific URL and a passcode by clicking or otherwise operating the minutes share button (icon) 3535 (the dialogue screen is similar to that of the first embodiment, and therefore description thereof is omitted). The user of the communication terminal 3(A) can access the record viewing and editing screen by inputting a specific URL and a passcode into this dialogue, as in the first embodiment.

<Editing process on the record viewing screen>
Next, the editing process of the record viewing screen according to the third embodiment will be described. Fig. 57 is a sequence diagram showing an example of the editing process of the record viewing screen according to the third embodiment. First, the display control unit 34 of the communication terminal 3 displays the record viewing and editing screen generated in step S106 on the display 318, and the operation acceptance unit 32 accepts a split operation as an example of an editing operation instruction by the user (creator) (step S311). Note that the "split" process in the third embodiment can also be treated as an example of the "edit" process shown in the first and second embodiments.

●Screen display example●
FIG. 58 is a screen display example of the record viewing and editing screen of the creator's communication terminal according to the third embodiment. The display control unit 34 displays the record viewing and editing screen 3361 on the display 318 of the communication terminal 3 by executing the process of step S311 (similar to the process of S106) described above. The record viewing and editing screen 3361 displays, for example, each input field of "Summary (topic)", "Participants", and "Meeting notes", an audio playback display section 3601, and a bookmark display field 3603, similar to the record viewing and editing screen 3151 shown in FIG. 25. The record viewing and editing screen 3361 further displays at least one or more text display fields. The record viewing and editing screen 3361 further displays screens 3, 4, and 5 captured by the screen capture process in association with each text displayed in each text display field for each capture time. The user of the communication terminal 3 on which the record viewing and editing screen 3361 is displayed is, for example, "Rikotaro", the creator of the audio record (minutes).

On the record viewing and editing screen 3361, for example, when the mouse 3701 is placed near each text display field (mouse-over operation), the display control unit 34 displays a "Play audio" button (icon) 3543, a "Delete" button (icon) 3544, and a "Bookmark" button (icon) 3602 near the text display field where the mouse was over. The "Bookmark" button (icon) 3602, which is newly provided in the third embodiment, is a button (icon) for displaying the desired text selected by the user as a bookmark in the bookmark display field 3603. By operating the "Bookmark" button (icon) 3602, the user can display the specific text selected by the user in the bookmark display field 3603.

When the user operates (clicks) the "Bookmark" button (icon) 3602, the communication terminal 3 transmits the text identification information, record identification information, and bookmark request to the voice storage management device 5, as described below. The voice storage management device 5 refers to the text information management DB 5303 (see FIG. 53) based on the text identification information and record identification information, identifies the text, and sets the item value of the "Bookmark" item to "true". The voice storage management device 5 then transmits a notification to the communication terminal 3 that the setting was successful. By receiving this notification, the communication terminal 3 is able to switch the display mode of the "Bookmark" button (icon) 3602 for the text from off (OFF) to on (ON), and display it in the bookmark display field 3603.

Furthermore, on the record viewing and editing screen 3361, when the mouse 3701 is placed (mouse-over operation) near a dividing line (date and time display area (e.g., "11:02:00")) that is a predetermined time interval (e.g., 30 seconds) that separates text groups and the images corresponding to those text groups, a split button (icon) 3604 in the shape of scissors, for example, is displayed. In this state, the user can, for example, click on the split button (icon) 3604, which causes the display control unit 34 to pop up a conference log split dialog box, which will be described next, near the split button (icon) 3604.

As an example of specific processing, in the audio recording management device 5, the split button (icon) 3604 and the date and time indicated by the corresponding dividing line (date and time display section) are associated in the screen data of the screen. When the user clicks the split button (icon) 3604 in this state, the communication terminal 3 acquires the date and time information corresponding to this split button (icon) 3604. Then, when the user clicks the "Split" button (icon) in the dialog (step S311), the acquired split date and time information, the recording identification information of the audio recording, and a split operation instruction request are sent to the audio recording management device 5 (step S312).

●Screen display example●
59 is another example of a display screen of the record viewing and editing screen of the creator's communication terminal according to the third embodiment. When an operation is performed on the split button (icon) 3604 on the display 318 of the communication terminal 3, the display control unit 34 pops up a conference log split dialog 3362 near the split button (icon) 3604. The user can perform an operation such as clicking the "Split" button (icon) of the conference log split dialog 3362 with the mouse 3701.

Returning to FIG. 57, the transmission/reception unit 31 transmits a split operation instruction request to the audio recording management device 5 (step S312). As a result, the transmission/reception unit 51 of the audio recording management device 5 receives the split operation instruction request transmitted by the communication terminal 3. At this time, the split operation instruction request includes at least one of the following three pieces of information. One of them includes text identification information that identifies the target text, image identification information that identifies the target image, and split operation button information performed on the split operation button described above. Note that the split operation instruction request is an example of a request to perform split processing on the record viewing screen that is transmitted by the communication terminal 3 to the audio recording management device 5.

Next, the audio recording management device 5 transmits a split operation instruction response as a response to the split operation instruction request received in step S312 (step S313). As a result, the transmission/reception unit 31 of the communication terminal 3 receives the split operation instruction response transmitted by the audio recording management device 5. At this time, the split operation instruction response includes the acceptance ID of the split operation instruction. This allows the communication terminal 3 to display a dialog screen shown in FIG. 60 (described later) on the display 318.

Next, the operation acceptance unit 32 accepts the operation of the "OK" button on the dialog screen (step S314). Specifically, the operation acceptance unit 32 accepts the operation of the "OK" button by the user on the dialog screen displayed by the voice recording management device 5 based on the acceptance ID of the split operation instruction transmitted and received in step S313. When the user operates the "OK" button, the display control unit 34 erases the split start notification dialog 3363 and the conference log split dialog 3362 via the operation acceptance unit 32 and transitions to the original screen. In the process of step S314, the communication terminal 3 may start a predetermined application and display the dialog screen generated by the communication terminal 3. In the process of step S314, the communication terminal 3 may further display the screen data related to the dialog screen transmitted by the voice recording management device 5 in step S313 on the communication terminal 3.

●Screen display example●
Fig. 60 shows another example of the record viewing and editing screen of the creator's communication terminal according to the third embodiment. When the "Split" button (icon) shown in Fig. 58 is operated on the display 318 of the communication terminal 3, the display control unit 34 pops up a split start notification dialog 3363 so as to be superimposed on the conference log split dialog 3362. This allows the user to perform an operation such as clicking the "OK" button (icon) of the split start notification dialog 3363 with the mouse 3701.

Returning to FIG. 57, the voice recording management device 5 performs a split process of the record viewing and editing screen (step S321). Here, the split process of the record viewing and editing screen includes the split process described above. In addition, in this split process of the record viewing and editing screen, the record bibliographic information management DB 5302 (see FIG. 53), the text information management DB 5303 (see FIG. 54), and the capture image management DB 5304 (see FIG. 55) are each used. Note that the process of step S321 is executed asynchronously with the process of step S314 described above, so it does not matter which process is executed first.

<<Details on splitting the record viewing and editing screen>>
Here, the details of the division process of the record viewing and editing screen in step S321 will be described. Fig. 61 is a flow chart showing an example of the division process of the record viewing and editing screen according to the third embodiment. First, the transmission/reception unit 51 of the voice recording management device 5 receives the recording identification information of the voice recording, the division time, and a division request (step S321-1). Note that the process of step S321-1 corresponds to the process of step S312 described above, and the division request includes text identification information, image identification information, and division operation button information.

The calculation identification unit 53 then identifies the text information management table (text information management DB 5303 (see FIG. 54)) that corresponds to the record identification information, and identifies records whose start times are before the division time and records whose start times are after the division time (step S321-2). Specifically, the calculation identification unit 53 searches the text information management DB 5303 using the text identification information and the division time as search keys, thereby identifying records whose start times are before the division time and records whose start times are after the division time.

Next, the setting registration unit 58 updates and registers the text information management table based on each (step S321-3). Specifically, the setting registration unit 58 updates and registers two text information management tables (text information management DB 5303) associated with the newly set tabs for the pre-split record identification information and the post-split record identification information based on the records with start times before the split time and the records with end times after the split time identified in step S321-2.

Figure 62 is a conceptual diagram showing an example of a text information management table after the split processing of the record viewing and editing screen according to the third embodiment, where (a) is a conceptual diagram of the text information management table constituting the first split voice recording screen, and (b) is a conceptual diagram of the text information management table constituting the second split voice recording screen. Figure 62 (a) shows the text information management table associated with the tab of the newly set post-split record identification information "R5301". Specifically, the end time corresponding to the text identification information "TX3006" is updated to "2 minutes 00 seconds" by the setting registration unit 58. At this time, "2 minutes 00 seconds" is the time corresponding to the date and time information when split at the date and time display unit ("11:02:00") shown in Figure 58. Note that all of the bookmark items corresponding to the text information identification information are set to "False", indicating that the bookmark processing was not performed for the spoken content managed in this table.

In FIG. 62(b), the text information management table associated with the tab of the newly set post-division record identification information "R5302" is shown. Specifically, the start time and end time corresponding to each of the text identification information "TX3007", "TX3008", "TX3009", ... are updated and registered by the setting registration unit 58. More specifically, the start time corresponding to the text identification information "TX3007" is "0 minutes 5 seconds" and the end time is "0 minutes 12 seconds". These times represent the start date and time (start time) when the speech of the content "Please look at the shared screen first" started and the end date and time (end time) when the speech of the content "Please look at the shared screen first" ended. Similarly, the start time corresponding to the text identification information "TX3008" is "0 minutes 18 seconds" and the end time is "0 minutes 23 seconds". These times represent the times when the speech of the content "Sales for the second half of fiscal year 2020 are XX" started and ended. Similarly, the start time corresponding to text identification information "TX3009" is "0 minutes 25 seconds" and the end time is "0 minutes 28 seconds." These times represent the times when the speech of "Operating profit is △△" started and ended. Note that, among the bookmark items corresponding to text information identification information, the bookmark corresponding to text identification information "TX3008" is set to "True." In other words, this indicates that for the speech content managed in this table, bookmark processing was performed on the text corresponding to text identification information "TX3008."

Returning to FIG. 61, the calculation identification unit 53 identifies the record identification information from the capture image management table (capture image management DB 5304 (see FIG. 55)) and identifies records whose acquisition time is before the division time and records whose acquisition time is after the division time (step S321-4). Specifically, the calculation identification unit 53 searches the capture image management DB 5304 using as a search key the acquisition time corresponding to the date and time displayed in the date and time display unit when the division button (icon) 3604 is operated, thereby reading and identifying the corresponding image data path. At this time, the calculation identification unit 53 calculates the actual start time based on, for example, the start date and time and the elapsed time in the record bibliographic information management DB 5002 (see FIG. 53). This is because the elapsed time from the start time of an event such as a meeting is recorded and managed in the text information management DB 5303 (see FIG. 54) and the capture image management DB 5304 (see FIG. 55), and the calculation identification unit 53 calculates the actual start time based on the start date and time and the elapsed time in the record bibliographic information management DB 5002.

Then, the setting registration unit 58 updates and registers the text information management table based on each (step S321-5). Specifically, the setting registration unit 58 updates and registers two capture image management tables (capture image management DB 5304) associated with the newly set tabs for the record identification information before the division and the record identification information after the division based on the records before the division time identified in step S321-4 and the records after the division time.

Figure 63 is a conceptual diagram showing an example of a capture image management table after splitting processing of the record viewing and editing screen according to the third embodiment, where (a) is a conceptual diagram of a capture image management table constituting the first split voice recording screen, and (b) is a conceptual diagram of a capture image management table constituting the second split voice recording screen. Figure 63(a) shows a capture image management table associated with the tab of the newly set post-split record identification information "R5301". Specifically, it shows a state in which each acquisition time and each image data path corresponding to image identification information "IM0003", "IM0004", ... are inherited from the contents previously registered by the setting registration unit 58.

Figure 63(b) shows the capture image management table associated with the tab for the newly set post-split recording identification information "R5302". Specifically, the acquisition time corresponding to the image identification information "IM0005" is updated to "0 minutes 00 seconds" by the setting registration unit 58. More specifically, for the second split audio recording, since the date and time display showing the split date and time showed "11:02:00", the acquisition time "2 minutes 00 seconds" corresponding to this date and time is updated to "0 minutes 00 seconds" by the setting registration unit 58.

Returning to FIG. 61, the calculation and identification unit 53 identifies the recorded bibliographic information corresponding to the record identification information from the recorded bibliographic information management table (recorded bibliographic information management DB 5302 (see FIG. 53)) (step S321-6). Specifically, since the date and time display portion showing the split date and time shows "11:02:00", the calculation and identification unit 53 identifies the tab of the recorded bibliographic information having the recorded bibliographic information including this date and time.

Next, the acquisition unit 52 acquires the audio data from the audio data path (step S321-7). Specifically, since the date and time display portion showing the split date and time indicates "11:02:00", the acquisition unit 52 acquires the audio data path managed in the tab of the record bibliographic information having the record bibliographic information including this date and time. In this case, the acquired audio data path is, for example, ".../00005006/record.mp3".

Next, the generation/processing unit 57 generates partial audio data before the division time and partial audio data after the division time, and generates event URLs corresponding to each (step S321-8).

Next, the setting registration unit 58 updates and registers the recorded bibliographic information management table based on the path of the audio data and the event URL, and then exits this flow (step S321-9).

Figure 64A is a conceptual diagram showing an example of a record bibliographic information management table after the division process of the record viewing and editing screen according to the third embodiment, and is a conceptual diagram of the record bibliographic information management table constituting the first divided voice recording screen. In Figure 64A, the record bibliographic information management table associated with the tab of the newly set post-division record identification information "R5301" is shown. Specifically, the setting registration unit 58 updates the record name to "Division 1_Healthcare Business Performance Reporting Meeting". Furthermore, the setting registration unit 58 updates the end date and time to "2021/03/31 11:02:00". Furthermore, the setting registration unit 58 updates the voice data path to ".../00005301/record.mp3". Furthermore, the setting registration unit 58 updates the event URL to "https://.../00005301".

Figure 64B is a conceptual diagram showing an example of a record bibliographic information management table after the division process of the record viewing and editing screen according to the third embodiment, which is a conceptual diagram of the record bibliographic information management table constituting the second divided audio recording screen. In Figure 64B, the record bibliographic information management table associated with the tab of the newly set post-division record identification information "R5302" is shown. Specifically, the setting registration unit 58 updates the record name to "Division 2_Healthcare Business Performance Reporting Meeting". Furthermore, the setting registration unit 58 updates the start date and time to "2021/03/31 11:02:00". Furthermore, the setting registration unit 58 updates the audio data path to ".../00005302/record.mp3". Furthermore, the setting registration unit 58 updates the event URL to "https://.../00005302".

The divided voice recording screen including each record identified and updated in step S321 described above is displayed on the communication terminal 3 when the user inputs the event URL indicating the desired divided event and the passcode (when the processing of step S101 described above is executed).

<Recording selection screen display after division>
Next, the selection of the divided records (events) and the display of each selected selection screen will be described.

●Screen display example●
FIG. 65 is an example of a screen display when selecting a record after the division process of the record viewing and editing screen in the communication terminal according to the third embodiment. By executing the process of step S89 described above, the display control unit 34 displays a record selection screen 3341 on the display 318 of the communication terminal 3. In addition to the contents described in FIG. 56, the record selection screen 3341 according to the third embodiment newly displays two divided events. In this embodiment, two event titles, "Division 1_Healthcare Business Performance Report Meeting" given the date information of "2021/3/31 11:00:00-11:02:00" and "Division 2_Healthcare Business Performance Report Meeting" given the date information of "2021/3/31 11:02:00-11:02:00", and the corresponding minutes sharing buttons (icons) are newly displayed. As a result, the user of the communication terminal 3(A) who creates the minutes can operate the minutes share button (icon) associated with the event title covered by the mouse pointer (cursor) 3701 by hovering the mouse over any of the event titles of the newly displayed events. The user of the communication terminal 3(A) can access a dialogue including a predetermined URL and a passcode by operating the minutes share button (icon) 3535 of a desired event by clicking or the like. Note that the dialogue screen that is displayed as a pop-up is the same as that of the first embodiment, and therefore a description thereof is omitted. The user of the communication terminal 3(A) can access the record viewing and editing screen by inputting a predetermined URL and a passcode into this dialogue, as in the first embodiment. In addition, the user can open the voice recording screen of the operated event by operating (clicking, tapping, etc.) the desired event title. FIG. 65 shows an example in which an operation is performed by a user on the event "Division 1_Healthcare Business Performance Reporting Meeting". Here, the audio recording management device 5 may manage the identification information of the user who operated the minutes share button (icon) and the users who participated in the divided audio recordings, thereby restricting the user who operated the minutes share button (icon) from viewing the event related to the divided minutes. Specifically, for example, if the user who operated the minutes share button (icon) did not participate in the event indicated in the divided minutes, the audio recording management device 5 may prohibit the user from viewing the divided minutes. Note that in this embodiment, the division of the audio recording related to the event is not limited to two, and it may be divided into three or more parts.

●Screen display example●
FIG. 66 is a screen display example of the first divided voice recording screen displayed on the communication terminal of a user other than the creator according to the third embodiment. When the user operates the event title "Division 1_Healthcare Business Performance Reporting Meeting" described above, the display control unit 34 displays a voice recording screen 3361 on the display 318 of the communication terminal 3. On the voice recording screen 3361, as in the record viewing and editing screen 3361 shown in FIG. 58, for example, each input field of "Summary (topic)", "Participants", and "Meeting Notes", a voice playback display section, and a bookmark display section 3603 are displayed. On the record viewing and editing screen 3361, at least one text display section is further displayed. On the record viewing and editing screen 3361, each text before the date and time display section of 11:01:30 shown in FIG. 58, and screens 3 and 4 associated with those texts are displayed. Note that the user of the communication terminal 3 on which the record viewing and editing screen 3361 is displayed is, for example, "Ebina Jiro", who is the viewer of the voice recording (minutes). It should be noted that the bookmark display field 3603 included in the voice recording screen 3361 indicates that there is no content to be displayed.

●Screen display example●
FIG. 67 is another example of a screen display when selecting a record after the division process on the record viewing and editing screen in the communication terminal according to the third embodiment. The display contents displayed in FIG. 67 are the same as those displayed on the record selection screen 3341 described in FIG. 65, and the user can perform the same procedure, so the description will be omitted. FIG. 67 shows an example in which the user performs an operation on the event "Division 2_Healthcare Business Performance Reporting Meeting". Here, the voice recording management device 5 may restrict the user who operated the minutes sharing button (icon) from viewing the event related to the divided minutes by managing the identification information of the user who operated the minutes sharing button (icon) and the users who participated in the divided voice recording. Specifically, for example, if the user who operated the minutes sharing button (icon) did not participate in the event shown in the divided minutes, the voice recording management device 5 may prohibit the user from viewing the divided minutes. Note that in this embodiment, the division of the voice recording related to the event is not limited to two, and may be three or more divisions.

●Screen display example●
FIG. 68 is a screen display example of the second divided voice recording screen displayed on the communication terminal of a user other than the creator according to the third embodiment. When the user operates the event title "Division 2_Healthcare Business Performance Reporting Meeting" described above, the display control unit 34 displays a voice recording screen 3361 on the display 318 of the communication terminal 3. On the voice recording screen 3361, as in the record viewing and editing screen 3361 shown in FIG. 58, for example, each input field of "Summary (topic)", "Participants", and "Meeting Notes", a voice playback display section, and a bookmark display section 3603 are displayed. On the record viewing and editing screen 3361, at least one text display section is further displayed. On the record viewing and editing screen 3361, each text after the date and time display section of 11:02:00 shown in FIG. 58, and screens 3 and 4 associated with those texts are displayed. Note that the user of the communication terminal 3 on which the record viewing and editing screen 3361 is displayed is, for example, "Ebina Jiro", who is the viewer of the voice recording (minutes). In addition, in the voice recording screen 3361 shown in Fig. 68, the text "Sales for the second half of fiscal year 2020 are XX" spoken on "2021/03/31 11:02:18" is displayed as a bookmark in the bookmark display field 3603. This is because the bookmarked part is included in the event "Division 2_Healthcare Business Performance Reporting", and is reflected on this screen.

As described above, the third embodiment is characterized in that bookmarks are also divided as a result of dividing the screen after editing. Furthermore, the communication terminal 3 can play back the divided audio related to the divided audio data transmitted by the audio recording management device 5 together with the display of the divided image.

[Major Effects of the Third Embodiment]
As described above, according to this embodiment, the voice recording management device 5 of the voice recording management system 2 receives a split operation instruction request for splitting a predetermined area displayed on the record viewing and editing screen as an editing request transmitted from the communication terminal 3 (S312), and transmits split edited screen data including split edited text data obtained by split processing of the predetermined text data and split edited image data obtained by split processing of the predetermined image data to the communication terminal 3 in response to the split operation instruction request (S105). After that, the communication terminal 3 displays the split edited screen (record viewing and editing screen) related to the split edited screen data transmitted from the voice recording management device 5 on the display 318 of the communication terminal 3 (S106). In this way, in addition to the effects of the first embodiment, it is possible to split and edit a screen that is unrelated to the participants of a meeting, etc., on the voice recording screen on which the created meeting minutes, etc. are displayed, and only the necessary information can be viewed by the participants who need that information. As a result, it is possible to improve the confidentiality of the display screen data displayed on the voice recording screen, the voice data played back, etc.

Furthermore, according to this embodiment, the voice recording management device 5 transmits each piece of split edited screen data including the specified bookmark information contained in each piece of split edited screen data, and the communication terminal 3 displays the bookmarks associated with the text and screens contained in each piece of split edited screen data in the bookmark display field 4603. This makes it possible to display or hide the bookmarked text in accordance with the split screens, in addition to the effect of the first embodiment. As a result, it is expected that the effect will be that it will be possible to further improve the convenience of editing on the voice recording screen for the user.

[Modification of the third embodiment]
Next, a modified example of the third embodiment will be described with reference to Fig. 69 to Fig. 71. In the third embodiment, the voice data is divided based on the division time, but if the utterance immediately before the division date and time (time) straddles the division date and time (time), the voice data accompanying the utterance will be divided in the middle of the utterance. On the other hand, the text data is divided based on the start date and time (time), so that the voice data and the text data cannot be matched.

Therefore, in a modified example of the third embodiment, if the end date and time (end time) of the text (audio) at the break on the record viewing and editing screen is later than the division date and time (division time), the division date and time (division time) is changed to the end date and time (end time) of the text (audio), and the text data, image data, and audio data are divided at the changed end date and time (end time). In this way, audio data can be appropriately divided without being interrupted even during the division process on the record viewing and editing screen. Note that, like the third embodiment, the modified example of the third embodiment is realized by the system configuration, hardware configuration, and each hardware resource used in the first embodiment, and each functional configuration using each hardware resource.

<<Details on splitting the record viewing and editing screen>>
Here, details of the modified example of the split process of the record viewing and editing screen in step S321 will be described. Fig. 69 is a flow chart showing an example of the split process of the record viewing and editing screen according to the modified example of the third embodiment. First, the transmission/reception unit 51 of the voice recording management device 5 receives the recording identification information of the voice recording, the split time, and a split request (step S321-21). Note that the process of step S321-1 corresponds to the process of step S312 described above, and the split request includes text identification information, image identification information, and split operation button information.

Next, the acquisition unit 52 identifies the text information management table (text information management DB 5303 (see FIG. 54)) corresponding to the record identification information, and acquires the end time information of the last record among the records whose start time is before the division time (step S321-22). Specifically, the acquisition unit 52 references the record of record identification information "R5006" from the text information management DB 5303 (see FIG. 54), and acquires the end time information of the last record among the records whose start time is before the division time. That is, the acquisition unit 52 acquires the time information of the end time "2 minutes 02 seconds" of the record of "Today's agenda is the performance of the healthcare business," which was uttered at a start time of "1 minute 59 seconds."

Next, the judgment unit 55 judges whether the acquired end time is later than the division time (step S321-23). If the acquired end time is later than the division time (step S321-23: YES), the calculation determination unit 53 sets the acquired end time as the division time and proceeds to the next step S321-25 (step S321-24). Specifically, the calculation determination unit 53 sets the "2 minutes 02 seconds" acquired in step S321-22 as the division time.

On the other hand, if the acquired end time is not later than the division time (step S321-23: NO), the process proceeds to step S321-25.

In step S321-25, the calculation and identification unit 53 identifies the text information management table (text information management DB 5303 (see FIG. 54)) that corresponds to the record identification information, and identifies records whose start times are before the division time and records whose start times are after the division time. This process is similar to the process of step S321-2 in the flowchart described in FIG. 61, so detailed description is omitted. As described above, in the text information management DB 5303 shown in FIG. 54, the end time of the utterance "Today's topic is the performance of the healthcare business" is "2 minutes and 2 seconds", which is two seconds later than the division time "11:02:00". As a result, "2 minutes and 2 seconds" is adopted as the division time obtained in step S321-24, in accordance with the process of step S321-22. In other words, in the subsequent processes, an offset (guaranteed time) of "2 seconds" is provided from the time of division processing to the end of the voice data.

Next, the setting registration unit 58 updates and registers the text information management table based on each of the settings (step S321-26). This process is similar to the process of step S321-3 in the flowchart described in FIG. 61, so a detailed description is omitted.

Figure 70 is a conceptual diagram showing an example of a text information management table after the division process of the record viewing and editing screen according to the modified example of the third embodiment, where (a) is a conceptual diagram of the text information management table constituting the first divided voice recording screen, and (b) is a conceptual diagram of the text information management table constituting the second divided voice recording screen. Figure 70(a) shows the text information management table associated with the tab of the newly set post-division record identification information "R5311". Specifically, the calculation identification unit 53 adds "2 seconds", which corresponds to the offset set in step S321-24, to the end time corresponding to the text identification information "TX3106". As a result, the setting registration unit 58 updates the end time to "2 minutes 02 seconds". At this time, "2 minutes 02 seconds" is the time obtained by adding the offset amount "2 seconds" to the date and time (time) divided in the date and time display unit ("11:02:00") shown in Figure 58. This process ensures that the audio "Today's topic is the performance of the healthcare business" will be heard to the end without any interruption, even if the split process is performed on the record viewing and editing screen. Note that all bookmark items corresponding to the text information identification information are set to "False," which indicates that bookmark processing was not performed on the spoken content managed in this table.

Figure 70 (b) shows the text information management table associated with the tab of the newly set post-division record identification information "R5312". Specifically, the start time and end time corresponding to each of the text identification information "TX3107", "TX3108", "TX3109", ... are updated and registered by the setting registration unit 58. More specifically, the start time corresponding to the text identification information "TX3107" is "0 minutes 3 seconds" and the end time is "0 minutes 10 seconds". These times represent the new times at which the speech of the content "Please look at the shared screen first" started and ended. In other words, even if the speech of the content "Please look at the shared screen first" actually started at "2 minutes 05 seconds", the offset of "2 seconds" to guarantee the previous speech content has already passed at the stage of the division process, so the voice recording management device 5 sets the new speech start time to "2 minutes 03 seconds", which is obtained by subtracting 2 seconds from "2 minutes 05 seconds". Similarly, for the speech end time, "2 minutes 12 seconds" is subtracted and "2 minutes 10 seconds" is set as the new speech end time. However, in the text information management table managed in the tab of the record identification information "R5302" of the text information management DB 5303, since there is no previous speech, "0 minutes 03 seconds" is set as the new speech start time and "0 minutes 10 seconds" is set as the new speech end time. Similarly, the start time corresponding to the text identification information "TX3108" is "0 minutes 16 seconds" and the end time is "0 minutes 21 seconds". These times represent the new times at which the speech of "Sales for the second half of fiscal year 2020 are XX" started and ended. Similarly, the start time corresponding to the text identification information "TX3109" is "0 minutes 23 seconds" and the end time is "0 minutes 26 seconds". These times represent the new times at which the speech of "Operating profit is △△" started and ended. Among the bookmark items corresponding to text information identification information, the bookmark corresponding to the text identification information "TX3108" is set to "True." This indicates that for the spoken content managed in this table, bookmark processing was performed on the text corresponding to the text identification information "TX3108."

Returning to FIG. 69, the calculation identification unit 53 identifies the record identification information from the capture image management table (capture image management DB 5304 (see FIG. 55)) and identifies records whose acquisition times are before the division time and records whose acquisition times are after the division time (step S321-27). This process is similar to the process in step S321-4 of the flowchart described in FIG. 61, so a detailed description will be omitted.

Then, the setting registration unit 58 updates and registers the text information management table based on each (step S321-28). Specifically, the setting registration unit 58 updates and registers two capture image management tables (capture image management DB 5304) associated with the newly set tabs for the record identification information before the division and the record identification information after the division based on the records before the division time identified in step S321-4 and the records after the division time.

Next, the calculation and identification unit 53 identifies the recorded bibliographic information corresponding to the record identification information from the recorded bibliographic information management table (recorded bibliographic information management DB 5302 (see FIG. 53)) (step S321-29). Specifically, since the date and time display portion showing the split date and time shows "11:02:00", the calculation and identification unit 53 identifies the tab of the recorded bibliographic information having the recorded bibliographic information including this date and time.

Next, the acquisition unit 52 acquires the audio data from the audio data path (step S321-30). Specifically, since the date and time display portion showing the split date and time indicates "11:02:00", the acquisition unit 52 acquires the audio data path managed in the tab of the record bibliographic information having the record bibliographic information including this date and time. In this case, the acquired audio data path is, for example, ".../00005006/record.mp3".

Next, the generation/processing unit 57 generates partial audio data before the division time and partial audio data after the division time, and generates event URLs corresponding to each (step S321-31).

Next, the setting registration unit 58 updates and registers the recorded bibliographic information management table based on the path of the audio data and the event URL, and then exits this flow (step S321-32).

Figure 71A is a conceptual diagram showing an example of a record bibliographic information management table after the division process of the record viewing and editing screen according to the third embodiment, and is a conceptual diagram of the record bibliographic information management table constituting the first divided voice recording screen. In Figure 71A, the record bibliographic information management table associated with the tab of the newly set post-division record identification information "R5301" is shown. Specifically, the setting registration unit 58 updates the record name to "Division 1_Healthcare Business Performance Reporting Meeting". Furthermore, the setting registration unit 58 updates the end date and time to "2021/03/31 11:02:02". Furthermore, the setting registration unit 58 updates the voice data path to ".../00005301/record.mp3". Furthermore, the setting registration unit 58 updates the event URL to "https://.../00005301".

Figure 71B is a conceptual diagram showing an example of a record bibliographic information management table after the division process of the record viewing and editing screen according to the third embodiment, and is a conceptual diagram of the record bibliographic information management table constituting the second divided audio recording screen. In Figure 71B, the record bibliographic information management table associated with the tab of the newly set post-division record identification information "R5302" is shown. Specifically, the setting registration unit 58 updates the record name to "Division 2_Healthcare Business Performance Reporting Meeting". Furthermore, the setting registration unit 58 updates the start date and time to "2021/03/31 11:02:02". Furthermore, the setting registration unit 58 updates the audio data path to ".../00005302/record.mp3". Furthermore, the setting registration unit 58 updates the event URL to "https://.../00005302".

The divided voice recording screen including each record identified and updated in step S321 described above is displayed on the communication terminal 3 when the user inputs the event URL indicating the desired divided event and the passcode (when the processing of step S101 is executed).

[Major Effects of the Modification of the Third Embodiment]
As described above, according to this embodiment, the voice recording management device 5 identifies the record identification information from the text information management table, obtains the end time information of the last record among the records whose start time is before the division time (step S321-22), and if the obtained end time is after the division time, sets the obtained end time as the division time (step S321-24). This has the effect of ensuring that the voice is not interrupted and is played to the end even if a division process is performed on the record viewing and editing screen.

[Supplementary description of the embodiment]
Each function of the above-mentioned embodiment can be realized by one or more processing circuits. Here, the term "processing circuit" in this specification includes a device programmed to execute each function by software, such as a processor implemented by an electronic circuit. This device refers to, for example, a processor, an ASIC (Application Specific Integrated Circuit), a DSP (Digital Signal Processor), an FPGA (Field Programmable Gate Array), an SOC (System On a Chip), a GPU (Graphics Processing Unit), and a conventional circuit module, etc.

Furthermore, the various texts and text information obtained by the above-mentioned embodiment may be obtained by the learning effect of machine learning using artificial intelligence (AI). In this case, the voice recognition server may obtain various texts and text information from voice information using machine learning, or a database different from the voice recognition server may obtain various texts and text information from voice information using machine learning. Here, machine learning is a technology for making a computer acquire human-like learning ability, and refers to a technology in which a computer autonomously creates an algorithm required for judgment such as data identification from learning data that is previously loaded, and applies this to new data to make predictions. The learning method for machine learning may be any of supervised learning, unsupervised learning, semi-supervised learning, reinforcement learning, and deep learning. Furthermore, the learning method for machine learning may be a combination of these learning methods, and any learning method for machine learning may be used.

So far, we have described an audio recording management system, an audio recording management device, an audio recording management method, and a program according to one embodiment of the present invention, but the present invention is not limited to the above-described embodiment, and can be modified within the scope of what a person skilled in the art can imagine, such as adding, changing, or deleting other embodiments, and any aspect is within the scope of the present invention as long as it achieves the functions and effects of the present invention.

■ Summary ■
For example, aspects of the present invention are as follows.

<First aspect>
A voice recording management system (e.g., voice recording management system 2; omitted below) as a first aspect includes a voice recording management device (e.g., voice recording management device 5; omitted below) that manages voice recording information obtained based on voice information, and one or more communication terminals (e.g., communication terminal 3; omitted below) that can display the voice recording information by communicating with the voice recording management device, and the voice recording management device includes an acquisition means (e.g., acquisition unit 52) that acquires voice data representing the voice information transmitted by a first communication terminal among the one or more communication terminals, and screen data representing a screen displayed on the first communication terminal, predetermined text data representing predetermined text obtained based on the acquired voice data, predetermined image data representing a predetermined image associated with the predetermined text, and a predetermined voice indicated by the predetermined text. The voice recording management device has a transmitting means (e.g., a transceiver unit 51, omitted below) for transmitting the edit request sent by the first communication terminal, and edited screen data including edited text data edited by editing the specified text data and edited image data edited by editing the specified image data, and edited audio data edited by editing the specified audio data, to a second communication terminal different from the first communication terminal in response to the editing request for the specified text or the specified image, and the second communication terminal has a display control means (e.g., a display control unit 34) for displaying an edited screen related to the edited screen data transmitted by the voice recording management device on a display means, and an audio playback means (e.g., an audio playback unit 36) for playing back the edited audio related to the edited audio data transmitted by the voice recording management device.

According to the first aspect, when editing a voice recording generated based on voice information, it is only necessary to edit the text data included in the voice recording or the image data corresponding to the text data, thereby improving the convenience of editing the voice recording.

<Second aspect>
In a second aspect of the voice recording management system, in the first aspect, the voice recording management device further includes a receiving means (e.g., a transmitting/receiving unit 51; omitted below) for receiving, as the editing request, either one of a text non-display request for non-displaying the specified text and an image non-display request for non-displaying the specified image, and when the received editing request is the text non-display request, the transmitting means transmits, as the editing process, a first non-display text data including first non-displayed text data obtained by non-displaying the specified text data and first non-displayed image data obtained by non-displaying the specified image data. The screen data and first muted audio data obtained by muting the specified audio data as the editing process are transmitted to the second communications terminal, and if the received editing request is a request to not display an image, second hidden screen data including second hidden image data obtained by not displaying the specified image data and second hidden text data obtained by not displaying one or more texts associated with the specified image as the editing process, and second muted audio data obtained by muting one or more specified audio data associated with the one or more texts as the editing process are transmitted to the second communications terminal.

According to the second aspect, like the first aspect, it is possible to improve the convenience of editing audio recordings.

<Third aspect>
A third aspect of the audio recording management system is such that, in the second aspect, the display control means displays a text hide operation unit operated to generate the text hide request near the specified text displayed on the screen, and displays an image hide operation unit operated to generate the image hide request near the specified image displayed on the screen.

According to the third aspect, in addition to improving the convenience of editing audio recordings according to the second aspect, it is possible to improve the operability of editing audio recordings.

<Fourth aspect>
A fourth aspect of the audio recording management system is, in the second or third aspect, the audio recording management device further has a generation means for generating the first hidden screen data and the first muted audio data in response to the request to not display text, and generating the second hidden screen data and the second muted audio data in response to the request to not display an image.

According to the fourth aspect, in addition to improving the convenience and operability of editing the audio recording according to the second or third aspect, it is possible to improve confidentiality during audio playback.

<Fifth aspect>
In the voice recording management system as a fifth aspect, in the second to fourth aspects, the first communication terminal further has an acquisition means for acquiring screen data related to the screen at a predetermined time interval, and when a request to hide an image for the specified image acquired at the acquisition time is received in a state in which specific text data obtained based on the voice data exists across the acquisition time at which the screen data was acquired by the acquisition means, the display control means displays on the display means a third hidden screen related to third hidden screen data obtained by hiding the specific text data in addition to the second hidden text data, and the voice playback means plays voice in which the voice data indicated by each text related to the second hidden text data and the specific text data has been muted.

According to the fifth aspect, in addition to improving the convenience and operability of editing audio recordings according to the second to fourth aspects, it is possible to further improve confidentiality during audio playback.

<Sixth aspect>
A sixth aspect of the audio recording management system is such that, in the fifth aspect, when an image display request to display the specified image again is received after the display control means has received the request to not display the specified image for the specified image, the display control means displays a screen in which specific text related to the specific text data is maintained in a hidden state, and the audio playback means plays audio in which specific audio related to the specific audio data indicated by the specific text is maintained in a muted state.

According to the sixth aspect, it is possible to further improve confidentiality during audio playback according to the fifth aspect.

<Seventh aspect>
In the seventh aspect of the audio recording management system, in the second to sixth aspects, the audio recording management device further has a processing means (e.g., a generation/processing unit 57) in which the request to not make any text visible and the request to not make any image visible are respectively requests to delete or make the specified text data and the specified image data private, and in the case of a request to delete, the specified text data and the specified image data are deleted, and in the case of a request to make private, the specified text data and the specified image data are not deleted, but are hidden on the first hidden screen represented by the first hidden screen data or the second hidden screen represented by the second hidden screen data.

According to the seventh aspect, it is possible to further improve the operability and confidentiality of editing the audio recordings according to the second to sixth aspects.

<Eighth aspect>
In the eighth aspect of the audio recording management system, in the first to seventh aspects, when the display control means selects a text group including a plurality of texts displayed on the screen, a summary input operation section (e.g., summary input dialog 3181; omitted below) for inputting a summary representing the text group is displayed near the text group.

According to the eighth aspect, it is possible to further improve operability and visibility when editing audio recordings.

<Ninth aspect>
A ninth aspect of the audio recording management system is the eighth aspect, in which the display control means displays the text hiding operation section (e.g., a "Private" button (icon) 3542, a "Delete" button (icon) 3544) in the vicinity of the one text group together with the summary input operation section.

According to the ninth aspect, it is possible to further improve the operability and visibility when editing the audio recording according to the eighth aspect.

<Tenth aspect>
In the audio recording management system of the 10th aspect, in the 1st to 9th aspects, the first hidden text data, the second hidden text data, the first hidden image data, and the second hidden image data which are processed to be hidden by the text non-display request or the image non-display request are each confidential data including performance information, sales information, profit information, and personal information of a specified business.

According to the tenth aspect, it becomes possible to manage the type of confidential data when editing the audio recording according to the first to ninth aspects.

<Eleventh aspect>
An eleventh aspect of the present invention is an audio recording management device that manages audio recording information obtained based on audio information, the audio recording management device having an acquisition means for acquiring audio data representing the audio information transmitted by a first communication terminal among one or more communication terminals capable of displaying the audio recording information by communicating with the audio recording management device, and screen data representing a screen displayed on the first communication terminal, and a transmission means for transmitting predetermined text data representing a predetermined text obtained based on the acquired audio data, predetermined image data representing a predetermined image that is included on the screen related to the acquired screen data and corresponds to the predetermined text, and predetermined audio data indicated by the predetermined text to the one or more communication terminals including the first communication terminal, wherein the transmission means transmits edited screen data including edited text data obtained by editing the predetermined text data and edited image data obtained by editing the predetermined image data, and edited audio data obtained by editing the predetermined audio data to a second communication terminal different from the first communication terminal, in response to an editing request transmitted by the first communication terminal for the predetermined text or the predetermined image.

According to the eleventh aspect, when editing a voice recording generated based on voice information, it is only necessary to edit the text data included in the voice recording or the image data corresponding to the text data, thereby improving the convenience of editing the voice recording.

<Twelfth aspect>
A voice recording management method executed by a voice recording management device that manages voice recording information obtained based on voice information as a twelfth aspect includes an acquisition step of acquiring voice data representing the voice information transmitted by a first communication terminal among one or more communication terminals capable of displaying the voice recording information by communicating with the voice recording management device, and screen data representing a screen displayed on the first communication terminal, and a transmission step of transmitting, to the one or more communication terminals including the first communication terminal, predetermined text data representing a predetermined text obtained based on the acquired voice data, predetermined image data representing a predetermined image that is included on the screen related to the acquired screen data and corresponds to the predetermined text, and predetermined voice data indicated by the predetermined text, wherein the transmission step is an editing request transmitted by the first communication terminal, and in response to the editing request for the predetermined text or the predetermined image, edited screen data including edited text data obtained by editing the predetermined text data and edited image data obtained by editing the predetermined image data, and edited voice data obtained by editing the predetermined voice data are transmitted to a second communication terminal different from the first communication terminal.

According to the twelfth aspect, when editing a voice recording generated based on voice information, it is only necessary to edit the text data included in the voice recording or the image data corresponding to the text data, thereby improving the convenience of editing the voice recording.

<Thirteenth aspect>
The program for causing a voice recording management device that manages voice recording information obtained based on voice information as a thirteenth aspect to execute the following processes includes an acquisition step of acquiring voice data representing the voice information transmitted by a first communication terminal among one or more communication terminals capable of displaying the voice recording information by communicating with the voice recording management device, and screen data representing a screen displayed on the first communication terminal, and a transmission step of transmitting, to the one or more communication terminals including the first communication terminal, predetermined text data representing a predetermined text obtained based on the acquired voice data, predetermined image data representing a predetermined image that is included on the screen related to the acquired screen data and corresponds to the predetermined text, and predetermined voice data indicated by the predetermined text, and the transmission step executes a process of transmitting, to a second communication terminal different from the first communication terminal, edited screen data including edited text data obtained by editing the predetermined text data and edited image data obtained by editing the predetermined image data, and edited voice data obtained by editing the predetermined voice data, in response to an editing request transmitted by the first communication terminal and an editing request for the predetermined text or the predetermined image, to a second communication terminal different from the first communication terminal.

According to the thirteenth aspect, when editing a voice recording generated based on voice information, it is only necessary to edit the text data included in the voice recording or the image data corresponding to the text data, thereby improving the convenience of editing the voice recording.

<14th aspect>
A fourteenth aspect of the voice recording management system is, in the first aspect, the receiving means of the voice recording management device receives a split operation instruction request to split a specified area displayed on the post-edit screen as the editing request, and the transmitting means transmits split edited screen data to the second communication terminal in response to the split operation instruction request, the split edited text data obtained by split processing the specified text data and split edited image data obtained by split processing the specified image data, and the display control means of the second communication terminal displays the split edited screen related to the split edited screen data transmitted by the voice recording management device on the display means.

According to the 14th aspect, it is possible to split and edit the screens that are not relevant to the participants of the meeting, etc., on the audio recording screen on which the created meeting minutes, etc. are displayed, and to allow the participants who need the information to view only the necessary information. As a result, it is possible to improve the confidentiality of the display screen data displayed on the audio recording screen, the audio data played back, etc.

<Fifteenth aspect>
In the voice recording management system of the fifteenth aspect, in the first aspect, the transmission means of the voice recording management device transmits specified bookmark information including specified text data contained in the split edited screen data to the second communication terminal in response to the split operation instruction request, and the display control means of the second communication terminal displays on the display means a split edited screen related to the split edited screen data including the specified bookmark information contained in the split edited screen data transmitted by the voice recording management device.

According to the fifteenth aspect, in addition to improving the confidentiality of the display screen data displayed on the voice recording screen according to the fourteenth aspect, the voice data played back, etc., it is also possible to display or hide bookmarked text in accordance with the divided screens. As a result, it is possible to further improve the convenience of editing on the voice recording screen for the user.

<16th aspect>
A 16th aspect of the voice recording management system is, in the first aspect, an audio playback means of the second communication terminal plays back the split edited audio related to the split edited audio data transmitted by the voice recording management device, which is split using the end date and time of the utterance as the split date and time, when the end date and time of an utterance that began at a date and time prior to the division date and time at which the specified text data was divided and that is closest to the division date and time straddles the division date and time.

According to the 16th aspect, in addition to improving the convenience of editing on the audio recording screen according to the 14th and 15th aspects, it is possible to guarantee the content of the audio data.

1 Communication system 2 Voice recording management system 3 Communication terminal 5 Voice recording management device 7 Voice recognition server (cloud service)
31 Transmitting/receiving unit (an example of a receiving means, an example of a transmitting means)
32 Operation reception unit (an example of a reception means)
33 Sound/image acquisition unit (an example of an acquisition means)
34 Display control unit (an example of a display control means)
36 Audio playback unit (an example of audio playback means)
51 Transmitting/receiving unit (an example of a receiving means, an example of a transmitting means)
52 Acquisition unit (an example of an acquisition means)
53 Calculation determination unit (an example of a calculation means, an example of a determination means)
54 Display control unit (an example of a display control means)
55 Determination unit (an example of a determination means)
56 Authentication unit (an example of an authentication means)
57 Generation/processing unit (an example of a generation means)
58 Setting registration unit (an example of a setting means, an example of a registration means)
59 Memory readout unit (an example of a memory readout means)

JP 2019-139572 A

Claims (16)

A voice recording management system including: a voice recording management device that manages voice recording information obtained based on voice information; and one or more communication terminals that can display the voice recording information by communicating with the voice recording management device,
The voice recording management device includes:
an acquisition means for acquiring voice data representing the voice information transmitted by a first communication terminal among the one or more communication terminals, and screen data representing a screen displayed on the first communication terminal;
a transmission means for transmitting, to the one or more communication terminals including the first communication terminal, predetermined text data representing a predetermined text obtained based on the acquired voice data, predetermined image data representing a predetermined image included on the screen related to the acquired screen data and associated with the predetermined text, and predetermined voice data represented by the predetermined text;
having
The transmitting means is
transmit edited screen data including edited text data obtained by editing the specified text data and edited image data obtained by editing the specified image data, and edited audio data obtained by editing the specified audio data, to a second communication terminal different from the first communication terminal in response to an editing request transmitted by the first communication terminal for the specified text or the specified image;
The second communication terminal comprises:
a display control means for displaying an edited screen related to the edited screen data transmitted from the voice recording management device on a display means;
a voice reproducing means for reproducing the edited voice related to the edited voice data transmitted from the voice recording management device;
having
1. A voice recording management system comprising: 2. The voice recording management system according to claim 1,
The voice recording management device further includes:
a receiving means for receiving, as the editing request, either one of a text non-display request for non-displaying the predetermined text and an image non-display request for non-displaying the predetermined image,
The transmitting means is
when the received editing request is the text non-display request, as the editing process, transmit to the second communication terminal first non-display screen data including first non-display text data obtained by non-display processing the predetermined text data and first non-display image data obtained by non-display processing the predetermined image data, and first muted audio data obtained by muting the predetermined audio data as the editing process;
When the received editing request is the non-display request for the image, second non-display screen data including second non-display image data obtained by non-display processing of the specified image data and second non-display text data obtained by non-display processing of one or more texts associated with the specified image, and second muted audio data obtained by muting one or more specified audio data associated with the one or more texts as the editing processing are transmitted to the second communication terminal as the editing processing.
2. The voice recording management system according to claim 1. The display control means
a text non-display operation unit operated to generate the text non-display request is displayed near the predetermined text displayed on the screen, and an image non-display operation unit operated to generate the image non-display request is displayed near the predetermined image displayed on the screen.
3. The voice recording management system according to claim 2. 4. The voice recording management system according to claim 2,
The voice recording management device further comprises:
a generating means for generating the first non-display screen data and the first muted audio data in response to the request to not display the text, and generating the second non-display screen data and the second muted audio data in response to the request to not display the image,
1. A voice recording management system comprising: The first communication terminal further comprises:
an acquisition means for acquiring screen data relating to the screen at a predetermined time interval;
The display control means
when the image non-display request for the predetermined image acquired at the acquisition time is received in a state in which specific text data obtained based on the voice data exists across an acquisition time at which the screen data is acquired by the acquisition means, displaying on the display means, in addition to the second non-display text data, a third non-display screen related to third non-display screen data obtained by non-display processing of the specific text data;
The audio reproducing means includes:
reproducing the audio in which the audio data indicated by the second non-display text data and each text related to the specific text data have been muted;
3. The voice recording management system according to claim 2. The display control means
displaying a screen on which a specific text related to the specific text data is kept in a non-display state when an image display request for displaying the specific image again is received after the image non-display request for the specific image is received;
The audio reproducing means includes:
reproducing a sound in which a specific sound related to the specific sound data indicated by the specific text is kept in a muted state;
6. The voice recording management system according to claim 5. 3. The voice recording management system according to claim 2,
The voice recording management device further comprises:
the text non-display request and the image non-display request are requests to delete or make private the specified text data and the specified image data, respectively, and the device has a processing means for deleting the specified text data and the specified image data in the case of the request to delete, and for making private the specified text data and the specified image data in the case of the request to make private, without deleting the specified text data and the specified image data, and for making them non-display on a first non-display screen represented by the first non-display screen data or a second non-display screen represented by the second non-display screen data.
1. A voice recording management system comprising: The display control means
when a text group including a plurality of texts displayed on the screen is selected, a summary input operation section for inputting a summary representing the text group is displayed near the text group.
2. The voice recording management system according to claim 1. The display control means
a text non-display operation section is displayed in the vicinity of the one text group together with the summary input operation section;
9. The voice recording management system according to claim 8. The first hidden text data, the second hidden text data , the first hidden image data , and the second hidden image data which are subjected to a hidden processing in response to a text hidden request or an image hidden request are confidential data including performance information, sales information, profit information, and personal information of a predetermined business, respectively.
2. The voice recording management system according to claim 1. A voice recording management device for managing voice recording information obtained based on voice information, comprising:
an acquisition means for acquiring voice data representing the voice information transmitted by a first communication terminal among one or more communication terminals capable of displaying the voice recording information by communicating with the voice recording management device, and screen data representing a screen displayed on the first communication terminal;
a transmission means for transmitting, to the one or more communication terminals including the first communication terminal, predetermined text data representing a predetermined text obtained based on the acquired voice data, predetermined image data representing a predetermined image included on the screen related to the acquired screen data and associated with the predetermined text, and predetermined voice data represented by the predetermined text;
having
The transmitting means is
the editing request being transmitted from the first communication terminal, in response to the editing request for the specified text or the specified image, transmitting edited screen data including edited text data obtained by editing the specified text data and edited image data obtained by editing the specified image data, and edited audio data obtained by editing the specified audio data, to a second communication terminal different from the first communication terminal;
1. A voice recording management device comprising: A voice recording management method executed by a voice recording management device that manages voice recording information obtained based on voice information, comprising:
an acquiring step of acquiring voice data representing the voice information transmitted by a first communication terminal among one or more communication terminals capable of displaying the voice recording information by communicating with the voice recording management device, and screen data representing a screen displayed on the first communication terminal;
a transmission step of transmitting, to the one or more communication terminals including the first communication terminal, predetermined text data representing a predetermined text obtained based on the acquired voice data, predetermined image data representing a predetermined image that is included on the screen related to the acquired screen data and corresponds to the predetermined text, and predetermined voice data indicated by the predetermined text;
Including,
The transmitting step includes:
the editing request being transmitted from the first communication terminal, in response to the editing request for the specified text or the specified image, transmitting edited screen data including edited text data obtained by editing the specified text data and edited image data obtained by editing the specified image data, and edited audio data obtained by editing the specified audio data, to a second communication terminal different from the first communication terminal;
A voice recording management method comprising: A voice recording management device for managing voice recording information obtained based on voice information,
an acquiring step of acquiring voice data representing the voice information transmitted by a first communication terminal among one or more communication terminals capable of displaying the voice recording information by communicating with the voice recording management device, and screen data representing a screen displayed on the first communication terminal;
a transmission step of transmitting, to the one or more communication terminals including the first communication terminal, predetermined text data representing a predetermined text obtained based on the acquired voice data, predetermined image data representing a predetermined image that is included on the screen related to the acquired screen data and corresponds to the predetermined text, and predetermined voice data indicated by the predetermined text;
Including,
In the transmitting step,
the editing request being transmitted from the first communication terminal, in response to the editing request for the specified text or the specified image, transmitting edited screen data including edited text data obtained by editing the specified text data and edited image data obtained by editing the specified image data, and edited audio data obtained by editing the specified audio data, to a second communication terminal different from the first communication terminal;
A program that executes a process. The receiving means includes:
receiving, as the editing request, a division operation instruction request for dividing a predetermined area displayed on the post-editing screen;
The transmitting means is
transmit, in response to the split operation instruction request, split-edited screen data including split-edited text data obtained by splitting the specified text data and split-edited image data obtained by splitting the specified image data, to the second communication terminal;
The display control means of the second communication terminal
a divided edited screen related to the divided edited screen data transmitted from the voice recording management device is displayed on the display means;
3. The voice recording management system according to claim 2. The transmitting means is
transmitting, in response to the split operation instruction request, predetermined bookmark information including predetermined text data included in the post-split edit screen data to the second communication terminal;
The display control means of the second communication terminal
a display means for displaying a post-edited screen related to the post-edited screen data including the predetermined bookmark information included in the post-edited screen data transmitted from the voice recording management device;
15. The voice recording management system according to claim 14. The audio reproducing means of the second communication terminal comprises:
When an end date and time of an utterance that started at a date and time prior to the division date and time at which the specified text data was divided and closest to the division date and time straddles the division date and time, the end date and time of the utterance is set as the division date and time, and the divided edited voice related to the divided edited voice data transmitted by the voice recording management device is played back.
16. The voice recording management system according to claim 14 or 15.