JP7791014B2 - Information processing device, information processing method, and information processing program - Google Patents

Description

The present invention relates to an information processing device, an information processing method, and an information processing program.

Technology has been known in the past that converts a user's speech into text information and provides the converted text information. For example, Patent Document 1 discloses a voice input device that includes a voice recognition unit that recognizes the user's voice, a character conversion unit that converts the voice recognized by the voice recognition unit into a character string, and a character string display unit that displays the character string converted by the character conversion unit.

Japanese Patent Application Laid-Open No. 2021-168020

However, the above-mentioned conventional technologies leave room for improvement in terms of improving user convenience. For example, the technology described in Patent Document 1 converts all of the user's speech into text and displays it. Therefore, if there is content that the user wants to keep secret for privacy reasons, for example, the displayed text must be edited, leaving room for improvement.

This application has been made in light of the above, and aims to provide an information processing device, information processing method, and information processing program that can improve user convenience.

The information processing device according to the present application includes a reception unit, a generation unit, and a provision unit. The reception unit receives a user's utterance. The generation unit masks portions of the utterance received by the reception unit that satisfy confidentiality conditions, and generates utterance text information, which is text information obtained by converting the user's utterance into text. The provision unit provides the utterance text information generated by the generation unit.

According to one aspect of the embodiment, it is possible to achieve an effect of improving user convenience.

FIG. 1 is a diagram illustrating an example of information processing according to an embodiment. FIG. 2 is a diagram illustrating an example of the configuration of the information processing apparatus according to the embodiment. FIG. 3 is a diagram illustrating an example of spoken text information displayed on the display unit of the information processing apparatus according to the embodiment. FIG. 4 is a diagram showing an example in which masking of a selected portion of spoken text information displayed on the display unit of the information processing apparatus according to the embodiment is released. FIG. 5 is a flowchart illustrating an example of information processing by the processing unit of the information processing apparatus according to the embodiment. FIG. 6 is a flowchart illustrating an example of information processing by the processing unit of the information processing apparatus according to the embodiment. FIG. 7 is a diagram illustrating another example of the configuration of the information processing device according to the embodiment. FIG. 8 is a hardware configuration diagram illustrating an example of a computer that realizes the functions of the information processing apparatus according to the embodiment.

Below, detailed explanations will be given of the information processing device, information processing method, and information processing program (hereinafter referred to as "embodiments") according to the present application, with reference to the drawings. Note that the information processing device, information processing method, and information processing program according to the present application are not limited to these embodiments. Furthermore, the embodiments can be combined as appropriate to the extent that the processing content is not contradictory. Furthermore, the same components in the following embodiments will be assigned the same reference numerals, and redundant explanations will be omitted.

[1. An example of information processing]
FIG. 1 is a diagram showing an example of information processing according to an embodiment, and in this embodiment, an information processing method is executed by an information processing device 1.

The information processing device 1 is a device that can utilize an AI (artificial intelligence) assistant function that supports interactive voice operation, and the user U can control peripheral devices and obtain various information by interacting with the information processing device 1. Peripheral devices include various appliances such as lighting equipment, refrigerators, washing machines, air conditioners, television sets, dishwashers, dish dryers, induction cookers, and microwave ovens.

In addition, when user U speaks to information processing device 1 to obtain various information, information processing device 1 transmits input information indicating instructions from user U to information providing device 2 (see Figure 2). Information processing device 1 can obtain content (e.g., various information such as news, traffic information, weather, and music) provided via network N (see Figure 2) from information providing device 2 in accordance with the input information, and display the obtained content on the display unit or output it from the speaker.

In addition, the information processing device 1 has a text conversion function that converts the speech of the user U into text and outputs the converted text information. This text conversion function allows the user U to create, for example, text to be used in emails or SNS (Social Networking Service), or text to be posted to electronic bulletin boards or review sites, by speaking to the information processing device 1. The text conversion function will be mainly described below with reference to Figure 1.

As shown in FIG. 1, when user U uses the text function of information processing device 1, he or she speaks to information processing device 1 (step S1). In the example shown in FIG. 1, user U's utterance is, "I recently moved into an apartment at 1-2-3, A-ku, Tokyo. It's a very comfortable place to live, and Tokkyo Taro and others were very helpful to me when I moved. I'm having a party soon, so please come along. My contact number is 090-0190-xxxx." Note that "090-0190-xxxx" is the phone number, and "xxxx" is a four-digit number combination.

The information processing device 1 accepts the user U's utterance (step S2). Then, the information processing device 1 masks the parts of the user U's utterance accepted in step S2 that satisfy the confidentiality conditions, and generates utterance text information, which is text information obtained by converting the user U's utterance into text (step S3). The confidentiality conditions are, for example, words that indicate privacy-related content such as address, name, and telephone number, and can be set or changed by the user U operating the information processing device 1.

In the example shown in Figure 1, "A-ku 1-2-3," "Takken Taro," and "090-0190-xxxx" are parts (words or groups of words, etc.) that satisfy the confidentiality conditions, and the information processing device 1 masks the parts that satisfy the confidentiality conditions. "A-ku 1-2-3" is information that indicates an address, and "Takken Taro" is information that indicates a name.

The information processing device 1 masks parts that satisfy confidentiality conditions during the process of converting user U's speech into text, but it can also mask parts that satisfy confidentiality conditions after converting user U's speech into text.

The information processing device 1 has a trained model that receives, for example, audio information or text information corresponding to the user U's utterance received in step S2 as input, and outputs a confidentiality score indicating the degree of confidentiality for each of the multiple element parts that make up the user U's utterance. The information processing device 1 inputs the audio information or text information corresponding to the user U's utterance received in step S2 into the trained model, and obtains from the trained model a confidentiality score that indicates the degree of confidentiality for each of the multiple element parts that make up the user U's utterance.

The information processing device 1 generates spoken text information by treating element parts among the multiple element parts whose confidentiality scores are equal to or greater than a threshold as parts that satisfy the confidentiality conditions. Note that if the input to the trained model is text information, the information processing device 1 converts the audio information corresponding to the utterance of user U received in step S2 into text information, and inputs the converted text information into the trained model.

In addition, the information processing device 1 may have a trained model that receives, for example, audio information or text information corresponding to the utterance of user U received in step S2 as input, and outputs, as spoken text information, speech text information in which parts of the utterance of user U that satisfy confidentiality conditions have been masked. In this case, the information processing device 1 can input, into the trained model, the audio information or text information corresponding to the utterance of user U received in step S2, and obtain the text information output from the trained model as speech text information.

In addition to or instead of the trained model, the information processing device 1 may have a confidential information table containing information indicating multiple words to be concealed. In this case, the information processing device 1 can generate spoken text information by treating the text information or audio information corresponding to the words included in the confidential information table as the portion that satisfies the confidentiality conditions.

The character strings included in the confidential information table may be character strings expressed using regular expressions. In this case, the text information or audio information specified by the regular expressions shown in the confidential information table is used as the part that satisfies the confidentiality conditions to generate the spoken text information.

Masking is performed by replacing parts of user U's utterance that satisfy the confidentiality conditions with, for example, at least one of specific characters, symbols, and patterns. In the example shown in Figure 1, spoken text information is generated in which parts of user U's utterance that satisfy the confidentiality conditions are replaced with the letter "X."

Specifically, in the example shown in Figure 1, the spoken text information is, "I recently moved into an apartment in XXXXXX, Tokyo. It's very comfortable, and XXXX and others were very helpful to me when I moved. I'm having a party soon, so please come along. My contact details are XXX-XXXX-XXXX."

The information processing device 1 provides the spoken text information generated in step S3 (step S4). For example, the information processing device 1 can provide the spoken text information to the user U by displaying the spoken text information on a display unit. The information processing device 1 can also provide the spoken text information to the user U by outputting the spoken text information as sound from a speaker.

When providing spoken text information by voice, the information processing device 1 can silence the masked portions or set a preset sound (for example, a beep). The information processing device 1 can also provide spoken text information to others by transmitting the spoken text information to their terminal device. This makes it possible to prevent others from learning the content of those portions, even if, for example, there is a portion in user U's speech that satisfies confidentiality conditions.

Furthermore, the information processing device 1 can unmask a masked portion of the spoken text information displayed on the display unit when the user U selects that portion. This allows the user U to easily find out the masked content by operating the information processing device 1 if they wish to know the masked content. Note that the information processing device 1 can also re-mask a portion of the spoken text information when the user U unselects the masked portion.

In this way, the information processing device 1 according to the embodiment generates utterance text information, which is text information obtained by converting user U's utterance into text by masking parts of the utterance that satisfy confidentiality conditions, and provides the generated utterance text information to user U. This allows the information processing device 1 to improve convenience for user U.

The following provides a detailed explanation of the configuration of an information processing system that includes an information processing device 1 that performs such processing.

2. Configuration of the information processing system
Next, the configuration of an information processing system including an information processing device 1 according to an embodiment will be described with reference to Fig. 2. Fig. 2 is a diagram showing an example of the configuration of the information processing device 1 according to an embodiment. As shown in Fig. 2, the information processing system 100 according to the embodiment includes the information processing device 1 and an information providing device 2. The information processing device 1 and the information providing device 2 are connected to each other via a network N so as to be able to communicate with each other via a wired or wireless connection. Note that the information processing system 100 shown in Fig. 2 may include a plurality of information processing devices 1 and a plurality of information providing devices 2.

The information processing device 1 may be, for example, a smart speaker, a desktop PC (Personal Computer), a notebook PC, a tablet terminal, a mobile phone, or a PDA (Personal Digital Assistant). Note that the information processing device 1 is not limited to the above examples and may also be, for example, a smart watch or a wearable device.

The information providing device 2 provides online services to the user U. Services provided by the information providing device 2 include, but are not limited to, online services such as search services, information providing services, e-commerce services, auction services, music distribution services, and video distribution services. Information providing services include a variety of services such as search services provided by search sites, news distribution services provided by news sites, traffic information providing services provided by traffic information sites, and weather information providing services provided by weather information sites.

The information providing device 2 is an information processing device capable of communicating with various devices via a predetermined network N such as the Internet, and is realized, for example, by a server device or a cloud system. For example, the information providing device 2 is connected to various other devices via the network N so that it can communicate with them.

3. Information Processing Device 1
As shown in Figure 2, the information processing device 1 according to the embodiment includes a communication unit 10, a display unit 11, an operation unit 12, a memory unit 13, an audio input unit 14, an audio output unit 15, a position detection unit 16, and a processing unit 17.

3.1. Communication Unit 10
The communication unit 10 is realized by, for example, a network interface card (NIC), etc. The communication unit 10 is connected to a network N by wire or wirelessly, and transmits and receives information to and from the information providing device 2 via the network N.

[3.2. Display section 11]
The display unit 11 is, for example, an LCD (Liquid Crystal Display) or an organic EL (Electro Luminescence) display.

[3.3. Operation unit 12]
The operation unit 12 includes, for example, a keyboard including keys for inputting letters, numbers, and spaces, an enter key, arrow keys, etc., a mouse, a power button, etc. When the display unit 11 is a touch panel display device, the operation unit 12 may include a touch panel.

[3.4. Storage unit 13]
The storage unit 13 is realized by, for example, a semiconductor memory element such as a random access memory (RAM) or a flash memory, or a storage device such as a hard disk or an optical disk.

The memory unit 13 stores various types of information. For example, the memory unit 13 stores information transmitted from the information providing device 2 and acquired by the processing unit 17 via the network N and the communication unit 10. The memory unit 13 also stores voice information and text information corresponding to the speech of the user U.

3.5. Audio Input Unit 14
The voice input unit 14 converts a voice signal, which is a signal of the voice uttered by the user U, into a digital signal and outputs the converted digital signal, which is a voice digital signal, as voice information to the processing unit 17. The voice input unit 14 includes, for example, a microphone and an AD (Analog to Digital) converter that converts the voice signal, which is an electrical analog signal output from the microphone, into a digital signal.

3.6. Audio Output Unit 15
The audio output unit 15 includes, for example, a DA (Digital to Analog) converter that converts a digital audio signal, which is audio information output from the processing unit 17, into an analog audio signal, and a speaker that converts the analog audio signal output from the DA converter into sound and outputs it.

3.7. Position detection unit 16
The position detection unit 16 detects, for example, the position of the information processing device 1, and outputs position data that is data on the detected position of the information processing device 1 to the processing unit 17. The position detection unit 16 receives a plurality of positioning signals transmitted from a plurality of positioning satellites in the GNSS (Global Navigation Satellite System), and detects the position of the information processing device 1 based on the received plurality of positioning signals.

3.8. Processing Unit 17
The processing unit 17 is a controller, and is realized, for example, by a CPU (Central Processing Unit) or an MPU (Micro Processing Unit) executing various programs stored in a storage device inside the information processing device 1 using RAM as a working area.

The processing unit 17 may also be implemented by an integrated circuit such as an ASIC (Application Specific Integrated Circuit) or FPGA (Field Programmable Gate Array). The processing unit 17 includes a reception unit 20, a generation unit 21, a provision unit 22, and a learning unit 23.

3.8.1. Reception unit 20
The reception unit 20 receives the speech of the user U based on the digital audio information output from the audio input unit 14. For example, when the user U performs a specific operation using the operation unit 12, the reception unit 20 receives the speech of the user U thereafter.

Specific operations differ, for example, for the information acquisition function, information transmission function, device control function, and text conversion function. The information acquisition function is a function for acquiring specific information from the information providing device 2 in response to user U's speech, the information transmission function is a function for transmitting information in response to user U's speech, and the device control function is a function for controlling peripheral devices in response to user U's speech. As described above, the text conversion function is a function for converting user U's speech into text and outputting the text information that has been converted into text.

Furthermore, when user U utters a specific keyword, the reception unit 20 can also receive subsequent utterances from user U. Whether user U has uttered a specific keyword is determined by voice recognition of the voice information signal output from the voice input unit 14. Note that the specific keyword differs for each of the information acquisition function, information transmission function, device control function, and text conversion function, for example.

The reception unit 20 has a voice recognition function that converts the voice information output from the voice input unit 14 in response to the user U's speech into text information. The reception unit 20 may also have a function that analyzes the meaning of the text information converted by the voice recognition function.

The reception unit 20 outputs voice information or text information corresponding to the user's utterance to the generation unit 21. The voice information corresponding to the user's utterance is voice information output from the voice input unit 14, and the text information corresponding to the user's utterance is information obtained by converting the voice information corresponding to the user's utterance into text using a voice recognition function. The reception unit 20, for example, associates the voice information and text information corresponding to the user U's utterance with each utterance from the user U and stores them in the storage unit 13.

The reception unit 20 also accepts the setting or change of confidentiality conditions that specify whether masking is required. For example, the reception unit 20 accepts the setting or change of confidentiality conditions when confidentiality condition information is input or changed in response to an operation by the user U on the operation unit 12.

[3.8.2. Generation unit 21]
In the text conversion function, the generation unit 21 generates utterance text information, which is text information obtained by converting the utterance of the user U received by the reception unit 20 into text by masking parts that satisfy confidentiality conditions.

The confidentiality conditions are, for example, words that indicate privacy-related content such as addresses, names, and telephone numbers, or words that indicate content that violates public order and morals. Words that indicate content that violates public order and morals include, for example, discriminatory or derogatory words, obscene words, and words that condone or encourage crime. As described above, the confidentiality conditions can be generated or changed by user U.

The generation unit 21 has a trained model that receives, for example, audio information or text information corresponding to user U's speech, and outputs a confidentiality score indicating the degree of confidentiality for each of the multiple element parts that make up user U's speech.

The generation unit 21 inputs the audio information or text information corresponding to the user U's utterance received by the reception unit 20 into the trained model, and obtains from the trained model a confidentiality score indicating the degree of confidentiality for each of the multiple element parts that make up the user U's utterance. The generation unit 21 generates utterance text information by regarding element parts among the multiple element parts whose confidentiality scores are equal to or greater than a threshold as parts that satisfy the confidentiality conditions.

The generation unit 21 may also have a trained model that receives, for example, audio information or text information corresponding to user U's speech as input, and outputs, as speech text information, speech text information in which parts of user U's speech that satisfy confidentiality conditions are masked.

In this case, the generation unit 21 inputs the voice information or text information corresponding to the user U's utterance received by the reception unit 20 into the trained model, and can obtain the text information output from the trained model as utterance text information.

The generation unit 21 may have multiple trained models according to the intended use of the spoken text information, for example. The intended use of the spoken text information may be for email, SNS, word-of-mouth posting, or electronic bulletin boards, for example. The generation unit 21 can determine the intended use of the spoken text information according to, for example, an operation by the user U on the operation unit 12, a specification made by the user U through speech, or the type of application displayed on the display unit 11.

The generation unit 21 may have, for example, multiple trained models according to the context of the user U. The context includes the current location of the user U or the movement state of the user U. The current location or movement state of the user U is determined based on, for example, the location detected by the location detection unit 16.

The trained model is generated by machine learning using a neural network, such as a convolutional neural network or a recurrent neural network, but is not limited to such examples. For example, instead of a neural network, the trained model may be generated using machine learning using other learning algorithms, such as learning algorithms for regression methods such as linear regression, multiple regression, or logistic regression.

Furthermore, instead of or in addition to the trained model, the generation unit 21 may have a confidential information table containing information indicating multiple words to be concealed. In this case, the information processing device 1 can generate spoken text information by treating text information or audio information corresponding to words included in the confidential information table as the part that satisfies the confidentiality conditions.

The character strings included in the confidential information table may be character strings expressed using regular expressions. In this case, the text information or audio information specified by the regular expressions in the confidential information table is used as the portion that satisfies the confidentiality conditions to generate the spoken text information. For example, in the case of a phone number, a character string expressed using a regular expression might be "[0-9]{3}-[0-9]{4}-[0-9]{4}".

Furthermore, instead of or in addition to the trained model, the generation unit 21 may have a confidentiality information table containing information on confidentiality for each different word. The confidentiality information is information indicating the confidentiality level, which is set to, for example, two or more levels. For example, the generation unit 21 can lower the confidentiality level to be masked as the proportion of words with set confidentiality levels included in the user U's speech increases.

For example, suppose there are three confidentiality levels, from level 1 to level 3, with the confidentiality levels increasing from level 1 to level 2 and level 3. In other words, the lowest confidentiality level is level 1, the next lowest confidentiality level is level 2, and the highest confidentiality level is level 3.

In this case, if the proportion of words with a confidentiality level set in the utterance of user U that are included is less than the first threshold, the generation unit 21 targets words with a confidentiality level of level 3 as masking targets. Furthermore, if the proportion of words with a confidentiality level set in the utterance of user U that are included is equal to or greater than the first threshold and less than the second threshold, the generation unit 21 targets words with a confidentiality level of level 2 or higher as masking targets. If the proportion of words with a confidentiality level set in the utterance of user U that are included is equal to or greater than the second threshold, the generation unit 21 targets words with a confidentiality level of level 1 or higher as masking targets.

In addition to or instead of the proportion of words set to a confidentiality level included in user U's speech, the generation unit 21 can also determine the confidentiality level to be masked depending on the purpose of the spoken text information and the context of user U. User U can also specify the confidentiality level to be masked by operating the operation unit 12 or by speaking to the information processing device 1. For example, the generation unit 21 can also mask words with a confidentiality level equal to or higher than the level specified by the reception unit 20.

Masking is performed by, for example, replacing portions of user U's speech that satisfy confidentiality conditions with at least one of specific characters, symbols, and patterns. Specific characters include, for example, "X," "・," "-," and "?", and symbols include, for example, "□," "◆," "※," and "◯." Patterns include, for example, checkerboard patterns and geometric patterns. Masking may also be performed by, for example, replacing portions of user U's speech that satisfy confidentiality conditions with spaces.

The generation unit 21 can also generate utterance text information by using both the trained model and the secret information table. The generation unit 21 also stores, in the storage unit 13, a combination of audio information and text information corresponding to the utterance of the user U for each utterance of the user U.

In addition, instead of masking with characters or symbols with the same number of characters as the part that satisfies the confidentiality conditions, the generation unit 21 can also mask with characters or symbols with a different number of characters than the part that satisfies the confidentiality conditions, for example, when the confidentiality level is equal to or higher than a threshold value.

[3.8.3. Providing Department 22]
The providing unit 22 provides the utterance text information generated by the generating unit 21. For example, the providing unit 22 provides the utterance text information to the user U by displaying the utterance text information on the display unit 11.

Figure 3 is a diagram showing an example of spoken text information displayed on the display unit 11 of the information processing device 1 according to the embodiment. The example shown in Figure 3 is an example of spoken text information displayed on the display unit 11 of the information processing device 1 when user U utters, "When I went shopping at the AAA supermarket near my house, I met and talked with Tokkyo Hanako, and she told me that her son, Jiro, joined BBB Shoji this year."

The spoken text information shown in Figure 3 is, "When I went shopping at the XXXXXX store near my house, I met and talked with Mr./Ms. XXX, and I learned that their son, XX, joined the XXXXXX store this year." In the spoken text information shown in Figure 3, the parts that satisfy the confidentiality conditions are masked with the characters "X," including "AAA Supermarket," "Hanako Patent," and "BBB Trading."

Masking is performed by replacing parts of user U's utterance that satisfy the confidentiality conditions with, for example, at least one of specific characters, symbols, and patterns. In the example shown in Figure 3, spoken text information is generated in which parts of user U's utterance that satisfy the confidentiality conditions are replaced with the letter "X."

The providing unit 22 can also convert the spoken text information into audio information using voice synthesis and output the converted audio information to the audio output unit 15. This allows the providing unit 22 to provide the spoken text information to the user U by voice. When converting the spoken text information into audio information using voice synthesis, the providing unit 22 can convert masked parts of the spoken text information into a specific sound (for example, a "beep" sound), and can also silence the masked parts of the spoken text information.

In addition, based on the dialogue with user U, the providing unit 22 can transmit information corresponding to user U's utterances to the information providing device 2, obtain information corresponding to user U's utterances from the information providing device 2, and control peripheral devices corresponding to user U's utterances.

Furthermore, when a masked portion of the speech text information displayed on the display unit 11 is selected, the providing unit 22 unmasks the masking. The user U can select the masked portion by, for example, operating the operating unit 12.

Figure 4 is a diagram showing an example in which masking of a selected portion of spoken text information displayed on the display unit 11 of the information processing device 1 according to the embodiment is removed. In the example shown in Figure 4, "XXXXXX" is selected by user U, and the masking of "XXXXXX" selected by user U is removed, displaying "BBB Trading."

The providing unit 22 can generate, as speech information, information including, for example, text information obtained by converting user U's speech into text without masking, and information indicating the location of parts of user U's speech that satisfy confidentiality conditions. In this case, the providing unit 22 can provide the speech text information to user U or transmit it to an external device via the communication unit 10 based on the generated speech information.

3.8.4. Learning Unit 23
The learning unit 23 can generate and update the trained model. For example, the learning unit 23 generates and updates the trained model using training data that includes speech information and text information corresponding to utterances of the user U that satisfy the confidentiality conditions.

The learning data is generated, for example, based on the user U's past speech history. For example, the learning unit 23 generates the learning data based on the user U's speech that has been accepted by the accepting unit 20 as words that satisfy the confidentiality conditions.

The learning unit 23 can also generate learning data by displaying the user U's past speech history on the display unit 11 and having the user U select words that satisfy the confidentiality conditions by operating the operation unit 12, etc.

For example, the learning unit 23 acquires user U's past speech history from the storage unit 13. User U's past speech history includes voice information and text information corresponding to user U's past utterances, and the learning unit 23 generates, as learning data, a combination of voice information and text information corresponding to words that satisfy the confidentiality conditions selected by user U.

The learning unit 23 can also generate, as training data, a combination of information indicating the position of words that satisfy the confidentiality conditions selected by user U and audio information or text information corresponding to user U's past utterances. The trained model generated or updated using such training data is a model that outputs a confidentiality score indicating the degree of confidentiality for each position (position from the first word) of a word included in the audio information or text information corresponding to user U's utterance.

In this case, the generation unit 21 inputs audio information or text information corresponding to the user U's utterance into the trained model, and generates the utterance text information by masking positions where the confidentiality score output from the trained model is equal to or greater than a threshold. Note that the generation unit 21 can, for example, set the threshold higher the higher the confidentiality level described above.

The learning unit 23 can generate multiple trained models according to the purpose of the spoken text information, and can also generate multiple trained models according to the context of the user U. Note that the learning unit 23 is only required to be able to generate the trained models described above, and the learning process performed by the learning unit 23 is not limited to the process described above.

4. Processing Procedure
Next, a procedure of information processing by the processing unit 17 of the information processing device 1 according to the embodiment will be described. Figures 5 and 6 are flowcharts showing an example of information processing by the processing unit 17 of the information processing device 1 according to the embodiment.

First, we will explain Figure 5. Figure 5 shows an example of the utterance text information generation process performed by the information processing device 1. The processing unit 17 of the information processing device 1 accepts an utterance from the user U (step S10).

Next, the processing unit 17 generates utterance text information, which is text information obtained by converting user U's utterance received in step S10 into text by masking parts that satisfy the confidentiality conditions (step S11). In the processing of step S11, the processing unit 17 inputs, for example, audio information or character information corresponding to user U's utterance into the trained model, and obtains from the trained model a confidentiality score indicating the degree of confidentiality for each of the multiple element parts that make up user U's utterance. The processing unit 17 generates utterance text information by treating element parts of the multiple element parts whose confidentiality scores are equal to or greater than a threshold as parts that satisfy the confidentiality conditions.

Then, the processing unit 17 provides the spoken text information generated in step S11 (step S12), and ends the processing shown in FIG. 5. For example, in the processing of step S12, the processing unit 17 can, for example, display the spoken text information on the display unit 11 or output the spoken text information as audio from the audio output unit 15.

Next, we will explain Figure 6. Figure 6 shows an example of the learning process performed by the information processing device 1. The processing unit 17 of the information processing device 1 generates learning data (step S20). In the process of step S20, for example, the processing unit 17 generates learning data based on the utterance of user U that has been accepted by the accepting unit 20 as words that satisfy the confidentiality conditions.

Next, a trained model is generated or updated using the training data generated in step S20 (step S21), and the process shown in Figure 6 is terminated.

5. Modifications
The above-described information processing device 1 has been described as a device such as a terminal device operated by a user U, but the information processing device 1 may also be a server device, etc. Fig. 7 is a diagram showing another example of the configuration of the information processing device 1 according to the embodiment.

The terminal device 3 shown in FIG. 7 converts the speech of the user U into voice information or text information, and transmits the user speech information including the converted voice information or text information to the information processing device 1 via the network N. The reception unit 20 of the information processing device 1 shown in FIG. 7 receives the user speech information transmitted from the terminal device 3 via the network N and the communication unit 10, thereby receiving the speech of the user U.

The generation unit 21 of the information processing device 1 generates spoken text information based on the audio information or text information included in the user utterance information. The provision unit 22 provides the spoken text information by transmitting the spoken text information generated by the generation unit 21 to the terminal device 3 via the communication unit 10 and the network N. The terminal device 3 receives the spoken text information transmitted from the information processing device 1 and displays the received spoken text information or outputs it as audio.

Note that some of the functions of the processing unit 17 in the information processing device 1 shown in FIG. 2 may be implemented by the terminal device 3 shown in FIG. 7. Also, some of the functions of the processing unit 17 in the information processing device 1 shown in FIG. 2 may be implemented by the information providing device 2.

Furthermore, the information processing device 1 shown in FIG. 2 has a trained model and a confidential information table for each user U, and in this case the trained model can also be called an on-device model. Furthermore, the information processing device 1 shown in FIG. 7 may have a trained model and a confidential information table that are common to all users U, or may have a trained model and a confidential information table for each user U.

In the example described above, the information processing device 1 shown in FIG. 2 has been described as having an AI assistant function that supports interactive voice operations, but the information processing device 1 may also be a device that does not have an AI assistant function. For example, the information processing device 1 may be a voice recorder, etc. The text conversion function of the information processing device 1 can also be used, for example, when converting meeting minutes into text.

6. Hardware Configuration
The information processing device 1 according to the embodiment described above is realized by, for example, a computer 80 configured as shown in Fig. 8. Fig. 8 is a hardware configuration diagram showing an example of the computer 80 that realizes the functions of the information processing device 1 according to the embodiment. The computer 80 has a CPU 81, a RAM 82, a ROM (Read Only Memory) 83, an HDD (Hard Disk Drive) 84, a communication interface (I/F) 85, an input/output interface (I/F) 86, and a media interface (I/F) 87.

The CPU 81 operates based on programs stored in the ROM 83 or HDD 84 and controls each component. The ROM 83 stores a boot program executed by the CPU 81 when the computer 80 starts up, as well as programs that depend on the computer 80's hardware.

The HDD 84 stores programs executed by the CPU 81 and data used by such programs. The communication interface 85 receives data from other devices via the network N (see Figure 2) and sends it to the CPU 81, and transmits data generated by the CPU 81 to other devices via the network N.

The CPU 81 controls output devices such as displays and printers, and input devices such as keyboards and mice, via the input/output interface 86. The CPU 81 acquires data from the input devices via the input/output interface 86. The CPU 81 also outputs data generated via the input/output interface 86 to the output devices.

The media interface 87 reads programs or data stored on the recording medium 88 and provides them to the CPU 81 via the RAM 82. The CPU 81 loads the programs from the recording medium 88 onto the RAM 82 via the media interface 87 and executes the loaded programs. The recording medium 88 may be, for example, an optical recording medium such as a DVD (Digital Versatile Disc) or a PD (Phase Change Rewritable Disk), a magneto-optical recording medium such as an MO (Magneto-Optical disk), a tape medium, a magnetic recording medium, or a semiconductor memory.

For example, when the computer 80 functions as the information processing device 1 according to the embodiment, the CPU 81 of the computer 80 executes programs loaded onto the RAM 82 to realize the functions of the processing unit 17. In addition, the HDD 84 stores data in the storage unit 13. The CPU 81 of the computer 80 reads and executes these programs from the recording medium 88, but as another example, these programs may be obtained from another device via the network N.

[7. Other]
Furthermore, among the processes described in the above embodiments and modifications, all or part of the processes described as being performed automatically can be performed manually, or all or part of the processes described as being performed manually can be performed automatically using known methods. In addition, the information including the processing procedures, specific names, various data, and parameters shown in the above documents and drawings can be changed as desired unless otherwise specified. For example, the various information shown in each drawing is not limited to the information shown in the drawings.

Furthermore, the components of each device shown in the figure are functional concepts and do not necessarily have to be physically configured as shown. In other words, the specific form of distribution and integration of each device is not limited to that shown, and all or part of them can be functionally or physically distributed and integrated in any unit depending on various loads, usage conditions, etc.

Furthermore, the above-described embodiments and variations can be combined as appropriate to the extent that the processing content is not contradictory.

8. Effects
As described above, the information processing device 1 according to the embodiment includes a receiving unit 20, a generating unit 21, and a providing unit 22. The receiving unit 20 receives an utterance from a user U. The generating unit 21 generates utterance text information, which is text information obtained by converting the utterance of the user U into text by masking a portion of the utterance received by the receiving unit 20 that satisfies a confidentiality condition. The providing unit 22 provides the utterance text information generated by the generating unit 21. This enables the information processing device 1 to improve convenience for the user U.

The generation unit 21 also has a trained model that receives as input audio information or text information corresponding to the user U's utterance received by the reception unit 20, and outputs a confidentiality score indicating the degree of confidentiality for each of multiple element parts that make up the user U's utterance. Using the trained model, the generation unit 21 generates utterance text information by treating element parts among the multiple element parts whose confidentiality scores are equal to or greater than a threshold as parts that satisfy the confidentiality conditions. This allows the information processing device 1 to appropriately detect parts that satisfy the confidentiality conditions.

The generation unit 21 also has a trained model that receives as input audio information or text information corresponding to the user U's utterance received by the reception unit 20, and outputs text information obtained by masking portions of the user U's utterance that satisfy the confidentiality conditions as utterance text information, and generates utterance text information using the trained model. This allows the information processing device 1 to appropriately detect portions that satisfy the confidentiality conditions.

The generation unit 21 also generates spoken text information using a confidential information table that includes information indicating multiple words to be concealed. This allows the information processing device 1 to appropriately detect parts that satisfy the confidentiality conditions.

The information processing device 1 also includes a learning unit 23 that updates the trained model based on the user U's past speech history. This allows the information processing device 1 to improve the accuracy of detecting parts that satisfy the confidentiality conditions using the trained model.

The generation unit 21 also masks the portions of the user U's speech that satisfy the confidentiality conditions by converting those portions into at least one of specific characters, symbols, and patterns. This allows the information processing device 1 to appropriately present the masked portions to the user U.

In addition, the providing unit 22 provides the utterance text information to the user U by displaying the utterance text information on the display unit 11. This allows the information processing device 1 to improve the convenience of the user U.

Furthermore, when a masked portion of the speech text information displayed on the display unit 11 is selected, the providing unit 22 removes the masking. This enables the information processing device 1 to improve the convenience of the user U.

The above describes in detail the embodiments of the present application based on the drawings, but this is merely an example, and the present invention can be implemented in other forms that incorporate various modifications and improvements based on the knowledge of those skilled in the art, including the aspects described in the Disclosure of the Invention section.

Furthermore, the "section, module, unit" mentioned above can be read as "means" or "circuit." For example, an acquisition unit can be read as an acquisition means or an acquisition circuit.

REFERENCE SIGNS LIST 1 Information processing device 2 Information providing device 3 Terminal device 10 Communication unit 11 Display unit 12 Operation unit 13 Storage unit 14 Voice input unit 15 Voice output unit 16 Position detection unit 17 Processing unit 20 Reception unit 21 Generation unit 22 Provision unit 23 Learning unit 100 Information processing system

Claims (11)

a reception unit that receives utterances from a user;
a generation unit that generates utterance text information, which is text information obtained by converting the user's utterance into text by masking a portion of the utterance that satisfies a confidentiality condition among the utterance received by the reception unit;
a providing unit that provides the utterance text information generated by the generating unit ,
The generation unit
Using a confidential information table including information indicating a plurality of words to be concealed and information indicating a confidentiality level for each of the words, the greater the proportion of words to which a confidentiality level is set that are included in the utterance, the lower the confidentiality level to be the target of the masking, and the speech text information is generated.
1. An information processing device comprising: a reception unit that receives utterances from a user;
a generation unit that generates utterance text information, which is text information obtained by converting the user's utterance into text by masking a portion of the utterance that satisfies a confidentiality condition among the utterance received by the reception unit;
a providing unit that provides the utterance text information to the user by displaying the utterance text information generated by the generating unit on a display unit,
The providing unit
When a masked portion of the speech text information displayed on the display unit is selected, the masking is released. The generation unit
2. The information processing device according to claim 1, further comprising a trained model that receives as input audio information or text information corresponding to the utterance received by the reception unit and outputs a confidentiality score indicating the degree of confidentiality for each of a plurality of element parts constituting the user's utterance, and that uses the trained model to generate the utterance text information by treating element parts among the plurality of element parts whose confidentiality score is equal to or greater than a threshold as parts that satisfy the confidentiality condition. The generation unit
2. The information processing device according to claim 1, further comprising a trained model that receives as input audio information or text information corresponding to the utterance received by the reception unit, and outputs as the utterance text information text information obtained by masking a portion of the user's utterance that satisfies the confidentiality conditions, and that generates the utterance text information using the trained model. The generation unit
The information processing apparatus according to claim 2 , wherein the speech text information is generated using a confidential information table including information indicating a plurality of words to be confidentialized. The information processing device according to claim 3 , further comprising a learning unit that updates the trained model based on a past speech history of the user. The generation unit
The information processing device according to any one of claims 1 to 5, characterized in that the part of the user's speech that satisfies the confidentiality condition is masked by converting the part that satisfies the confidentiality condition into at least one of a specific character, symbol, and pattern. 1. A computer-implemented information processing method, comprising:
a reception step of receiving an utterance from a user;
a generating step of generating utterance text information, which is text information obtained by converting the user's utterance into text by masking a portion of the utterance that satisfies a confidentiality condition among the utterance received by the receiving step;
a providing step of providing the utterance text information generated by the generating step ,
The generating step includes:
Using a confidential information table including information indicating a plurality of words to be concealed and information indicating a confidentiality level for each of the words, the greater the proportion of words to which a confidentiality level is set that are included in the utterance, the lower the confidentiality level to be the target of the masking, and the speech text information is generated.
1. An information processing method comprising: A computer-implemented information processing method, comprising:
a reception step of receiving an utterance from a user;
a generating step of generating utterance text information, which is text information obtained by converting the user's utterance into text by masking a portion of the utterance that satisfies a confidentiality condition among the utterance received by the receiving step;
a providing step of providing the utterance text information to the user by displaying the utterance text information generated by the generating step on a display unit,
The providing step includes:
When a masked portion of the speech text information displayed on the display unit is selected, the masking is released.
An information processing method comprising: A reception procedure for receiving a user's utterance;
a generation step of generating utterance text information, which is text information obtained by converting the user's utterance into text by masking a portion of the utterance that satisfies a confidentiality condition among the utterance received by the reception step;
a providing step of providing the utterance text information generated by the generating step ;
The generating procedure includes:
Using a confidential information table including information indicating a plurality of words to be concealed and information indicating a confidentiality level for each of the words, the greater the proportion of words to which a confidentiality level is set that are included in the utterance, the lower the confidentiality level to be the target of the masking, and the speech text information is generated.
An information processing program characterized by: A reception procedure for receiving a user's utterance;
a generation step of generating utterance text information, which is text information obtained by converting the user's utterance into text by masking a portion of the utterance that satisfies a confidentiality condition among the utterance received by the reception step;
a providing step of providing the utterance text information to the user by displaying the utterance text information generated by the generating step on a display unit;
The providing step comprises:
When a masked portion of the speech text information displayed on the display unit is selected, the masking is released.
An information processing program characterized by: