JP7781997B2 - system - Google Patents

Description

The technology disclosed herein relates to a system.

Patent document 1 discloses a persona chatbot control method executed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot's character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Japanese Patent Application Laid-Open No. 2022-180282

With conventional technology, emotional voices and negative comments during online calls can lead to reduced work efficiency.

The system according to the embodiment aims to convert emotional voices and negative comments into rational voices during online conversations.

The system according to the embodiment includes an acquisition unit, an analysis unit, a filtering unit, and a provision unit. The acquisition unit acquires voice data. The analysis unit analyzes the voice data acquired by the acquisition unit and classifies the speaker's emotion. The filtering unit converts negative emotional voice into rational voice based on the emotion classified by the analysis unit. The provision unit provides the user with the voice converted by the filtering unit.

The system according to the embodiment can convert emotional voices and negative comments into rational voices during online calls.

1 is a conceptual diagram illustrating an example of the configuration of a data processing system according to a first embodiment. 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. FIG. 10 is a conceptual diagram illustrating an example of the configuration of a data processing system according to a second embodiment. FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. FIG. 10 is a conceptual diagram illustrating an example of the configuration of a data processing system according to a third embodiment. FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. 1 shows an emotion map onto which multiple emotions are mapped. 1 shows an emotion map onto which multiple emotions are mapped.

Below, an example of an embodiment of a system relating to the technology disclosed herein will be described with reference to the accompanying drawings.

First, let me explain the terminology used in the following explanation.

In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Furthermore, a processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose Computing on Graphics Processing Units), an APU (Accelerated Processing Unit), or a TPU (Tensor Processing Unit).

In the following embodiments, coded random access memory (RAM) is memory in which information is temporarily stored and is used as work memory by the processor.

In the following embodiments, the coded storage refers to one or more non-volatile storage devices that store various programs, parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

In the following embodiments, a communication I/F (Interface) is an interface that includes a communication processor, an antenna, and the like. The communication I/F controls communication between multiple computers. Examples of communication standards that can be applied to the communication I/F include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

In the following embodiments, "A and/or B" is synonymous with "at least one of A and B." In other words, "A and/or B" means that it may be A alone, B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and/or B" also applies when three or more things are expressed connected by "and/or."

[First embodiment]
FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

As shown in FIG. 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

The data processing device 12 includes a computer 22, a database 24, and a communication I/F 26. The computer 22 includes a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and communication I/F 26 are also connected to the bus 34. The communication I/F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and/or a LAN (Local Area Network).

The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I/F 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

The reception device 38 includes a touch panel 38A and a microphone 38B, and receives user input. The touch panel 38A detects contact with a pointer (e.g., a pen or a finger) to receive user input via the pointer. The microphone 38B detects the user's voice to receive user input via voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 (see Figure 2) acquires the data indicating the user input.

The output device 40 includes a display 40A and a speaker 40B, and presents data to the user by outputting the data in a form that the user can perceive (e.g., audio and/or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

The communication I/F 44 is connected to the network 54. The communication I/Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54.

Figure 2 shows an example of the main functions of the data processing device 12 and smart device 14.

As shown in FIG. 2, in the data processing device 12, specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

Storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion. The emotion estimation function (emotion identification function) using the emotion identification model 59 performs various estimations and predictions regarding the user's emotions, including estimation and prediction of the user's emotion, but is not limited to these examples. Furthermore, emotion estimation and prediction also include, for example, emotion analysis.

In the smart device 14, the specific processing is performed by the processor 46. The storage 50 stores the specific processing program 60. The specific processing program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the specific processing program 60 from the storage 50 and executes the read specific processing program 60 on the RAM 48. The specific processing is realized by the processor 46 operating as the control unit 46A in accordance with the specific processing program 60 executed on the RAM 48. The smart device 14 also has a data generation model and emotion identification model similar to the data generation model 58 and emotion identification model 59, and can use these models to perform processing similar to that of the specific processing unit 290.

Note that a device other than the data processing device 12 may have the data generation model 58. For example, a server device (e.g., a generation server) may have the data generation model 58. In this case, the data processing device 12 communicates with the server device having the data generation model 58 to obtain processing results (prediction results, etc.) using the data generation model 58. The data processing device 12 may also be a server device, or a terminal device owned by a user (e.g., a mobile phone, robot, home appliance, etc.). Next, an example of processing by the data processing system 10 according to the first embodiment will be described.

(Example 1)
An online call filtering system according to an embodiment of the present invention filters emotional voices and negative comments during online calls. This system acquires audio data from online calls and uses emotion recognition AI to analyze the audio data and classify the speaker's emotions. For example, emotions such as anger, sadness, and joy are identified. The emotion recognition AI determines the emotion based on characteristics such as the tone, pitch, and speed of the voice. Next, negative emotional voices are filtered into rational voices. For example, angry voices are converted into calm voices. This filtering process is performed based on the emotions determined by the emotion recognition AI. Furthermore, for negative comments, a generation AI is used to present audio that omits comments unrelated to the main topic of the discussion. For example, emotional comments and irrelevant topics are removed, and only important information is extracted and presented. This process allows users to work efficiently without feeling stressed. This mechanism eliminates the need to overhear emotional voices and negative comments during online calls, improving work efficiency. Furthermore, a stress-free work environment is created, improving employee satisfaction. This allows the online call filtering system to filter out emotional voices and negative comments during online calls, improving work efficiency.

An online call filtering system according to an embodiment includes an acquisition unit, an analysis unit, a filtering unit, and a providing unit. The acquisition unit acquires audio data from online calls. The acquisition unit collects audio data in real time using, for example, a microphone. The acquisition unit can also read recorded audio files. For example, the acquisition unit supports audio data formats such as audio file formats, sampling rates, and bit rates. The analysis unit uses emotion recognition AI to analyze the audio data acquired by the acquisition unit and classify the speaker's emotions. The analysis unit determines emotions based on features such as tone, pitch, and speed of the audio. The emotion recognition AI receives audio data as input and outputs an emotion classification result. For example, the analysis unit analyzes the tone of the audio data and identifies emotions such as anger, sadness, and joy. The filtering unit converts negative emotional voice into rational voice based on the emotions classified by the analysis unit. For example, the filtering unit converts voice containing angry emotions into a calm tone. The filtering unit receives the determination result of the emotion recognition AI as input and outputs the converted voice. For example, the filtering unit adjusts the tone of the voice and converts it to a calm tone. The providing unit provides the voice converted by the filtering unit to the user. The providing unit plays the voice using, for example, a speaker. The providing unit can also display the converted voice as text data. For example, the providing unit converts voice data into text and displays it on a screen. In this way, the online call filtering system according to the embodiment can filter out emotional voices and negative comments during online calls, improving work efficiency.

The acquisition unit acquires audio data from online calls. The acquisition unit collects audio data in real time using, for example, a microphone. Specifically, the acquisition unit acquires clear audio data by using a high-sensitivity microphone and incorporating noise-canceling technology to reduce ambient noise. The acquisition unit can also read recorded audio files. For example, the acquisition unit supports audio data formats such as audio file formats, sampling rates, and bit rates. This allows the acquisition unit to flexibly handle audio data in various formats. The acquisition unit also acquires metadata for the audio data and provides it to the analysis and filtering units. The metadata includes the recording date and time, speaker identification information, and call context. This makes it easier for the acquisition unit to understand the context of the audio data and improve the accuracy of subsequent analysis and filtering. The acquisition unit stores the audio data in cloud storage or local storage and makes it accessible as needed. This allows the acquisition unit to efficiently manage not only real-time audio data but also past audio data and use it for analysis and filtering.

The analysis unit uses emotion recognition AI to analyze the voice data acquired by the acquisition unit and classify the speaker's emotions. The analysis unit determines emotions based on voice characteristics such as tone, pitch, and speed. The emotion recognition AI takes voice data as input and outputs emotion classification results. Specifically, the emotion recognition AI uses deep learning technology to extract features of the voice data and classify emotions based on a pre-trained model. For example, the analysis unit analyzes the tone of the voice data and identifies emotions such as anger, sadness, and joy. Furthermore, the analysis unit takes into account temporal changes in the voice data and can track emotional fluctuations in real time. This allows the analysis unit to quickly detect changes in the speaker's emotions and provide information for appropriate filtering. To improve the accuracy of the emotion recognition AI, the analysis unit regularly retrains the model and incorporates new data. This allows the analysis unit to always utilize the latest emotion recognition technology and achieve highly accurate emotion classification.

The filtering unit converts negative emotional speech into rational speech based on the emotions classified by the analysis unit. For example, the filtering unit converts speech containing angry emotions into a calm tone. Specifically, the filtering unit adjusts the tone, pitch, and speed of the speech to soften the intensity of the emotion. For example, if the emotion is strong, the filtering unit converts the speech into a calm speech by lowering the pitch, slowing the speed, and softening the tone. The filtering unit inputs the judgment results of the emotion recognition AI and outputs the converted speech. Furthermore, the filtering unit utilizes natural language processing technology to adjust the emotional expression while preserving the content of the speech. This allows the filtering unit to convert emotional expressions into rational expressions without compromising the speaker's intention. The filtering unit can adjust the filtering strength and criteria according to user settings. For example, it is possible to set it to filter only specific emotions or to change the degree of filtering depending on the intensity of the emotion. This allows the filtering unit to provide flexible filtering tailored to the user's needs.

The providing unit provides the user with the audio converted by the filtering unit. The providing unit plays the audio using, for example, a speaker. Specifically, the providing unit uses a high-quality speaker to clearly play the converted audio. The providing unit can also display the converted audio as text data. For example, the providing unit converts audio data into text and displays it on a screen. This allows the user to view information not only as audio but also as text. Furthermore, the providing unit can customize audio playback and text display settings through a user interface. For example, it is possible to adjust the audio playback speed and volume, and change the font size and display position of the text. This allows the providing unit to flexibly provide information tailored to the user's preferences. The providing unit can also save the converted audio and text data so that it can be played back or referenced later. This allows the user to review past call content and reuse it as needed. The providing unit can also link with other systems and devices to share the converted audio and text data. For example, it can link with business chat tools or email systems to automatically record and share call content. This allows the provider to offer a variety of information provision methods to improve business efficiency.

The analysis unit can determine emotions based on the tone, pitch, and speed characteristics of the voice. The analysis unit, for example, analyzes the tone of the voice to determine emotions. For example, it identifies emotions based on the pitch and intensity of the tone. The analysis unit can also analyze the pitch of the voice to determine emotions. For example, it can identify emotions based on the frequency of the pitch. The analysis unit can also analyze the speed of the voice to determine emotions. For example, it can identify emotions based on the speaking rate. This allows the analysis unit to accurately determine emotions based on the voice characteristics. Some or all of the above-mentioned processing in the analysis unit may be performed using AI, for example, or may be performed without using AI. For example, the analysis unit can input voice data into AI and have the AI perform emotion determination.

The filtering unit can convert voice containing angry emotions into a calm tone. The filtering unit, for example, converts voice containing angry emotions into a calm tone. For example, it adjusts the tone of the voice to convert it into a calm tone. The filtering unit can also adjust the pitch of the voice to convert it into a calm tone. For example, it stabilizes the pitch frequency. The filtering unit can also adjust the speed of the voice to convert it into a calm tone. For example, it keeps the speaking rate constant. In this way, the filtering unit can reduce the emotional impact by converting angry emotions into a calm tone. Some or all of the above-mentioned processing in the filtering unit may be performed using AI, or may be performed without using AI. For example, the filtering unit can input voice data into AI and have the AI perform the tone conversion.

The system is equipped with a generation unit that can eliminate negative comments and extract and present only important information. The generation unit uses a generation AI to eliminate negative comments and extract and present only important information. The generation unit, for example, removes emotional comments and irrelevant topics and extracts important information. The generation AI receives audio data as input and outputs audio data from which important information has been extracted. For example, the generation unit detects emotional comments and removes them. The generation unit can also detect irrelevant topics and remove them. For example, the generation unit detects comments unrelated to the main topic of the discussion and removes them. This allows the generation unit to efficiently obtain only important information by omitting negative comments. Some or all of the above-mentioned processing in the generation unit may be performed using, or without, the generation AI. For example, the generation unit may input audio data to the generation AI and have the generation AI extract important information.

The providing unit can provide the converted voice to the user. The providing unit, for example, provides the converted voice to the user using a speaker. For example, the providing unit plays the converted voice in real time. The providing unit can also display the converted voice as text data. For example, the providing unit converts voice data into text and displays it on a screen. In this way, the providing unit can provide the converted voice to the user, thereby realizing a stress-free work environment. Some or all of the above-mentioned processing in the providing unit may be performed using AI, for example, or may be performed without using AI. For example, the providing unit can input the converted voice data to AI and have the AI convert it into text data.

The acquisition unit can analyze the user's past call history and select the optimal acquisition method. The acquisition unit, for example, analyzes the user's past call history and selects the optimal acquisition method. For example, the acquisition unit prioritizes acquisition of calling apps that the user has used frequently in the past. The acquisition unit can also predict the audio data to be acquired during a specific time period from the user's past call history and select the optimal acquisition method. For example, the acquisition unit analyzes the user's past call history and prioritizes acquisition of only important calls. In this way, the acquisition unit can select the optimal acquisition method by analyzing the past call history. Some or all of the above-mentioned processing in the acquisition unit may be performed using AI, for example, or may be performed without using AI. For example, the acquisition unit can input call history data into AI and have the AI select the optimal acquisition method.

When acquiring voice data, the acquisition unit can filter the voice data based on the user's current work situation or areas of interest. For example, if the user is in a meeting, the acquisition unit acquires only voice data related to the meeting. For example, the acquisition unit prioritizes acquiring voice data related to the meeting agenda. Furthermore, if the user is concentrating on a particular project, the acquisition unit can also prioritize acquiring voice data related to that project. For example, the acquisition unit filters relevant voice data based on the progress of the project. Furthermore, the acquisition unit can filter and acquire relevant voice data based on the user's areas of interest. For example, the acquisition unit prioritizes acquiring voice data related to the user's area of expertise or topics of interest. In this way, the acquisition unit can acquire highly relevant data by filtering based on the work situation or areas of interest. Some or all of the above-mentioned processing in the acquisition unit may be performed using, or without, AI. For example, the acquisition unit can input data on the work situation and areas of interest into AI and have the AI perform the filtering.

When acquiring voice data, the acquisition unit can prioritize acquiring highly relevant data based on the user's geographical location information. For example, if the user is in a specific area, the acquisition unit prioritizes acquiring voice data related to that area. For example, the acquisition unit acquires voice data related to local events and news. Furthermore, if the user is on the move, the acquisition unit can also acquire highly relevant voice data based on the user's current location. For example, the acquisition unit acquires voice data containing information relevant to the user while on the move. Furthermore, the acquisition unit can filter and acquire optimal voice data based on the user's geographical location information. For example, the acquisition unit filters voice data based on geographical relevance. This allows the acquisition unit to acquire highly relevant data based on geographical location information, enabling efficient data collection. Some or all of the above-mentioned processing in the acquisition unit may be performed using, or without, AI. For example, the acquisition unit may input geographical location information to AI and cause the AI to acquire highly relevant data.

When acquiring voice data, the acquisition unit can analyze the user's social media activity and acquire related data. The acquisition unit, for example, acquires voice data related to topics frequently mentioned by the user on social media. For example, the acquisition unit analyzes the content of the user's social media posts and acquires related voice data. The acquisition unit can also analyze topics of interest from the user's social media activity and acquire related voice data. For example, the acquisition unit acquires voice data related to topics of interest based on the user's social media activity. The acquisition unit can also filter and acquire optimal voice data based on the content of the user's social media posts. For example, the acquisition unit filters voice data based on trends and topics on social media. In this way, the acquisition unit can acquire highly relevant data by analyzing social media activity. Some or all of the above-mentioned processing by the acquisition unit may be performed using, or without, AI. For example, the acquisition unit can input social media data into AI and have the AI acquire related data.

During analysis, the analysis unit can adjust the level of detail of the analysis based on the importance of the audio data. For example, the analysis unit performs a detailed analysis of important audio data. For example, the analysis unit analyzes the content of the audio data in detail and extracts important information. The analysis unit can also perform a simplified analysis of audio data with low importance. For example, the analysis unit analyzes the overview of the audio data and provides concise information. The analysis unit can also dynamically adjust the level of detail of the analysis based on the importance of the audio data. For example, the analysis unit adjusts the depth and scope of the analysis based on the importance of the audio data. This allows the analysis unit to adjust the level of detail of the analysis based on the importance of the audio data, enabling efficient analysis. Some or all of the above-mentioned processing in the analysis unit may be performed using AI, for example, or may be performed without AI. For example, the analysis unit can input the importance of the audio data to AI and have the AI adjust the level of detail of the analysis.

During analysis, the analysis unit can apply different analysis algorithms depending on the category of audio data. For example, the analysis unit applies a minutes generation algorithm to conference audio data. For example, the analysis unit automatically summarizes the contents of the conference and generates minutes. The analysis unit can also apply a customer satisfaction analysis algorithm to customer support audio data. For example, the analysis unit analyzes the content of customer comments and evaluates satisfaction. The analysis unit can also apply a learning progress analysis algorithm to educational audio data. For example, the analysis unit analyzes educational content and evaluates learning progress. In this way, the analysis unit can apply an appropriate analysis algorithm depending on the category of audio data, thereby improving analysis accuracy. Some or all of the above-mentioned processing in the analysis unit may be performed using AI, for example, or may be performed without using AI. For example, the analysis unit can input the category of audio data into AI and have the AI apply the analysis algorithm.

During analysis, the analysis unit can determine analysis priorities based on the time the audio data was acquired. The analysis unit, for example, prioritizes analysis of the most recent audio data. For example, the analysis unit can quickly analyze the most recent audio data and provide results in real time. The analysis unit can also prioritize analysis of audio data acquired immediately after an important event. For example, the analysis unit can prioritize analysis of audio data acquired immediately after the event to extract important information. The analysis unit can also dynamically adjust analysis priorities based on the user's schedule. For example, the analysis unit can prioritize analysis of audio data related to important tasks based on the user's schedule. This enables the analysis unit to determine analysis priorities based on the time the audio data was acquired, enabling efficient analysis. Some or all of the above-mentioned processing in the analysis unit may be performed using AI, for example, or without AI. For example, the analysis unit can input the time the audio data was acquired into AI and have the AI determine the analysis priorities.

During analysis, the analysis unit can adjust the order of analysis based on the relevance of the audio data. For example, the analysis unit prioritizes analysis of highly relevant audio data. For example, the analysis unit quickly analyzes highly relevant audio data and extracts important information. The analysis unit can also postpone analysis of less relevant audio data. For example, the analysis unit postpones analysis of less relevant audio data and prioritizes analysis of more important data. The analysis unit can also dynamically adjust the order of analysis based on the relevance of the audio data. For example, the analysis unit dynamically adjusts the order of analysis based on the relevance of the audio data and performs efficient analysis. In this way, the analysis unit can adjust the order of analysis based on the relevance of the audio data, enabling efficient analysis. Some or all of the above-mentioned processing in the analysis unit may be performed using AI, for example, or may be performed without using AI. For example, the analysis unit can input the relevance of the audio data to AI and have the AI adjust the order of analysis.

The filtering unit can improve the accuracy of filtering based on the interrelationships between audio data during filtering. The filtering unit, for example, takes into account the context of the audio data and filters related audio collectively. For example, the filtering unit analyzes the context of the audio data and extracts related information. The filtering unit can also analyze the interrelationships between audio data and extract and filter important parts. For example, the filtering unit extracts important information based on the interrelationships between audio data. The filtering unit can also dynamically adjust the accuracy of filtering based on the interrelationships between audio data. For example, the filtering unit improves the accuracy of filtering based on the interrelationships between audio data. In this way, the filtering unit improves the accuracy of filtering by taking the interrelationships between audio data into consideration. Some or all of the above-mentioned processing in the filtering unit may be performed using AI, for example, or may be performed without using AI. For example, the filtering unit can input the interrelationships between audio data to AI and have the AI improve the accuracy of filtering.

When filtering, the filtering unit can take into account the attribute information of the person who submitted the audio data. The filtering unit, for example, prioritizes filtering of important audio data based on the submitter's job title. For example, the filtering unit prioritizes filtering of audio data containing important information based on the submitter's job title. The filtering unit can also filter related audio data based on the submitter's field of expertise. For example, the filtering unit filters audio data containing information related to the submitter's field of expertise. The filtering unit can also dynamically adjust the filtering criteria based on the submitter's attribute information. For example, the filtering unit adjusts the filtering criteria based on the submitter's attribute information. This enables the filtering unit to perform appropriate filtering by taking the submitter's attribute information into consideration. Some or all of the above-mentioned processing in the filtering unit may be performed using AI, for example, or may be performed without using AI. For example, the filtering unit can input the submitter's attribute information into AI and have the AI perform the filtering.

The filtering unit can perform filtering while taking into account the geographical distribution of the audio data. For example, the filtering unit prioritizes filtering of audio data related to a specific region. For example, the filtering unit filters audio data related to local events or news. The filtering unit can also filter highly relevant audio data based on the geographical distribution. For example, the filtering unit filters audio data based on geographical relevance. The filtering unit can also improve the accuracy of filtering by taking the geographical distribution of the audio data into account. For example, the filtering unit improves the accuracy of filtering based on the geographical distribution. In this way, the filtering unit can efficiently filter highly relevant audio data by taking the geographical distribution into account. Some or all of the above-mentioned processing in the filtering unit may be performed using AI, for example, or may be performed without using AI. For example, the filtering unit can input the geographical distribution data to AI and have the AI perform the filtering.

When filtering, the filtering unit can improve the accuracy of filtering by referring to literature related to the audio data. For example, the filtering unit prioritizes filtering of important audio data based on related literature. For example, the filtering unit adjusts the filtering criteria for the audio data by referring to related literature. The filtering unit can also analyze literature related to the audio data and dynamically improve the accuracy of filtering. For example, the filtering unit improves the accuracy of filtering based on related literature. In this way, the filtering unit improves the accuracy of filtering by referring to related literature. Some or all of the above-mentioned processing in the filtering unit may be performed using, or without, AI, for example. For example, the filtering unit can input related literature data into AI and have the AI improve the accuracy of filtering.

When providing the display method, the providing unit can select the optimal display method by referring to the user's past operation history. For example, the providing unit prioritizes providing display methods that the user has used in the past. For example, the providing unit selects the optimal display method based on the user's past operation history. The providing unit can also predict and provide the optimal display method from the user's past operation history. For example, the providing unit analyzes past operation patterns and provides the optimal display method. The providing unit can also analyze the user's past operation history and provide a display method with high visibility. For example, the providing unit selects a display method with high visibility based on the past operation history. In this way, the providing unit can provide the optimal display method by referring to the past operation history. Some or all of the above-mentioned processing in the providing unit may be performed using AI, for example, or may be performed without using AI. For example, the providing unit can input operation history data to AI and have the AI select the optimal display method.

When providing the display, the providing unit can select the optimal display method by taking into account the user's device information. For example, if the user is using a smartphone, the providing unit provides a display method that matches the screen size. For example, the providing unit provides a display method optimized for the smartphone screen size. Furthermore, if the user is using a tablet, the providing unit can also provide a display method optimized for a large screen. For example, the providing unit provides a display method optimized for the tablet screen size. Furthermore, if the user is using a smartwatch, the providing unit can also provide a display method that is simple and highly visible. For example, the providing unit provides a display method optimized for the smartwatch screen size. In this way, the providing unit can provide the optimal display method by taking the device information into account. Some or all of the above-mentioned processing in the providing unit may be performed using AI, for example, or may be performed without using AI. For example, the providing unit can input device information into AI and have the AI select the optimal display method.

The generation unit can improve the accuracy of generation by taking into account the interrelationships between audio data during generation. For example, the generation unit generates related audio all at once, taking into account the context of the audio data. For example, the generation unit analyzes the context of the audio data and extracts and generates related information. The generation unit can also analyze the interrelationships between audio data and extract and generate important parts. For example, the generation unit extracts and generates important information based on the interrelationships between audio data. The generation unit can also dynamically adjust the accuracy of generation based on the interrelationships between audio data. For example, the generation unit improves the accuracy of generation based on the interrelationships between audio data. In this way, the generation unit improves the accuracy of generation by taking the interrelationships between audio data into account. Some or all of the above-mentioned processing in the generation unit may be performed using, or without, AI, for example. For example, the generation unit can input the interrelationships between audio data into AI and have the AI improve the accuracy of generation.

When generating audio data, the generation unit can take into consideration the attribute information of the person who submitted the audio data. For example, the generation unit prioritizes the generation of important audio data based on the submitter's job title. For example, the generation unit prioritizes the generation of audio data containing important information based on the submitter's job title. The generation unit can also generate related audio data based on the submitter's field of expertise. For example, the generation unit generates audio data containing information related to the submitter's field of expertise. The generation unit can also dynamically adjust the generation criteria based on the submitter's attribute information. For example, the generation unit adjusts the generation criteria based on the submitter's attribute information. This enables the generation unit to generate appropriate audio by taking the submitter's attribute information into consideration. Some or all of the above-mentioned processing in the generation unit may be performed using AI, for example, or may be performed without using AI. For example, the generation unit can input the submitter's attribute information into AI and have the AI perform the generation.

The generation unit can generate the audio data while taking into account the geographical distribution of the audio data. For example, the generation unit prioritizes the generation of audio data related to a specific region. For example, the generation unit generates audio data related to local events or news. The generation unit can also generate highly relevant audio data based on the geographical distribution. For example, the generation unit generates audio data based on geographical relevance. The generation unit can also improve the accuracy of generation by taking the geographical distribution of the audio data into account. For example, the generation unit improves the accuracy of generation based on the geographical distribution. In this way, the generation unit can efficiently generate highly relevant audio data by taking the geographical distribution into account. Some or all of the above-mentioned processing in the generation unit may be performed using AI, for example, or may be performed without using AI. For example, the generation unit can input geographical distribution data to AI and have the AI perform the generation.

During generation, the generation unit can improve the accuracy of generation by referring to literature related to the audio data. For example, the generation unit prioritizes the generation of important audio data based on related literature. For example, the generation unit adjusts the generation criteria for the audio data by referring to related literature. The generation unit can also analyze literature related to the audio data and dynamically improve the accuracy of generation. For example, the generation unit improves the accuracy of generation based on related literature. In this way, the generation unit improves the accuracy of generation by referring to related literature. Some or all of the above-mentioned processing in the generation unit may be performed using, or without, AI, for example. For example, the generation unit can input related literature data into AI and have the AI improve the accuracy of generation.

The system according to the embodiment is not limited to the example described above, and various modifications are possible, for example, as follows:

The acquisition unit can analyze the user's past call history and select the optimal acquisition method. For example, the acquisition unit can prioritize acquisition of calling apps that the user has used frequently in the past. The acquisition unit can also predict the audio data to be acquired during a specific time period from the user's past call history and select the optimal acquisition method. For example, the acquisition unit can analyze the user's past call history and prioritize acquisition of only important calls. In this way, the acquisition unit can select the optimal acquisition method by analyzing the past call history. Some or all of the above-mentioned processing in the acquisition unit may be performed using AI, for example, or may be performed without using AI. For example, the acquisition unit can input call history data into AI and have the AI select the optimal acquisition method.

The analysis unit can determine analysis priorities based on the time when the audio data was acquired. For example, the analysis unit prioritizes analysis of the most recent audio data. For example, the analysis unit can quickly analyze the most recent audio data and provide results in real time. The analysis unit can also prioritize analysis of audio data acquired immediately after an important event. For example, the analysis unit prioritizes analysis of audio data acquired immediately after the event to extract important information. The analysis unit can also dynamically adjust analysis priorities based on the user's schedule. For example, the analysis unit prioritizes analysis of audio data related to important tasks based on the user's schedule. This enables the analysis unit to determine analysis priorities based on the time when the audio data was acquired, enabling efficient analysis. Some or all of the above-mentioned processing in the analysis unit may be performed using AI, for example, or without AI. For example, the analysis unit can input the time when the audio data was acquired into AI and have the AI determine the analysis priorities.

The filtering unit can improve the accuracy of filtering based on the interrelationships between audio data. For example, the filtering unit filters related audio collectively, taking into account the context of the audio data. For example, the filtering unit analyzes the context of the audio data and extracts related information. The filtering unit can also analyze the interrelationships between audio data, extract important parts, and filter them. For example, the filtering unit extracts important information based on the interrelationships between audio data. The filtering unit can also dynamically adjust the accuracy of filtering based on the interrelationships between audio data. For example, the filtering unit improves the accuracy of filtering based on the interrelationships between audio data. In this way, the filtering unit improves the accuracy of filtering by taking the interrelationships between audio data into consideration. Some or all of the above-mentioned processing in the filtering unit may be performed using AI, for example, or may be performed without using AI. For example, the filtering unit can input the interrelationships between audio data to AI and have the AI improve the accuracy of filtering.

The providing unit can select the optimal display method by referring to the user's past operation history. For example, the providing unit prioritizes providing display methods that the user has used in the past. For example, the providing unit selects the optimal display method based on the user's past operation history. The providing unit can also predict and provide the optimal display method from the user's past operation history. For example, the providing unit analyzes past operation patterns and provides the optimal display method. The providing unit can also analyze the user's past operation history and provide a display method with high visibility. For example, the providing unit selects a display method with high visibility based on the past operation history. In this way, the providing unit can provide the optimal display method by referring to the past operation history. Some or all of the above-mentioned processing in the providing unit may be performed using AI, for example, or may be performed without using AI. For example, the providing unit can input operation history data to AI and have the AI select the optimal display method.

The generation unit can improve the accuracy of generation by taking into account the interrelationships between audio data. For example, the generation unit generates related audio all at once, taking into account the context of the audio data. For example, the generation unit analyzes the context of the audio data and extracts and generates related information. The generation unit can also analyze the interrelationships between audio data and extract and generate important parts. For example, the generation unit extracts and generates important information based on the interrelationships between audio data. The generation unit can also dynamically adjust the accuracy of generation based on the interrelationships between audio data. For example, the generation unit improves the accuracy of generation based on the interrelationships between audio data. In this way, the generation unit improves the accuracy of generation by taking the interrelationships between audio data into account. Some or all of the above-mentioned processing in the generation unit may be performed using, or without, AI, for example. For example, the generation unit can input the interrelationships between audio data into AI and have the AI improve the accuracy of generation.

The processing flow for Example 1 is briefly explained below.

Step 1: The acquisition unit acquires audio data of an online call. For example, the acquisition unit collects audio data in real time using a microphone. The acquisition unit can also read recorded audio files. For example, the acquisition unit supports audio data formats such as audio file formats, sampling rates, and bit rates.
Step 2: The analysis unit uses emotion recognition AI to analyze the voice data acquired by the acquisition unit and classify the speaker's emotion. The analysis unit determines the emotion based on features such as voice tone, pitch, and speed. The emotion recognition AI takes the voice data as input and outputs an emotion classification result. For example, the analysis unit analyzes the tone of the voice data and identifies emotions such as anger, sadness, and joy.
Step 3: The filtering unit converts negative emotional voice into rational voice based on the emotions classified by the analysis unit. For example, the filtering unit converts voice containing angry emotions into a calm tone. The filtering unit receives the judgment result of the emotion recognition AI as input and outputs the converted voice. For example, the filtering unit adjusts the tone of the voice to convert it into a calm tone.
Step 4: The providing unit provides the user with the voice converted by the filtering unit. The providing unit, for example, plays the voice using a speaker. The providing unit can also display the converted voice as text data. For example, the providing unit converts the voice data into text and displays it on a screen.

(Example 2)
An online call filtering system according to an embodiment of the present invention filters emotional voices and negative comments during online calls. This system acquires audio data from online calls and uses emotion recognition AI to analyze the audio data and classify the speaker's emotions. For example, emotions such as anger, sadness, and joy are identified. The emotion recognition AI determines the emotion based on characteristics such as the tone, pitch, and speed of the voice. Next, negative emotional voices are filtered into rational voices. For example, angry voices are converted into calm voices. This filtering process is performed based on the emotions determined by the emotion recognition AI. Furthermore, for negative comments, a generation AI is used to present audio that omits comments unrelated to the main topic of the discussion. For example, emotional comments and irrelevant topics are removed, and only important information is extracted and presented. This process allows users to work efficiently without feeling stressed. This mechanism eliminates the need to overhear emotional voices and negative comments during online calls, improving work efficiency. Furthermore, a stress-free work environment is created, improving employee satisfaction. This allows the online call filtering system to filter out emotional voices and negative comments during online calls, improving work efficiency.

An online call filtering system according to an embodiment includes an acquisition unit, an analysis unit, a filtering unit, and a providing unit. The acquisition unit acquires audio data from online calls. The acquisition unit collects audio data in real time using, for example, a microphone. The acquisition unit can also read recorded audio files. For example, the acquisition unit supports audio data formats such as audio file formats, sampling rates, and bit rates. The analysis unit uses emotion recognition AI to analyze the audio data acquired by the acquisition unit and classify the speaker's emotions. The analysis unit determines emotions based on features such as tone, pitch, and speed of the audio. The emotion recognition AI receives audio data as input and outputs an emotion classification result. For example, the analysis unit analyzes the tone of the audio data and identifies emotions such as anger, sadness, and joy. The filtering unit converts negative emotional voice into rational voice based on the emotions classified by the analysis unit. For example, the filtering unit converts voice containing angry emotions into a calm tone. The filtering unit receives the determination result of the emotion recognition AI as input and outputs the converted voice. For example, the filtering unit adjusts the tone of the voice and converts it to a calm tone. The providing unit provides the voice converted by the filtering unit to the user. The providing unit plays the voice using, for example, a speaker. The providing unit can also display the converted voice as text data. For example, the providing unit converts voice data into text and displays it on a screen. In this way, the online call filtering system according to the embodiment can filter out emotional voices and negative comments during online calls, improving work efficiency.

The acquisition unit acquires audio data from online calls. The acquisition unit collects audio data in real time using, for example, a microphone. Specifically, the acquisition unit uses a high-sensitivity microphone and incorporates noise-canceling technology to reduce ambient noise, thereby acquiring clear audio data. The acquisition unit can also read recorded audio files. For example, the acquisition unit supports audio data formats such as audio file formats, sampling rates, and bit rates. This allows the acquisition unit to flexibly handle audio data in various formats. The acquisition unit also acquires metadata for the audio data and provides it to the analysis and filtering units. The metadata includes the recording date and time, speaker identification information, and call context. This makes it easier for the acquisition unit to understand the context of the audio data and improve the accuracy of subsequent analysis and filtering. The acquisition unit stores the audio data in cloud storage or local storage, allowing it to be accessed as needed. This allows the acquisition unit to efficiently manage not only real-time audio data but also past audio data and use it for analysis and filtering.

The analysis unit uses emotion recognition AI to analyze the voice data acquired by the acquisition unit and classify the speaker's emotions. The analysis unit determines emotions based on voice characteristics such as tone, pitch, and speed. The emotion recognition AI takes voice data as input and outputs emotion classification results. Specifically, the emotion recognition AI uses deep learning technology to extract features of the voice data and classify emotions based on a pre-trained model. For example, the analysis unit analyzes the tone of the voice data and identifies emotions such as anger, sadness, and joy. Furthermore, the analysis unit takes into account temporal changes in the voice data and can track emotional fluctuations in real time. This allows the analysis unit to quickly detect changes in the speaker's emotions and provide information for appropriate filtering. To improve the accuracy of the emotion recognition AI, the analysis unit regularly retrains the model and incorporates new data. This allows the analysis unit to always utilize the latest emotion recognition technology and achieve highly accurate emotion classification.

The filtering unit converts negative emotional speech into rational speech based on the emotions classified by the analysis unit. For example, the filtering unit converts speech containing angry emotions into a calm tone. Specifically, the filtering unit adjusts the tone, pitch, and speed of the speech to soften the intensity of the emotion. For example, if the emotion is strong, the filtering unit converts the speech into a calm speech by lowering the pitch, slowing the speed, and softening the tone. The filtering unit inputs the judgment results of the emotion recognition AI and outputs the converted speech. Furthermore, the filtering unit utilizes natural language processing technology to adjust the emotional expression while preserving the content of the speech. This allows the filtering unit to convert emotional expressions into rational expressions without compromising the speaker's intention. The filtering unit can adjust the filtering strength and criteria according to user settings. For example, it is possible to set it to filter only specific emotions or to change the degree of filtering depending on the intensity of the emotion. This allows the filtering unit to provide flexible filtering tailored to the user's needs.

The providing unit provides the user with the audio converted by the filtering unit. The providing unit plays the audio using, for example, a speaker. Specifically, the providing unit uses a high-quality speaker to clearly play the converted audio. The providing unit can also display the converted audio as text data. For example, the providing unit converts audio data into text and displays it on a screen. This allows the user to view information not only as audio but also as text. Furthermore, the providing unit can customize audio playback and text display settings through a user interface. For example, it is possible to adjust the audio playback speed and volume, and change the font size and display position of the text. This allows the providing unit to flexibly provide information tailored to the user's preferences. The providing unit can also save the converted audio and text data so that it can be played back or referenced later. This allows the user to review past call content and reuse it as needed. The providing unit can also link with other systems and devices to share the converted audio and text data. For example, it can link with business chat tools or email systems to automatically record and share call content. This allows the provider to offer a variety of information provision methods to improve business efficiency.

The analysis unit can determine emotions based on the tone, pitch, and speed characteristics of the voice. The analysis unit, for example, analyzes the tone of the voice to determine emotions. For example, it identifies emotions based on the pitch and intensity of the tone. The analysis unit can also analyze the pitch of the voice to determine emotions. For example, it can identify emotions based on the frequency of the pitch. The analysis unit can also analyze the speed of the voice to determine emotions. For example, it can identify emotions based on the speaking rate. This allows the analysis unit to accurately determine emotions based on the voice characteristics. Some or all of the above-mentioned processing in the analysis unit may be performed using AI, for example, or may be performed without using AI. For example, the analysis unit can input voice data into AI and have the AI perform emotion determination.

The filtering unit can convert voice containing angry emotions into a calm tone. The filtering unit, for example, converts voice containing angry emotions into a calm tone. For example, it adjusts the tone of the voice to convert it into a calm tone. The filtering unit can also adjust the pitch of the voice to convert it into a calm tone. For example, it stabilizes the pitch frequency. The filtering unit can also adjust the speed of the voice to convert it into a calm tone. For example, it keeps the speaking rate constant. In this way, the filtering unit can reduce the emotional impact by converting angry emotions into a calm tone. Some or all of the above-mentioned processing in the filtering unit may be performed using AI, or may be performed without using AI. For example, the filtering unit can input voice data into AI and have the AI perform the tone conversion.

The system is equipped with a generation unit that can eliminate negative comments and extract and present only important information. The generation unit uses a generation AI to eliminate negative comments and extract and present only important information. The generation unit, for example, removes emotional comments and irrelevant topics and extracts important information. The generation AI receives audio data as input and outputs audio data from which important information has been extracted. For example, the generation unit detects emotional comments and removes them. The generation unit can also detect irrelevant topics and remove them. For example, the generation unit detects comments unrelated to the main topic of the discussion and removes them. This allows the generation unit to efficiently obtain only important information by omitting negative comments. Some or all of the above-mentioned processing in the generation unit may be performed using, or without, the generation AI. For example, the generation unit may input audio data to the generation AI and have the generation AI extract important information.

The providing unit can provide the converted voice to the user. The providing unit, for example, provides the converted voice to the user using a speaker. For example, the providing unit plays the converted voice in real time. The providing unit can also display the converted voice as text data. For example, the providing unit converts voice data into text and displays it on a screen. In this way, the providing unit can provide the converted voice to the user, thereby realizing a stress-free work environment. Some or all of the above-mentioned processing in the providing unit may be performed using AI, for example, or may be performed without using AI. For example, the providing unit can input the converted voice data to AI and have the AI convert it into text data.

The acquisition unit can estimate the user's emotions and adjust the timing of voice data acquisition based on the estimated user emotions. The acquisition unit, for example, estimates the user's emotions and adjusts the timing of voice data acquisition based on the estimated user emotions. For example, if the user is feeling stressed, the acquisition unit reduces the frequency of voice data acquisition and acquires only important parts. The acquisition unit can also increase the frequency of voice data acquisition and acquire more detailed information if the user is relaxed. For example, if the user is in a hurry, the acquisition unit speeds up the timing of voice data acquisition and acquires it in real time. This allows the acquisition unit to adjust the timing of voice data acquisition according to the user's emotions, enabling efficient data acquisition. Emotion estimation is achieved using an emotion estimation function, for example, an emotion engine or generative AI. Generative AI includes, but is not limited to, text generation AI (e.g., LLM) and multimodal generation AI. Some or all of the above-mentioned processing in the acquisition unit may be performed using AI, for example, or without AI. For example, the acquisition unit can input the user's emotion data into the generation AI and have the generation AI perform emotion estimation.

The acquisition unit can analyze the user's past call history and select the optimal acquisition method. The acquisition unit, for example, analyzes the user's past call history and selects the optimal acquisition method. For example, the acquisition unit prioritizes acquisition of calling apps that the user has used frequently in the past. The acquisition unit can also predict the audio data to be acquired during a specific time period from the user's past call history and select the optimal acquisition method. For example, the acquisition unit analyzes the user's past call history and prioritizes acquisition of only important calls. In this way, the acquisition unit can select the optimal acquisition method by analyzing the past call history. Some or all of the above-mentioned processing in the acquisition unit may be performed using AI, for example, or may be performed without using AI. For example, the acquisition unit can input call history data into AI and have the AI select the optimal acquisition method.

When acquiring voice data, the acquisition unit can filter the voice data based on the user's current work situation or areas of interest. For example, if the user is in a meeting, the acquisition unit acquires only voice data related to the meeting. For example, the acquisition unit prioritizes acquiring voice data related to the meeting agenda. Furthermore, if the user is concentrating on a particular project, the acquisition unit can also prioritize acquiring voice data related to that project. For example, the acquisition unit filters relevant voice data based on the progress of the project. Furthermore, the acquisition unit can filter and acquire relevant voice data based on the user's areas of interest. For example, the acquisition unit prioritizes acquiring voice data related to the user's area of expertise or topics of interest. In this way, the acquisition unit can acquire highly relevant data by filtering based on the work situation or areas of interest. Some or all of the above-mentioned processing in the acquisition unit may be performed using, or without, AI. For example, the acquisition unit can input data on the work situation and areas of interest into AI and have the AI perform the filtering.

The acquisition unit can estimate the user's emotions and prioritize the voice data to be acquired based on the estimated user emotions. For example, when the user is stressed, the acquisition unit prioritizes acquiring only important voice data. For example, the acquisition unit prioritizes acquiring important voice data related to work. Furthermore, when the user is relaxed, the acquisition unit can prioritize acquiring detailed voice data. For example, the acquisition unit prioritizes acquiring voice data containing detailed information related to work. Furthermore, when the user is in a hurry, the acquisition unit can prioritize acquiring voice data that can be acquired quickly. For example, the acquisition unit prioritizes acquiring voice data that can be acquired in real time. Thus, the acquisition unit prioritizes the acquisition of important data by prioritizing the voice data according to the user's emotions. Emotion estimation is achieved using an emotion estimation function, such as an emotion engine or generative AI. Generative AI can be, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI. Some or all of the above-described processing in the acquisition unit may be performed using AI, or may be performed without AI. For example, the acquisition unit can input the user's emotion data into the generation AI and have the generation AI perform emotion estimation.

When acquiring voice data, the acquisition unit can prioritize acquiring highly relevant data based on the user's geographical location information. For example, if the user is in a specific area, the acquisition unit prioritizes acquiring voice data related to that area. For example, the acquisition unit acquires voice data related to local events and news. Furthermore, if the user is on the move, the acquisition unit can also acquire highly relevant voice data based on the user's current location. For example, the acquisition unit acquires voice data containing information relevant to the user while on the move. Furthermore, the acquisition unit can filter and acquire optimal voice data based on the user's geographical location information. For example, the acquisition unit filters voice data based on geographical relevance. This allows the acquisition unit to acquire highly relevant data based on geographical location information, enabling efficient data collection. Some or all of the above-mentioned processing in the acquisition unit may be performed using, or without, AI. For example, the acquisition unit may input geographical location information to AI and cause the AI to acquire highly relevant data.

When acquiring voice data, the acquisition unit can analyze the user's social media activity and acquire related data. The acquisition unit, for example, acquires voice data related to topics frequently mentioned by the user on social media. For example, the acquisition unit analyzes the content of the user's social media posts and acquires related voice data. The acquisition unit can also analyze topics of interest from the user's social media activity and acquire related voice data. For example, the acquisition unit acquires voice data related to topics of interest based on the user's social media activity. The acquisition unit can also filter and acquire optimal voice data based on the content of the user's social media posts. For example, the acquisition unit filters voice data based on trends and topics on social media. In this way, the acquisition unit can acquire highly relevant data by analyzing social media activity. Some or all of the above-mentioned processing by the acquisition unit may be performed using, or without, AI. For example, the acquisition unit can input social media data into AI and have the AI acquire related data.

The analysis unit can estimate the user's emotions and adjust the presentation method of the analysis based on the estimated user emotions. For example, if the user is nervous, the analysis unit provides simple, highly visible analysis results. For example, the analysis unit displays the analysis results in a visually easy-to-understand format using graphs and charts. The analysis unit can also provide detailed analysis results if the user is relaxed. For example, the analysis unit displays analysis results including detailed text and data. The analysis unit can also provide analysis results that focus on the main points if the user is in a hurry. For example, the analysis unit displays concise analysis results that highlight important points. This allows the analysis unit to adjust the presentation method of the analysis according to the user's emotions and provide highly visible analysis results. Emotion estimation is achieved using an emotion estimation function, such as an emotion engine or generative AI. Generative AI can include, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI. Some or all of the above-described processing in the analysis unit may be performed using AI, or without AI. For example, the analysis unit can input the user's emotional data into the generation AI and have the generation AI adjust the way the analysis is expressed.

During analysis, the analysis unit can adjust the level of detail of the analysis based on the importance of the audio data. For example, the analysis unit performs a detailed analysis of important audio data. For example, the analysis unit analyzes the content of the audio data in detail and extracts important information. The analysis unit can also perform a simplified analysis of audio data with low importance. For example, the analysis unit analyzes an overview of the audio data and provides concise information. The analysis unit can also dynamically adjust the level of detail of the analysis based on the importance of the audio data. For example, the analysis unit adjusts the depth and scope of the analysis based on the importance of the audio data. This allows the analysis unit to adjust the level of detail of the analysis based on the importance of the audio data, enabling efficient analysis. Some or all of the above-mentioned processing in the analysis unit may be performed using AI, or may be performed without AI. For example, the analysis unit can input the importance of the audio data to AI and have the AI adjust the level of detail of the analysis.

During analysis, the analysis unit can apply different analysis algorithms depending on the category of audio data. For example, the analysis unit applies a minutes generation algorithm to conference audio data. For example, the analysis unit automatically summarizes the contents of the conference and generates minutes. The analysis unit can also apply a customer satisfaction analysis algorithm to customer support audio data. For example, the analysis unit analyzes the content of customer comments and evaluates satisfaction. The analysis unit can also apply a learning progress analysis algorithm to educational audio data. For example, the analysis unit analyzes educational content and evaluates learning progress. In this way, the analysis unit can apply an appropriate analysis algorithm depending on the category of audio data, thereby improving analysis accuracy. Some or all of the above-mentioned processing in the analysis unit may be performed using AI, for example, or may be performed without using AI. For example, the analysis unit can input the category of audio data to AI and have the AI apply the analysis algorithm.

The analysis unit can estimate the user's emotions and adjust the length of the analysis based on the estimated user emotions. For example, if the user is in a hurry, the analysis unit can perform a short, to-the-point analysis. For example, the analysis unit can provide concise analysis results that emphasize important points. The analysis unit can also perform a detailed analysis if the user is relaxed. For example, the analysis unit can provide analysis results that include detailed text and data. The analysis unit can also perform an analysis that adds visually stimulating effects if the user is excited. For example, the analysis unit can display the analysis results with visual effects in graphs and charts. This allows the analysis unit to adjust the length of the analysis according to the user's emotions, enabling efficient analysis. Emotion estimation is achieved using an emotion estimation function, such as an emotion engine or generative AI. Generative AI can include, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI. Some or all of the above-described processing in the analysis unit can be performed using AI, or without AI. For example, the analysis unit can input the user's emotion data into the generation AI and have the generation AI adjust the length of the analysis.

During analysis, the analysis unit can determine analysis priorities based on the time the audio data was acquired. The analysis unit, for example, prioritizes analysis of the most recent audio data. For example, the analysis unit can quickly analyze the most recent audio data and provide results in real time. The analysis unit can also prioritize analysis of audio data acquired immediately after an important event. For example, the analysis unit can prioritize analysis of audio data acquired immediately after the event to extract important information. The analysis unit can also dynamically adjust analysis priorities based on the user's schedule. For example, the analysis unit can prioritize analysis of audio data related to important tasks based on the user's schedule. This enables the analysis unit to determine analysis priorities based on the time the audio data was acquired, enabling efficient analysis. Some or all of the above-mentioned processing in the analysis unit may be performed using AI, for example, or without AI. For example, the analysis unit can input the time the audio data was acquired into AI and have the AI determine the analysis priorities.

During analysis, the analysis unit can adjust the order of analysis based on the relevance of the audio data. For example, the analysis unit prioritizes analysis of highly relevant audio data. For example, the analysis unit quickly analyzes highly relevant audio data and extracts important information. The analysis unit can also postpone analysis of less relevant audio data. For example, the analysis unit postpones analysis of less relevant audio data and prioritizes analysis of more important data. The analysis unit can also dynamically adjust the order of analysis based on the relevance of the audio data. For example, the analysis unit dynamically adjusts the order of analysis based on the relevance of the audio data and performs efficient analysis. In this way, the analysis unit can adjust the order of analysis based on the relevance of the audio data, enabling efficient analysis. Some or all of the above-mentioned processing in the analysis unit may be performed using AI, for example, or may be performed without using AI. For example, the analysis unit can input the relevance of the audio data to AI and have the AI adjust the order of analysis.

The filtering unit can estimate the user's emotions and adjust the filtering criteria based on the estimated user emotions. For example, if the user is stressed, the filtering unit applies strict filtering criteria to remove negative voices. For example, the filtering unit removes emotional comments and irrelevant topics. The filtering unit can also apply lenient filtering criteria to provide detailed voices when the user is relaxed. For example, the filtering unit provides voice data containing detailed information. The filtering unit can also quickly filter and provide only important voices when the user is in a hurry. For example, the filtering unit quickly provides voice data containing important information. This allows the filtering unit to adjust the filtering criteria according to the user's emotions, enabling appropriate filtering. Emotion estimation is achieved using an emotion estimation function, such as an emotion engine or generative AI. Generative AI can include, but is not limited to, text generation AI (e.g., LLM) and multimodal generation AI. Some or all of the above-described processing in the filtering unit may be performed using AI, or without AI. For example, the filtering unit can input the user's emotional data into the generation AI and have the generation AI adjust the filtering criteria.

The filtering unit can improve the accuracy of filtering based on the interrelationships between audio data during filtering. The filtering unit, for example, takes into account the context of the audio data and filters related audio collectively. For example, the filtering unit analyzes the context of the audio data and extracts related information. The filtering unit can also analyze the interrelationships between audio data and extract and filter important parts. For example, the filtering unit extracts important information based on the interrelationships between audio data. The filtering unit can also dynamically adjust the accuracy of filtering based on the interrelationships between audio data. For example, the filtering unit improves the accuracy of filtering based on the interrelationships between audio data. In this way, the filtering unit improves the accuracy of filtering by taking the interrelationships between audio data into consideration. Some or all of the above-mentioned processing in the filtering unit may be performed using AI, for example, or may be performed without using AI. For example, the filtering unit can input the interrelationships between audio data to AI and have the AI improve the accuracy of filtering.

When filtering, the filtering unit can take into account the attribute information of the person who submitted the audio data. The filtering unit, for example, prioritizes filtering of important audio data based on the submitter's job title. For example, the filtering unit prioritizes filtering of audio data containing important information based on the submitter's job title. The filtering unit can also filter related audio data based on the submitter's field of expertise. For example, the filtering unit filters audio data containing information related to the submitter's field of expertise. The filtering unit can also dynamically adjust the filtering criteria based on the submitter's attribute information. For example, the filtering unit adjusts the filtering criteria based on the submitter's attribute information. This enables the filtering unit to perform appropriate filtering by taking the submitter's attribute information into consideration. Some or all of the above-mentioned processing in the filtering unit may be performed using AI, for example, or may be performed without using AI. For example, the filtering unit can input the submitter's attribute information into AI and have the AI perform the filtering.

The filtering unit can estimate the user's emotions and adjust the display order of the filtering results based on the estimated user emotions. For example, if the user is feeling stressed, the filtering unit can prioritize displaying important audio data. For example, the filtering unit can prioritize displaying audio data containing important information. Furthermore, if the user is relaxed, the filtering unit can prioritize displaying detailed audio data. For example, the filtering unit can prioritize displaying audio data containing detailed information. Furthermore, if the user is in a hurry, the filtering unit can quickly display the filtering results. For example, the filtering unit can quickly display important information. Thus, the filtering unit can prioritize providing important information by adjusting the display order of the filtering results according to the user's emotions. Emotion estimation is achieved using an emotion estimation function, such as an emotion engine or generative AI. Generative AI can include, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI. Some or all of the above-described processing in the filtering unit may be performed using AI, or without AI. For example, the filtering unit can input user emotion data into the generation AI and have the generation AI adjust the display order of the filtering results.

The filtering unit can perform filtering while taking into account the geographical distribution of the audio data. For example, the filtering unit prioritizes filtering of audio data related to a specific region. For example, the filtering unit filters audio data related to local events or news. The filtering unit can also filter highly relevant audio data based on the geographical distribution. For example, the filtering unit filters audio data based on geographical relevance. The filtering unit can also improve the accuracy of filtering by taking the geographical distribution of the audio data into account. For example, the filtering unit improves the accuracy of filtering based on the geographical distribution. In this way, the filtering unit can efficiently filter highly relevant audio data by taking the geographical distribution into account. Some or all of the above-mentioned processing in the filtering unit may be performed using AI, for example, or may be performed without using AI. For example, the filtering unit can input the geographical distribution data to AI and have the AI perform the filtering.

When filtering, the filtering unit can improve the accuracy of filtering by referring to literature related to the audio data. For example, the filtering unit prioritizes filtering of important audio data based on related literature. For example, the filtering unit adjusts the filtering criteria for the audio data by referring to related literature. The filtering unit can also analyze literature related to the audio data and dynamically improve the accuracy of filtering. For example, the filtering unit improves the accuracy of filtering based on related literature. In this way, the filtering unit improves the accuracy of filtering by referring to related literature. Some or all of the above-mentioned processing in the filtering unit may be performed using, or without, AI, for example. For example, the filtering unit can input related literature data into AI and have the AI improve the accuracy of filtering.

The providing unit can estimate the user's emotions and adjust the display method based on the estimated user emotions. For example, if the user is nervous, the providing unit can provide a simple, highly visible display method. For example, the providing unit can display information in a visually easy-to-understand format using graphs and charts. Furthermore, if the user is relaxed, the providing unit can provide a display method including detailed information. For example, the providing unit can provide a display method including detailed text and data. Furthermore, if the user is in a hurry, the providing unit can provide a display method that focuses on the main points. For example, the providing unit can provide a concise display method that emphasizes important points. This allows the providing unit to adjust the display method according to the user's emotions, thereby providing highly visible information. Emotion estimation is achieved using an emotion estimation function, such as an emotion engine or generation AI. Generation AI can include, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI. Some or all of the above-described processing in the providing unit can be performed using AI, for example, or without AI. For example, the providing unit can input the user's emotion data into the generation AI and have the generation AI adjust the display method.

When providing the display method, the providing unit can select the optimal display method by referring to the user's past operation history. For example, the providing unit prioritizes providing display methods that the user has used in the past. For example, the providing unit selects the optimal display method based on the user's past operation history. The providing unit can also predict and provide the optimal display method from the user's past operation history. For example, the providing unit analyzes past operation patterns and provides the optimal display method. The providing unit can also analyze the user's past operation history and provide a display method with high visibility. For example, the providing unit selects a display method with high visibility based on the past operation history. In this way, the providing unit can provide the optimal display method by referring to the past operation history. Some or all of the above-mentioned processing in the providing unit may be performed using AI, for example, or may be performed without using AI. For example, the providing unit can input operation history data to AI and have the AI select the optimal display method.

The providing unit can estimate the user's emotions and adjust the provided operating procedures based on the estimated user emotions. For example, if the user is nervous, the providing unit can provide simple operating procedures. For example, the providing unit can provide step-by-step procedures to allow the user to operate easily. The providing unit can also provide detailed operating procedures if the user is relaxed. For example, the providing unit can provide operating procedures including detailed explanations and guides. The providing unit can also provide procedures that allow the user to operate quickly if the user is in a hurry. For example, the providing unit can provide shortcuts or simplified procedures. This allows the providing unit to adjust the operating procedures according to the user's emotions, enabling efficient operation. Emotion estimation is achieved using an emotion estimation function, for example, using an emotion engine or generative AI. Generative AI can be, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI. Some or all of the above-mentioned processing in the providing unit can be performed using AI, for example, or without AI. For example, the providing unit can input the user's emotional data into the generation AI and have the generation AI adjust the operating procedure.

When providing the display, the providing unit can select the optimal display method by taking into account the user's device information. For example, if the user is using a smartphone, the providing unit provides a display method that matches the screen size. For example, the providing unit provides a display method optimized for the smartphone screen size. Furthermore, if the user is using a tablet, the providing unit can also provide a display method optimized for a large screen. For example, the providing unit provides a display method optimized for the tablet screen size. Furthermore, if the user is using a smartwatch, the providing unit can also provide a display method that is simple and highly visible. For example, the providing unit provides a display method optimized for the smartwatch screen size. In this way, the providing unit can provide the optimal display method by taking the device information into account. Some or all of the above-mentioned processing in the providing unit may be performed using AI, for example, or may be performed without using AI. For example, the providing unit can input device information into AI and have the AI select the optimal display method.

The generation unit can estimate the user's emotions and prioritize the audio to be generated based on the estimated user emotions. For example, when the user is stressed, the generation unit prioritizes generating important audio. For example, the generation unit prioritizes generating important audio data related to work. Furthermore, when the user is relaxed, the generation unit can prioritize generating detailed audio. For example, the generation unit prioritizes generating audio data containing detailed information related to work. Furthermore, when the user is in a hurry, the generation unit can prioritize generating audio that can be generated quickly. For example, the generation unit prioritizes generating audio data that can be generated in real time. Thus, the generation unit can prioritize generating important audio by prioritizing audio according to the user's emotions. Emotion estimation is achieved using an emotion estimation function, such as an emotion engine or generation AI. Generation AI can include, but is not limited to, text generation AI (e.g., LLM) and multimodal generation AI. Some or all of the above-described processing in the generation unit may be performed using AI, or without AI. For example, the generation unit can input the user's emotional data into the generation AI and have the generation AI determine the priority of the voices.

The generation unit can improve the accuracy of generation by taking into account the interrelationships between audio data during generation. For example, the generation unit generates related audio all at once, taking into account the context of the audio data. For example, the generation unit analyzes the context of the audio data and extracts and generates related information. The generation unit can also analyze the interrelationships between audio data and extract and generate important parts. For example, the generation unit extracts and generates important information based on the interrelationships between audio data. The generation unit can also dynamically adjust the accuracy of generation based on the interrelationships between audio data. For example, the generation unit improves the accuracy of generation based on the interrelationships between audio data. In this way, the generation unit improves the accuracy of generation by taking into account the interrelationships between audio data. Some or all of the above-mentioned processing in the generation unit may be performed using, or without, AI. For example, the generation unit can input the interrelationships between audio data into AI and have the AI improve the accuracy of generation.

When generating audio data, the generation unit can take into consideration the attribute information of the person who submitted the audio data. For example, the generation unit prioritizes the generation of important audio data based on the submitter's job title. For example, the generation unit prioritizes the generation of audio data containing important information based on the submitter's job title. The generation unit can also generate related audio data based on the submitter's field of expertise. For example, the generation unit generates audio data containing information related to the submitter's field of expertise. The generation unit can also dynamically adjust the generation criteria based on the submitter's attribute information. For example, the generation unit adjusts the generation criteria based on the submitter's attribute information. This enables the generation unit to generate appropriate audio by taking the submitter's attribute information into consideration. Some or all of the above-mentioned processing in the generation unit may be performed using AI, for example, or may be performed without using AI. For example, the generation unit can input the submitter's attribute information into AI and have the AI perform the generation.

The generation unit can estimate the user's emotions and adjust the display method of the generated audio based on the estimated user emotions. For example, if the user is nervous, the generation unit provides a simple, highly visible display method. For example, the generation unit displays information in a visually easy-to-understand format using graphs and charts. The generation unit can also provide a display method including detailed information if the user is relaxed. For example, the generation unit provides a display method including detailed text and data. The generation unit can also provide a display method that focuses on the main points if the user is in a hurry. For example, the generation unit provides a concise display method that emphasizes important points. This allows the generation unit to adjust the display method according to the user's emotions, thereby providing highly visible information. Emotion estimation is achieved using an emotion estimation function, such as an emotion engine or generation AI. Generation AI can include, but is not limited to, text generation AI (e.g., LLM) and multimodal generation AI. Some or all of the above-described processing in the generation unit may be performed using AI, or without AI. For example, the generation unit can input the user's emotion data into the generation AI and have the generation AI adjust the display method.

The generation unit can generate the audio data while taking into account the geographical distribution of the audio data. For example, the generation unit prioritizes the generation of audio data related to a specific region. For example, the generation unit generates audio data related to local events or news. The generation unit can also generate highly relevant audio data based on the geographical distribution. For example, the generation unit generates audio data based on geographical relevance. The generation unit can also improve the accuracy of generation by taking the geographical distribution of the audio data into account. For example, the generation unit improves the accuracy of generation based on the geographical distribution. In this way, the generation unit can efficiently generate highly relevant audio data by taking the geographical distribution into account. Some or all of the above-mentioned processing in the generation unit may be performed using AI, for example, or may be performed without using AI. For example, the generation unit can input geographical distribution data to AI and have the AI perform the generation.

During generation, the generation unit can improve the accuracy of generation by referring to literature related to the audio data. For example, the generation unit prioritizes the generation of important audio data based on related literature. For example, the generation unit adjusts the generation criteria for the audio data by referring to related literature. The generation unit can also analyze literature related to the audio data and dynamically improve the accuracy of generation. For example, the generation unit improves the accuracy of generation based on related literature. In this way, the generation unit improves the accuracy of generation by referring to related literature. Some or all of the above-mentioned processing in the generation unit may be performed using, or without, AI, for example. For example, the generation unit can input related literature data into AI and have the AI improve the accuracy of generation.

The system according to the embodiment is not limited to the example described above, and various modifications are possible, for example, as follows:

The acquisition unit can analyze the user's past call history and select the optimal acquisition method. For example, the acquisition unit can prioritize acquisition of calling apps that the user has used frequently in the past. The acquisition unit can also predict the audio data to be acquired during a specific time period from the user's past call history and select the optimal acquisition method. For example, the acquisition unit can analyze the user's past call history and prioritize acquisition of only important calls. In this way, the acquisition unit can select the optimal acquisition method by analyzing the past call history. Some or all of the above-mentioned processing in the acquisition unit may be performed using AI, for example, or may be performed without using AI. For example, the acquisition unit can input call history data into AI and have the AI select the optimal acquisition method.

The analysis unit can determine analysis priorities based on the time when the audio data was acquired. For example, the analysis unit prioritizes analysis of the most recent audio data. For example, the analysis unit can quickly analyze the most recent audio data and provide results in real time. The analysis unit can also prioritize analysis of audio data acquired immediately after an important event. For example, the analysis unit prioritizes analysis of audio data acquired immediately after the event to extract important information. The analysis unit can also dynamically adjust analysis priorities based on the user's schedule. For example, the analysis unit prioritizes analysis of audio data related to important tasks based on the user's schedule. This enables the analysis unit to determine analysis priorities based on the time when the audio data was acquired, enabling efficient analysis. Some or all of the above-mentioned processing in the analysis unit may be performed using AI, for example, or without AI. For example, the analysis unit can input the time when the audio data was acquired into AI and have the AI determine the analysis priorities.

The filtering unit can improve the accuracy of filtering based on the interrelationships between audio data. For example, the filtering unit filters related audio collectively, taking into account the context of the audio data. For example, the filtering unit analyzes the context of the audio data and extracts related information. The filtering unit can also analyze the interrelationships between audio data, extract important parts, and filter them. For example, the filtering unit extracts important information based on the interrelationships between audio data. The filtering unit can also dynamically adjust the accuracy of filtering based on the interrelationships between audio data. For example, the filtering unit improves the accuracy of filtering based on the interrelationships between audio data. In this way, the filtering unit improves the accuracy of filtering by taking the interrelationships between audio data into consideration. Some or all of the above-mentioned processing in the filtering unit may be performed using AI, for example, or may be performed without using AI. For example, the filtering unit can input the interrelationships between audio data to AI and have the AI improve the accuracy of filtering.

The providing unit can select the optimal display method by referring to the user's past operation history. For example, the providing unit prioritizes providing display methods that the user has used in the past. For example, the providing unit selects the optimal display method based on the user's past operation history. The providing unit can also predict and provide the optimal display method from the user's past operation history. For example, the providing unit analyzes past operation patterns and provides the optimal display method. The providing unit can also analyze the user's past operation history and provide a display method with high visibility. For example, the providing unit selects a display method with high visibility based on the past operation history. In this way, the providing unit can provide the optimal display method by referring to the past operation history. Some or all of the above-mentioned processing in the providing unit may be performed using AI, for example, or may be performed without using AI. For example, the providing unit can input operation history data to AI and have the AI select the optimal display method.

The generation unit can improve the accuracy of generation by taking into account the interrelationships between audio data. For example, the generation unit generates related audio all at once, taking into account the context of the audio data. For example, the generation unit analyzes the context of the audio data and extracts and generates related information. The generation unit can also analyze the interrelationships between audio data and extract and generate important parts. For example, the generation unit extracts and generates important information based on the interrelationships between audio data. The generation unit can also dynamically adjust the accuracy of generation based on the interrelationships between audio data. For example, the generation unit improves the accuracy of generation based on the interrelationships between audio data. In this way, the generation unit improves the accuracy of generation by taking the interrelationships between audio data into account. Some or all of the above-mentioned processing in the generation unit may be performed using, or without, AI, for example. For example, the generation unit can input the interrelationships between audio data into AI and have the AI improve the accuracy of generation.

The acquisition unit can estimate the user's emotions and adjust the timing of voice data acquisition based on the estimated user emotions. For example, if the user is feeling stressed, the acquisition unit can reduce the frequency of voice data acquisition and acquire only important parts. Furthermore, if the user is relaxed, the acquisition unit can increase the frequency of voice data acquisition to acquire more detailed information. For example, if the user is in a hurry, the acquisition unit can speed up the timing of voice data acquisition and acquire it in real time. This allows the acquisition unit to adjust the timing of voice data acquisition according to the user's emotions, enabling efficient data acquisition. Emotion estimation is achieved using an emotion estimation function, for example, using an emotion engine or generative AI. Generative AI includes, but is not limited to, text generation AI (e.g., LLM) and multimodal generation AI. Some or all of the above-mentioned processing in the acquisition unit may be performed using AI, or may be performed without AI. For example, the acquisition unit can input the user's emotion data into the generative AI and have the generative AI perform emotion estimation.

The analysis unit can estimate the user's emotions and adjust the presentation of the analysis based on the estimated user emotions. For example, if the user is nervous, the analysis unit can provide simple, highly visible analysis results. For example, the analysis unit can display the analysis results in a visually easy-to-understand format using graphs and charts. The analysis unit can also provide detailed analysis results if the user is relaxed. For example, the analysis unit can display analysis results including detailed text and data. The analysis unit can also provide analysis results that focus on the main points if the user is in a hurry. For example, the analysis unit can display concise analysis results that highlight important points. This allows the analysis unit to adjust the presentation of the analysis based on the user's emotions and provide highly visible analysis results. Emotion estimation is achieved using an emotion estimation function, such as an emotion engine or generative AI. Generative AI can include, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI. Some or all of the above-described processing in the analysis unit can be performed using AI, or without AI. For example, the analysis unit can input the user's emotional data into the generation AI and have the generation AI adjust the way the analysis is expressed.

The filtering unit can estimate the user's emotions and adjust the filtering criteria based on the estimated user emotions. For example, if the user is stressed, strict filtering criteria can be applied to remove negative voices. For example, the filtering unit can remove emotional comments and irrelevant topics. Furthermore, if the user is relaxed, the filtering unit can apply looser filtering criteria to provide more detailed voices. For example, the filtering unit can provide voice data containing detailed information. Furthermore, if the user is in a hurry, the filtering unit can quickly filter and provide only important voices. For example, the filtering unit can quickly provide voice data containing important information. This allows the filtering unit to adjust the filtering criteria according to the user's emotions, enabling appropriate filtering. Emotion estimation is achieved using an emotion estimation function, such as an emotion engine or generative AI. Generative AI can include, but is not limited to, text generation AI (e.g., LLM) and multimodal generation AI. Some or all of the above-described processing in the filtering unit may be performed using AI, or without AI. For example, the filtering unit can input the user's emotional data into the generation AI and have the generation AI adjust the filtering criteria.

The providing unit can estimate the user's emotions and adjust the display method based on the estimated user emotions. For example, if the user is nervous, the providing unit can provide a simple, highly visible display method. For example, the providing unit can display information in a visually easy-to-understand format using graphs and charts. Furthermore, if the user is relaxed, the providing unit can provide a display method that includes detailed information. For example, the providing unit can provide a display method that includes detailed text and data. Furthermore, if the user is in a hurry, the providing unit can provide a display method that focuses on the main points. For example, the providing unit can provide a concise display method that highlights important points. This allows the providing unit to adjust the display method according to the user's emotions, thereby providing highly visible information. Emotion estimation is achieved using an emotion estimation function, such as an emotion engine or generation AI. Generation AI can include, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI. Some or all of the above-described processing in the providing unit can be performed using AI, for example, or without AI. For example, the providing unit can input the user's emotion data into the generation AI and have the generation AI adjust the display method.

The generation unit can estimate the user's emotions and prioritize the audio to be generated based on the estimated user emotions. For example, if the user is stressed, it prioritizes the generation of important audio. For example, the generation unit prioritizes the generation of important work-related audio data. Furthermore, if the user is relaxed, the generation unit can prioritize the generation of detailed audio. For example, the generation unit prioritizes the generation of audio data containing detailed work-related information. Furthermore, if the user is in a hurry, the generation unit can prioritize the generation of audio that can be generated quickly. For example, the generation unit prioritizes the generation of audio data that can be generated in real time. Thus, the generation unit can prioritize the generation of important audio by prioritizing audio according to the user's emotions. Emotion estimation is achieved using an emotion estimation function, such as an emotion engine or generation AI. Generation AI can include, but is not limited to, text generation AI (e.g., LLM) and multimodal generation AI. Some or all of the above-described processing in the generation unit may be performed using AI, or without AI. For example, the generation unit can input the user's emotional data into the generation AI and have the generation AI determine the priority of the voices.

The processing flow for Example 2 is briefly explained below.

Step 1: The acquisition unit acquires audio data of an online call. For example, the acquisition unit collects audio data in real time using a microphone. The acquisition unit can also read recorded audio files. For example, the acquisition unit supports audio data formats such as audio file formats, sampling rates, and bit rates.
Step 2: The analysis unit uses emotion recognition AI to analyze the voice data acquired by the acquisition unit and classify the speaker's emotion. The analysis unit determines the emotion based on features such as voice tone, pitch, and speed. The emotion recognition AI takes the voice data as input and outputs an emotion classification result. For example, the analysis unit analyzes the tone of the voice data and identifies emotions such as anger, sadness, and joy.
Step 3: The filtering unit converts negative emotional voice into rational voice based on the emotions classified by the analysis unit. For example, the filtering unit converts voice containing angry emotions into a calm tone. The filtering unit receives the judgment result of the emotion recognition AI as input and outputs the converted voice. For example, the filtering unit adjusts the tone of the voice to convert it into a calm tone.
Step 4: The providing unit provides the user with the voice converted by the filtering unit. The providing unit, for example, plays the voice using a speaker. The providing unit can also display the converted voice as text data. For example, the providing unit converts the voice data into text and displays it on a screen.

The specific processing unit 290 transmits the results of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the results of the specific processing. The microphone 38B acquires audio indicating the user input regarding the results of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

The data generation model 58 is what is known as generative AI (artificial intelligence). An example of a data generation model 58 is generative AI such as ChatGPT (registered trademark) (Internet search <URL: https://openai.com/blog/chatgpt>). The data generation model 58 is obtained by performing deep learning on a neural network. A prompt containing an instruction is input to the data generation model 58, and inference data such as voice data indicating speech, text data indicating text, and image data indicating an image (e.g., still image data or video data) is also input. The data generation model 58 performs inference on the input inference data in accordance with the instructions indicated by the prompt and outputs the inference results in one or more data formats, such as voice data, text data, and image data. The data generation model 58 includes, for example, text generation AI, image generation AI, and multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and/or summarization. The identification processing unit 290 performs the above-mentioned identification processing using the data generation model 58. The data generation model 58 may be a fine-tuned model that outputs inference results from prompts that do not include instructions. In this case, the data generation model 58 can output inference results from prompts that do not include instructions. The data processing device 12 or the like includes multiple types of data generation models 58, and the data generation model 58 includes AI other than the generation AI. Examples of AI other than the generation AI include linear regression, logistic regression, decision trees, random forests, support vector machines (SVMs), k-means clustering, convolutional neural networks (CNNs), recurrent neural networks (RNNs), generative adversarial networks (GANs), and naive Bayes, and can perform various processes, but are not limited to these examples. The AI may also be an AI agent. When the processing of each of the above-mentioned parts is performed by AI, the processing may be performed in part or in whole by AI, but is not limited to these examples. In addition, processing performed by AI, including generation AI, may be replaced with rule-based processing, and rule-based processing may be replaced with processing performed by AI, including generation AI.

Furthermore, the processing by the data processing system 10 described above is executed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the smart device 14, but may also be executed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the smart device 14. Furthermore, the specific processing unit 290 of the data processing device 12 acquires or collects information necessary for processing from the smart device 14 or an external device, etc., and the smart device 14 acquires or collects information necessary for processing from the data processing device 12 or an external device, etc.

Each of the multiple elements, including the acquisition unit, analysis unit, filtering unit, provision unit, and generation unit, described above, is realized, for example, by at least one of the smart device 14 and the data processing device 12. For example, the acquisition unit collects voice data using the microphone of the smart device 14, which is then analyzed by the specific processing unit 290 of the data processing device 12. The analysis unit, realized, for example, by the specific processing unit 290 of the data processing device 12, analyzes the voice data using emotion recognition AI. The filtering unit, realized, for example, by the specific processing unit 290 of the data processing device 12, converts negative emotional voice into rational voice. The provision unit, for example, plays the converted voice using the speaker of the smart device 14. The generation unit, realized, for example, by the specific processing unit 290 of the data processing device 12, omits negative comments and extracts and presents only important information. The acquisition unit, for example, estimates the user's emotions using the control unit 46A of the smart device 14 and adjusts the timing of voice data acquisition. The correspondence between each part and the device or control unit is not limited to the example described above, and various modifications are possible.

Second Embodiment
FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

As shown in FIG. 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

The data processing device 12 includes a computer 22, a database 24, and a communication I/F 26. The computer 22 includes a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and communication I/F 26 are also connected to the bus 34. The communication I/F 26 is connected to a network 54. Examples of the network 54 include a WAN and/or a LAN.

The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I/F 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

The microphone 238 receives instructions and other information from the user by receiving voice uttered by the user. The microphone 238 captures the voice uttered by the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the user's surroundings (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

The communication I/F 44 is connected to the network 54. The communication I/Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I/Fs 44 and 26 is carried out in a secure manner.

Figure 4 shows an example of the main functions of the data processing device 12 and smart glasses 214. As shown in Figure 4, in the data processing device 12, specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32.

The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as the specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

Storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion. The emotion estimation function (emotion identification function) using the emotion identification model 59 performs various estimations and predictions regarding the user's emotions, including estimation and prediction of the user's emotion, but is not limited to these examples. Furthermore, emotion estimation and prediction also include, for example, emotion analysis.

In the smart glasses 214, specific processing is performed by the processor 46. A specific processing program 60 is stored in the storage 50. The processor 46 reads the specific processing program 60 from the storage 50 and executes the read specific processing program 60 on the RAM 48. The specific processing is realized by the processor 46 operating as the control unit 46A in accordance with the specific processing program 60 executed on the RAM 48. The smart glasses 214 also have a data generation model and emotion identification model similar to the data generation model 58 and emotion identification model 59, and can use these models to perform processing similar to that of the specific processing unit 290.

Note that a device other than the data processing device 12 may have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 communicates with the server device having the data generation model 58 to obtain processing results (such as prediction results) using the data generation model 58. The data processing device 12 may also be a server device, or a terminal device owned by a user (for example, a mobile phone, robot, home appliance, etc.).

The specific processing unit 290 transmits the results of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the results of the specific processing. The microphone 238 acquires audio indicating the user input regarding the results of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

The data generation model 58 is what is known as generative AI. An example of a data generation model 58 is generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt containing an instruction is input to the data generation model 58, and inference data such as voice data indicating speech, text data indicating text, and image data indicating an image (e.g., still image data or video data) is input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in one or more data formats, such as voice data, text data, and image data. The data generation model 58 includes, for example, text generation AI, image generation AI, and multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and/or summarization. The identification processing unit 290 performs the above-mentioned identification processing using the data generation model 58. The data generation model 58 may be a fine-tuned model that outputs inference results from prompts that do not include instructions. In this case, the data generation model 58 can output inference results from prompts that do not include instructions. The data processing device 12 or the like includes multiple types of data generation models 58, and the data generation model 58 includes AI other than the generation AI. Examples of AI other than the generation AI include linear regression, logistic regression, decision trees, random forests, support vector machines (SVMs), k-means clustering, convolutional neural networks (CNNs), recurrent neural networks (RNNs), generative adversarial networks (GANs), and naive Bayes, and can perform various processes, but are not limited to these examples. The AI may also be an AI agent. When the processing of each of the above-mentioned parts is performed by AI, the processing may be performed in part or in whole by AI, but is not limited to these examples. In addition, processing performed by AI, including generation AI, may be replaced with rule-based processing, and rule-based processing may be replaced with processing performed by AI, including generation AI.

The data processing system 210 according to the second embodiment performs processing similar to that of the data processing system 10 according to the first embodiment. Processing by the data processing system 210 is executed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the smart glasses 214, but may also be executed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the smart glasses 214. In addition, the specific processing unit 290 of the data processing device 12 acquires or collects information necessary for processing from the smart glasses 214 or an external device, etc., and the smart glasses 214 acquires or collects information necessary for processing from the data processing device 12 or an external device, etc.

Each of the multiple elements, including the acquisition unit, analysis unit, filtering unit, provision unit, and generation unit, described above, is realized, for example, by at least one of the smart glasses 214 and the data processing device 12. For example, the acquisition unit collects voice data using the microphone of the smart glasses 214, which is analyzed by the specific processing unit 290 of the data processing device 12. The analysis unit, realized, for example, by the specific processing unit 290 of the data processing device 12, analyzes the voice data using emotion recognition AI. The filtering unit, realized, for example, by the specific processing unit 290 of the data processing device 12, converts negative emotional voice into rational voice. The provision unit, for example, plays the converted voice using the speaker of the smart glasses 214. The generation unit, realized, for example, by the specific processing unit 290 of the data processing device 12, omits negative comments and extracts and presents only important information. The acquisition unit, for example, estimates the user's emotions using the control unit 46A of the smart glasses 214 and adjusts the timing of voice data acquisition. The correspondence between each part and the device or control unit is not limited to the example described above, and various modifications are possible.

[Third embodiment]
FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

As shown in FIG. 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

The data processing device 12 includes a computer 22, a database 24, and a communication I/F 26. The computer 22 includes a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and communication I/F 26 are also connected to the bus 34. The communication I/F 26 is connected to a network 54. Examples of the network 54 include a WAN and/or a LAN.

The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I/F 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

The microphone 238 receives instructions and other information from the user by receiving voice uttered by the user. The microphone 238 captures the voice uttered by the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the user's surroundings (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

The communication I/F 44 is connected to the network 54. The communication I/Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I/Fs 44 and 26 is carried out in a secure manner.

Figure 6 shows an example of the main functions of the data processing device 12 and headset terminal 314. As shown in Figure 6, in the data processing device 12, specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32.

The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as the specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

Storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion. The emotion estimation function (emotion identification function) using the emotion identification model 59 performs various estimations and predictions regarding the user's emotions, including estimation and prediction of the user's emotion, but is not limited to these examples. Furthermore, emotion estimation and prediction also include, for example, emotion analysis.

In the headset type terminal 314, the specific processing is performed by the processor 46. A specific program 60 is stored in the storage 50. The processor 46 reads the specific program 60 from the storage 50 and executes the read specific program 60 on the RAM 48. The specific processing is realized by the processor 46 operating as the control unit 46A in accordance with the specific program 60 executed on the RAM 48. The headset type terminal 314 also has a data generation model and emotion identification model similar to the data generation model 58 and emotion identification model 59, and can use these models to perform processing similar to that of the identification processing unit 290.

Note that a device other than the data processing device 12 may have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 communicates with the server device having the data generation model 58 to obtain processing results (such as prediction results) using the data generation model 58. The data processing device 12 may also be a server device, or a terminal device owned by a user (for example, a mobile phone, robot, home appliance, etc.).

The specific processing unit 290 transmits the results of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the results of the specific processing. The microphone 238 acquires audio indicating the user input regarding the results of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

The data generation model 58 is what is known as generative AI. An example of a data generation model 58 is generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt containing an instruction is input to the data generation model 58, and inference data such as voice data indicating speech, text data indicating text, and image data indicating an image (e.g., still image data or video data) is input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in one or more data formats, such as voice data, text data, and image data. The data generation model 58 includes, for example, text generation AI, image generation AI, and multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and/or summarization. The identification processing unit 290 performs the above-mentioned identification processing using the data generation model 58. The data generation model 58 may be a fine-tuned model that outputs inference results from prompts that do not include instructions. In this case, the data generation model 58 can output inference results from prompts that do not include instructions. The data processing device 12 or the like includes multiple types of data generation models 58, and the data generation model 58 includes AI other than the generation AI. Examples of AI other than the generation AI include linear regression, logistic regression, decision trees, random forests, support vector machines (SVMs), k-means clustering, convolutional neural networks (CNNs), recurrent neural networks (RNNs), generative adversarial networks (GANs), and naive Bayes, and can perform various processes, but are not limited to these examples. The AI may also be an AI agent. When the processing of each of the above-mentioned parts is performed by AI, the processing may be performed in part or in whole by AI, but is not limited to these examples. In addition, processing performed by AI, including generation AI, may be replaced with rule-based processing, and rule-based processing may be replaced with processing performed by AI, including generation AI.

The data processing system 310 according to the third embodiment performs the same processing as the data processing system 10 according to the first embodiment. Processing by the data processing system 310 is executed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the headset terminal 314, but may also be executed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the headset terminal 314. In addition, the specific processing unit 290 of the data processing device 12 acquires or collects information required for processing from the headset terminal 314 or an external device, and the headset terminal 314 acquires or collects information required for processing from the data processing device 12 or an external device.

Each of the multiple elements, including the acquisition unit, analysis unit, filtering unit, provision unit, and generation unit, described above, is realized, for example, by at least one of the headset-type terminal 314 and the data processing device 12. For example, the acquisition unit collects voice data using the microphone of the headset-type terminal 314, which is then analyzed by the specific processing unit 290 of the data processing device 12. The analysis unit, realized, for example, by the specific processing unit 290 of the data processing device 12, analyzes the voice data using emotion recognition AI. The filtering unit, realized, for example, by the specific processing unit 290 of the data processing device 12, converts negative emotional voice into rational voice. The provision unit plays the converted voice using, for example, the speaker of the headset-type terminal 314. The generation unit, realized, for example, by the specific processing unit 290 of the data processing device 12, omits negative comments and extracts and presents only important information. The acquisition unit, for example, estimates the user's emotions using the control unit 46A of the headset-type terminal 314, and adjusts the timing of voice data acquisition. The correspondence between each part and the device or control unit is not limited to the example described above, and various modifications are possible.

[Fourth embodiment]
FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

As shown in FIG. 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

The data processing device 12 includes a computer 22, a database 24, and a communication I/F 26. The computer 22 includes a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and communication I/F 26 are also connected to the bus 34. The communication I/F 26 is connected to a network 54. Examples of the network 54 include a WAN and/or a LAN.

The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I/F 44, and a control target 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

The microphone 238 receives instructions and other information from the user by receiving voice uttered by the user. The microphone 238 captures the voice uttered by the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS image sensor or CCD image sensor, and captures images of the user's surroundings (e.g., an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

The communication I/F 44 is connected to the network 54. The communication I/Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I/Fs 44 and 26 is carried out in a secure manner.

The control object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, in the data processing device 12, specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32.

The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as the specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

Storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion. The emotion estimation function (emotion identification function) using the emotion identification model 59 performs various estimations and predictions regarding the user's emotions, including estimation and prediction of the user's emotion, but is not limited to these examples. Furthermore, emotion estimation and prediction also include, for example, emotion analysis.

In robot 414, specific processing is performed by processor 46. Specific program 60 is stored in storage 50. Processor 46 reads specific program 60 from storage 50 and executes the read specific program 60 on RAM 48. The specific processing is realized by processor 46 operating as control unit 46A in accordance with specific program 60 executed on RAM 48. Robot 414 also has a data generation model and emotion identification model similar to data generation model 58 and emotion identification model 59, and can use these models to perform processing similar to that of identification processing unit 290.

Note that a device other than the data processing device 12 may have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 communicates with the server device having the data generation model 58 to obtain processing results (such as prediction results) using the data generation model 58. The data processing device 12 may also be a server device, or a terminal device owned by a user (for example, a mobile phone, robot, home appliance, etc.).

The specific processing unit 290 transmits the results of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the results of the specific processing. The microphone 238 acquires audio indicating the user input regarding the results of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

The data generation model 58 is what is known as generative AI. An example of a data generation model 58 is generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt containing an instruction is input to the data generation model 58, and inference data such as voice data indicating speech, text data indicating text, and image data indicating an image (e.g., still image data or video data) is input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in one or more data formats, such as voice data, text data, and image data. The data generation model 58 includes, for example, text generation AI, image generation AI, and multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and/or summarization. The identification processing unit 290 performs the above-mentioned identification processing using the data generation model 58. The data generation model 58 may be a fine-tuned model that outputs inference results from prompts that do not include instructions. In this case, the data generation model 58 can output inference results from prompts that do not include instructions. The data processing device 12 or the like includes multiple types of data generation models 58, and the data generation model 58 includes AI other than the generation AI. Examples of AI other than the generation AI include linear regression, logistic regression, decision trees, random forests, support vector machines (SVMs), k-means clustering, convolutional neural networks (CNNs), recurrent neural networks (RNNs), generative adversarial networks (GANs), and naive Bayes, and can perform various processes, but are not limited to these examples. The AI may also be an AI agent. When the processing of each of the above-mentioned parts is performed by AI, the processing may be performed in part or in whole by AI, but is not limited to these examples. In addition, processing performed by AI, including generation AI, may be replaced with rule-based processing, and rule-based processing may be replaced with processing performed by AI, including generation AI.

The data processing system 410 according to the fourth embodiment performs processing similar to that of the data processing system 10 according to the first embodiment. Processing by the data processing system 410 is executed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the robot 414, but may also be executed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the robot 414. Furthermore, the specific processing unit 290 of the data processing device 12 acquires or collects information required for processing from the robot 414 or an external device, etc., and the robot 414 acquires or collects information required for processing from the data processing device 12 or an external device, etc.

Each of the multiple elements, including the acquisition unit, analysis unit, filtering unit, provision unit, and generation unit, described above, is realized, for example, by at least one of the robot 414 and the data processing device 12. For example, the acquisition unit collects voice data using the microphone of the robot 414, which is analyzed by the specific processing unit 290 of the data processing device 12. The analysis unit is realized, for example, by the specific processing unit 290 of the data processing device 12, and analyzes the voice data using emotion recognition AI. The filtering unit is realized, for example, by the specific processing unit 290 of the data processing device 12, and converts negative emotional voice into rational voice. The provision unit plays the converted voice using, for example, the speaker of the robot 414. The generation unit is realized, for example, by the specific processing unit 290 of the data processing device 12, and omits negative comments and extracts and presents only important information. The acquisition unit, for example, estimates the user's emotions using the control unit 46A of the robot 414, and adjusts the timing of voice data acquisition. The correspondence between each part and the device or control unit is not limited to the example described above, and various modifications are possible.

The emotion identification model 59, which serves as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see Figure 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

Figure 9 shows an emotion map 400 on which multiple emotions are mapped. In emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both emotions and mental states. Emotions that are generally generated from reactions that occur in the brain are arranged on the left side of the concentric circles. Emotions that are generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions that are generally generated from reactions that occur in the brain and are induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, emotion map 400 maps multiple emotions based on the structure by which emotions are generated, with emotions that tend to occur simultaneously being mapped close together.

These emotions are distributed in the 3 o'clock direction on emotion map 400, and usually fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness takes precedence over internal sensations, resulting in a sense of calm.

The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further out you go on emotion map 400, the more visible (expressed in behavior) the emotion becomes.

Here, human emotions are based on various balances such as posture and blood sugar levels, and when these balances deviate from the ideal, it indicates discomfort, and when they approach the ideal, it indicates pleasure. Emotions can also be created for robots, cars, motorcycles, etc., based on various balances such as posture and remaining battery life, and when these balances deviate from the ideal, it indicates discomfort, and when they approach the ideal, it indicates pleasure. Emotion maps can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on speech emotion recognition and emotional brain physiological signal analysis systems, Tokushima University, doctoral dissertation: https://ci.nii.ac.jp/naid/500000375379). The left half of the emotion map is lined with emotions belonging to an area called "reaction," where sensation is dominant. The right half of the emotion map is lined with emotions belonging to an area called "situation," where situational awareness is dominant.

The emotion map defines two emotions that encourage learning. One is the negative emotion around the middle of "repentance" or "reflection" on the situation side. In other words, this is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the positive emotion around "desire" on the response side. In other words, this is when the robot experiences positive feelings such as "I want more" or "I want to know more."

The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are close to each other have similar values, as in the emotion map 900 shown in Figure 10. Figure 10 shows an example in which multiple emotions, such as "relieved," "calm," and "reassuring," have similar emotion values.

In the above embodiment, an example was given in which a specific process is performed by a single computer 22, but the technology disclosed herein is not limited to this, and distributed processing of the specific process may also be performed by multiple computers, including computer 22.

In the above embodiment, an example was described in which the specific processing program 56 is stored in the storage 32, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

It is not necessary to store the entire specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entire specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

The following types of processors can be used as hardware resources for executing specific processes. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource for executing specific processes by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which are processors with a circuit configuration designed specifically for executing specific processes. All processors have built-in or connected memory, and all use the memory to execute specific processes.

The hardware resource that executes the specific processing may be composed of one of these various processors, or may be composed of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). The hardware resource that executes the specific processing may also be a single processor.

As an example of a configuration using a single processor, first, there is a configuration in which one processor is configured using a combination of one or more CPUs and software, and this processor functions as a hardware resource that executes specific processing. Second, there is a configuration in which a processor is used to realize the functions of an entire system, including multiple hardware resources that execute specific processing, on a single IC chip, as typified by SoC (System-on-a-chip). In this way, specific processing is realized using one or more of the various processors listed above as hardware resources.

More specifically, the hardware structure of these various processors can be an electrical circuit that combines circuit elements such as semiconductor devices. Furthermore, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps can be added, or the processing order can be rearranged, all within the scope of the spirit of the invention.

Furthermore, in the above examples, the first to fourth embodiments have been described separately, but some or all of these embodiments may be combined. Furthermore, the smart device 14, smart glasses 214, headset terminal 314, and robot 414 are only examples, and they may be combined, or other devices may be used. Furthermore, in the above examples, the first and second embodiments have been described separately, but these may also be combined.

The above-described written content and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above explanation of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or substitutions may be made to the above-described written content and illustrations, as long as they do not deviate from the spirit of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described written content and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

All publications, patent applications, and technical standards mentioned in this specification are incorporated herein by reference to the same extent as if each individual publication, patent application, and technical standard was specifically and individually indicated to be incorporated by reference.

(Appendix 1)
an acquisition unit that acquires voice data;
an analysis unit that analyzes the voice data acquired by the acquisition unit and classifies the emotion of the speaker;
a filtering unit that converts negative emotional voice into rational voice based on the emotion classified by the analysis unit;
a providing unit that provides the voice converted by the filtering unit to a user.
(Appendix 2)
The analysis unit
2. The system of claim 1, wherein emotion is determined based on characteristics of tone, pitch, and speed of speech.
(Appendix 3)
The filtering unit
2. The system of claim 1, wherein the system converts voice containing an angry emotion into a calm tone.
(Appendix 4)
A generating unit is provided,
The generation unit
The system described in Appendix 1 is characterized in that it eliminates negative comments and extracts and presents only important information.
(Appendix 5)
The providing unit
2. The system of claim 1, further comprising: providing the converted audio to the user.
(Appendix 6)
The acquisition unit
The system according to claim 1, further comprising: estimating a user's emotion; and adjusting timing for acquiring voice data based on the estimated user's emotion.
(Appendix 7)
The acquisition unit
The system according to claim 1, further comprising: analyzing a user's past call history and selecting an optimal acquisition method.
(Appendix 8)
The acquisition unit
The system according to claim 1, wherein when acquiring voice data, filtering is performed based on the user's current work situation or area of interest.
(Appendix 9)
The acquisition unit
The system according to claim 1, further comprising: estimating a user's emotion; and determining a priority order of voice data to be acquired based on the estimated user's emotion.
(Appendix 10)
The acquisition unit
The system according to claim 1, wherein when acquiring voice data, highly relevant data is acquired preferentially based on the user's geographical location information.
(Appendix 11)
The acquisition unit
The system described in Appendix 1, characterized in that when acquiring voice data, the system analyzes the user's SNS activity and acquires related data.
(Appendix 12)
The analysis unit
2. The system of claim 1, further comprising: estimating a user's emotion; and adjusting a method of expressing the analysis based on the estimated user's emotion.
(Appendix 13)
The analysis unit
The system according to claim 1, wherein the level of detail of the analysis is adjusted based on the importance of the audio data during the analysis.
(Appendix 14)
The analysis unit
2. The system of claim 1, wherein during analysis, different analysis algorithms are applied depending on the category of the audio data.
(Appendix 15)
The analysis unit
10. The system of claim 1, further comprising: estimating a user's emotion; and adjusting the length of the analysis based on the estimated user's emotion.
(Appendix 16)
The analysis unit
The system according to claim 1, wherein, during analysis, the system determines the priority of analysis based on the time when the voice data was acquired.
(Appendix 17)
The analysis unit
2. The system of claim 1, wherein during analysis, the order of analysis is adjusted based on the relevance of the audio data.
(Appendix 18)
The filtering unit
2. The system of claim 1, further comprising: estimating a user's emotion; and adjusting filtering criteria based on the estimated user's emotion.
(Appendix 19)
The filtering unit
The system of claim 1, wherein the accuracy of filtering is improved based on correlations between audio data during filtering.
(Appendix 20)
The filtering unit
The system according to claim 1, wherein the filtering is performed taking into consideration attribute information of the person who submitted the voice data.
(Appendix 21)
The filtering unit
The system according to claim 1, further comprising: estimating a user's emotion; and adjusting an order in which filtering results are displayed based on the estimated user's emotion.
(Appendix 22)
The filtering unit
The system according to claim 1, wherein filtering is performed taking into account the geographical distribution of the audio data.
(Appendix 23)
The filtering unit
The system according to claim 1, wherein the accuracy of filtering is improved based on related literature of the speech data during filtering.
(Appendix 24)
The providing unit
10. The system of claim 1, further comprising: estimating a user's emotion; and adjusting a presentation method of the offering based on the estimated user's emotion.
(Appendix 25)
The providing unit
The system according to claim 1, wherein the system selects the most suitable display method by referring to the user's past operation history when providing the information.
(Appendix 26)
The providing unit
The system according to claim 1, further comprising: estimating a user's emotion; and adjusting the provided operation procedure based on the estimated user's emotion.
(Appendix 27)
The providing unit
The system according to claim 1, wherein the system selects the most suitable display method in consideration of the user's device information when providing the information.
(Appendix 28)
The generation unit
The system according to claim 1, further comprising: estimating a user's emotion; and determining a priority of speech to be generated based on the estimated user's emotion.
(Appendix 29)
The generation unit
The system according to claim 1, characterized in that during generation, correlations between speech data are taken into account to improve generation accuracy.
(Appendix 30)
The generation unit
The system according to claim 1, wherein the generation is performed taking into consideration attribute information of the person who submitted the voice data.
(Appendix 31)
The generation unit
The system according to claim 1, further comprising: estimating a user's emotion; and adjusting a display method of the generated voice based on the estimated user's emotion.
(Appendix 32)
The generation unit
The system according to claim 1, wherein the generation is performed taking into consideration the geographical distribution of the voice data.
(Appendix 33)
The generation unit
The system according to claim 1, characterized in that, during generation, the accuracy of generation is improved by referring to literature related to the speech data.

10, 210, 310, 410 Data processing system 12 Data processing device 14 Smart device 214 Smart glasses 314 Headset type terminal 414 Robot

Claims (4)

an acquisition unit that acquires voice data;
an analysis unit that analyzes the voice data acquired by the acquisition unit and classifies the emotion of the speaker;
a filtering unit that converts negative emotional voice into rational voice based on the emotion classified by the analysis unit;
a providing unit that provides the voice converted by the filtering unit to a user ,
The acquisition unit
The user's emotion is estimated, and the timing of acquiring voice data is adjusted based on the estimated user's emotion.
A system characterized by: The analysis unit
The system of claim 1, wherein emotion is determined based on tone, pitch, and rate characteristics of the voice. The filtering unit
The system of claim 1, wherein the system converts angry speech into a calm tone. A generating unit is provided,
The generation unit
The system according to claim 1, characterized in that it omits negative comments and extracts and presents only important information.