JP7778201B2 - system - Google Patents

Description

The technology disclosed herein relates to a system.

Patent document 1 discloses a persona chatbot control method executed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot's character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Japanese Patent Application Laid-Open No. 2022-180282

Special frauds have become a social problem, and it is often difficult for victims to realize they are being scammed. In particular, with telephone fraud, it can take time for victims to realize they are being scammed, and in the meantime they may suffer significant losses.

This invention detects incoming fraudulent calls and listens to the conversation through a smartphone application or a microphone installed on a home phone. The conversation data is converted into text and sent to a generative AI. The generative AI determines from the conversation data that fraud is suspected, and if fraud is suspected, an audio alert is issued from the microphone. This allows users to become aware of the possibility of fraud early on and prevent damage.

1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. FIG. 10 is a conceptual diagram illustrating an example of the configuration of a data processing system according to a second embodiment. FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. FIG. 10 is a conceptual diagram illustrating an example of the configuration of a data processing system according to a third embodiment. FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. FIG. 10 is a conceptual diagram illustrating an example of the configuration of a data processing system according to a fourth embodiment. FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. 1 shows an emotion map onto which multiple emotions are mapped. 1 shows an emotion map onto which multiple emotions are mapped. FIG. 10 is a sequence diagram showing a flow of processing in the data processing system according to the first embodiment of the first form example. FIG. 10 is a sequence diagram showing a processing flow of the data processing system in Application Example 1 of Form Example 1. FIG. 10 is a sequence diagram showing a processing flow of a data processing system in a second embodiment of the second form example. FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 of Form Example 2. FIG. 10 is a sequence diagram showing a processing flow of a data processing system in a third embodiment of the third form example. FIG. 13 is a sequence diagram showing the flow of processing in the data processing system in Application Example 3 of Form Example 3. FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in the first embodiment of the first form example when an emotion engine is combined. FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1 of Form Example 1 when an emotion engine is combined. FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in the second embodiment of the second form example when an emotion engine is combined. FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 of Form Example 2 when an emotion engine is combined. FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in the third embodiment of the third form example when an emotion engine is combined. FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 3 of Form Example 3 when an emotion engine is combined.

Below, an example of an embodiment of a system relating to the technology disclosed herein will be described with reference to the accompanying drawings.

First, let me explain the terminology used in the following explanation.

In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Furthermore, a processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose Computing on Graphics Processing Units), an APU (Accelerated Processing Unit), or a TPU (TENSOR PROCESSING UNIT (registered trademark)).

In the following embodiments, coded random access memory (RAM) is memory in which information is temporarily stored and is used as work memory by the processor.

In the following embodiments, the coded storage refers to one or more non-volatile storage devices that store various programs, parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

In the following embodiments, a communication I/F (Interface) with a symbol is an interface that includes a communication processor, an antenna, etc. The communication I/F controls communication between multiple computers. Examples of communication standards that can be applied to the communication I/F include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

In the following embodiments, "A and/or B" is synonymous with "at least one of A and B." In other words, "A and/or B" means that it may be just A, just B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and/or B" also applies when three or more things are expressed connected by "and/or."

[First embodiment]

Figure 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

As shown in FIG. 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

The data processing device 12 includes a computer 22, a database 24, and a communication I/F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and communication I/F 26 are also connected to the bus 34. The communication I/F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and/or a LAN (Local Area Network).

The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I/F 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

The reception device 38 is equipped with a touch panel 38A, a microphone 38B, etc., and receives user input. The touch panel 38A detects contact with an indicator (e.g., a pen or finger) to receive user input via the indicator. The microphone 38B detects the user's voice to receive user input via voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

The output device 40 is equipped with a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible by the user 20 (e.g., audio and/or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

The communication I/F 44 is connected to the network 54. The communication I/Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54.

Figure 2 shows an example of the main functions of the data processing device 12 and smart device 14.

As shown in FIG. 2, in the data processing device 12, specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

Storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

Next, we will explain the identification process performed by the identification processing unit 290 of the data processing device 12.

"Example 1"

One embodiment of the present invention uses a smartphone application. In this case, the application installed on the smartphone detects an incoming phone call and listens to the conversation through the microphone. The listened-to conversation data is converted into text and sent to the generative AI. The generative AI determines from the conversation data that fraud is suspected, and if fraud is suspected, issues an audio alert from the smartphone's speaker.

"Example 2"

Another embodiment of the present invention involves installing a microphone on a home phone. In this case, the home phone's microphone listens to telephone conversations, and the listened conversation data is transcribed and sent to the generative AI. The generative AI determines from the conversation data that fraud is suspected, and if fraud is suspected, issues an audio alert from the home phone's speaker.

"Example 3"

A further embodiment of the present invention is one in which the generative AI detects specific phrases or patterns that indicate the possibility of specialized fraud. Specifically, the generative AI learns typical phrases for "bank transfer fraud" and typical patterns for "it's me" fraud, and by detecting these, determines the possibility of fraud.

The processing flow for each example is explained below.

"Example 1"

Step 1: The application installed on the smartphone detects an incoming call.

Step 2: The application listens to your conversation through your smartphone's microphone.

Step 3: The conversation data that was heard is transcribed and sent to the generative AI.

Step 4: The generative AI determines that fraud is suspected based on the conversation data.

Step 5: If fraud is suspected, an audio alert will be issued through your smartphone's speaker.

"Example 2"

Step 1: A microphone installed on a home phone listens to the phone conversation.

Step 2: The conversation data that is heard is transcribed and sent to the generative AI.

Step 3: The generative AI determines that fraud is suspected based on the conversation data.

Step 4: If fraud is suspected, an audio alert will be issued over the home phone speaker.

"Example 3"

Step 1: The generative AI learns specific phrases or patterns that indicate potential fraud.

Step 2: The generative AI detects specific phrases or patterns it has learned to identify suspected fraud in the conversation data.

Step 3: If fraud is suspected, an audio alert will be issued.

(Example 1)

Next, we will explain Example 1 of Form Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the smart device 14 will be referred to as the "terminal."

In recent years, telephone fraud has been on the rise, with special frauds targeting the elderly becoming a particular social problem. Conventional countermeasures require users to identify fraudulent calls themselves, but as fraud methods become more sophisticated, this becomes increasingly difficult. Therefore, there is a need for a system that automatically detects potential fraud and issues a warning when users receive a fraudulent call.

The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means. In this invention, the server includes means for detecting incoming fraudulent calls, means for listening to the conversation from a microphone installed in the communication terminal, means for converting the conversation data into text and sending it to the generative AI model, and means for issuing an audio alert from the speaker of the communication terminal if the generative AI model determines that fraud is suspected based on the conversation data. This makes it possible to automatically detect the possibility of fraud when a user receives a fraudulent call and immediately issue a warning.

"Means for detecting incoming fraudulent calls" refers to functions or devices in a communications terminal for detecting incoming calls.

"Means for listening to conversations using a microphone installed in a communications terminal" refers to a function or device that uses a microphone built into or connected to a communications terminal to collect audio during a call in real time.

"Means for converting conversation data into text and sending it to the generative AI model" refers to functions or devices for converting collected voice data into text data and sending that text data to the generative AI model.

A "generative AI model" is an artificial intelligence model that analyzes input data and detects specific patterns and phrases.

"Means for issuing an audio alert from the speaker of the communication terminal when it is determined that fraud is suspected" refers to a function or device for playing an alert sound or message through the speaker of the communication terminal when the generative AI model detects the possibility of fraud.

"Specific phrases or patterns that indicate the possibility of specialized fraud" are specific words or sentence structures associated with fraudulent acts, and the generative AI model's detection of these serves as the basis for determining the possibility of fraud.

"Audio alert" is a message or sound that warns the user audibly.

This invention is a system that automatically detects fraudulent calls and issues a warning to the user using an application installed on a communication terminal. A specific embodiment of this system is described below.

First, the user installs a dedicated application on their communication device. This application has the function of detecting incoming phone calls. Specifically, it uses the communication device's phone API to catch incoming call events.

When a user answers a call, the device listens to the conversation in real time through the microphone. During this time, the application runs in the background and captures the conversation. The listened conversation data is transcribed using the Google (registered trademark) Speech-to-Text API. The audio data is sent to the API and received as text data.

The transcribed conversation data is sent to a server via the internet. The HTTPS protocol is used for transmission to ensure data security. The server inputs the received conversation data into a generative AI model (e.g., OpenAI's GPT-4 (registered trademark)). The generative AI model uses prompt text to analyze the possibility of fraud. Specific examples of prompt text are as follows:

Analyze the conversation data below to determine if it may be a scam.

Conversation data: "Your bank account has been compromised. Please provide your account number for verification."

The generative AI model analyzes the conversation data and determines whether fraud is suspected. If fraud is suspected, the server sends that information back to the device, again using the HTTPS protocol.

The device analyzes the results received from the server, and if fraud is suspected, an audio alert is emitted from the communication device's speaker. Specifically, an audio message such as "There is a possibility of fraud. Please be careful" is played. This alert makes the user aware of the possibility of fraud and allows them to take appropriate action.

As a concrete example, consider the case where a user receives a phone call saying, "Your bank account has been fraudulently used. Please provide your account number to confirm." This conversation is picked up by a microphone and transcribed. The transcribed data becomes, "Your bank account has been fraudulently used. Please provide your account number to confirm." This data is sent to a generative AI model, which determines that there is a high possibility of fraud. The server sends the result back to the device, which then issues an audio alert. The user hears this alert and becomes aware of the possibility of fraud.

In this way, the invention can automatically detect possible fraud when a user receives a fraudulent call and immediately issue a warning.

The flow of the identification process in Example 1 will be explained using Figure 11.

Step 1: Detect incoming calls

The terminal detects incoming calls through an application installed on the communication terminal. Specifically, it uses the communication terminal's telephone API to catch incoming call events. The input is the incoming call signal, and the output is the incoming call detection event.

Step 2: Listen to the conversation

When a user answers a call, the device listens to the conversation in real time through the microphone. The application runs in the background and captures the conversation. The input is the audio data during the call, and the output is the collected audio data.

Step 3: Transcribe the conversation data

The device uses the Google Speech-to-Text API to transcribe the conversation data it hears. It sends the audio data to the API and receives it as text data. The input is the collected audio data, and the output is the transcribed text data.

Step 4: Send the transcribed data

The terminal sends the transcribed conversation data to the server. The HTTPS protocol is used for transmission to ensure data security. The input is transcribed text data, and the output is a data transmission event to the server.

Step 5: Fraud Detection

The server inputs the received conversation data into a generative AI model. The generative AI model uses prompts to analyze the possibility of fraud. Specific examples of prompts are as follows:

Analyze the conversation data below to determine if it may be a scam.

Conversation data: "Your bank account has been compromised. Please provide your account number for verification."

The input is transcribed text data, and the output is a judgment result regarding the possibility of fraud.

Step 6: Receive the results

The server receives the judgment results from the generative AI model and returns them to the device. The return is again done using the HTTPS protocol. The input is the judgment result from the generative AI model, and the output is a data transmission event to the device.

Step 7: Trigger an audio alert

The device analyzes the judgment results received from the server, and if fraud is suspected, it issues an audio alert from the communication device's speaker. Specifically, it plays an audio message such as "There is a possibility of fraud. Please be careful." The input is the judgment result from the server, and the output is the generation of an audio alert.

(Application Example 1)

Next, we will explain Application Example 1 of Form Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the smart device 14 will be referred to as the "terminal."

In recent years, telephone fraud has been on the rise, with special fraud targeting the elderly becoming a particular social problem. Conventional fraud prevention measures make it difficult for users to detect potential fraud, making it difficult to prevent damage before it occurs. Therefore, there is a need for a system that can analyze telephone conversations in real time and immediately issue a warning if there is a possibility of fraud.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

In this invention, the server includes means for detecting incoming fraudulent calls, means for listening to conversations from a voice input device installed in the communication terminal, means for converting the conversation data into text and sending it to the generative AI, means for issuing an audio alert from the voice output device if the generative AI determines that fraud is suspected from the conversation data, means for the generative AI to detect specific phrases or patterns that indicate the possibility of fraud, and means for the audio alert to contain content that encourages the user to become aware of the possibility of fraud. This allows the user to immediately become aware of the possibility of fraud and prevent damage from occurring.

A "fraudulent call" is a call made with the intent of defrauding someone of money or personal information through fraudulent means.

"Incoming call" refers to an incoming phone call.

A "communications terminal" is a device for sending and receiving voice and data, including smartphones and home telephones.

An "audio input device" is a device that converts audio into digital data, such as a microphone.

"Conversation data" refers to data that has been converted into text from speech captured by a voice input device.

"Generative AI" is artificial intelligence that analyzes input data and detects specific patterns and phrases.

An "audio output device" is a device for outputting digital data as audio, and a speaker is an example of this.

"Audio alert" is a warning sound or message emitted from an audio output device.

"Specific phrases or patterns" are characteristic words or sentence structures that indicate possible fraud.

"User" refers to an individual or organization that uses this system.

A system for implementing this invention has the ability to detect incoming fraudulent calls and listen to the conversation from a voice input device installed on the communication terminal. The conversation data is transcribed in real time and sent to a generative AI. The generative AI detects specific phrases or patterns from the conversation data that indicate the possibility of fraud, and if fraud is suspected, issues a voice alert from a voice output device.

Hardware and software used

Hardware: Communication devices such as smartphones and home phones, microphones (audio input devices), speakers (audio output devices)

Software:

SpeechRecognition: A Python library for converting speech to text

requests: A Python library for sending HTTP requests

Generative AI: Artificial intelligence that analyzes text data using external APIs

System Operation

1. Incoming call detection: When the communication terminal detects an incoming call, it begins recording the conversation.

2. Conversation transcription: Recorded audio data is transcribed in real time using the SpeechRecognition library.

3. Generative AI Analysis: The transcribed conversation data is sent to a generative AI via an HTTP request. The generative AI detects specific phrases or patterns that indicate potential fraud.

4. Alert generation: If the generative AI detects the possibility of fraud, an audio alert will be emitted from the communication device's speaker. This audio alert will alert the user to the possibility of fraud.

Specific examples

For example, if a user receives a phone call and the caller says, "Your bank account has been fraudulently accessed. Please provide your account details immediately," the system will transcribe this conversation in real time and send it to the generative AI. The generative AI will detect that this phrase matches a specific pattern that indicates potential fraud and immediately issue an audio alert.

Example prompt

Examples of prompts for generative AI models include:

"Determine whether the following text is potentially fraudulent: 'Your bank account has been compromised. Please provide your account information immediately.'"

In this way, users can immediately become aware of potential fraud and prevent damage before it occurs.

The flow of the specific processing in Application Example 1 will be explained using Figure 12.

Step 1: Incoming call detection

The communication terminal detects an incoming call. The input is the incoming call signal, and the output is a trigger to start recording. Specifically, the communication terminal's phone application detects the incoming call and activates the recording module.

Step 2: Record the conversation

The communication terminal's audio input device (microphone) records the conversation. The input is the audio data during the call, and the output is the recorded audio file. Specifically, the recording module captures the audio during the call and saves it as an audio file.

Step 3: Transcribe the conversation

Recorded audio data is converted into text using the SpeechRecognition library. The input is an audio file, and the output is text data. Specifically, the speech recognition engine analyzes the audio file and generates the corresponding text.

Step 4: Analysis using generative AI

The transcribed text data is sent to the generative AI via an HTTP request. The input is text data, and the output is a flag indicating the possibility of fraud. Specifically, a request containing the text data is sent to the generative AI's API endpoint, and the analysis results are received.

Step 5: Determine the possibility of fraud

Generative AI analyzes text data to detect specific phrases or patterns that indicate potential fraud. The input is text data, and the output is a flag indicating potential fraud. Specifically, generative AI analyzes text data and flags cases where there is a high probability of fraud.

Step 6: Send an alert

If possible fraud is detected, an audio alert is emitted from the communication terminal's audio output device (speaker). The input is a flag indicating possible fraud, and the output is an audio alert. Specifically, the audio alert module receives a flag indicating possible fraud and plays an audio message to warn the user.

In this way, users can immediately become aware of potential fraud and prevent damage before it occurs.

(Example 2)

Next, we will explain Example 2 of Form Example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the smart device 14 will be referred to as the "terminal."

In recent years, fraudulent phone scams have become more sophisticated, resulting in many people falling victim to them. Special frauds targeting the elderly in particular are on the rise, creating a need for effective countermeasures. Conventional countermeasures lack systems that can detect fraudulent phone calls in real time and issue warnings to users, making it difficult to prevent fraud before it occurs.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

In this invention, the server includes means for detecting incoming fraudulent calls, means for collecting conversation from a voice input device installed in the communication device, means for converting the conversation data into text data, means for transmitting the text data to the generative AI model, and means for issuing a voice alert from the voice input device if the generative AI model determines that fraud is suspected based on the conversation data. This makes it possible to detect fraudulent calls in real time and issue an immediate warning to the user.

"Means for detecting incoming fraudulent calls" refers to a function that identifies and detects incoming calls that may be fraudulent to a communications device.

An "audio input device" is a microphone or other audio capture device installed on a communications device to capture a user's speech.

"Means for converting conversational data into text data" refers to speech recognition software or algorithms used to analyze collected voice data and convert it into text format.

A "generative AI model" is an artificial intelligence model that analyzes input text data and determines the possibility of fraud based on specific patterns and phrases.

"Means for issuing audio alerts" refers to a function that plays an audio message to warn the user if fraud is suspected.

This invention is a system that detects incoming fraudulent calls in real time and issues a warning to users. A specific embodiment of this system is described below.

1. System Configuration

The system consists of the following main components:

Methods for detecting incoming fraudulent calls

Voice input device

Method of converting conversation data into text data

Generative AI model

Method for issuing audio alerts

2. Hardware and Software Use

The voice input device uses a microphone installed in a home telephone. This microphone collects the user's speech in real time.

To convert conversation data into text data, we use voice recognition software (e.g., Google Speech-to-Text API). This software analyzes the collected voice data and converts it into text data.

The generative AI model uses advanced artificial intelligence models such as OpenAI GPT-4. This model analyzes the input text data and determines the likelihood of fraud.

The audio alert is generated using the speaker on a home phone, which will emit an audio alert if fraud is suspected.

3. System Operation

When a user starts a conversation on their home phone, a voice input device collects the conversation. The collected voice data is converted into text data using voice recognition software. This text data is sent to a server and input into a generative AI model. The generative AI model analyzes the conversation data and, if it determines that fraud is suspected, the server sends an alert signal to the home phone. The home phone's speaker emits an audio alert to warn the user.

4. Specific Examples

For example, suppose a user is speaking on their home phone, "Hello, your bank account has been compromised. Immediate action is required." A voice input device collects this speech, and speech recognition software converts it into text data. This text data is sent to a server, where it is analyzed by a generative AI model. If the generative AI model determines that there is a high possibility of fraud, the server sends an alert signal to the home phone. The speaker on the home phone will emit an audio alert saying, "Possible fraud. Please be careful."

5. Prompt Sentence Examples

"Analyze the conversation data below and determine whether or not you suspect fraud. If you suspect fraud, please explain why.

Conversation data: 'Hello, your bank account has been compromised. We need your immediate attention.'

In this way, users can be aware of their fraud risk in real time.

The flow of the identification process in Example 2 will be explained using Figure 13.

Step 1:

Audio collection

Device: A voice input device (microphone) installed on a home phone collects the user's conversation in real time.

Input: User's voice conversation

Output: Digital audio data

What it does: When a user starts a phone conversation, the microphone automatically captures the audio and stores it as digital audio data.

Step 2:

Transcription of audio data

Device: Built-in voice recognition software (e.g., Google Speech-to-Text API) in home phones converts collected voice data into text data.

Input: Digital audio data

Output: Text data

What it does: Speech recognition software analyzes the audio data and generates text data such as, "Hello, your bank account has been compromised. Immediate action is required."

Step 3:

Send text data

Device: The home phone sends the transcribed conversation data to the server.

Input: Text data

Output: Send data to the server

How it works: A home phone sends data over the internet to a server, which then passes the data to a generative AI model.

Step 4:

Fraud detection

Server: A generative AI model (e.g., OpenAI GPT-4) analyzes the received conversation data and determines whether fraud is suspected.

Input: Text data

Output: Judgment result regarding the likelihood of fraud

Specific behavior: The generative AI model analyzes the text "Hello, your bank account has been compromised. Immediate action is required" and determines that it is likely to be fraudulent.

Step 5:

Audio alert occurs

Server: If fraud is suspected, the server sends an alert signal to the home phone.

Device: The speaker on your home phone receives the alert signal from the server and plays an audio alert.

Input: Alert signal

Output: Audio alert

Specific behavior: The speaker on your home phone will play a voice alert saying, "This may be a scam. Be careful."

(Application Example 2)

Next, we will explain Application Example 2 of Form Example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the smart device 14 will be referred to as the "terminal."

In recent years, the number of victims of fraudulent phone calls has been increasing, and special frauds targeting the elderly in particular have become a social problem. Conventional fraud prevention systems have been inadequate in detecting fraudulent calls, making it difficult for users to recognize the possibility of fraud. Furthermore, there is a demand for systems that can detect fraud in real time and issue warnings to users on home phones and smartphones.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

In this invention, the server includes means for detecting incoming fraudulent calls, means for listening to conversations using a voice recognition device, means for converting the conversation data into text and sending it to the generative AI, means for issuing an audio alert from a voice output device if the generative AI determines that fraud is suspected based on the conversation data, and means including an application to be installed on a smartphone. This enables real-time detection of fraudulent calls and prompt warnings to users.

A "fraudulent call" is a call made with the intent of defrauding someone of money or personal information through fraudulent means.

"Incoming call" refers to an incoming phone call.

A "voice recognition device" is a device that converts voice into text data.

"Conversation data" refers to the content of a conversation converted into text by a voice recognition device.

"Generative AI" is artificial intelligence that generates new information based on input data.

An "audio output device" is a device for outputting audio.

"Voice alert" is a function that issues a warning via voice.

"Applications installed on smartphones" are software programs that run on smartphones.

The following hardware and software are required to implement this invention. Hardware requires a smartphone, a voice recognition device, and a voice output device. Software requires a voice recognition library (e.g., speech_recognition), a generative AI library (e.g., openai), and a text-to-speech library (e.g., pyttsx3).

First, the phone call is recorded in real time using the smartphone's microphone. The recorded voice data is converted into text data using a voice recognition device. The speech_recognition library is used for this voice recognition.

The converted text data is then sent to the generative AI. The generative AI analyzes the input text data and determines whether it is fraudulent. This analysis is performed using the OpenAI library. Specifically, the following prompt is sent to the generative AI:

Please determine if the following conversation is a scam:

"Hello, this is the bank's security department. Your account has been compromised. Please provide your account number and PIN."

If you suspect it may be a scam, please reply 'scam'.

If the generative AI determines that there is a possibility of fraud, it will issue an audio alert using the smartphone's audio output device. This audio alert uses the pyttsx3 library. The content of the audio alert is intended to alert the user to the possibility of fraud.

For example, if a user is on a call on their smartphone and a conversation that could be fraudulent takes place, the application automatically converts the conversation into text and sends it to a generative AI to determine whether it is fraudulent. If it determines that there is a possibility of fraud, an audio alert will sound from the smartphone speaker saying, "Possible fraud!"

In this way, the invention enables real-time detection of fraudulent calls and prompt warning to users.

The flow of the specific processing in Application Example 2 will be explained using Figure 14.

Step 1:

The user initiates a call on their smartphone. The smartphone's microphone records the call in real time.

Input: Call audio

Output: Recorded audio data

Specific operation: The smartphone's microphone captures the call audio and saves it as audio data.

Step 2:

The device sends the recorded voice data to a voice recognition device, which converts the voice data into text data.

Input: Recorded audio data

Output: Text data

Specific operation: The speech recognition library (e.g., speech_recognition) analyzes the audio data and generates corresponding text data.

Step 3:

The device sends the generated text data to the generative AI, which analyzes the text data and determines whether it is fraudulent.

Input: Text data

Output: Judgment result regarding the possibility of fraud

Specific operation: A generative AI library (e.g., openai) analyzes text data based on the prompt and determines whether it is likely to be fraudulent.

Step 4:

The server receives the judgment results from the generative AI and generates an audio alert if there is a possibility of fraud.

Input: Judgment result regarding the possibility of fraud

Output: Audio alert content

Specific behavior: The generative AI generates a warning message saying, "Possible fraud!"

Step 5:

The device sends the contents of the audio alert to the audio output device, which then issues the audio alert.

Input: Audio alert content

Output: Audio alert

Specific operation: A text-to-speech library (e.g., pyttsx3) converts the contents of the audio alert into speech, and the warning sound is output from the smartphone speaker.

(Example 3)

Next, we will explain Example 3 of Form Example 3. In the following explanation, the data processing device 12 will be referred to as the "server" and the smart device 14 will be referred to as the "terminal."

Conventional fraud detection systems were limited to detecting incoming fraudulent phone calls and were inadequate at detecting fraud in text messages and emails received by users. Furthermore, there were limited ways to notify users of potential fraud, which could delay users' realization of the risk of fraud. This made it difficult to prevent fraud from occurring.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Example 3 is realized by the following means.

In this invention, the server includes means for transmitting text data entered by the user to the server, means for the server to analyze the text data using a generative artificial intelligence model and determine the possibility of fraud, and means for the server to transmit the analysis results to the communication terminal. This makes it possible to quickly detect the possibility of fraud even in text messages or emails received by the user and notify the user.

"Means for detecting incoming fraudulent calls" refers to equipment or software that identifies and detects potentially fraudulent calls from among calls received over a communication line.

"A voice input device installed on a communication terminal" refers to a microphone or related hardware for collecting voice that is attached to communication devices such as smartphones and home telephones.

"Means for converting conversation data into text and sending it to the generative AI" refers to software or hardware for converting voice data into text data and sending that text data to the generative AI.

"Generative AI" is an AI model that uses natural language processing technology to analyze text data and detect specific patterns and phrases.

"A means for issuing an audio alert" is a device or software that issues an audio warning if it determines that there is a possibility of fraud.

"Means for transmitting text data entered by a user to a server" refers to software or hardware for transmitting text data entered by a user into a communication terminal to a server via a communication line such as the Internet.

"Means for the server to analyze text data using a generative artificial intelligence model and determine the possibility of fraud" refers to software or hardware that inputs the text data received by the server into a generative artificial intelligence model and evaluates the possibility of fraud from the analysis results.

"Means by which the server transmits the analysis results to the communications terminal" refers to software or hardware for transmitting the analysis results from the generative artificial intelligence model to the communications terminal.

"Means by which the communications terminal displays the analysis results to the user" refers to software or hardware that visually or audibly notifies the user of the analysis results received from the server.

This invention relates to a system that uses a generative artificial intelligence model to detect the possibility of special fraud. A specific embodiment of this system is described below.

System Configuration

Subject: Server

The server uses a generative artificial intelligence model to analyze text data and determine the likelihood of fraud. Specifically, the server uses the following hardware and software:

Hardware: High-performance processor, memory, and storage devices

Software: Generative AI models (e.g., AI models using natural language processing technology)

The server receives text data sent by the user and inputs it into a generative artificial intelligence model. The model detects specific phrases and patterns that indicate potential fraud and returns the results to the server. The server then sends the analysis results to the communication device.

Subject: Terminal

The device is responsible for sending text data entered by the user to the server and displaying the analysis results from the server to the user. Specifically, the device uses the following hardware and software:

Hardware: Communication devices such as smartphones, tablets, and PCs

Software: Dedicated application

The device sends the text data entered by the user to the server and displays the analysis results received from the server to the user. If the analysis results indicate the possibility of fraud, the device displays a warning message.

Subject: User

The user is responsible for entering text data through the device and checking the analysis results. Specifically, the user performs the following operations:

Enter message or email content into your device

Check the analysis results and take appropriate action as necessary.

Specific examples

Example 1

When a user receives a message saying, "Mom, please transfer the money now," the system behaves as follows:

1. The user enters a message into a dedicated application on the device.

2. The device sends this message to the server.

3. The server receives the message and inputs it as a prompt into the generative artificial intelligence model.

4. The generative artificial intelligence model analyzes the message and returns the result "Possible fraud."

5. The server sends the analysis results to the device.

6. The device displays the analysis results to the user, and the user confirms the warning message.

Example 2

When a user receives an email saying, "Your bank account has been compromised. Contact us immediately," the system works as follows:

1. The user enters the email content into a dedicated application on their device.

2. The device sends the contents of this email to the server.

3. The server receives the email content and inputs it as a prompt into the generative artificial intelligence model.

4. The generative artificial intelligence model analyzes the contents of the email and returns the result "Possible fraud."

5. The server sends the analysis results to the device.

6. The device displays the analysis results to the user, and the user confirms the warning message.

Example prompt

Below is an example of a prompt to input to a generative artificial intelligence model:

"Please determine if the following message is a potential scam: 'Mom, please transfer the money now.'"

"Determine whether the following email is a potential scam: 'Your bank account has been compromised. Contact us immediately.'"

This is how the specialized fraud detection system using a generative artificial intelligence model operates. The flow of the identification process in Example 3 will be explained using Figure 15.

Step 1:

The user enters text data.

The user enters the contents of the message or email they receive into a dedicated application on their device. For example, they might copy and paste the message "Mom, please transfer the money now" into the application. The input data is saved in text format on the device.

Step 2:

The device sends the text data to the server.

The terminal sends the text data entered by the user to the server. Specifically, the terminal application generates an HTTP request and sends a payload containing the text data to the server. At this time, the data is encrypted before being sent. The input is the text data entered by the user, and the output is the text data sent to the server.

Step 3:

The server receives the text data and inputs it into the generative artificial intelligence model.

The server receives text data sent from the terminal. The received data is first stored in a database. The server then inputs the text data as a prompt sentence into the generative AI model. An example of a prompt sentence is "Please determine whether the following message is likely to be fraudulent: 'Mom, please transfer the money now.'" The input is the text data received from the terminal, and the output is the prompt sentence input into the generative AI model.

Step 4:

A generative artificial intelligence model analyzes the text data and returns the results.

The generative artificial intelligence model analyzes the input prompt text and determines whether it is likely to be fraud. Specifically, the model analyzes it by comparing it with typical phrases and patterns of "bank transfer fraud" and "it's my son" fraud. If the analysis result indicates a high possibility of fraud, it returns a warning message stating "Possible fraud." The input is the prompt text, and the output is the analysis result.

Step 5:

The server sends the analysis results to the device.

The server sends the analysis results obtained from the generative AI model to the terminal. Specifically, the server generates an HTTP response and sends a payload containing the analysis results to the terminal. At this time, the data is encrypted before being sent. The input is the analysis results from the generative AI model, and the output is the analysis results sent to the terminal.

Step 6:

The device displays the analysis results to the user.

The device displays the analysis results received from the server to the user. For example, if the analysis result is a warning message stating "Possible fraud," the device application displays this message to the user as a pop-up window or notification. The input is the analysis result received from the server, and the output is the warning message displayed to the user. The user can check this warning message and take appropriate action.

(Application Example 3)

Next, we will explain Application Example 3 of Form Example 3. In the following explanation, the data processing device 12 will be referred to as the "server" and the smart device 14 will be referred to as the "terminal."

In recent years, special fraud methods have become more sophisticated, with many people falling victim to them. In particular, "bank transfer fraud" and "it's me" frauds targeting the elderly have become a social problem. In order to prevent these frauds, a system is needed that can detect possible fraud early and issue a warning to users. However, current systems have low fraud detection accuracy, making it difficult for users to recognize the possibility of fraud.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 3 is realized by the following means.

In this invention, the server includes means for detecting incoming fraudulent calls, means for listening to conversations from a voice input device installed in the communication terminal, means for converting the conversation data into text and sending it to the generative AI, means for issuing a voice alert from the voice input device if the generative AI determines that fraud is suspected from the conversation data, means for the generative AI to detect specific phrases or patterns that indicate the possibility of fraud, and means for issuing a warning to the user if the generative AI detects specific phrases or patterns that indicate the possibility of fraud. This makes it possible to detect the possibility of fraud with high accuracy and issue a warning to the user quickly.

"Means for detecting incoming fraudulent calls" refers to a function that identifies and notifies a communication terminal of incoming calls that may be fraudulent.

"A voice input device installed on a communication terminal" is a device for collecting voice that is attached to communication devices such as smartphones and home telephones.

"Means for converting conversation data into text and sending it to generative AI" refers to a function for converting voice data collected by a voice input device into text data and sending that text data to generative AI.

"Means for issuing a voice alert from the voice input device when the generative AI determines that fraud is suspected based on the conversation data" refers to a function that issues a warning sound through the voice input device when the generative AI determines that there is a possibility of fraud based on the conversation data analyzed.

"Means for the generative AI to detect specific phrases or patterns that indicate the possibility of fraud" refers to a function that enables the generative AI to detect typical fraudulent phrases and patterns that it has previously learned from conversation data.

"Means for issuing a warning to the user when the generative AI detects a specific phrase or pattern that indicates the possibility of fraud" refers to a function that issues a warning to the user when the generative AI detects a phrase or pattern that indicates the possibility of fraud.

As an embodiment of this invention, we will explain how to install a fraud detection assistant system on a smartphone.

First, the system is equipped with a means for detecting incoming fraudulent calls. This means has the function of identifying and notifying the communication terminal of incoming calls that may be fraudulent. Specifically, the content of the call is collected in real time using a voice input device installed on the communication terminal.

The collected voice data is then converted into text data through a means of transcribing the conversation data and sending it to the generative AI. This conversion is done using voice recognition software (e.g., Google's voice recognition API). The converted text data is then sent to the generative AI model.

Generative AI models are pre-trained to detect specific phrases and patterns that indicate potential fraud. These models are built using, for example, the Hugging Face transformers library. If the generative AI determines from the conversation data that fraud is suspected, it will warn the user by issuing an audio alert via a voice input device.

Furthermore, if the generative AI detects certain phrases or patterns that indicate potential fraud, a user alert mechanism will be activated. This mechanism will alert the user to the potential fraud by providing a visual or audio warning.

As a concrete example, consider the following audio file:

Voice: "Mom, please transfer the money right now. It's urgent."

Example prompt for parsing this audio file:

Judge whether the phrase "Mom, please transfer the money now. It's urgent." could be a scam.

By feeding this prompt into a generative AI model, it can analyze the possibility of fraud and issue a warning to the user.

The flow of the specific processing in Application Example 3 will be explained using Figure 16.

Step 1:

Detect incoming fraudulent calls

The server identifies and notifies the communication terminal of incoming calls that may be fraudulent. Specifically, it monitors the communication terminal's call log and detects possible fraud based on specific numbers or patterns. The input is call log data, and the output is a notification of a potentially fraudulent call.

Step 2:

Listen to conversations from a voice input device

The terminal collects call content in real time using an audio input device installed on the communication terminal. Specifically, audio data is acquired through a microphone and saved as an audio file. The input is the audio during the call, and the output is an audio file.

Step 3:

Conversation data is transcribed and sent to generative AI.

The device converts the collected voice data into text data and sends it to the generative AI. Specifically, it uses voice recognition software (e.g., Google's voice recognition API) to transcribe the voice data. The input is an audio file, and the output is text data.

Step 4:

Generative AI analyzes conversation data

The server uses a generative AI model to detect specific phrases and patterns in text data that indicate potential fraud. Specifically, it analyzes the text data using a generative AI model (e.g., Hugging Face's transformers library). The input is the text data, and the output is a determination of the likelihood of fraud.

Step 5:

Audible alerts when fraud is suspected

If the generative AI determines that there is a possibility of fraud, the device will issue an audio alert from the voice input device. Specifically, it will play a warning sound or message. The input is the result of the determination of the possibility of fraud, and the output is an audio alert.

Step 6:

Warn the user

If the generative AI detects certain phrases or patterns that indicate potential fraud, the device will alert the user by displaying a warning message on the screen or sending a notification. The input is the result of the determination of potential fraud, and the output is a warning to the user.

Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

"Example 1"

One embodiment of the present invention is a system that combines a generative AI with an emotion engine. This system analyzes emotions from the user's voice tone, volume, speaking rate, etc. Specifically, if the user's voice tone or volume suddenly increases while they are receiving a scam call, the system senses that the user may be panicking. Based on this information, the generative AI determines that the possibility of a scam has increased further and issues an audio alert.

"Example 2"

In another embodiment, the system could issue an audio alert if the emotion engine exceeds a certain threshold. For example, if the user's tone of voice exceeds a certain threshold, or if the user's speaking speed exceeds a certain rate, the emotion engine could determine that the user is experiencing strong emotion. Based on this determination, the generative AI could issue an audio alert, warning the user of possible fraud.

"Example 3"

Furthermore, as another embodiment, a system in which the emotion engine and generative AI work in conjunction with each other is also possible. In this system, the emotion engine analyzes the user's emotions and sends the results to the generative AI. The generative AI combines information from the emotion engine with conversation data it has analyzed to determine the possibility of fraud. In this way, more accurate fraud detection is possible.

The processing flow for each example form is explained below.

"Example 1"

Step 1: The user receives a scam call.

Step 2: A smartphone app or a microphone installed on a home phone listens to the conversation.

Step 3: Convert the conversation data into text and send it to the generative AI.

Step 4: Generative AI analyzes the text data and determines the likelihood of fraud.

Step 5: The emotion engine analyzes the user's emotions based on their tone of voice, volume, speaking rate, etc.

Step 6: Based on the information from the emotion engine, the generative AI determines that the likelihood of fraud has increased and issues an audio alert.

"Example 2"

Step 1: The user receives a scam call.

Step 2: A smartphone app or a microphone installed on a home phone listens to the conversation.

Step 3: Convert the conversation data into text and send it to the generative AI.

Step 4: Generative AI analyzes the text data and determines the likelihood of fraud.

Step 5: The emotion engine analyzes the user's emotions based on their tone of voice, volume, speaking rate, etc.

Step 6: If the emotion engine exceeds a certain threshold, it notifies the generative AI.

Step 7: Based on the notification from the emotion engine, the generative AI determines that the likelihood of fraud has increased and issues an audio alert.

"Example 3"

Step 1: The user receives a scam call.

Step 2: A smartphone app or a microphone installed on a home phone listens to the conversation.

Step 3: Convert the conversation data into text and send it to the generative AI.

Step 4: Generative AI analyzes the text data and determines the likelihood of fraud.

Step 5: The emotion engine analyzes the user's emotions based on their tone of voice, volume, speaking rate, etc., and sends the results to the generative AI.

Step 6: The generative AI combines information from the emotion engine with the conversation data it has analyzed to determine the likelihood of fraud.

Step 7: If it determines there is a high possibility of fraud, the generative AI will issue an audio alert.

(Example 1)

Next, we will explain Example 1 of Form Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the smart device 14 will be referred to as the "terminal."

Conventional fraud call detection systems analyze only the content of conversations to determine the possibility of fraud, which means they are unable to take the user's emotional state into account, resulting in low fraud detection accuracy. Furthermore, they are unable to respond adequately when the user panics, making it difficult to prevent fraud from occurring.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

In this invention, the server includes means for detecting incoming fraudulent calls, means for listening to conversations from a voice input device installed in the communication terminal, means for converting the conversation data into text and transmitting it to the generative artificial intelligence, means for issuing a voice alert from the voice input device if the generative artificial intelligence determines that fraud is suspected from the conversation data, means for collecting parameters such as the user's voice tone, volume, and speaking rate and analyzing their emotions, and means for reassessing the possibility of fraud based on the results of the emotion analysis. This makes it possible to detect fraud while taking the user's emotional state into account, thereby preventing fraud damage before it occurs.

"Means for detecting incoming fraudulent calls" refers to equipment or software that detects whether an incoming call to a communications terminal is likely to be fraudulent.

"A voice input device installed on a communication terminal" refers to a device for inputting voice, such as a microphone attached to a communication device such as a smartphone or home telephone.

"Means for converting conversation data into text and sending it to generative AI" refers to a device or software that converts voice data into text data and sends that text data to generative AI.

"Generative AI" is an artificial intelligence system that analyzes received data and determines the likelihood of fraud by detecting specific patterns and phrases.

"A means for issuing an audio alert" is a device or software that plays an audio message to warn the user if it determines that there is a high possibility of fraud.

"Means for collecting parameters such as the user's voice tone, volume, and speaking rate, and analyzing emotions" refers to a device or software that collects information such as tone, volume, and speaking rate from the user's voice data, and analyzes the user's emotional state based on that information.

"Means for reassessing the possibility of fraud based on the results of sentiment analysis" refers to a device or software that inputs the results of sentiment analysis into generative artificial intelligence and reassess the possibility of fraud.

Form for implementing the invention

This invention is a system that detects fraudulent phone calls and warns users, and is implemented using an application and server installed on a communications terminal. This system detects incoming fraudulent calls, listens to the conversation, transcribes the conversation data, and sends it to a generative artificial intelligence. Furthermore, it aims to prevent fraud by analyzing the user's emotional state and reassessing the possibility of fraud.

Hardware and software used

1. Communication terminal

Communication devices such as smartphones and home phones.

A microphone as an audio input device.

2. Server

Speech recognition software (e.g., Google Cloud Speech-to-Text API) is used to transcribe audio data.

Sentiment analysis uses an emotion engine (e.g., IBM Watson (registered trademark) Tone Analyzer).

For generative artificial intelligence, we use AI models with natural language processing technology (e.g., OpenAI's GPT-4).

Specific system operation

1. Incoming phone call detection

The server detects incoming phone calls through an application installed on the communication device. The application uses the communication device's native API to catch incoming phone call events.

2. Listening to Conversations

The device uses a microphone to listen to phone conversations. The application captures the microphone's audio input in real time and saves it as audio data.

3. Transcription of audio data

The server converts the audio data it hears into text data. This process involves calling the Google Cloud Speech-to-Text API to convert the audio data into text.

4. Sending to generative AI

The server sends the transcribed conversation data to the generative AI. The server sends the text data to the generative AI using an HTTP request.

5. Fraud Detection

The generative AI analyzes the received conversation data and determines whether fraud is suspected. Using natural language processing technology, the generative AI analyzes the content of the conversation and evaluates whether it matches any fraudulent patterns.

6. Sentiment Analysis

The server collects parameters such as the user's voice tone, volume, and speaking rate and analyzes them using an emotion engine. The server uses voice analysis software to extract emotion parameters from the voice data.

7. Panic Detection

If the user's tone or volume of voice suddenly increases, the server determines that the user may be panicking. The server analyzes data from the emotion engine and evaluates whether there is a sudden change.

8. Reassessing the possibility of fraud

Based on information from the emotion engine, the generative AI determines that the likelihood of fraud has increased. The generative AI receives the emotion data as additional input and reassess the likelihood of fraud.

9. Audio alerts

If the server determines that there is a high possibility of fraud, it instructs the communication device to play an audio alert from its speaker. The server then sends a command to the communication device's application to play the audio alert.

Examples and prompts

As a concrete example, imagine a user is receiving a phone call on their smartphone. If the person on the other end of the line says, "Your bank account has been fraudulently used. Please tell us your account number and password immediately," the user's voice tone will suddenly rise and the volume will also increase. Based on this information, the generative artificial intelligence will determine that there is a high possibility of fraud, and will emit an audio alert from the communication device's speaker saying, "This may be a fraud. Please be careful."

Example prompt:

"Analyze the conversation data below to determine if it may be a scam. Also consider changes in the user's tone and volume of voice.

Conversation data: 'Your bank account has been compromised. Please provide your account number and password immediately.'

User's voice tone: Sudden rise

User volume: Increased"

The flow of the identification process in Example 1 will be explained using Figure 17.

Step 1:

Incoming phone call detection

The server detects incoming phone calls through an application installed on the communication terminal.

Input: Incoming call event from communication terminal.

Data processing/calculation: Capture incoming call events using the communication device's native API.

Output: Event information indicating that an incoming call was detected.

Specific operation: The application uses ANDROID (registered trademark) or iOS native APIs to catch incoming phone calls and notify the server.

Step 2:

Listening to conversations

The device uses a microphone to listen to the conversation during the call.

Input: Audio data during the call.

Data processing/calculation: Captures microphone audio input in real time and saves it as audio data.

Output: Audio data.

Specific behavior: The application captures audio input from the microphone in real time and saves it as audio data.

Step 3:

Transcription of audio data

The server converts the audio data it hears into text data.

Input: Audio data.

Data processing/calculation: Call the Google Cloud Speech-to-Text API to convert voice data into text.

Output: Character data.

Specific operation: The server converts the audio data into text using the Google Cloud Speech-to-Text API.

Step 4:

Send to generative AI

The server sends the transcribed conversation data to the generative artificial intelligence.

Input: Character data.

Data processing/calculation: Send text data to generative artificial intelligence using an HTTP request.

Output: Result of request sent to generative artificial intelligence.

Specific operation: The server sends text data to the generative artificial intelligence using an HTTP request.

Step 5:

Fraud detection

The generative artificial intelligence analyzes the received conversation data and determines whether fraud is suspected.

Input: Character data.

Data processing/calculation: Using natural language processing technology, the content of the conversation is analyzed and evaluated to see if it matches a fraud pattern.

Output: Assessment result regarding likelihood of fraud.

Specific operation: The generative artificial intelligence uses natural language processing technology to analyze the content of the conversation and evaluate whether it matches any fraud patterns.

Step 6:

Emotion analysis

The server collects parameters such as the user's voice tone, volume, and speaking speed, and analyzes them using an emotion engine.

Input: Audio data.

Data processing/calculation: Use voice analysis software to extract emotional parameters from voice data.

Output: Emotion parameters.

Specific operation: The server uses voice analysis software (e.g., IBM Watson Tone Analyzer) to extract emotion parameters from the voice data.

Step 7:

Detecting panic states

If the user's voice tone or volume suddenly increases, the server will determine that the user may be panicking.

Input: Emotion parameters.

Data processing/calculation: Analyze data from the emotion engine and evaluate whether there are any sudden changes.

Output: Panic state assessment results.

Specific behavior: The server analyzes data from the emotion engine and evaluates whether there are any sudden changes.

Step 8:

Reassessing the possibility of fraud

Based on information from the emotion engine, the generative artificial intelligence determines that the likelihood of fraud has increased.

Input: Emotion parameters.

Data processing/calculation: Emotional data is taken as additional input and the likelihood of fraud is reassessed.

Output: Reassessed fraud probability.

Specific behavior: The generative AI receives emotional data as additional input and reassesss the likelihood of fraud.

Step 9:

Send audio alerts

If the server determines that there is a high possibility of fraud, it will instruct the communication device to emit an audio alert from its speaker.

Input: Reassessed potential fraud.

Data processing/calculation: Sends a command to the communication terminal application to play an audio alert.

Output: Plays an audio alert.

Specific operation: The server sends a command to the application on the communication device to play an audio alert, which is then played from the communication device's speaker.

(Application Example 1)

Next, we will explain Application Example 1 of Form Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the smart device 14 will be referred to as the "terminal."

In recent years, fraudulent phone scams have become more sophisticated, with many people falling victim to them. Elderly people and those unfamiliar with technology are particularly prone to falling victim to these scams, as they are less wary of them. Furthermore, when users receive a fraudulent call, they often panic, making it difficult for them to make a calm judgment. There is a need for effective measures to improve this situation and prevent fraudulent incidents.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

In this invention, the server includes means for detecting incoming fraudulent calls, means for listening to conversations from a voice input device installed in the communication terminal, means for converting the conversation data into text and sending it to the generative AI, means for issuing a voice alert from the voice input device if the generative AI determines that fraud is suspected from the conversation data, means including an emotion engine for analyzing emotions from the user's voice tone, volume, speaking speed, etc., and means for further increasing the possibility of fraud based on the analysis results of the emotion engine. In this way, by combining fraudulent call detection with user emotion analysis, it is possible to prevent fraud victims before they occur.

A "fraudulent call" is a call made with the intent of defrauding someone of money or personal information through fraudulent means.

"Incoming call" refers to an incoming phone call.

A "communications terminal" is an electronic device used to send and receive voice and data.

An "audio input device" is a device for converting audio into digital data.

"Conversation data" refers to data that has been converted into text from speech captured by a voice input device.

"Generative AI" is artificial intelligence that makes specific decisions and generates results based on input data.

"Audio alert" is a means of warning or notifying someone by voice.

An "emotion engine" is software or a system for analyzing emotions from voice data.

"Tone" refers to the pitch and texture of the voice.

"Volume" refers to the loudness of the sound.

"Speech rate" refers to the speed at which you speak.

The following system configuration is used to implement this invention.

System Configuration

1. Hardware

Communication terminal: A device capable of voice communication, such as a smartphone or home telephone.

Audio input device: A microphone built into the communication terminal.

Audio output device: Speaker built into the communication terminal.

2. Software

Speech Recognition Library: Use the speech_recognition library to convert speech to text.

Generative AI model: An AI model that uses the transformers library to determine the likelihood of fraud.

Emotion Engine: A model for analyzing user emotions using the transformers library.

Audio Alert Library: Uses the pyttsx3 library to generate audio alerts.

Processing flow

1. Incoming call detection

The server detects that a call has been received by the communication terminal.

The communication terminal's voice input device listens to the conversation and acquires voice data.

2. Voice Recognition

The server uses a voice recognition library to convert the acquired voice data into text data.

3. Fraud Detection Using Generative AI

The server inputs the text data into a generative AI model to determine the likelihood of fraud.

If certain phrases or patterns are detected, it is determined that there is a high possibility of fraud.

4. Sentiment analysis

The server uses an emotion engine to analyze the user's emotions based on their tone of voice, volume, speaking speed, etc.

If a user is perceived to be in a panic, the likelihood of fraud increases even more.

5. Audio alerts

If the server determines that there is a high possibility of fraud, it uses the audio alert library to issue an audio alert from the communication terminal's audio output device.

The alert is intended to alert users to potential fraud.

Specific examples

For example, if a user receives a phone call and it is determined that the call may be fraudulent, an audio alert will sound from the communication device's speaker saying, "Warning! Possible fraud." This alert will make the user aware of the possibility of fraud and allow them to respond calmly.

Example prompt

"Could this call be a scam?"

In this way, this invention combines the detection of fraudulent calls with user sentiment analysis, making it possible to prevent fraudulent incidents.

The flow of the specific processing in Application Example 1 is explained using Figure 18.

Step 1:

The server detects that a call has been received by the communication terminal.

Input: Incoming signal from communication terminal.

Data processing: Receives an incoming call signal and begins processing when the call starts.

Output: Call start flag.

Specific operation: When a communication terminal receives an incoming call, the server detects the signal and recognizes that a call has started.

Step 2:

The device's voice input device listens to the conversation and captures the voice data.

Input: Audio during a call.

Data processing: The voice input device collects voice data in real time.

Output: Captured audio data.

Specific operation: Audio during a call is collected through the microphone and saved as audio data.

Step 3:

The server uses a voice recognition library to convert the acquired voice data into text data.

Input: Acquired audio data.

Data processing: Use the speech recognition library (speech_recognition) to convert audio data into text data.

Output: Converted text data.

Specific operation: Voice data is input into the voice recognition library and output as text data.

Step 4:

The server inputs the text data into a generative AI model to determine the likelihood of fraud.

Input: Converted text data.

Data processing: Text data is fed into generative AI models (transformers) to assess the likelihood of fraud.

Output: Assessment results regarding the likelihood of fraud.

Specific operation: Text data is input into a generative AI model, which outputs an evaluation result indicating whether the likelihood of fraud is high.

Step 5:

The server uses an emotion engine to analyze the user's emotions based on their tone of voice, volume, speaking speed, etc.

Input: Converted text and audio data.

Data processing: Analyze user emotions using emotion engines (transformers).

Output: Analysis results on user sentiment.

Specific operation: Voice and text data are input into the emotion engine, and the user's emotional state is analyzed.

Step 6:

If the server determines that there is a high possibility of fraud, it uses the audio alert library to issue an audio alert from the communication terminal's audio output device.

Input: Fraud likelihood assessment results and user sentiment analysis results.

Data processing: If a high possibility of fraud is determined, an audio alert is generated using the audio alert library (pyttsx3).

Output: Audio alert.

Specific operation: If it is determined that there is a high possibility of fraud, an audio alert will be emitted from the communication device's speaker saying, "Warning! Possible fraud."

(Example 2)

Next, we will explain Example 2 of Form Example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the smart device 14 will be referred to as the "terminal."

Conventional fraud prevention systems had low accuracy in detecting potential fraud, making it impossible to completely eliminate the risk of users falling victim to fraud. Furthermore, warnings were issued without taking the user's emotional state into consideration, which sometimes led to users being unable to respond appropriately. This made it difficult to prevent fraud from occurring.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

In this invention, the server includes means for detecting incoming fraudulent calls, means for listening to conversations from a voice input device installed in the communication device, means for converting the conversation data into text and sending it to the generative AI model, means for issuing an audio alert from the voice input device if the generative AI model determines that fraud is suspected from the conversation data, means for an emotion analysis engine to analyze the user's tone of voice and speaking rate, and means for the generative AI model to issue an audio alert if the emotion analysis engine exceeds a specific threshold. This makes it possible to detect the possibility of fraud with high accuracy and issue an appropriate warning according to the user's emotional state.

"Means for detecting incoming fraudulent calls" refers to a function that identifies and notifies a communication device of incoming calls that may be fraudulent.

A "voice input device" is a device that records conversations between a user and another person in real time and stores them in digital format.

"Means for converting conversation data into text and sending it to the generative AI model" refers to a function for converting voice data into text data and sending that text data to the generative AI model.

A "generative AI model" is an artificial intelligence system that analyzes input data, detects specific patterns and phrases, and determines the likelihood of fraud.

"Means for issuing audio alerts" refers to an audio notification function that warns users if a potential fraud is detected.

The "emotion analysis engine" is an engine that analyzes the user's tone of voice and speaking speed to determine their emotional state.

"When a specific threshold is exceeded" refers to when the emotion analysis engine analyzes the user's emotional state and it exceeds a pre-set reference value.

"Communication equipment" refers to devices used for making voice calls, such as home telephones and smartphones.

Form for implementing the invention

This invention is a system that detects incoming fraudulent calls and issues a warning to the user. A specific embodiment of this system is described below.

Hardware and software used

Communication devices: devices used for voice calls, such as home phones and smartphones.

Audio input device: A microphone installed in the communication device

Server: A computer system for processing voice data and running generative AI models and sentiment analysis engines.

Generative AI model: An artificial intelligence system that analyzes conversation data and determines the likelihood of fraud.

Emotion analysis engine: An engine that analyzes the user's tone of voice and speaking rate to determine their emotional state.

Speech recognition software: Software for converting voice data into text data (e.g., Google Speech-to-Text API)

System Operation

1. When a user starts a conversation on a communication device, the device's voice input device records the conversation in real time. The recorded voice data is saved in digital format.

2. The server uses speech recognition software to convert the collected voice data into text. For example, the speech "Hello, this is a bank representative" is converted into text "Hello, this is a bank representative."

3. The server sends the converted text data to the generative AI model. The generative AI model receives this data and prepares it for analysis.

4. The generative AI model analyzes the conversation data and detects specific keywords and phrases (e.g., "bank account" or "password"), which determines whether there is potential for fraud.

5. If the generative AI model determines that there is a high possibility of fraud, the server will send instructions to the terminal and an audio alert will be issued from the communication device's speaker saying, "This is likely fraud. Please be careful."

6. The emotion analysis engine analyzes the user's tone of voice and speaking rate in real time. For example, if the user's voice suddenly gets higher in pitch or their speaking rate increases, the emotion analysis engine will detect this.

7. If the sentiment analysis engine determines that the user's emotions exceed a certain threshold, the generative AI model will issue an audio alert saying, "This may be a scam. Please remain calm."

Specific examples

Example 1: When a user says over the phone, "Please tell me your bank account information," the generative AI model detects the keyword "bank account" and determines that there is a possibility of fraud. The speaker on the communication device issues a voice alert saying, "There is a possibility of fraud. Please be careful."

Example 2: If a user says "Please tell me my password urgently" over the phone and their voice tone gets higher and faster, the emotion analysis engine determines that the user is experiencing strong emotions. The generative AI model issues a voice alert, warning, "This may be a scam. Please remain calm."

Example prompt

Prompt 1: "If someone calls and asks for your bank account information, determine whether it's a scam."

Prompt 2: "Please explain how the emotion engine determines if the user's voice tone gets higher and their speaking rate gets faster."

The system is designed to reduce the risk of users falling victim to telephone fraud. By combining a generative AI model with a sentiment analysis engine, it can detect potential fraud with greater accuracy and alert users.

The flow of the identification process in Example 2 will be explained using Figure 19.

Step 1:

The user begins a conversation on a communication device.

Specific operations involve a user initiating a call using a home phone or smartphone. The input is the user's voice, and the output is voice data collected by a voice input device.

Step 2:

The device's voice input device listens to the conversation and collects voice data.

Specifically, the device's microphone records the conversation between the user and the other party in real time. The input is the user's and the other party's voice, and the output is audio data stored in digital format.

Step 3:

The server converts the audio data into text data.

Specifically, the server uses speech recognition software (e.g., Google Speech-to-Text API) to convert the collected voice data into text data. The input is digital voice data, and the output is text conversation data.

Step 4:

The server sends the transcribed conversation data to the generative AI model.

Specific operations involve the server sending the converted text data to the generative AI model. The input is text-format conversation data, and the output is the data sent to the generative AI model.

Step 5:

A generative AI model analyzes conversation data to determine the likelihood of fraud.

Specifically, the generative AI model analyzes conversation data and detects specific keywords and phrases (e.g., "bank account" or "password"). The input is text-based conversation data, and the output is a judgment about the likelihood of fraud.

Step 6:

If there is a possibility of fraud, an audio alert will be issued via the device's voice input device.

Specifically, if the generative AI model determines that there is a high possibility of fraud, the server sends instructions to the terminal and an audio alert is emitted from the communication device's speaker saying, "There is a possibility of fraud. Please be careful." The input is the judgment result of the generative AI model, and the output is an audio alert.

Step 7:

The emotion analysis engine analyzes the user's tone of voice and speaking speed.

Specifically, the emotion analysis engine analyzes the user's tone of voice and speaking rate in real time. The input is the user's voice data, and the output is the analysis results regarding their emotional state.

Step 8:

If the sentiment analysis engine exceeds a certain threshold, the generative AI model will issue an audio alert.

Specifically, if the emotion analysis engine determines that the user's emotion exceeds a certain threshold, the generative AI model issues an audio alert saying, "This may be a scam. Please remain calm." The input is the analysis result of the emotion analysis engine, and the output is an audio alert.

(Application Example 2)

Next, we will explain Application Example 2 of Form Example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the smart device 14 will be referred to as the "terminal."

In recent years, fraudulent phone scams have become increasingly sophisticated, leaving many people vulnerable. Elderly people and those unfamiliar with technology are particularly vulnerable to these scams, as they are less vigilant against them. While systems for detecting fraudulent calls exist, few warning systems take the user's emotional state into account. This increases the risk of users becoming victims of fraudulent calls without realizing they are being scammed. Therefore, there is a need for a system that detects fraudulent calls and takes the user's emotional state into account.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

In this invention, the server includes means for detecting incoming fraudulent calls, means for listening to conversations from a voice input device installed in the smart device, means for converting the conversation data into text and sending it to the generative AI, means for issuing a voice alert from the voice input device if the generative AI determines that fraud is suspected from the conversation data, and means for analyzing the user's emotions using an emotion engine and issuing a warning if a specific threshold is exceeded. This makes it possible to detect fraudulent calls and issue warnings that take into account the user's emotional state.

A "fraudulent call" is a call made with the intent of defrauding someone of money or personal information through fraudulent means.

"Incoming call" refers to an incoming phone call.

A "smart device" is an electronic device with internet connectivity, such as a smartphone or tablet.

An "audio input device" is a device such as a microphone that converts audio into digital data.

"Conversation data" refers to data that has been converted into text from speech acquired through a voice input device.

"Generative AI" is a system that uses artificial intelligence to analyze data and make specific decisions or generate results.

"Audio alert" is a means of warning or notifying the user using audio.

An "emotion engine" is an algorithm or system that analyzes a user's emotional state from voice and text data.

A "specific threshold" is a numerical value or condition that serves as a benchmark when the emotion engine evaluates the user's emotional state.

A "warning" is a notification or alert that alerts the user.

A system for implementing this invention includes a series of means for detecting incoming fraudulent calls and issuing a warning to the user. A specific embodiment of the system is described below.

First, a voice input device (microphone) is installed on a smart device (e.g., a smartphone or tablet). This voice input device captures the user's call content in real time. The captured voice data is converted into text data using voice recognition software (e.g., the speech_recognition library).

The transcribed conversation data is then sent to a generative AI model (e.g., a model using the transformers library). This generative AI model analyzes the conversation data and determines whether there is a possibility of fraud. If a fraudulent activity is detected, software for issuing audio alerts (e.g., the pyttsx3 library) is activated to warn the user.

Furthermore, an emotion engine (e.g., an emotion analysis model using the transformers library) is used to analyze the user's emotional state. The emotion engine evaluates the user's tone of voice, speaking rate, etc., and issues a warning if certain thresholds are exceeded. This warning is also notified to the user as an audio alert.

For example, if a user calls and says, "Please tell me your bank account information," the generative AI model will detect the possibility of fraud and issue a voice alert. Also, if the user is feeling angry or anxious, the emotion engine will detect this and issue a voice alert.

Examples of prompt statements include:

User's call: "Please tell me my bank account information."

Prompt for generative AI model: "Is this call potentially fraudulent?"

In this way, it is possible to create a system that can detect fraudulent calls and issue warnings that take into account the user's emotional state.

The flow of the specific processing in Application Example 2 is explained using Figure 20.

Step 1:

A user initiates a call on a smart device. The smart device's audio input device (microphone) captures the call content in real time. The input is the user's voice data, and the output is a stream of voice data.

Step 2:

The device uses speech recognition software (speech_recognition library) to convert the acquired voice data into text data. The input is a stream of voice data, and the output is transcribed conversation data.

Step 3:

The device sends the transcribed conversation data to a generative AI model (a model using the transformers library). The generative AI model analyzes the conversation data and determines whether there is a possibility of fraud. The input is the transcribed conversation data, and the output is a judgment result regarding the possibility of fraud.

Step 4:

The server receives the judgment results of the generative AI model, and if it determines that there is a possibility of fraud, it launches software (pyttsx3 library) to issue an audio alert. The input is the judgment result regarding the possibility of fraud, and the output is an audio alert.

Step 5:

The device uses an emotion engine (an emotion analysis model using the Transformers library) to analyze the user's emotional state. The emotion engine evaluates the user's tone of voice, speaking rate, etc., and issues a warning if certain thresholds are exceeded. The input is transcribed conversation data, and the output is an evaluation result regarding the user's emotional state.

Step 6:

The server receives the emotion engine's evaluation results, and if a certain threshold is exceeded, it launches software (pyttsx3 library) to issue an audio alert. The input is the evaluation result regarding the emotional state, and the output is an audio alert.

Step 7:

The user receives an audio alert and acknowledges the warning about potential fraud or emotional state. The input is the audio alert and the output is the user's acknowledgment.

(Example 3)

Next, we will explain Example 3 of Form Example 3. In the following explanation, the data processing device 12 will be referred to as the "server" and the smart device 14 will be referred to as the "terminal."

Conventional fraud detection systems have the problem of low accuracy because they only analyze conversation data to determine the possibility of fraud. Furthermore, because they do not take the user's emotional state into account, there is a high risk of overlooking possible fraud. Furthermore, there are limited ways to notify users of possible fraud, which can lead to users not realizing the risk of fraud.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Example 3 is realized by the following means.

In this invention, the server includes means for detecting incoming fraudulent calls, means for listening to conversations from a voice input device installed in the communication terminal, means for converting the conversation data into text and transmitting it to the generative artificial intelligence, means for issuing a voice alert from the voice input device if the generative artificial intelligence determines that fraud is suspected from the conversation data, means for an emotion analysis engine to analyze the user's emotions, and means for the generative artificial intelligence to integrate information from the emotion analysis engine and the conversation data to determine the possibility of fraud. This makes it possible to detect the possibility of fraud with high accuracy and notify the user promptly.

"Means for detecting incoming fraudulent calls" refers to devices or software that monitor communication lines and identify incoming calls that may be fraudulent.

"A voice input device installed on a communication terminal" refers to a microphone attached to a communication device such as a smartphone or home telephone for collecting voice.

"Means for converting conversation data into text and sending it to generative AI" refers to software or a device for converting voice data into text data and sending that text data to generative AI.

"Generative AI" is an artificial intelligence model that uses natural language processing technology to analyze text data and detect specific patterns and phrases.

"Means for issuing audio alerts" refers to an audio output device that issues a warning to the user if the generative artificial intelligence detects the possibility of fraud.

An "emotion analysis engine" is software or a device that analyzes a user's voice and text data to identify their emotional state.

"Means for determining the likelihood of fraud" refers to an algorithm or software that uses generative artificial intelligence to integrate conversational data and information from a sentiment analysis engine to assess the likelihood of fraud.

This invention is a system that detects incoming fraudulent calls and notifies users of the possibility of fraud. A specific embodiment of this system is described below.

First, while a user is using a communication terminal (such as a smartphone or home phone), a means for detecting incoming fraudulent calls is activated. This means is a device or software that monitors communication lines and identifies incoming calls that may be fraudulent.

Next, a voice input device (microphone) installed on the communication terminal listens to the conversation between the user and the caller making the fraudulent call. This voice data is converted into text data by transcribing the conversation data and sending it to generative artificial intelligence. This conversion uses voice recognition technology. Specifically, voice recognition services such as Google Speech-to-Text API and IBM Watson Speech to Text can be used.

The server analyzes the text data using generative artificial intelligence (e.g., natural language processing models such as BERT or GPT-3 (registered trademark)). The generative artificial intelligence detects specific phrases and patterns that may be fraudulent and evaluates the results. For example, if the phrase "Please transfer money" is included, it will determine that there is a high possibility of fraud.

Furthermore, the server uses an emotion analysis engine (for example, IBM Watson or Microsoft® Azure® emotion analysis API) to analyze the user's emotions. It analyzes the user's voice and text data to identify emotional states such as anxiety or tension. The emotion data obtained by the emotion analysis engine is sent to the generative artificial intelligence.

Generative AI combines information from the sentiment analysis engine with conversational data to more accurately assess the likelihood of fraud. For example, if the message is "Please transfer money" and the user's emotions are anxious, it will conclude that there is a very high possibility of fraud.

Finally, if the server determines that the message is likely to be fraudulent, it will notify the user of this result. The notification is made by issuing an audio alert. Specifically, an audio alert will be issued from the communication device's speaker saying, "This message may be fraudulent. Please be careful."

As a concrete example, consider the case where a user receives a message saying, "Please transfer money." This message is picked up by the voice input device of the communication device and converted into text data. The server analyzes this text data using generative artificial intelligence and determines that there is a high possibility of fraud. At the same time, the emotion analysis engine detects the user's anxiety and sends this information to the generative artificial intelligence. The server combines this information and concludes that there is a very high possibility of fraud. Finally, the server sends an audio alert to the user's communication device saying, "This message may be fraudulent. Please be careful."

Example prompt:

"Please determine whether the following message is likely a scam. Message: 'Please transfer money.'"

In this way, the server, terminal, and user work together to detect the possibility of fraud with high accuracy and quickly notify the user. The flow of the identification process in Example 3 will be explained using Figure 21.

Step 1:

The user enters a message.

The user uses a communication device (such as a smartphone or home phone) to input the received message. For example, they might input a message such as "Please transfer money." The input message is saved on the device as voice data.

Step 2:

The device sends a message to the server.

The terminal sends the voice data entered by the user to the server. At this time, the terminal sends the voice data to the server via the Internet. The input is voice data, and the output is the transfer of voice data to the server.

Step 3:

The server converts the audio data into text data.

The server converts the received voice data into text data using voice recognition technology. Specifically, it uses voice recognition services such as Google Speech-to-Text API and IBM Watson Speech to Text. The input is voice data, and the output is text data.

Step 4:

The server analyzes the text data using generative artificial intelligence.

The server analyzes the text data using generative artificial intelligence (e.g., natural language processing models such as BERT or GPT-3). The generative artificial intelligence detects specific phrases and patterns that may be fraudulent and evaluates the results. The input is the text data, and the output is an evaluation result regarding the likelihood of fraud.

Step 5:

The server analyzes the user's emotions using an emotion analysis engine.

The server uses an emotion analysis engine (for example, IBM Watson or Microsoft Azure's emotion analysis API) to analyze the user's emotions. It analyzes the user's voice and text data to identify emotional states such as anxiety or tension. The input is voice or text data, and the output is emotion data.

Step 6:

The server combines the results of the generative artificial intelligence and the sentiment analysis engine to determine the likelihood of fraud.

The server integrates the analysis results from the generative AI with the emotional data from the emotion analysis engine. This allows for a more accurate assessment of the likelihood of fraud. For example, if the message is "Please transfer money" and the user's emotions are anxious, it will conclude that there is a very high likelihood of fraud. The input is the analysis results and emotional data, and the output is a final assessment of the likelihood of fraud.

Step 7:

The server will notify the user of possible fraud.

If the server determines that there is a high possibility of fraud, it notifies the user of the result. The notification is done by issuing an audio alert. Specifically, an audio alert saying "This message may be fraudulent. Please be careful" is issued from the speaker of the communication terminal. The input is the final assessment of the possibility of fraud, and the output is the audio alert.

(Application Example 3)

Next, we will explain Application Example 3 of Form Example 3. In the following explanation, the data processing device 12 will be referred to as the "server" and the smart device 14 will be referred to as the "terminal."

Traditional fraud detection systems only detect specific phrases and patterns and do not take the user's emotional state into account, making it difficult to accurately determine the likelihood of fraud. Furthermore, there was a lack of means to provide appropriate warnings to users when fraud was likely.

The identification processing by the identification processing unit 290 of the data processing device 12 in Application Example 3 is realized by the following means. In this invention, the server includes means for detecting incoming fraudulent calls, means for listening to conversations from a smartphone application or a microphone installed in a home phone, means for converting the conversation data into text and sending it to the generative AI, means for issuing an audio alert from the microphone if the generative AI determines that fraud is suspected based on the conversation data, means for analyzing the user's emotions using an emotion engine, means for determining the possibility of fraud by combining information from the emotion engine with the conversation data analyzed by the generative AI, and means for displaying a warning to the user if there is a high possibility of fraud. This makes it possible to more accurately determine the possibility of fraud and provide the user with an appropriate warning.

"Means for detecting incoming fraudulent calls" refers to devices or software that monitor telephone lines or communications networks and detect incoming calls that may be fraudulent.

"A microphone installed in a smartphone application or home phone" is a voice input device attached to a smartphone or home phone to collect audio.

"Means for listening to conversations" refers to devices or software that analyze audio data collected through a microphone and recognize the content of the conversation.

"Means for converting conversation data into text and sending it to the generative AI" refers to a device or software that converts voice data into text data and sends that text data to the generative AI.

"Generative AI" is an artificial intelligence model trained to perform a specific task, in this case to determine the likelihood of fraud.

"Means for issuing an audio alert from the microphone when it is determined that fraud is suspected" refers to a device or software that issues an alarm or message through the microphone when the generative AI detects the possibility of fraud.

An "emotion engine" is software or a device that analyzes a user's emotional state from their voice or text data.

"Means for analyzing user emotions" refers to devices or software that use an emotion engine to analyze the user's emotional state.

"Means for determining the possibility of fraud by combining information from the emotion engine with conversation data analyzed by generative AI" refers to devices or software that integrate emotional information obtained from the emotion engine with the results of analysis of conversation data by generative AI, and make a comprehensive determination of the possibility of fraud.

"Means for displaying a warning to the user in the event of a high likelihood of fraud" refers to devices or software that provide a visual or audible warning to the user in the event that a high likelihood of fraud is determined.

A system for implementing this invention includes a series of means for detecting incoming fraudulent calls and providing a warning to the user. Specific embodiments are described below.

First, the server is equipped with a means for detecting incoming fraudulent calls. This means is a device or software that monitors telephone lines and communication networks to detect potentially fraudulent calls. For example, specific phone numbers or callers can be registered on a blacklist, and fraudulent calls can be detected based on this.

Next, the device (smartphone application or home phone) is equipped with a microphone to pick up on the conversation. This microphone collects the voice during the call and sends it to the server as conversation data. The conversation data is then converted into text using voice recognition technology and sent to the generative AI.

Generative AI analyzes conversation data to determine the likelihood of fraud. In doing so, it uses models trained to detect specific phrases and patterns. For example, it can detect typical phrases such as "bank transfer fraud" and "it's me" fraud.

Furthermore, an emotion engine analyzes the user's emotions. The emotion engine is software that analyzes the user's emotional state from voice and text data. The emotional information obtained from the emotion engine is integrated with the results of analysis of conversation data by generative AI to make a comprehensive judgment on the possibility of fraud.

If it determines that there is a high possibility of fraud, the device will display a warning to the user. This warning can be provided visually or audibly. For example, a warning message can be displayed on the smartphone screen or an audio alert can be issued.

For example, consider the following user input:

"Your account has been frozen. Please transfer the funds immediately."

"Hey, I need money right now."

By inputting these prompts into a generative AI model, it can analyze the possibility of fraud and, in conjunction with an emotion engine, display a warning to the user.

The flow of the specific processing in Application Example 3 will be explained using Figure 22.

Step 1:

The server monitors telephone lines and communication networks to detect incoming calls that may be fraudulent. The input is a signal from the telephone line or communication network, and the output is a flag indicating an incoming fraudulent call. Specifically, it registers specific phone numbers and callers on a blacklist and detects fraudulent calls based on that.

Step 2:

The device (smartphone application or home phone) collects audio during a call through a microphone. The input is the audio signal during the call, and the output is audio data. Specifically, the microphone converts the audio into a digital signal and sends that data to a server.

Step 3:

The server uses voice recognition technology to convert the voice data into text. The input is voice data and the output is text data. Specifically, the voice recognition software analyzes the voice data and converts it into corresponding text.

Step 4:

The server sends text data to the generative AI, which analyzes it for the possibility of fraud. The input is text data, and the output is a score or flag indicating the possibility of fraud. Specifically, the generative AI model detects specific phrases and patterns and determines the possibility of fraud.

Step 5:

The server uses an emotion engine to analyze the user's emotions. The input is text data, and the output is data indicating the user's emotional state. Specifically, the emotion engine analyzes the text data and identifies the user's emotional state.

Step 6:

The server combines information from the emotion engine with conversation data analyzed by the generative AI to make a comprehensive judgment on the possibility of fraud. The input is emotional state data and a score or flag indicating the possibility of fraud, and the output is a final flag indicating the possibility of fraud. Specifically, it integrates emotional information with the analysis results of the generative AI to make a highly accurate judgment on the possibility of fraud.

Step 7:

If it determines that there is a high possibility of fraud, the device will display a warning to the user. The input is a flag indicating the final possibility of fraud, and the output is a warning message or audio alert. Specifically, a warning message is displayed on the smartphone screen or an audio alert is issued.

The specific processing unit 290 transmits the results of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the results of the specific processing. The microphone 38B acquires audio indicating the user input regarding the results of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

Data generation model 58 is what is known as generative AI (artificial intelligence). An example of data generation model 58 is generative AI such as ChatGPT (registered trademark) (Internet search <URL: https://openai.com/blog/chatgpt>). Data generation model 58 is obtained by performing deep learning on a neural network. A prompt containing an instruction is input to data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. Data generation model 58 performs inference on the input inference data in accordance with the instructions indicated by the prompt, and outputs the inference results in the form of data such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and/or summarization.

Another example of generative AI is Gemini (registered trademark) (Internet search <URL: https://gemini.google.com/?hl=ja>).

In the above embodiment, an example was given in which the specific processing was performed by the data processing device 12, but the technology disclosed herein is not limited to this, and the specific processing may also be performed by the smart device 14.

[Second embodiment]

Figure 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

As shown in FIG. 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

The data processing device 12 includes a computer 22, a database 24, and a communication I/F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and communication I/F 26 are also connected to the bus 34. The communication I/F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and/or a LAN (Local Area Network).

The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I/F 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the user 20's surroundings (e.g., an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

The communication I/F 44 is connected to the network 54. The communication I/Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I/Fs 44 and 26 is carried out in a secure manner.

Figure 4 shows an example of the main functions of the data processing device 12 and smart glasses 214. As shown in Figure 4, in the data processing device 12, specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32.

The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as the specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

Storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

Next, we will explain the identification process performed by the identification processing unit 290 of the data processing device 12.

"Example 1"

One embodiment of the present invention uses a smartphone application. In this case, the application installed on the smartphone detects an incoming phone call and listens to the conversation through the microphone. The listened-to conversation data is converted into text and sent to the generative AI. The generative AI determines from the conversation data that fraud is suspected, and if fraud is suspected, issues an audio alert from the smartphone's speaker.

"Example 2"

Another embodiment of the present invention involves installing a microphone on a home phone. In this case, the home phone's microphone listens to telephone conversations, and the listened conversation data is transcribed and sent to the generative AI. The generative AI determines from the conversation data that fraud is suspected, and if fraud is suspected, issues an audio alert from the home phone's speaker.

"Example 3"

A further embodiment of the present invention is one in which the generative AI detects specific phrases or patterns that indicate the possibility of specialized fraud. Specifically, the generative AI learns typical phrases for "bank transfer fraud" and typical patterns for "it's me" fraud, and by detecting these, determines the possibility of fraud.

The processing flow for each example form is explained below.

"Example 1"

Step 1: The application installed on the smartphone detects an incoming call.

Step 2: The application listens to your conversation through your smartphone's microphone.

Step 3: The conversation data heard is transcribed and sent to the generative AI.

Step 4: The generative AI determines that fraud is suspected based on the conversation data.

Step 5: If fraud is suspected, an audio alert will be issued through your smartphone's speaker.

"Example 2"

Step 1: A microphone installed on a home phone listens to the phone conversation.

Step 2: The conversation data that is heard is transcribed and sent to the generative AI.

Step 3: The generative AI determines that fraud is suspected based on the conversation data.

Step 4: If fraud is suspected, an audio alert will be issued over the home phone speaker.

"Example 3"

Step 1: The generative AI learns specific phrases or patterns that indicate potential fraud.

Step 2: The generative AI detects specific phrases or patterns it has learned to identify suspected fraud in the conversation data.

Step 3: If fraud is suspected, an audio alert will be issued.

(Example 1)

Next, Example 1 of Form Example 1 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

In recent years, telephone fraud has been on the rise, with special frauds targeting the elderly becoming a particular social problem. Conventional countermeasures require users to identify fraudulent calls themselves, but as fraud methods become more sophisticated, this becomes increasingly difficult. Therefore, there is a need for a system that automatically detects potential fraud and issues a warning when users receive a fraudulent call.

The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means. In this invention, the server includes means for detecting incoming fraudulent calls, means for listening to the conversation from a microphone installed in the communication terminal, means for converting the conversation data into text and sending it to the generative AI model, and means for issuing an audio alert from the speaker of the communication terminal if the generative AI model determines that fraud is suspected based on the conversation data. This makes it possible to automatically detect the possibility of fraud when a user receives a fraudulent call and immediately issue a warning.

"Means for detecting incoming fraudulent calls" refers to functions or devices in a communications terminal for detecting incoming calls.

"Means for listening to conversations using a microphone installed in a communications terminal" refers to a function or device that uses a microphone built into or connected to a communications terminal to collect audio during a call in real time.

"Means for converting conversation data into text and sending it to the generative AI model" refers to functions or devices for converting collected voice data into text data and sending that text data to the generative AI model.

A "generative AI model" is an artificial intelligence model that analyzes input data and detects specific patterns and phrases.

"Means for issuing an audio alert from the speaker of the communication terminal when it is determined that fraud is suspected" refers to a function or device for playing an alert sound or message through the speaker of the communication terminal when the generative AI model detects the possibility of fraud.

"Specific phrases or patterns that indicate the possibility of specialized fraud" are specific words or sentence structures associated with fraudulent acts, and the generative AI model's detection of these serves as the basis for determining the possibility of fraud.

"Audio alert" is a message or sound that warns the user audibly.

This invention is a system that automatically detects fraudulent calls and issues a warning to the user using an application installed on a communication terminal. A specific embodiment of this system is described below.

First, the user installs a dedicated application on their communication device. This application has the function of detecting incoming phone calls. Specifically, it uses the communication device's phone API to catch incoming call events.

When a user answers a call, the device listens to the conversation in real time through the microphone. At this time, the application runs in the background and captures the conversation. The listened conversation data is transcribed using the Google Speech-to-Text API. The audio data is sent to the API and received as text data.

The transcribed conversation data is sent to a server via the internet. The HTTPS protocol is used for transmission to ensure data security. The server inputs the received conversation data into a generative AI model (e.g., OpenAI's GPT-4). The generative AI model uses prompt text to analyze the possibility of fraud. Specific examples of prompt text are as follows:

Analyze the conversation data below to determine if it may be a scam.

Conversation data: "Your bank account has been compromised. Please provide your account number for verification."

The generative AI model analyzes the conversation data and determines whether fraud is suspected. If fraud is suspected, the server sends that information back to the device, again using the HTTPS protocol.

The device analyzes the results received from the server, and if fraud is suspected, an audio alert is emitted from the communication device's speaker. Specifically, an audio message such as "There is a possibility of fraud. Please be careful" is played. This alert makes the user aware of the possibility of fraud and allows them to take appropriate action.

As a concrete example, consider the case where a user receives a phone call saying, "Your bank account has been fraudulently used. Please provide your account number to confirm." This conversation is picked up by a microphone and transcribed. The transcribed data becomes, "Your bank account has been fraudulently used. Please provide your account number to confirm." This data is sent to a generative AI model, which determines that there is a high possibility of fraud. The server sends the result back to the device, which then issues an audio alert. The user hears this alert and becomes aware of the possibility of fraud.

In this way, the invention can automatically detect possible fraud when a user receives a fraudulent call and immediately issue a warning.

The flow of the identification process in Example 1 will be explained using Figure 11.

Step 1: Detect incoming calls

The terminal detects incoming calls through an application installed on the communication terminal. Specifically, it uses the communication terminal's telephone API to catch incoming call events. The input is the incoming call signal, and the output is the incoming call detection event.

Step 2: Listen to the conversation

When a user answers a call, the device listens to the conversation in real time through the microphone. The application runs in the background and captures the conversation. The input is the audio data during the call, and the output is the collected audio data.

Step 3: Transcribe the conversation data

The device uses the Google Speech-to-Text API to transcribe the conversation data it hears. It sends the audio data to the API and receives it as text data. The input is the collected audio data, and the output is the transcribed text data.

Step 4: Send the transcribed data

The terminal sends the transcribed conversation data to the server. The HTTPS protocol is used for transmission to ensure data security. The input is transcribed text data, and the output is a data transmission event to the server.

Step 5: Fraud Detection

The server inputs the received conversation data into a generative AI model. The generative AI model uses prompts to analyze the possibility of fraud. Specific examples of prompts are as follows:

Analyze the conversation data below to determine if it may be a scam.

Conversation data: "Your bank account has been compromised. Please provide your account number for verification."

The input is transcribed text data, and the output is a judgment result regarding the possibility of fraud.

Step 6: Receive the results

The server receives the judgment results from the generative AI model and returns them to the device. The return is again done using the HTTPS protocol. The input is the judgment result from the generative AI model, and the output is a data transmission event to the device.

Step 7: Receive an audio alert

The device analyzes the judgment results received from the server, and if fraud is suspected, it issues an audio alert from the communication device's speaker. Specifically, it plays an audio message such as "There is a possibility of fraud. Please be careful." The input is the judgment result from the server, and the output is the generation of an audio alert.

(Application Example 1)

Next, we will explain Application Example 1 of Form Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

In recent years, telephone fraud has been on the rise, with special fraud targeting the elderly becoming a particular social problem. Conventional fraud prevention measures make it difficult for users to detect potential fraud, making it difficult to prevent damage before it occurs. Therefore, there is a need for a system that can analyze telephone conversations in real time and immediately issue a warning if there is a possibility of fraud.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

In this invention, the server includes means for detecting incoming fraudulent calls, means for listening to conversations from a voice input device installed in the communication terminal, means for converting the conversation data into text and sending it to the generative AI, means for issuing an audio alert from the voice output device if the generative AI determines that fraud is suspected from the conversation data, means for the generative AI to detect specific phrases or patterns that indicate the possibility of fraud, and means for the audio alert to contain content that encourages the user to become aware of the possibility of fraud. This allows the user to immediately become aware of the possibility of fraud and prevent damage from occurring.

A "fraudulent call" is a call made with the intent of defrauding someone of money or personal information through fraudulent means.

"Incoming call" refers to an incoming phone call.

A "communications terminal" is a device for sending and receiving voice and data, including smartphones and home telephones.

An "audio input device" is a device that converts audio into digital data, such as a microphone.

"Conversation data" refers to data that has been converted into text from speech captured by a voice input device.

"Generative AI" is artificial intelligence that analyzes input data and detects specific patterns and phrases.

An "audio output device" is a device for outputting digital data as audio, and a speaker is an example of this.

"Audio alert" is a warning sound or message emitted from an audio output device.

"Specific phrases or patterns" are characteristic words or sentence structures that indicate possible fraud.

"User" refers to an individual or organization that uses this system.

A system for implementing this invention has the ability to detect incoming fraudulent calls and listen to the conversation from a voice input device installed on the communication terminal. The conversation data is transcribed in real time and sent to a generative AI. The generative AI detects specific phrases or patterns from the conversation data that indicate the possibility of fraud, and if fraud is suspected, issues a voice alert from a voice output device.

Hardware and software used

Hardware: Communication devices such as smartphones and home phones, microphones (audio input devices), speakers (audio output devices)

Software:

SpeechRecognition: A Python library for converting speech to text

requests: A Python library for sending HTTP requests

Generative AI: Artificial intelligence that analyzes text data using external APIs

System Operation

1. Incoming call detection: When the communication terminal detects an incoming call, it begins recording the conversation.

2. Conversation transcription: Recorded audio data is transcribed in real time using the SpeechRecognition library.

3. Generative AI Analysis: The transcribed conversation data is sent to a generative AI via an HTTP request. The generative AI detects specific phrases or patterns that indicate potential fraud.

4. Alert generation: If the generative AI detects the possibility of fraud, an audio alert will be emitted from the communication device's speaker. This audio alert will alert the user to the possibility of fraud.

Specific examples

For example, if a user receives a phone call and the caller says, "Your bank account has been fraudulently accessed. Please provide your account details immediately," the system will transcribe this conversation in real time and send it to the generative AI. The generative AI will detect that this phrase matches a specific pattern that indicates potential fraud and immediately issue an audio alert.

Example prompt

Examples of prompts for generative AI models include:

"Determine whether the following text is potentially fraudulent: 'Your bank account has been compromised. Please provide your account information immediately.'"

In this way, users can immediately become aware of potential fraud and prevent damage before it occurs.

The flow of the identification process in Application Example 1 will be explained using Figure 12.

Step 1: Incoming call detection

The communication terminal detects an incoming call. The input is the incoming call signal, and the output is a trigger to start recording. Specifically, the communication terminal's phone application detects the incoming call and activates the recording module.

Step 2: Record the conversation

The communication terminal's audio input device (microphone) records the conversation. The input is the audio data during the call, and the output is the recorded audio file. Specifically, the recording module captures the audio during the call and saves it as an audio file.

Step 3: Transcribe the conversation

Recorded audio data is converted into text using the SpeechRecognition library. The input is an audio file, and the output is text data. Specifically, the speech recognition engine analyzes the audio file and generates the corresponding text.

Step 4: Analysis using generative AI

The transcribed text data is sent to the generative AI via an HTTP request. The input is text data, and the output is a flag indicating the possibility of fraud. Specifically, a request containing the text data is sent to the generative AI's API endpoint, and the analysis results are received.

Step 5: Determine the possibility of fraud

Generative AI analyzes text data to detect specific phrases or patterns that indicate potential fraud. The input is text data, and the output is a flag indicating potential fraud. Specifically, generative AI analyzes text data and flags cases where there is a high probability of fraud.

Step 6: Send an alert

If possible fraud is detected, an audio alert is emitted from the communication terminal's audio output device (speaker). The input is a flag indicating possible fraud, and the output is an audio alert. Specifically, the audio alert module receives a flag indicating possible fraud and plays an audio message to warn the user.

In this way, users can immediately become aware of potential fraud and prevent damage before it occurs.

(Example 2)

Next, Example 2 of Form Example 2 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

In recent years, fraudulent phone scams have become more sophisticated, resulting in many people falling victim to them. Special frauds targeting the elderly in particular are on the rise, creating a need for effective countermeasures. Conventional countermeasures lack systems that can detect fraudulent phone calls in real time and issue warnings to users, making it difficult to prevent fraud before it occurs.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

In this invention, the server includes means for detecting incoming fraudulent calls, means for collecting conversation from a voice input device installed in the communication device, means for converting the conversation data into text data, means for transmitting the text data to the generative AI model, and means for issuing a voice alert from the voice input device if the generative AI model determines that fraud is suspected based on the conversation data. This makes it possible to detect fraudulent calls in real time and issue an immediate warning to the user.

"Means for detecting incoming fraudulent calls" refers to a function that identifies and detects incoming calls that may be fraudulent to a communications device.

An "audio input device" is a microphone or other audio capture device installed on a communications device to capture a user's speech.

"Means for converting speech data into text data" refers to speech recognition software or algorithms used to analyze collected voice data and convert it into text format.

A "generative AI model" is an artificial intelligence model that analyzes input text data and determines the possibility of fraud based on specific patterns and phrases.

"Means for issuing audio alerts" refers to a function that plays an audio message to warn the user if fraud is suspected.

This invention is a system that detects incoming fraudulent calls in real time and issues a warning to users. A specific embodiment of this system is described below.

1. System Configuration

The system consists of the following main components:

Methods for detecting incoming fraudulent calls

Voice input device

Method of converting conversation data into text data

Generative AI model

Method for issuing audio alerts

2. Hardware and Software Use

The voice input device uses a microphone installed in a home telephone. This microphone collects the user's speech in real time.

To convert conversation data into text data, we use voice recognition software (e.g., Google Speech-to-Text API). This software analyzes the collected voice data and converts it into text data.

The generative AI model uses advanced artificial intelligence models such as OpenAI GPT-4. This model analyzes the input text data and determines the likelihood of fraud.

The audio alert is generated using the speaker on a home phone, which will emit an audio alert if fraud is suspected.

3. System Operation

When a user starts a conversation on their home phone, a voice input device collects the conversation. The collected voice data is converted into text data using voice recognition software. This text data is sent to a server and input into a generative AI model. The generative AI model analyzes the conversation data and, if it determines that fraud is suspected, the server sends an alert signal to the home phone. The home phone's speaker emits an audio alert to warn the user.

4. Specific Examples

For example, suppose a user is speaking on their home phone, "Hello, your bank account has been compromised. Immediate action is required." A voice input device collects this speech, and speech recognition software converts it into text data. This text data is sent to a server, where it is analyzed by a generative AI model. If the generative AI model determines that there is a high possibility of fraud, the server sends an alert signal to the home phone. The speaker on the home phone will emit an audio alert saying, "Possible fraud. Please be careful."

5. Prompt Sentence Examples

"Analyze the conversation data below and determine whether or not you suspect fraud. If you suspect fraud, please explain why.

Conversation data: 'Hello, your bank account has been compromised. We need your immediate attention.'

In this way, users can be aware of their fraud risk in real time.

The flow of the identification process in Example 2 will be explained using Figure 13.

Step 1:

Audio collection

Device: A voice input device (microphone) installed on a home phone collects the user's conversation in real time.

Input: User's voice conversation

Output: Digital audio data

What it does: When a user starts a phone conversation, the microphone automatically captures the audio and stores it as digital audio data.

Step 2:

Transcription of audio data

Device: Built-in voice recognition software (e.g., Google Speech-to-Text API) in home phones converts collected voice data into text data.

Input: Digital audio data

Output: Text data

What it does: Speech recognition software analyzes the audio data and generates text data such as, "Hello, your bank account has been compromised. Immediate action is required."

Step 3:

Send text data

Device: The home phone sends the transcribed conversation data to the server.

Input: Text data

Output: Send data to the server

How it works: A home phone sends data over the internet to a server, which then passes the data to a generative AI model.

Step 4:

Fraud detection

Server: A generative AI model (e.g., OpenAI GPT-4) analyzes the received conversation data and determines whether fraud is suspected.

Input: Text data

Output: Judgment result regarding the likelihood of fraud

Specific behavior: The generative AI model analyzes the text "Hello, your bank account has been compromised. Immediate action is required" and determines that it is likely to be fraudulent.

Step 5:

Audio alert occurs

Server: If fraud is suspected, the server sends an alert signal to the home phone.

Device: The speaker on your home phone receives the alert signal from the server and plays an audio alert.

Input: Alert signal

Output: Audio alert

Specific action: The speaker on your home phone will play a voice alert saying, "This may be a scam. Be careful."

(Application Example 2)

Next, we will explain Application Example 2 of Form Example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

In recent years, the number of victims of fraudulent phone calls has been increasing, and special frauds targeting the elderly in particular have become a social problem. Conventional fraud prevention systems have been inadequate in detecting fraudulent calls, making it difficult for users to recognize the possibility of fraud. Furthermore, there is a demand for systems that can detect fraud in real time and issue warnings to users on home phones and smartphones.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

In this invention, the server includes means for detecting incoming fraudulent calls, means for listening to conversations using a voice recognition device, means for converting the conversation data into text and sending it to the generative AI, means for issuing an audio alert from a voice output device if the generative AI determines that fraud is suspected based on the conversation data, and means including an application to be installed on a smartphone. This enables real-time detection of fraudulent calls and prompt warnings to users.

A "fraudulent call" is a call made with the intent of defrauding someone of money or personal information through fraudulent means.

"Incoming call" refers to an incoming phone call.

A "voice recognition device" is a device that converts voice into text data.

"Conversation data" refers to the content of a conversation converted into text by a voice recognition device.

"Generative AI" is artificial intelligence that generates new information based on input data.

An "audio output device" is a device for outputting audio.

"Voice alert" is a function that issues a warning via voice.

"Applications installed on smartphones" are software programs that run on smartphones.

The following hardware and software are required to implement this invention. Hardware requires a smartphone, a voice recognition device, and a voice output device. Software requires a voice recognition library (e.g., speech_recognition), a generative AI library (e.g., openai), and a text-to-speech library (e.g., pyttsx3).

First, the phone call is recorded in real time using the smartphone's microphone. The recorded voice data is converted into text data using a voice recognition device. The speech_recognition library is used for this voice recognition.

The converted text data is then sent to the generative AI. The generative AI analyzes the input text data and determines whether it is fraudulent. This analysis is performed using the OpenAI library. Specifically, the following prompt is sent to the generative AI:

Please determine if the following conversation is a scam:

"Hello, this is the bank's security department. Your account has been compromised. Please provide your account number and PIN."

If you suspect it may be a scam, please reply 'scam'.

If the generative AI determines that there is a possibility of fraud, it will issue an audio alert using the smartphone's audio output device. This audio alert uses the pyttsx3 library. The content of the audio alert is intended to alert the user to the possibility of fraud.

For example, if a user is on a call on their smartphone and a conversation that could be fraudulent takes place, the application automatically converts the conversation into text and sends it to a generative AI to determine whether it is fraudulent. If it determines that there is a possibility of fraud, an audio alert will sound from the smartphone speaker saying, "Possible fraud!"

In this way, the invention enables real-time detection of fraudulent calls and prompt warning to users.

The flow of the specific processing in Application Example 2 will be explained using Figure 14.

Step 1:

The user initiates a call on their smartphone. The smartphone's microphone records the call in real time.

Input: Call audio

Output: Recorded audio data

Specific operation: The smartphone's microphone captures the call audio and saves it as audio data.

Step 2:

The device sends the recorded voice data to a voice recognition device, which converts the voice data into text data.

Input: Recorded audio data

Output: Text data

Specific operation: The speech recognition library (e.g., speech_recognition) analyzes the audio data and generates corresponding text data.

Step 3:

The device sends the generated text data to the generative AI, which analyzes the text data and determines whether it is fraudulent.

Input: Text data

Output: Judgment result regarding the possibility of fraud

Specific operation: A generative AI library (e.g., openai) analyzes text data based on the prompt and determines whether it is likely to be fraudulent.

Step 4:

The server receives the judgment results from the generative AI and generates an audio alert if there is a possibility of fraud.

Input: Judgment result regarding the possibility of fraud

Output: Audio alert content

Specific behavior: The generative AI generates a warning message saying, "Possible fraud!"

Step 5:

The device sends the contents of the audio alert to the audio output device, which then issues the audio alert.

Input: Audio alert content

Output: Audio alert

Specific operation: A text-to-speech library (e.g., pyttsx3) converts the contents of the audio alert into speech, and the warning sound is output from the smartphone speaker.

(Example 3)

Next, we will explain Example 3 of Form Example 3. In the following explanation, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

Conventional fraud detection systems were limited to detecting incoming fraudulent phone calls and were inadequate at detecting fraud in text messages and emails received by users. Furthermore, there were limited ways to notify users of potential fraud, which could delay users' realization of the risk of fraud. This made it difficult to prevent fraud from occurring.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Example 3 is realized by the following means.

In this invention, the server includes means for transmitting text data entered by the user to the server, means for the server to analyze the text data using a generative artificial intelligence model and determine the possibility of fraud, and means for the server to transmit the analysis results to the communication terminal. This makes it possible to quickly detect the possibility of fraud even in text messages or emails received by the user and notify the user.

"Means for detecting incoming fraudulent calls" refers to equipment or software that identifies and detects potentially fraudulent calls from among calls received over a communication line.

"A voice input device installed on a communication terminal" refers to a microphone or related hardware for collecting voice that is attached to communication devices such as smartphones and home telephones.

"Means for converting conversation data into text and sending it to the generative AI" refers to software or hardware for converting voice data into text data and sending that text data to the generative AI.

"Generative AI" is an AI model that uses natural language processing technology to analyze text data and detect specific patterns and phrases.

"A means for issuing an audio alert" is a device or software that issues an audio warning if it determines that there is a possibility of fraud.

"Means for transmitting text data entered by a user to a server" refers to software or hardware for transmitting text data entered by a user into a communication terminal to a server via a communication line such as the Internet.

"Means for the server to analyze text data using a generative artificial intelligence model and determine the possibility of fraud" refers to software or hardware that inputs the text data received by the server into a generative artificial intelligence model and evaluates the possibility of fraud from the analysis results.

"Means by which the server transmits the analysis results to the communications terminal" refers to software or hardware for transmitting the analysis results from the generative artificial intelligence model to the communications terminal.

"Means by which the communications terminal displays the analysis results to the user" refers to software or hardware that visually or audibly notifies the user of the analysis results received from the server.

This invention relates to a system that uses a generative artificial intelligence model to detect the possibility of special fraud. A specific embodiment of this system is described below.

System Configuration

Subject: Server

The server uses a generative artificial intelligence model to analyze text data and determine the likelihood of fraud. Specifically, the server uses the following hardware and software:

Hardware: High-performance processor, memory, and storage devices

Software: Generative AI models (e.g., AI models using natural language processing technology)

The server receives text data sent by the user and inputs it into a generative artificial intelligence model. The model detects specific phrases and patterns that indicate potential fraud and returns the results to the server. The server then sends the analysis results to the communication device.

Subject: Terminal

The device is responsible for sending text data entered by the user to the server and displaying the analysis results from the server to the user. Specifically, the device uses the following hardware and software:

Hardware: Communication devices such as smartphones, tablets, and PCs

Software: Dedicated application

The device sends the text data entered by the user to the server and displays the analysis results received from the server to the user. If the analysis results indicate the possibility of fraud, the device displays a warning message.

Subject: User

The user is responsible for entering text data through the device and checking the analysis results. Specifically, the user performs the following operations:

Enter message or email content into your device

Check the analysis results and take appropriate action as necessary.

Specific examples

Example 1

When a user receives a message saying, "Mom, please transfer the money now," the system works as follows:

1. The user enters a message into a dedicated application on the device.

2. The device sends this message to the server.

3. The server receives the message and inputs it as a prompt into the generative artificial intelligence model.

4. The generative artificial intelligence model analyzes the message and returns the result "Possible fraud."

5. The server sends the analysis results to the device.

6. The device displays the analysis results to the user, and the user confirms the warning message.

Example 2

When a user receives an email saying, "Your bank account has been compromised. Contact us immediately," the system works as follows:

1. The user enters the email content into a dedicated application on their device.

2. The device sends the contents of this email to the server.

3. The server receives the email content and inputs it as a prompt into the generative artificial intelligence model.

4. The generative artificial intelligence model analyzes the contents of the email and returns the result "Possible fraud."

5. The server sends the analysis results to the device.

6. The device displays the analysis results to the user, and the user confirms the warning message.

Example prompt

Below is an example of a prompt to input to a generative artificial intelligence model:

"Please determine if the following message is a potential scam: 'Mom, please transfer the money now.'"

"Determine whether the following email is a potential scam: 'Your bank account has been compromised. Contact us immediately.'"

This is how the specialized fraud detection system using a generative artificial intelligence model operates. The flow of the identification process in Example 3 will be explained using Figure 15.

Step 1:

The user enters text data.

The user enters the contents of the message or email they receive into a dedicated application on their device. For example, they might copy and paste the message "Mom, please transfer the money now" into the application. The input data is saved in text format on the device.

Step 2:

The device sends the text data to the server.

The terminal sends the text data entered by the user to the server. Specifically, the terminal application generates an HTTP request and sends a payload containing the text data to the server. At this time, the data is encrypted before being sent. The input is the text data entered by the user, and the output is the text data sent to the server.

Step 3:

The server receives the text data and inputs it into the generative artificial intelligence model.

The server receives text data sent from the terminal. The received data is first stored in a database. The server then inputs the text data as a prompt sentence into the generative AI model. An example of a prompt sentence is "Please determine whether the following message is likely to be fraudulent: 'Mom, please transfer the money now.'" The input is the text data received from the terminal, and the output is the prompt sentence input into the generative AI model.

Step 4:

A generative artificial intelligence model analyzes the text data and returns the results.

The generative artificial intelligence model analyzes the input prompt text and determines whether it is likely to be fraud. Specifically, the model analyzes it by comparing it with typical phrases and patterns of "bank transfer fraud" and "it's my son" fraud. If the analysis result indicates a high possibility of fraud, it returns a warning message stating "Possible fraud." The input is the prompt text, and the output is the analysis result.

Step 5:

The server sends the analysis results to the device.

The server sends the analysis results obtained from the generative AI model to the terminal. Specifically, the server generates an HTTP response and sends a payload containing the analysis results to the terminal. At this time, the data is encrypted before being sent. The input is the analysis results from the generative AI model, and the output is the analysis results sent to the terminal.

Step 6:

The device displays the analysis results to the user.

The device displays the analysis results received from the server to the user. For example, if the analysis result is a warning message stating "Possible fraud," the device application displays this message to the user as a pop-up window or notification. The input is the analysis result received from the server, and the output is the warning message displayed to the user. The user can check this warning message and take appropriate action.

(Application Example 3)

Next, we will explain Application Example 3 of Form Example 3. In the following explanation, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

In recent years, special fraud methods have become more sophisticated, with many people falling victim to them. In particular, "bank transfer fraud" and "it's me" frauds targeting the elderly have become a social problem. In order to prevent these frauds, a system is needed that can detect possible fraud early and issue a warning to users. However, current systems have low fraud detection accuracy, making it difficult for users to recognize the possibility of fraud.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 3 is realized by the following means.

In this invention, the server includes means for detecting incoming fraudulent calls, means for listening to conversations from a voice input device installed in the communication terminal, means for converting the conversation data into text and sending it to the generative AI, means for issuing a voice alert from the voice input device if the generative AI determines that fraud is suspected from the conversation data, means for the generative AI to detect specific phrases or patterns that indicate the possibility of fraud, and means for issuing a warning to the user if the generative AI detects specific phrases or patterns that indicate the possibility of fraud. This makes it possible to detect the possibility of fraud with high accuracy and issue a warning to the user quickly.

"Means for detecting incoming fraudulent calls" refers to a function that identifies and notifies a communication terminal of incoming calls that may be fraudulent.

"A voice input device installed on a communication terminal" is a device for collecting voice that is attached to communication devices such as smartphones and home telephones.

"Means for converting conversation data into text and sending it to generative AI" refers to a function for converting voice data collected by a voice input device into text data and sending that text data to generative AI.

"Means for issuing a voice alert from the voice input device when the generative AI determines that fraud is suspected based on the conversation data" refers to a function that issues a warning sound through the voice input device when the generative AI determines that there is a possibility of fraud based on the conversation data analyzed.

"Means for the generative AI to detect specific phrases or patterns that indicate the possibility of fraud" refers to a function that enables the generative AI to detect typical fraudulent phrases and patterns that it has previously learned from conversation data.

"Means for issuing a warning to the user when the generative AI detects a specific phrase or pattern that indicates the possibility of fraud" refers to a function that issues a warning to the user when the generative AI detects a phrase or pattern that indicates the possibility of fraud.

As an embodiment of this invention, we will explain how to install a fraud detection assistant system on a smartphone.

First, the system is equipped with a means for detecting incoming fraudulent calls. This means has the function of identifying and notifying the communication terminal of incoming calls that may be fraudulent. Specifically, the content of the call is collected in real time using a voice input device installed on the communication terminal.

The collected voice data is then converted into text data through a means of transcribing the conversation data and sending it to the generative AI. This conversion is done using voice recognition software (e.g., Google's voice recognition API). The converted text data is then sent to the generative AI model.

Generative AI models are pre-trained to detect specific phrases and patterns that indicate potential fraud. These models are built using, for example, the Hugging Face transformers library. If the generative AI determines from the conversation data that fraud is suspected, it will warn the user by issuing an audio alert via a voice input device.

Furthermore, if the generative AI detects certain phrases or patterns that indicate potential fraud, a user alert mechanism will be activated. This mechanism will alert the user to the potential fraud by providing a visual or audio warning.

As a concrete example, consider the following audio file:

Voice: "Mom, please transfer the money right now. It's urgent."

Example prompt for parsing this audio file:

Judge whether the phrase "Mom, please transfer the money now. It's urgent." could be a scam.

By feeding this prompt into a generative AI model, it can analyze the possibility of fraud and issue a warning to the user.

The flow of the specific processing in Application Example 3 will be explained using Figure 16.

Step 1:

Detect incoming fraudulent calls

The server identifies and notifies the communication terminal of incoming calls that may be fraudulent. Specifically, it monitors the communication terminal's call log and detects possible fraud based on specific numbers or patterns. The input is call log data, and the output is a notification of a potentially fraudulent call.

Step 2:

Listen to conversations from a voice input device

The terminal collects call content in real time using an audio input device installed on the communication terminal. Specifically, audio data is acquired through a microphone and saved as an audio file. The input is the audio during the call, and the output is an audio file.

Step 3:

Conversation data is transcribed and sent to generative AI.

The device converts the collected voice data into text data and sends it to the generative AI. Specifically, it uses voice recognition software (e.g., Google's voice recognition API) to transcribe the voice data. The input is an audio file, and the output is text data.

Step 4:

Generative AI analyzes conversation data

The server uses a generative AI model to detect specific phrases and patterns in text data that indicate potential fraud. Specifically, it analyzes the text data using a generative AI model (e.g., Hugging Face's transformers library). The input is the text data, and the output is a determination of the likelihood of fraud.

Step 5:

Audible alerts when fraud is suspected

If the generative AI determines that there is a possibility of fraud, the device will issue an audio alert from the voice input device. Specifically, it will play a warning sound or message. The input is the result of the determination of the possibility of fraud, and the output is an audio alert.

Step 6:

Warn the user

If the generative AI detects certain phrases or patterns that indicate potential fraud, the device will alert the user by displaying a warning message on the screen or sending a notification. The input is the result of the determination of potential fraud, and the output is a warning to the user.

It is also possible to further combine an emotion engine that estimates the user's emotion. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

"Example 1"

One embodiment of the present invention is a system that combines a generative AI with an emotion engine. This system analyzes emotions from the user's voice tone, volume, speaking rate, etc. Specifically, if the user's voice tone or volume suddenly increases while they are receiving a scam call, the system senses that the user may be panicking. Based on this information, the generative AI determines that the possibility of a scam has increased further and issues an audio alert.

"Example 2"

In another embodiment, the system could issue an audio alert if the emotion engine exceeds a certain threshold. For example, if the user's tone of voice exceeds a certain threshold, or if the user's speaking speed exceeds a certain rate, the emotion engine could determine that the user is experiencing strong emotion. Based on this determination, the generative AI could issue an audio alert, warning the user of possible fraud.

"Example 3"

Furthermore, as another embodiment, a system in which the emotion engine and generative AI work in conjunction with each other is also possible. In this system, the emotion engine analyzes the user's emotions and sends the results to the generative AI. The generative AI combines information from the emotion engine with conversation data it has analyzed to determine the possibility of fraud. In this way, more accurate fraud detection is possible.

The processing flow for each example form is explained below.

"Example 1"

Step 1: The user receives a scam call.

Step 2: A smartphone app or a microphone installed on a home phone listens to the conversation.

Step 3: Convert the conversation data into text and send it to the generative AI.

Step 4: Generative AI analyzes the text data and determines the likelihood of fraud.

Step 5: The emotion engine analyzes the user's emotions based on their tone of voice, volume, speaking rate, etc.

Step 6: The generative AI uses information from the emotion engine to further increase the likelihood of fraud.

and issues an audio alert.

"Example 2"

Step 1: The user receives a scam call.

Step 2: A smartphone app or a microphone installed on a home phone listens to the conversation.

Step 3: Convert the conversation data into text and send it to the generative AI.

Step 4: Generative AI analyzes the text data and determines the likelihood of fraud.

Step 5: The emotion engine analyzes the user's emotions based on their tone of voice, volume, speaking rate, etc.

Step 6: If the emotion engine exceeds a certain threshold, it notifies the generative AI.

Step 7: Based on the notification from the emotion engine, the generative AI determines that the likelihood of fraud has increased and issues an audio alert.

"Example 3"

Step 1: The user receives a scam call.

Step 2: A smartphone app or a microphone installed on a home phone listens to the conversation.

Step 3: Convert the conversation data into text and send it to the generative AI.

Step 4: Generative AI analyzes the text data and determines the likelihood of fraud.

Step 5: The emotion engine analyzes the user's emotions based on their tone of voice, volume, speaking rate, etc., and sends the results to the generative AI.

Step 6: The generative AI combines information from the emotion engine with the conversation data it has analyzed to determine the likelihood of fraud.

Step 7: If it determines there is a high possibility of fraud, the generative AI will issue an audio alert.

(Example 1)

Next, Example 1 of Form Example 1 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

Conventional fraud call detection systems analyze only the content of conversations to determine the possibility of fraud, which means they are unable to take the user's emotional state into account, resulting in low fraud detection accuracy. Furthermore, they are unable to respond adequately when the user panics, making it difficult to prevent fraud from occurring.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

In this invention, the server includes means for detecting incoming fraudulent calls, means for listening to conversations from a voice input device installed in the communication terminal, means for converting the conversation data into text and transmitting it to the generative artificial intelligence, means for issuing a voice alert from the voice input device if the generative artificial intelligence determines that fraud is suspected from the conversation data, means for collecting parameters such as the user's voice tone, volume, and speaking rate and analyzing their emotions, and means for reassessing the possibility of fraud based on the results of the emotion analysis. This makes it possible to detect fraud while taking the user's emotional state into account, thereby preventing fraud damage before it occurs.

"Means for detecting incoming fraudulent calls" refers to equipment or software that detects whether an incoming call to a communications terminal is likely to be fraudulent.

"A voice input device installed on a communication terminal" refers to a device for inputting voice, such as a microphone attached to a communication device such as a smartphone or home telephone.

"Means for converting conversation data into text and sending it to generative AI" refers to a device or software that converts voice data into text data and sends that text data to generative AI.

"Generative AI" is an artificial intelligence system that analyzes received data and determines the likelihood of fraud by detecting specific patterns and phrases.

"A means for issuing an audio alert" is a device or software that plays an audio message to warn the user if it determines that there is a high possibility of fraud.

"Means for collecting parameters such as the user's voice tone, volume, and speaking rate, and analyzing emotions" refers to a device or software that collects information such as tone, volume, and speaking rate from the user's voice data, and analyzes the user's emotional state based on that information.

"Means for reassessing the possibility of fraud based on the results of sentiment analysis" refers to a device or software that inputs the results of sentiment analysis into generative artificial intelligence and reassess the possibility of fraud.

Form for implementing the invention

This invention is a system that detects fraudulent phone calls and warns users, and is implemented using an application and server installed on a communications terminal. This system detects incoming fraudulent calls, listens to the conversation, transcribes the conversation data, and sends it to a generative artificial intelligence. Furthermore, it aims to prevent fraud by analyzing the user's emotional state and reassessing the possibility of fraud.

Hardware and software used

1. Communication terminal

Communication devices such as smartphones and home phones.

A microphone as an audio input device.

2. Server

Speech recognition software (e.g., Google Cloud Speech-to-Text API) is used to transcribe audio data.

For sentiment analysis, we use an emotion engine (e.g., IBM Watson Tone Analyzer).

For generative artificial intelligence, we use AI models with natural language processing technology (e.g., OpenAI's GPT-4).

Specific system operation

1. Incoming phone call detection

The server detects incoming phone calls through an application installed on the communication device. The application uses the communication device's native API to catch incoming phone call events.

2. Listening to Conversations

The device uses a microphone to listen to phone conversations. The application captures the microphone's audio input in real time and saves it as audio data.

3. Transcription of audio data

The server converts the audio data it hears into text data. This process involves calling the Google Cloud Speech-to-Text API to convert the audio data into text.

4. Sending to generative AI

The server sends the transcribed conversation data to the generative AI. The server sends the text data to the generative AI using an HTTP request.

5. Fraud Detection

The generative AI analyzes the received conversation data and determines whether fraud is suspected. Using natural language processing technology, the generative AI analyzes the content of the conversation and evaluates whether it matches any fraudulent patterns.

6. Sentiment Analysis

The server collects parameters such as the user's voice tone, volume, and speaking rate and analyzes them using an emotion engine. The server uses voice analysis software to extract emotion parameters from the voice data.

7. Panic Detection

If the user's tone or volume of voice suddenly increases, the server determines that the user may be panicking. The server analyzes data from the emotion engine and evaluates whether there is a sudden change.

8. Reassessing the possibility of fraud

Based on information from the emotion engine, the generative AI determines that the likelihood of fraud has increased. The generative AI receives the emotion data as additional input and reassess the likelihood of fraud.

9. Audio alerts

If the server determines that there is a high possibility of fraud, it instructs the communication device to play an audio alert from its speaker. The server then sends a command to the communication device's application to play the audio alert.

Examples and prompts

As a concrete example, imagine a user is receiving a phone call on their smartphone. If the person on the other end of the line says, "Your bank account has been fraudulently used. Please tell us your account number and password immediately," the user's voice tone will suddenly rise and the volume will also increase. Based on this information, the generative artificial intelligence will determine that there is a high possibility of fraud, and will emit an audio alert from the communication device's speaker saying, "This may be a fraud. Please be careful."

Example prompt:

"Analyze the conversation data below to determine if it may be a scam. Also consider changes in the user's tone and volume of voice.

Conversation data: 'Your bank account has been compromised. Please provide your account number and password immediately.'

User's voice tone: Sudden rise

User volume: Increased"

The flow of the identification process in Example 1 will be explained using Figure 17.

Step 1:

Incoming phone call detection

The server detects incoming phone calls through an application installed on the communication terminal.

Input: Incoming call event from communication terminal.

Data processing/calculation: Capture incoming call events using the communication device's native API.

Output: Event information indicating that an incoming call was detected.

Specific behavior: The application uses native Android and iOS APIs to catch incoming phone calls and notify the server.

Step 2:

Listening to conversations

The device uses a microphone to listen to the conversation during the call.

Input: Audio data during the call.

Data processing/calculation: Captures microphone audio input in real time and saves it as audio data.

Output: Audio data.

Specific behavior: The application captures audio input from the microphone in real time and saves it as audio data.

Step 3:

Transcription of audio data

The server converts the audio data it hears into text data.

Input: Audio data.

Data processing/calculation: Call the Google Cloud Speech-to-Text API to convert voice data into text.

Output: Character data.

Specific operation: The server converts the audio data into text using the Google Cloud Speech-to-Text API.

Step 4:

Send to generative AI

The server sends the transcribed conversation data to the generative artificial intelligence.

Input: Character data.

Data processing/calculation: Send text data to generative artificial intelligence using an HTTP request.

Output: Result of request sent to generative artificial intelligence.

Specific operation: The server sends text data to the generative artificial intelligence using an HTTP request.

Step 5:

Fraud detection

The generative artificial intelligence analyzes the received conversation data and determines whether fraud is suspected.

Input: Character data.

Data processing/calculation: Using natural language processing technology, the content of the conversation is analyzed and evaluated to see if it matches a fraud pattern.

Output: Assessment result regarding likelihood of fraud.

Specific operation: The generative artificial intelligence uses natural language processing technology to analyze the content of the conversation and evaluate whether it matches any fraud patterns.

Step 6:

Emotion analysis

The server collects parameters such as the user's voice tone, volume, and speaking speed, and analyzes them using an emotion engine.

Input: Audio data.

Data processing/calculation: Use voice analysis software to extract emotional parameters from voice data.

Output: Emotion parameters.

Specific operation: The server uses voice analysis software (e.g., IBM Watson Tone Analyzer) to extract emotion parameters from the voice data.

Step 7:

Detecting panic states

If the user's voice tone or volume suddenly increases, the server will determine that the user may be panicking.

Input: Emotion parameters.

Data processing/calculation: Analyze data from the emotion engine and evaluate whether there are any sudden changes.

Output: Panic state assessment results.

Specific behavior: The server analyzes data from the emotion engine and evaluates whether there are any sudden changes.

Step 8:

Reassessing the possibility of fraud

Based on information from the emotion engine, the generative artificial intelligence determines that the likelihood of fraud has increased.

Input: Emotion parameters.

Data processing/calculation: Emotional data is taken as additional input and the likelihood of fraud is reassessed.

Output: Reassessed fraud probability.

Specific behavior: The generative AI receives emotional data as additional input and reassesss the likelihood of fraud.

Step 9:

Send audio alerts

If the server determines that there is a high possibility of fraud, it will instruct the communication device to emit an audio alert from its speaker.

Input: Reassessed potential fraud.

Data processing/calculation: Sends a command to the communication terminal application to play an audio alert.

Output: Plays an audio alert.

Specific operation: The server sends a command to the application on the communication device to play an audio alert, which is then played from the communication device's speaker.

(Application Example 1)

Next, we will explain Application Example 1 of Form Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

In recent years, fraudulent phone scams have become more sophisticated, with many people falling victim to them. Elderly people and those unfamiliar with technology are particularly prone to falling victim to these scams, as they are less wary of them. Furthermore, when users receive a fraudulent call, they often panic, making it difficult for them to make a calm judgment. There is a need for effective measures to improve this situation and prevent fraudulent incidents.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

In this invention, the server includes means for detecting incoming fraudulent calls, means for listening to conversations from a voice input device installed in the communication terminal, means for converting the conversation data into text and sending it to the generative AI, means for issuing a voice alert from the voice input device if the generative AI determines that fraud is suspected from the conversation data, means including an emotion engine for analyzing emotions from the user's voice tone, volume, speaking speed, etc., and means for further increasing the possibility of fraud based on the analysis results of the emotion engine. In this way, by combining fraudulent call detection with user emotion analysis, it is possible to prevent fraud victims before they occur.

A "fraudulent call" is a call made with the intent of defrauding someone of money or personal information through fraudulent means.

"Incoming call" refers to an incoming phone call.

A "communications terminal" is an electronic device used to send and receive voice and data.

An "audio input device" is a device for converting audio into digital data.

"Conversation data" refers to data that has been converted into text from speech captured by a voice input device.

"Generative AI" is artificial intelligence that makes specific decisions and generates results based on input data.

"Audio alert" is a means of warning or notifying someone by voice.

An "emotion engine" is software or a system for analyzing emotions from voice data.

"Tone" refers to the pitch and texture of the voice.

"Volume" refers to the loudness of the sound.

"Speech rate" refers to the speed at which you speak.

The following system configuration is used to implement this invention.

System Configuration

1. Hardware

Communication terminal: A device capable of voice communication, such as a smartphone or home telephone.

Audio input device: A microphone built into the communication terminal.

Audio output device: Speaker built into the communication terminal.

2. Software

Speech Recognition Library: Use the speech_recognition library to convert speech to text.

Generative AI model: An AI model that uses the transformers library to determine the likelihood of fraud.

Emotion Engine: A model for analyzing user emotions using the transformers library.

Audio Alert Library: Uses the pyttsx3 library to emit audio alerts.

Processing flow

1. Incoming call detection

The server detects that a call has been received by the communication terminal.

The communication terminal's voice input device listens to the conversation and acquires voice data.

2. Voice Recognition

The server uses a voice recognition library to convert the acquired voice data into text data.

3. Fraud Detection Using Generative AI

The server inputs the text data into a generative AI model to determine the likelihood of fraud.

If certain phrases or patterns are detected, it is determined that there is a high possibility of fraud.

4. Sentiment analysis

The server uses an emotion engine to analyze the user's emotions based on their tone of voice, volume, speaking speed, etc.

If a user is perceived to be in a panic, the likelihood of fraud increases even more.

5. Audio alerts

If the server determines that there is a high possibility of fraud, it uses the audio alert library to issue an audio alert from the communication terminal's audio output device.

The alert is intended to alert users to potential fraud.

Specific examples

For example, if a user receives a phone call and it is determined that the call may be a scam, an audio alert will sound from the communication device's speaker saying, "Warning! Possible scam." This alert will make the user aware of the possibility of fraud and allow them to respond calmly.

Example prompt

"Could this call be a scam?"

In this way, this invention combines the detection of fraudulent calls with user sentiment analysis, making it possible to prevent fraudulent incidents.

The flow of the specific processing in Application Example 1 is explained using Figure 18.

Step 1:

The server detects that a call has been received by the communication terminal.

Input: Incoming signal from communication terminal.

Data processing: Receives incoming call signals and begins processing when the call starts.

Output: Call start flag.

Specific operation: When a communication terminal receives an incoming call, the server detects the signal and recognizes that a call has started.

Step 2:

The device's voice input device listens to the conversation and captures the voice data.

Input: Audio during a call.

Data processing: The voice input device collects voice data in real time.

Output: Captured audio data.

Specific operation: Audio during a call is collected through the microphone and saved as audio data.

Step 3:

The server uses a voice recognition library to convert the acquired voice data into text data.

Input: Acquired audio data.

Data processing: Use the speech recognition library (speech_recognition) to convert audio data into text data.

Output: Converted text data.

Specific operation: Voice data is input into the voice recognition library and output as text data.

Step 4:

The server inputs the text data into a generative AI model to determine the likelihood of fraud.

Input: Converted text data.

Data processing: Text data is fed into generative AI models (transformers) to assess the likelihood of fraud.

Output: Assessment results regarding the likelihood of fraud.

Specific operation: Text data is input into a generative AI model, which outputs an evaluation result indicating whether the likelihood of fraud is high.

Step 5:

The server uses an emotion engine to analyze the user's emotions based on their tone of voice, volume, speaking speed, etc.

Input: Converted text and audio data.

Data processing: Analyze user emotions using emotion engines (transformers).

Output: Analysis results on user sentiment.

Specific operation: Voice and text data are input into the emotion engine, and the user's emotional state is analyzed.

Step 6:

If the server determines that there is a high possibility of fraud, it uses the audio alert library to issue an audio alert from the communication terminal's audio output device.

Input: Fraud likelihood assessment results and user sentiment analysis results.

Data processing: If a high possibility of fraud is determined, an audio alert is generated using the audio alert library (pyttsx3).

Output: Audio alert.

Specific operation: If it is determined that there is a high possibility of fraud, an audio alert will be emitted from the communication device's speaker saying, "Warning! Possible fraud."

(Example 2)

Next, we will explain Example 2 of Form Example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

Conventional fraud prevention systems had low accuracy in detecting potential fraud, making it impossible to completely eliminate the risk of users falling victim to fraud. Furthermore, warnings were issued without taking the user's emotional state into consideration, which sometimes led to users being unable to respond appropriately. This made it difficult to prevent fraud from occurring.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

In this invention, the server includes means for detecting incoming fraudulent calls, means for listening to conversations from a voice input device installed in the communication device, means for converting the conversation data into text and sending it to the generative AI model, means for issuing an audio alert from the voice input device if the generative AI model determines that fraud is suspected from the conversation data, means for an emotion analysis engine to analyze the user's tone of voice and speaking rate, and means for the generative AI model to issue an audio alert if the emotion analysis engine exceeds a specific threshold. This makes it possible to detect the possibility of fraud with high accuracy and issue an appropriate warning according to the user's emotional state.

"Means for detecting incoming fraudulent calls" refers to a function that identifies and notifies a communication device of incoming calls that may be fraudulent.

A "voice input device" is a device that records conversations between a user and another person in real time and stores them in digital format.

"Means for converting conversation data into text and sending it to the generative AI model" refers to a function for converting voice data into text data and sending that text data to the generative AI model.

A "generative AI model" is an artificial intelligence system that analyzes input data, detects specific patterns and phrases, and determines the likelihood of fraud.

"Means for issuing audio alerts" refers to an audio notification function that warns users if a potential fraud is detected.

The "emotion analysis engine" is an engine that analyzes the user's tone of voice and speaking speed to determine their emotional state.

"When a specific threshold is exceeded" refers to when the emotion analysis engine analyzes the user's emotional state and it exceeds a pre-set reference value.

"Communication equipment" refers to devices used for making voice calls, such as home telephones and smartphones.

Form for implementing the invention

This invention is a system that detects incoming fraudulent calls and issues a warning to the user. A specific embodiment of this system is described below.

Hardware and software used

Communication devices: devices used for voice calls, such as home phones and smartphones.

Audio input device: A microphone installed in the communication device

Server: A computer system for processing voice data and running generative AI models and sentiment analysis engines.

Generative AI model: An artificial intelligence system that analyzes conversation data and determines the likelihood of fraud.

Emotion analysis engine: An engine that analyzes the user's tone of voice and speaking rate to determine their emotional state.

Speech recognition software: Software for converting voice data into text data (e.g., Google Speech-to-Text API)

System Operation

1. When a user starts a conversation on a communication device, the device's voice input device records the conversation in real time. The recorded voice data is saved in digital format.

2. The server uses speech recognition software to convert the collected voice data into text. For example, the speech "Hello, this is a bank representative" is converted into text "Hello, this is a bank representative."

3. The server sends the converted text data to the generative AI model. The generative AI model receives this data and prepares it for analysis.

4. The generative AI model analyzes the conversation data and detects specific keywords and phrases (e.g., "bank account" or "password"), which determines whether there is potential for fraud.

5. If the generative AI model determines that there is a high possibility of fraud, the server will send instructions to the terminal and an audio alert will be issued from the communication device's speaker saying, "There is a possibility of fraud. Please be careful."

6. The emotion analysis engine analyzes the user's tone of voice and speaking rate in real time. For example, if the user's voice suddenly gets higher in pitch or their speaking rate increases, the emotion analysis engine will detect this.

7. If the sentiment analysis engine determines that the user's emotions exceed a certain threshold, the generative AI model will issue an audio alert saying, "This may be a scam. Please remain calm."

Specific examples

Example 1: When a user says over the phone, "Please tell me your bank account information," the generative AI model detects the keyword "bank account" and determines that there is a possibility of fraud. The speaker on the communication device issues a voice alert saying, "There is a possibility of fraud. Please be careful."

Example 2: If a user says "Please tell me my password urgently" over the phone and their voice tone gets higher and faster, the emotion analysis engine determines that the user is experiencing strong emotions. The generative AI model issues a voice alert, warning, "This may be a scam. Please remain calm."

Example prompt

Prompt 1: "If someone calls and asks for your bank account information, determine whether it's a scam."

Prompt 2: "Please explain how the emotion engine determines if the user's voice tone gets higher and their speaking rate gets faster."

The system is designed to reduce the risk of users falling victim to telephone fraud. By combining a generative AI model with a sentiment analysis engine, it can detect potential fraud with greater accuracy and alert users.

The flow of the identification process in Example 2 will be explained using Figure 19.

Step 1:

The user begins a conversation on a communication device.

Specific operations involve a user initiating a call using a home phone or smartphone. The input is the user's voice, and the output is voice data collected by a voice input device.

Step 2:

The device's voice input device listens to the conversation and collects voice data.

Specifically, the device's microphone records the conversation between the user and the other party in real time. The input is the user's and the other party's voice, and the output is audio data stored in digital format.

Step 3:

The server converts the audio data into text data.

Specifically, the server uses speech recognition software (e.g., Google Speech-to-Text API) to convert the collected voice data into text data. The input is digital voice data, and the output is text conversation data.

Step 4:

The server sends the transcribed conversation data to the generative AI model.

Specific operations involve the server sending the converted text data to the generative AI model. The input is text-format conversation data, and the output is the data sent to the generative AI model.

Step 5:

A generative AI model analyzes conversation data to determine the likelihood of fraud.

Specifically, the generative AI model analyzes conversation data and detects specific keywords and phrases (e.g., "bank account" or "password"). The input is text-based conversation data, and the output is a judgment about the likelihood of fraud.

Step 6:

If there is a possibility of fraud, an audio alert will be issued via the device's voice input device.

Specifically, if the generative AI model determines that there is a high possibility of fraud, the server sends instructions to the terminal and an audio alert is emitted from the communication device's speaker saying, "There is a possibility of fraud. Please be careful." The input is the judgment result of the generative AI model, and the output is an audio alert.

Step 7:

The emotion analysis engine analyzes the user's tone of voice and speaking speed.

Specifically, the emotion analysis engine analyzes the user's tone of voice and speaking rate in real time. The input is the user's voice data, and the output is the analysis results regarding their emotional state.

Step 8:

If the sentiment analysis engine exceeds a certain threshold, the generative AI model will issue an audio alert.

Specifically, if the emotion analysis engine determines that the user's emotion exceeds a certain threshold, the generative AI model issues an audio alert saying, "This may be a scam. Please remain calm." The input is the analysis result of the emotion analysis engine, and the output is an audio alert.

(Application Example 2)

Next, we will explain Application Example 2 of Form Example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

In recent years, fraudulent phone scams have become increasingly sophisticated, leaving many people vulnerable. Elderly people and those unfamiliar with technology are particularly vulnerable to these scams, as they are less vigilant against them. While systems for detecting fraudulent calls exist, few warning systems take the user's emotional state into account. This increases the risk of users becoming victims of fraudulent calls without realizing they are being scammed. Therefore, there is a need for a system that detects fraudulent calls and takes the user's emotional state into account.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

In this invention, the server includes means for detecting incoming fraudulent calls, means for listening to conversations from a voice input device installed in the smart device, means for converting the conversation data into text and sending it to the generative AI, means for issuing a voice alert from the voice input device if the generative AI determines that fraud is suspected from the conversation data, and means for analyzing the user's emotions using an emotion engine and issuing a warning if a specific threshold is exceeded. This makes it possible to detect fraudulent calls and issue warnings that take into account the user's emotional state.

A "fraudulent call" is a call made with the intent of defrauding someone of money or personal information through fraudulent means.

"Incoming call" refers to an incoming phone call.

A "smart device" is an electronic device with internet connectivity, such as a smartphone or tablet.

An "audio input device" is a device such as a microphone that converts audio into digital data.

"Conversation data" refers to data that has been converted into text from speech acquired through a voice input device.

"Generative AI" is a system that uses artificial intelligence to analyze data and make specific decisions or generate results.

"Audio alert" is a means of warning or notifying the user using audio.

An "emotion engine" is an algorithm or system that analyzes a user's emotional state from voice and text data.

A "specific threshold" is a numerical value or condition that serves as a benchmark when the emotion engine evaluates the user's emotional state.

A "warning" is a notification or alert that alerts the user.

A system for implementing this invention includes a series of means for detecting incoming fraudulent calls and issuing a warning to the user. A specific embodiment of the system is described below.

First, a voice input device (microphone) is installed on a smart device (e.g., a smartphone or tablet). This voice input device captures the user's call content in real time. The captured voice data is converted into text data using voice recognition software (e.g., the speech_recognition library).

The transcribed conversation data is then sent to a generative AI model (e.g., a model using the transformers library). This generative AI model analyzes the conversation data and determines whether there is a possibility of fraud. If a fraudulent activity is detected, software for issuing audio alerts (e.g., the pyttsx3 library) is activated to warn the user.

Furthermore, an emotion engine (e.g., an emotion analysis model using the transformers library) is used to analyze the user's emotional state. The emotion engine evaluates the user's tone of voice, speaking rate, etc., and issues a warning if certain thresholds are exceeded. This warning is also notified to the user as an audio alert.

For example, if a user calls and says, "Please tell me your bank account information," the generative AI model will detect the possibility of fraud and issue a voice alert. Also, if the user is feeling angry or anxious, the emotion engine will detect this and issue a voice alert.

Examples of prompt statements include:

User's call: "Please tell me my bank account information."

Prompt for generative AI model: "Is this call potentially fraudulent?"

In this way, it is possible to create a system that can detect fraudulent calls and issue warnings that take into account the user's emotional state.

The flow of the specific processing in Application Example 2 is explained using Figure 20.

Step 1:

A user initiates a call on a smart device. The smart device's audio input device (microphone) captures the call content in real time. The input is the user's voice data, and the output is a stream of voice data.

Step 2:

The device uses speech recognition software (speech_recognition library) to convert the acquired voice data into text data. The input is a stream of voice data, and the output is transcribed conversation data.

Step 3:

The device sends the transcribed conversation data to a generative AI model (a model using the transformers library). The generative AI model analyzes the conversation data and determines whether there is a possibility of fraud. The input is the transcribed conversation data, and the output is a judgment result regarding the possibility of fraud.

Step 4:

The server receives the judgment results of the generative AI model, and if it determines that there is a possibility of fraud, it launches software (pyttsx3 library) to issue an audio alert. The input is the judgment result regarding the possibility of fraud, and the output is an audio alert.

Step 5:

The device uses an emotion engine (an emotion analysis model using the Transformers library) to analyze the user's emotional state. The emotion engine evaluates the user's tone of voice, speaking rate, etc., and issues a warning if certain thresholds are exceeded. The input is transcribed conversation data, and the output is an evaluation result regarding the user's emotional state.

Step 6:

The server receives the emotion engine's evaluation results, and if a certain threshold is exceeded, it launches software (pyttsx3 library) to issue an audio alert. The input is the evaluation result regarding the emotional state, and the output is an audio alert.

Step 7:

The user receives an audio alert and acknowledges the warning about potential fraud or emotional state. The input is the audio alert and the output is the user's acknowledgment.

(Example 3)

Next, we will explain Example 3 of Form Example 3. In the following explanation, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

Conventional fraud detection systems have the problem of low accuracy because they only analyze conversation data to determine the possibility of fraud. Furthermore, because they do not take the user's emotional state into account, there is a high risk of overlooking possible fraud. Furthermore, there are limited ways to notify users of possible fraud, which can lead to users not realizing the risk of fraud.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Example 3 is realized by the following means.

In this invention, the server includes means for detecting incoming fraudulent calls, means for listening to conversations from a voice input device installed in the communication terminal, means for converting the conversation data into text and transmitting it to the generative artificial intelligence, means for issuing a voice alert from the voice input device if the generative artificial intelligence determines that fraud is suspected from the conversation data, means for an emotion analysis engine to analyze the user's emotions, and means for the generative artificial intelligence to integrate information from the emotion analysis engine and the conversation data to determine the possibility of fraud. This makes it possible to detect the possibility of fraud with high accuracy and notify the user promptly.

"Means for detecting incoming fraudulent calls" refers to devices or software that monitor communication lines and identify incoming calls that may be fraudulent.

"A voice input device installed on a communication terminal" refers to a microphone attached to a communication device such as a smartphone or home telephone for collecting voice.

"Means for converting conversation data into text and sending it to generative AI" refers to software or a device for converting voice data into text data and sending that text data to generative AI.

"Generative AI" is an artificial intelligence model that uses natural language processing technology to analyze text data and detect specific patterns and phrases.

"Means for issuing audio alerts" refers to an audio output device that issues a warning to the user if the generative artificial intelligence detects the possibility of fraud.

An "emotion analysis engine" is software or a device that analyzes a user's voice and text data to identify their emotional state.

"Means for determining the likelihood of fraud" refers to an algorithm or software that uses generative artificial intelligence to integrate conversational data and information from a sentiment analysis engine to assess the likelihood of fraud.

This invention is a system that detects incoming fraudulent calls and notifies users of the possibility of fraud. A specific embodiment of this system is described below.

First, while a user is using a communication terminal (such as a smartphone or home phone), a means for detecting incoming fraudulent calls is activated. This means is a device or software that monitors communication lines and identifies incoming calls that may be fraudulent.

Next, a voice input device (microphone) installed on the communication terminal listens to the conversation between the user and the caller making the fraudulent call. This voice data is converted into text data by transcribing the conversation data and sending it to generative artificial intelligence. This conversion uses voice recognition technology. Specifically, voice recognition services such as Google Speech-to-Text API and IBM Watson Speech to Text can be used.

The server analyzes the text data using generative artificial intelligence (for example, natural language processing models such as BERT or GPT-3). The generative artificial intelligence detects specific phrases and patterns that may be fraudulent and evaluates the results. For example, if the phrase "Please transfer money" is included, it will determine that there is a high possibility of fraud.

Furthermore, the server uses an emotion analysis engine (for example, IBM Watson or Microsoft Azure's emotion analysis API) to analyze the user's emotions. It analyzes the user's voice and text data to identify emotional states such as anxiety or tension. The emotion data obtained by the emotion analysis engine is sent to the generative artificial intelligence.

Generative AI combines information from the sentiment analysis engine with conversational data to more accurately assess the likelihood of fraud. For example, if the message is "Please transfer money" and the user's emotions are anxious, it will conclude that there is a very high possibility of fraud.

Finally, if the server determines that the message is likely to be fraudulent, it will notify the user of this result. The notification is made by issuing an audio alert. Specifically, an audio alert will be issued from the communication device's speaker saying, "This message may be fraudulent. Please be careful."

As a concrete example, consider the case where a user receives a message saying, "Please transfer money." This message is picked up by the voice input device of the communication device and converted into text data. The server analyzes this text data using generative artificial intelligence and determines that there is a high possibility of fraud. At the same time, the emotion analysis engine detects the user's anxiety and sends this information to the generative artificial intelligence. The server combines this information and concludes that there is a very high possibility of fraud. Finally, the server sends an audio alert to the user's communication device saying, "This message may be fraudulent. Please be careful."

Example prompt:

"Please determine whether the following message is likely a scam. Message: 'Please transfer money.'"

In this way, the server, terminal, and user work together to detect the possibility of fraud with high accuracy and quickly notify the user. The flow of the identification process in Example 3 will be explained using Figure 21.

Step 1:

The user enters a message.

The user uses a communication device (such as a smartphone or home phone) to input the received message. For example, they might input a message such as "Please transfer money." The input message is saved on the device as voice data.

Step 2:

The device sends a message to the server.

The terminal sends the voice data entered by the user to the server. At this time, the terminal sends the voice data to the server via the Internet. The input is voice data, and the output is the transfer of voice data to the server.

Step 3:

The server converts the audio data into text data.

The server converts the received voice data into text data using voice recognition technology. Specifically, it uses voice recognition services such as Google Speech-to-Text API and IBM Watson Speech to Text. The input is voice data, and the output is text data.

Step 4:

The server analyzes the text data using generative artificial intelligence.

The server analyzes the text data using generative artificial intelligence (e.g., natural language processing models such as BERT or GPT-3). The generative artificial intelligence detects specific phrases and patterns that may be fraudulent and evaluates the results. The input is the text data, and the output is an evaluation result regarding the likelihood of fraud.

Step 5:

The server analyzes the user's emotions using an emotion analysis engine.

The server uses an emotion analysis engine (for example, IBM Watson or Microsoft Azure's emotion analysis API) to analyze the user's emotions. It analyzes the user's voice and text data to identify emotional states such as anxiety or tension. The input is voice or text data, and the output is emotion data.

Step 6:

The server combines the results of the generative artificial intelligence and the sentiment analysis engine to determine the likelihood of fraud.

The server integrates the analysis results from the generative AI with the emotional data from the emotion analysis engine. This allows for a more accurate assessment of the likelihood of fraud. For example, if the message is "Please transfer money" and the user's emotions are anxious, it will conclude that there is a very high likelihood of fraud. The input is the analysis results and emotional data, and the output is a final assessment of the likelihood of fraud.

Step 7:

The server will notify the user of possible fraud.

If the server determines that there is a high possibility of fraud, it notifies the user of the result. The notification is done by issuing an audio alert. Specifically, an audio alert saying "This message may be fraudulent. Please be careful" is issued from the speaker of the communication terminal. The input is the final assessment of the possibility of fraud, and the output is the audio alert.

(Application Example 3)

Next, we will explain Application Example 3 of Form Example 3. In the following explanation, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

Traditional fraud detection systems only detect specific phrases and patterns and do not take the user's emotional state into account, making it difficult to accurately determine the likelihood of fraud. Furthermore, there was a lack of means to provide appropriate warnings to users when fraud was likely.

The identification processing by the identification processing unit 290 of the data processing device 12 in Application Example 3 is realized by the following means. In this invention, the server includes means for detecting incoming fraudulent calls, means for listening to conversations from a smartphone application or a microphone installed in a home phone, means for converting the conversation data into text and sending it to the generative AI, means for issuing an audio alert from the microphone if the generative AI determines that fraud is suspected based on the conversation data, means for analyzing the user's emotions using an emotion engine, means for determining the possibility of fraud by combining information from the emotion engine with the conversation data analyzed by the generative AI, and means for displaying a warning to the user if there is a high possibility of fraud. This makes it possible to more accurately determine the possibility of fraud and provide the user with an appropriate warning.

"Means for detecting incoming fraudulent calls" refers to devices or software that monitor telephone lines or communications networks and detect incoming calls that may be fraudulent.

"A microphone installed in a smartphone application or home phone" is a voice input device attached to a smartphone or home phone to collect audio.

"Means for listening to conversations" refers to devices or software that analyze audio data collected through a microphone and recognize the content of the conversation.

"Means for converting conversation data into text and sending it to the generative AI" refers to a device or software that converts voice data into text data and sends that text data to the generative AI.

"Generative AI" is an artificial intelligence model trained to perform a specific task, in this case to determine the likelihood of fraud.

"Means for issuing an audio alert from the microphone when it is determined that fraud is suspected" refers to a device or software that issues an alarm or message through the microphone when the generative AI detects the possibility of fraud.

An "emotion engine" is software or a device that analyzes a user's emotional state from their voice or text data.

"Means for analyzing user emotions" refers to devices or software that use an emotion engine to analyze the user's emotional state.

"Means for determining the possibility of fraud by combining information from the emotion engine with conversation data analyzed by generative AI" refers to devices or software that integrate emotional information obtained from the emotion engine with the results of analysis of conversation data by generative AI, and make a comprehensive determination of the possibility of fraud.

"Means for displaying a warning to the user in the event of a high likelihood of fraud" refers to devices or software that provide a visual or audible warning to the user in the event that a high likelihood of fraud is determined.

A system for implementing this invention includes a series of means for detecting incoming fraudulent calls and providing a warning to the user. Specific embodiments are described below.

First, the server is equipped with a means for detecting incoming fraudulent calls. This means is a device or software that monitors telephone lines and communication networks to detect potentially fraudulent calls. For example, specific phone numbers or callers can be registered on a blacklist, and fraudulent calls can be detected based on this.

Next, the device (smartphone application or home phone) is equipped with a microphone for listening to the conversation. This microphone collects the voice during the call and sends it to the server as conversation data. The conversation data is converted into text using voice recognition technology and sent to the generative AI.

Generative AI analyzes conversation data to determine the likelihood of fraud. In doing so, it uses models trained to detect specific phrases and patterns. For example, it can detect typical phrases such as "bank transfer fraud" and "it's me" fraud.

Furthermore, an emotion engine analyzes the user's emotions. The emotion engine is software that analyzes the user's emotional state from voice and text data. The emotional information obtained from the emotion engine is integrated with the results of analysis of conversation data by generative AI to make a comprehensive judgment on the possibility of fraud.

If it determines that there is a high possibility of fraud, the device will display a warning to the user. This warning can be provided visually or audibly. For example, a warning message can be displayed on the smartphone screen or an audio alert can be issued.

For example, consider the following user input:

"Your account has been frozen. Please transfer the funds immediately."

"Hey, I need money right now."

By inputting these prompts into a generative AI model, it can analyze the possibility of fraud and, in conjunction with an emotion engine, display a warning to the user.

The flow of the specific processing in Application Example 3 will be explained using Figure 22.

Step 1:

The server monitors telephone lines and communication networks to detect incoming calls that may be fraudulent. The input is a signal from the telephone line or communication network, and the output is a flag indicating an incoming fraudulent call. Specifically, it registers specific phone numbers and callers on a blacklist and detects fraudulent calls based on that.

Step 2:

The device (smartphone application or home phone) collects audio during a call through a microphone. The input is the audio signal during the call, and the output is audio data. Specifically, the microphone converts the audio into a digital signal and sends that data to a server.

Step 3:

The server uses voice recognition technology to convert the voice data into text. The input is voice data and the output is text data. Specifically, the voice recognition software analyzes the voice data and converts it into the corresponding text.

Step 4:

The server sends text data to the generative AI, which analyzes it for the possibility of fraud. The input is text data, and the output is a score or flag indicating the possibility of fraud. Specifically, the generative AI model detects specific phrases and patterns and determines the possibility of fraud.

Step 5:

The server uses an emotion engine to analyze the user's emotions. The input is text data, and the output is data indicating the user's emotional state. Specifically, the emotion engine analyzes the text data and identifies the user's emotional state.

Step 6:

The server combines information from the emotion engine with conversation data analyzed by the generative AI to make a comprehensive judgment on the possibility of fraud. The input is emotional state data and a score or flag indicating the possibility of fraud, and the output is a final flag indicating the possibility of fraud. Specifically, it integrates emotional information with the analysis results of the generative AI to make a highly accurate judgment on the possibility of fraud.

Step 7:

If it determines that there is a high possibility of fraud, the device will display a warning to the user. The input is a flag indicating the final possibility of fraud, and the output is a warning message or audio alert. Specifically, a warning message is displayed on the smartphone screen or an audio alert is issued.

The specific processing unit 290 transmits the results of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the results of the specific processing. The microphone 238 acquires audio indicating the user input regarding the results of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

Data generation model 58 is what is known as generative AI (artificial intelligence). An example of data generation model 58 is generative AI such as ChatGPT (Internet search <URL: https://openai.com/blog/chatgpt>). Data generation model 58 is obtained by performing deep learning on a neural network. A prompt containing an instruction is input to data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. Data generation model 58 performs inference on the input inference data in accordance with the instructions indicated by the prompt, and outputs the inference results in the form of data such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and/or summarization.

Another example of generative AI is Gemini (Internet search <URL: https://gemini.google.com/?hl=ja>).

In the above embodiment, an example was given in which the specific processing was performed by the data processing device 12, but the technology disclosed herein is not limited to this, and the specific processing may also be performed by the smart glasses 214.

[Third embodiment]

Figure 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

As shown in FIG. 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

The data processing device 12 includes a computer 22, a database 24, and a communication I/F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and communication I/F 26 are also connected to the bus 34. The communication I/F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and/or a LAN (Local Area Network).

The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I/F 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the user 20's surroundings (e.g., an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

The communication I/F 44 is connected to the network 54. The communication I/Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I/Fs 44 and 26 is carried out in a secure manner.

Figure 6 shows an example of the main functions of the data processing device 12 and headset terminal 314. As shown in Figure 6, in the data processing device 12, specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32.

The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as the specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

Storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

In the headset terminal 314, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

Next, we will explain the identification process performed by the identification processing unit 290 of the data processing device 12.

"Example 1"

One embodiment of the present invention uses a smartphone application. In this case, the application installed on the smartphone detects an incoming phone call and listens to the conversation through the microphone. The listened-to conversation data is converted into text and sent to the generative AI. The generative AI determines from the conversation data that fraud is suspected, and if fraud is suspected, issues an audio alert from the smartphone's speaker.

"Example 2"

Another embodiment of the present invention involves installing a microphone on a home phone. In this case, the home phone's microphone listens to telephone conversations, and the listened conversation data is transcribed and sent to the generative AI. The generative AI determines from the conversation data that fraud is suspected, and if fraud is suspected, issues an audio alert from the home phone's speaker.

"Example 3"

A further embodiment of the present invention is one in which the generative AI detects specific phrases or patterns that indicate the possibility of specialized fraud. Specifically, the generative AI learns typical phrases for "bank transfer fraud" and typical patterns for "it's me" fraud, and by detecting these, determines the possibility of fraud.

The processing flow for each example form is explained below.

"Example 1"

Step 1: The application installed on the smartphone detects an incoming call.

Step 2: The application listens to your conversation through your smartphone's microphone.

Step 3: The conversation data heard is transcribed and sent to the generative AI.

Step 4: The generative AI determines that fraud is suspected based on the conversation data.

Step 5: If fraud is suspected, an audio alert will be issued through your smartphone's speaker.

"Example 2"

Step 1: A microphone installed on a home phone listens to the phone conversation.

Step 2: The conversation data that is heard is transcribed and sent to the generative AI.

Step 3: The generative AI determines that fraud is suspected based on the conversation data.

Step 4: If fraud is suspected, an audio alert will be issued over the home phone speaker.

"Example 3"

Step 1: The generative AI learns specific phrases or patterns that indicate potential fraud.

Step 2: The generative AI detects specific phrases or patterns it has learned to identify suspected fraud in the conversation data.

Step 3: If fraud is suspected, an audio alert will be issued.

(Example 1)

Next, Example 1 of Form Example 1 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset-type terminal 314 will be referred to as the "terminal."

In recent years, telephone fraud has been on the rise, with special frauds targeting the elderly becoming a particular social problem. Conventional countermeasures require users to identify fraudulent calls themselves, but as fraud methods become more sophisticated, this becomes increasingly difficult. Therefore, there is a need for a system that automatically detects potential fraud and issues a warning when users receive a fraudulent call.

The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means. In this invention, the server includes means for detecting incoming fraudulent calls, means for listening to the conversation from a microphone installed in the communication terminal, means for converting the conversation data into text and sending it to the generative AI model, and means for issuing an audio alert from the speaker of the communication terminal if the generative AI model determines that fraud is suspected based on the conversation data. This makes it possible to automatically detect the possibility of fraud when a user receives a fraudulent call and immediately issue a warning.

"Means for detecting incoming fraudulent calls" refers to functions or devices in a communications terminal for detecting incoming calls.

"Means for listening to conversations using a microphone installed in a communications terminal" refers to a function or device that uses a microphone built into or connected to a communications terminal to collect audio during a call in real time.

"Means for converting conversation data into text and sending it to the generative AI model" refers to functions or devices for converting collected voice data into text data and sending that text data to the generative AI model.

A "generative AI model" is an artificial intelligence model that analyzes input data and detects specific patterns and phrases.

"Means for issuing an audio alert from the speaker of the communication device when it is determined that fraud is suspected" refers to a function or device that plays an alert sound or message through the speaker of the communication device when the generative AI model detects the possibility of fraud.

"Specific phrases or patterns that indicate the possibility of specialized fraud" are specific words or sentence structures associated with fraudulent acts, and the generative AI model's detection of these serves as the basis for determining the possibility of fraud.

"Audio alert" is a message or sound that warns the user audibly.

This invention is a system that automatically detects fraudulent calls and issues a warning to the user using an application installed on a communication terminal. A specific embodiment of this system is described below.

First, the user installs a dedicated application on their communication device. This application has the function of detecting incoming phone calls. Specifically, it uses the communication device's phone API to catch incoming call events.

When a user answers a call, the device listens to the conversation in real time through the microphone. At this time, the application runs in the background and captures the conversation. The listened conversation data is transcribed using the Google Speech-to-Text API. The audio data is sent to the API and received as text data.

The transcribed conversation data is sent to a server via the internet. The HTTPS protocol is used for transmission to ensure data security. The server inputs the received conversation data into a generative AI model (e.g., OpenAI's GPT-4). The generative AI model uses prompt text to analyze the possibility of fraud. Specific examples of prompt text are as follows:

Analyze the conversation data below to determine if it may be a scam.

Conversation data: "Your bank account has been compromised. Please provide your account number for verification."

The generative AI model analyzes the conversation data and determines whether fraud is suspected. If fraud is suspected, the server sends that information back to the device, again using the HTTPS protocol.

The device analyzes the results received from the server, and if fraud is suspected, an audio alert is emitted from the communication device's speaker. Specifically, an audio message such as "There is a possibility of fraud. Please be careful" is played. This alert makes the user aware of the possibility of fraud and allows them to take appropriate action.

As a concrete example, consider the case where a user receives a phone call saying, "Your bank account has been fraudulently used. Please provide your account number to confirm." This conversation is picked up by a microphone and transcribed. The transcribed data becomes, "Your bank account has been fraudulently used. Please provide your account number to confirm." This data is sent to a generative AI model, which determines that there is a high possibility of fraud. The server sends the result back to the device, which then issues an audio alert. The user hears this alert and becomes aware of the possibility of fraud.

In this way, the invention can automatically detect possible fraud when a user receives a fraudulent call and immediately issue a warning.

The flow of the identification process in Example 1 will be explained using Figure 11.

Step 1: Detect incoming calls

The terminal detects incoming calls through an application installed on the communication terminal. Specifically, it uses the communication terminal's telephone API to catch incoming call events. The input is the incoming call signal, and the output is the incoming call detection event.

Step 2: Listen to the conversation

When a user answers a call, the device listens to the conversation in real time through the microphone. The application runs in the background and captures the conversation. The input is the audio data during the call, and the output is the collected audio data.

Step 3: Transcribe the conversation data

The device uses the Google Speech-to-Text API to transcribe the conversation data it hears. It sends the audio data to the API and receives it as text data. The input is the collected audio data, and the output is the transcribed text data.

Step 4: Send the transcribed data

The terminal sends the transcribed conversation data to the server. The HTTPS protocol is used for transmission to ensure data security. The input is transcribed text data, and the output is a data transmission event to the server.

Step 5: Fraud Detection

The server inputs the received conversation data into a generative AI model. The generative AI model uses prompts to analyze the possibility of fraud. Specific examples of prompts are as follows:

Analyze the conversation data below to determine if it may be a scam.

Conversation data: "Your bank account has been compromised. Please provide your account number for verification."

The input is transcribed text data, and the output is a judgment result regarding the possibility of fraud.

Step 6: Receive the results

The server receives the judgment results from the generative AI model and returns them to the device. The return is again done using the HTTPS protocol. The input is the judgment result from the generative AI model, and the output is a data transmission event to the device.

Step 7: Receive an audio alert

The device analyzes the judgment results received from the server, and if fraud is suspected, it issues an audio alert from the communication device's speaker. Specifically, it plays an audio message such as "There is a possibility of fraud. Please be careful." The input is the judgment result from the server, and the output is the generation of an audio alert.

(Application Example 1)

Next, we will explain Application Example 1 of Form Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the headset-type terminal 314 will be referred to as the "terminal."

In recent years, telephone fraud has been on the rise, with special fraud targeting the elderly becoming a particular social problem. Conventional fraud prevention measures make it difficult for users to detect potential fraud, making it difficult to prevent damage before it occurs. Therefore, there is a need for a system that can analyze telephone conversations in real time and immediately issue a warning if there is a possibility of fraud.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

In this invention, the server includes means for detecting incoming fraudulent calls, means for listening to conversations from a voice input device installed in the communication terminal, means for converting the conversation data into text and sending it to the generative AI, means for issuing an audio alert from the voice output device if the generative AI determines that fraud is suspected from the conversation data, means for the generative AI to detect specific phrases or patterns that indicate the possibility of fraud, and means for the audio alert to contain content that encourages the user to become aware of the possibility of fraud. This allows the user to immediately become aware of the possibility of fraud and prevent damage from occurring.

A "fraudulent call" is a call made with the intent of defrauding someone of money or personal information through fraudulent means.

"Incoming call" refers to an incoming phone call.

A "communications terminal" is a device for sending and receiving voice and data, including smartphones and home telephones.

An "audio input device" is a device that converts audio into digital data, such as a microphone.

"Conversation data" refers to data that has been converted into text from speech captured by a voice input device.

"Generative AI" is artificial intelligence that analyzes input data and detects specific patterns and phrases.

An "audio output device" is a device for outputting digital data as audio, and a speaker is an example of this.

"Audio alert" is a warning sound or message emitted from an audio output device.

"Specific phrases or patterns" are characteristic words or sentence structures that indicate possible fraud.

"User" refers to an individual or organization that uses this system.

A system for implementing this invention has the ability to detect incoming fraudulent calls and listen to the conversation from a voice input device installed on the communication terminal. The conversation data is transcribed in real time and sent to a generative AI. The generative AI detects specific phrases or patterns from the conversation data that indicate the possibility of fraud, and if fraud is suspected, issues a voice alert from a voice output device.

Hardware and software used

Hardware: Communication devices such as smartphones and home phones, microphones (audio input devices), speakers (audio output devices)

Software:

SpeechRecognition: A Python library for converting speech to text

requests: A Python library for sending HTTP requests

Generative AI: Artificial intelligence that analyzes text data using external APIs

System Operation

1. Incoming call detection: When the communication terminal detects an incoming call, it begins recording the conversation.

2. Conversation transcription: Recorded audio data is transcribed in real time using the SpeechRecognition library.

3. Generative AI Analysis: The transcribed conversation data is sent to a generative AI via an HTTP request. The generative AI detects specific phrases or patterns that indicate potential fraud.

4. Alert generation: If the generative AI detects the possibility of fraud, an audio alert will be emitted from the communication device's speaker. This audio alert will alert the user to the possibility of fraud.

Specific examples

For example, if a user receives a phone call and the caller says, "Your bank account has been fraudulently accessed. Please provide your account details immediately," the system will transcribe this conversation in real time and send it to the generative AI. The generative AI will detect that this phrase matches a specific pattern that indicates potential fraud and immediately issue an audio alert.

Example prompt

Examples of prompts for generative AI models include:

"Determine whether the following text is potentially fraudulent: 'Your bank account has been compromised. Please provide your account information immediately.'"

In this way, users can immediately become aware of potential fraud and prevent damage before it occurs.

The flow of the specific processing in Application Example 1 will be explained using Figure 12.

Step 1: Incoming call detection

The communication terminal detects an incoming call. The input is the incoming call signal, and the output is a trigger to start recording. Specifically, the communication terminal's phone application detects the incoming call and activates the recording module.

Step 2: Record the conversation

The communication terminal's audio input device (microphone) records the conversation. The input is the audio data during the call, and the output is the recorded audio file. Specifically, the recording module captures the audio during the call and saves it as an audio file.

Step 3: Transcribe the conversation

Recorded audio data is converted into text using the SpeechRecognition library. The input is an audio file, and the output is text data. Specifically, the speech recognition engine analyzes the audio file and generates the corresponding text.

Step 4: Analysis using generative AI

The transcribed text data is sent to the generative AI via an HTTP request. The input is text data, and the output is a flag indicating the possibility of fraud. Specifically, a request containing the text data is sent to the generative AI's API endpoint, and the analysis results are received.

Step 5: Determine the possibility of fraud

Generative AI analyzes text data to detect specific phrases or patterns that indicate potential fraud. The input is text data, and the output is a flag indicating potential fraud. Specifically, generative AI analyzes text data and flags cases where there is a high probability of fraud.

Step 6: Send an alert

If possible fraud is detected, an audio alert is emitted from the communication terminal's audio output device (speaker). The input is a flag indicating possible fraud, and the output is an audio alert. Specifically, the audio alert module receives a flag indicating possible fraud and plays an audio message to warn the user.

In this way, users can immediately become aware of potential fraud and prevent damage before it occurs.

(Example 2)

Next, Example 2 of Form Example 2 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset-type terminal 314 will be referred to as the "terminal."

In recent years, fraudulent phone scams have become more sophisticated, resulting in many people falling victim to them. Special frauds targeting the elderly in particular are on the rise, creating a need for effective countermeasures. Conventional countermeasures lack systems that can detect fraudulent phone calls in real time and issue warnings to users, making it difficult to prevent fraud before it occurs.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

In this invention, the server includes means for detecting incoming fraudulent calls, means for collecting conversation from a voice input device installed in the communication device, means for converting the conversation data into text data, means for transmitting the text data to the generative AI model, and means for issuing a voice alert from the voice input device if the generative AI model determines that fraud is suspected based on the conversation data. This makes it possible to detect fraudulent calls in real time and issue an immediate warning to the user.

"Means for detecting incoming fraudulent calls" refers to a function that identifies and detects incoming calls that may be fraudulent to a communications device.

An "audio input device" is a microphone or other audio capture device installed on a communications device to capture a user's speech.

"Means for converting speech data into text data" refers to speech recognition software or algorithms used to analyze collected voice data and convert it into text format.

A "generative AI model" is an artificial intelligence model that analyzes input text data and determines the possibility of fraud based on specific patterns and phrases.

"Means for issuing audio alerts" refers to a function that plays an audio message to warn the user if fraud is suspected.

This invention is a system that detects incoming fraudulent calls in real time and issues a warning to users. A specific embodiment of this system is described below.

1. System Configuration

The system consists of the following main components:

Methods for detecting incoming fraudulent calls

Voice input device

Method of converting conversation data into text data

Generative AI model

Method for issuing audio alerts

2. Hardware and Software Use

The voice input device uses a microphone installed in a home telephone. This microphone collects the user's speech in real time.

To convert conversation data into text data, we use voice recognition software (e.g., Google Speech-to-Text API). This software analyzes the collected voice data and converts it into text data.

The generative AI model uses advanced artificial intelligence models such as OpenAI GPT-4. This model analyzes the input text data and determines the likelihood of fraud.

The audio alert is generated using the speaker on a home phone, which will emit an audio alert if fraud is suspected.

3. System Operation

When a user starts a conversation on their home phone, a voice input device collects the conversation. The collected voice data is converted into text data using voice recognition software. This text data is sent to a server and input into a generative AI model. The generative AI model analyzes the conversation data and, if it determines that fraud is suspected, the server sends an alert signal to the home phone. The home phone's speaker emits an audio alert to warn the user.

4. Specific Examples

For example, suppose a user is speaking on their home phone, "Hello, your bank account has been compromised. Immediate action is required." A voice input device collects this speech, and speech recognition software converts it into text data. This text data is sent to a server, where it is analyzed by a generative AI model. If the generative AI model determines that there is a high possibility of fraud, the server sends an alert signal to the home phone. The speaker on the home phone will emit an audio alert saying, "Possible fraud. Please be careful."

5. Prompt Sentence Examples

"Analyze the conversation data below and determine whether or not you suspect fraud. If you suspect fraud, please explain why.

Conversation data: 'Hello, your bank account has been compromised. We need your immediate attention.'

In this way, users can be aware of their fraud risk in real time.

The flow of the identification process in Example 2 will be explained using Figure 13.

Step 1:

Audio collection

Device: A voice input device (microphone) installed on a home phone collects the user's conversation in real time.

Input: User's voice conversation

Output: Digital audio data

What it does: When a user starts a phone conversation, the microphone automatically captures the audio and stores it as digital audio data.

Step 2:

Transcription of audio data

Device: Built-in voice recognition software (e.g., Google Speech-to-Text API) in home phones converts collected voice data into text data.

Input: Digital audio data

Output: Text data

What it does: Speech recognition software analyzes the audio data and generates text data such as, "Hello, your bank account has been compromised. Immediate action is required."

Step 3:

Send text data

Device: The home phone sends the transcribed conversation data to the server.

Input: Text data

Output: Send data to the server

How it works: A home phone sends data over the internet to a server, which then passes the data to a generative AI model.

Step 4:

Fraud detection

Server: A generative AI model (e.g., OpenAI GPT-4) analyzes the received conversation data and determines whether fraud is suspected.

Input: Text data

Output: Judgment result regarding the likelihood of fraud

Specific behavior: The generative AI model analyzes the text "Hello, your bank account has been compromised. Immediate action is required" and determines that it is likely to be fraudulent.

Step 5:

Audio alert occurs

Server: If fraud is suspected, the server sends an alert signal to the home phone.

Device: The speaker on your home phone receives the alert signal from the server and plays an audio alert.

Input: Alert signal

Output: Audio alert

Specific behavior: The speaker on your home phone will play a voice alert saying, "This may be a scam. Be careful."

(Application Example 2)

Next, we will explain Application Example 2 of Form Example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the headset-type terminal 314 will be referred to as the "terminal."

In recent years, the number of victims of fraudulent phone calls has been increasing, and special frauds targeting the elderly in particular have become a social problem. Conventional fraud prevention systems have been inadequate in detecting fraudulent calls, making it difficult for users to recognize the possibility of fraud. Furthermore, there is a demand for systems that can detect fraud in real time and issue warnings to users on home phones and smartphones.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

In this invention, the server includes means for detecting incoming fraudulent calls, means for listening to conversations using a voice recognition device, means for converting the conversation data into text and sending it to the generative AI, means for issuing an audio alert from a voice output device if the generative AI determines that fraud is suspected based on the conversation data, and means including an application to be installed on a smartphone. This enables real-time detection of fraudulent calls and prompt warnings to users.

A "fraudulent call" is a call made with the intent of defrauding someone of money or personal information through fraudulent means.

"Incoming call" refers to an incoming phone call.

A "voice recognition device" is a device that converts voice into text data.

"Conversation data" refers to the content of a conversation converted into text by a voice recognition device.

"Generative AI" is artificial intelligence that generates new information based on input data.

An "audio output device" is a device for outputting audio.

"Voice alert" is a function that issues a warning via voice.

"Applications installed on smartphones" are software programs that run on smartphones.

The following hardware and software are required to implement this invention. Hardware requires a smartphone, a voice recognition device, and a voice output device. Software requires a voice recognition library (e.g., speech_recognition), a generative AI library (e.g., openai), and a text-to-speech library (e.g., pyttsx3).

First, the phone call is recorded in real time using the smartphone's microphone. The recorded voice data is converted into text data using a voice recognition device. The speech_recognition library is used for this voice recognition.

The converted text data is then sent to the generative AI. The generative AI analyzes the input text data and determines whether it is fraudulent. This analysis is performed using the OpenAI library. Specifically, the following prompt is sent to the generative AI:

Please determine if the following conversation is a scam:

"Hello, this is the bank's security department. Your account has been compromised. Please provide your account number and PIN."

If you suspect it may be a scam, please reply 'scam'.

If the generative AI determines that there is a possibility of fraud, it will issue an audio alert using the smartphone's audio output device. This audio alert uses the pyttsx3 library. The content of the audio alert is intended to alert the user to the possibility of fraud.

For example, if a user is on a call on their smartphone and a conversation that could be fraudulent takes place, the application automatically converts the conversation into text and sends it to a generative AI to determine whether it is fraudulent. If it determines that there is a possibility of fraud, an audio alert will sound from the smartphone speaker saying, "Possible fraud!"

In this way, the invention enables real-time detection of fraudulent calls and prompt warning to users.

The flow of the specific processing in Application Example 2 will be explained using Figure 14.

Step 1:

The user initiates a call on their smartphone. The smartphone's microphone records the call in real time.

Input: Call audio

Output: Recorded audio data

Specific operation: The smartphone's microphone captures the call audio and saves it as audio data.

Step 2:

The device sends the recorded voice data to a voice recognition device, which converts the voice data into text data.

Input: Recorded audio data

Output: Text data

Specific operation: The speech recognition library (e.g., speech_recognition) analyzes the audio data and generates corresponding text data.

Step 3:

The device sends the generated text data to the generative AI, which analyzes the text data and determines whether it is fraudulent.

Input: Text data

Output: Judgment result regarding the possibility of fraud

Specific operation: A generative AI library (e.g., openai) analyzes text data based on the prompt and determines whether it is likely to be fraudulent.

Step 4:

The server receives the judgment results from the generative AI and generates an audio alert if there is a possibility of fraud.

Input: Judgment result regarding the possibility of fraud

Output: Audio alert content

Specific behavior: The generative AI generates a warning message saying, "Possible fraud!"

Step 5:

The device sends the contents of the audio alert to the audio output device, which then issues the audio alert.

Input: Audio alert content

Output: Audio alert

Specific operation: A text-to-speech library (e.g., pyttsx3) converts the contents of the audio alert into speech, and the warning sound is output from the smartphone speaker.

(Example 3)

Next, we will explain Example 3 of Form Example 3. In the following explanation, the data processing device 12 will be referred to as the "server" and the headset-type terminal 314 will be referred to as the "terminal."

Conventional fraud detection systems were limited to detecting incoming fraudulent phone calls and were inadequate at detecting fraud in text messages and emails received by users. Furthermore, there were limited ways to notify users of potential fraud, which could delay users' realization of the risk of fraud. This made it difficult to prevent fraud from occurring.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Example 3 is realized by the following means.

In this invention, the server includes means for transmitting text data entered by the user to the server, means for the server to analyze the text data using a generative artificial intelligence model and determine the possibility of fraud, and means for the server to transmit the analysis results to the communication terminal. This makes it possible to quickly detect the possibility of fraud even in text messages or emails received by the user and notify the user.

"Means for detecting incoming fraudulent calls" refers to equipment or software that identifies and detects potentially fraudulent calls from among calls received over a communication line.

"A voice input device installed on a communication terminal" refers to a microphone or related hardware for collecting voice that is attached to communication devices such as smartphones and home telephones.

"Means for converting conversation data into text and sending it to the generative AI" refers to software or hardware for converting voice data into text data and sending that text data to the generative AI.

"Generative AI" is an AI model that uses natural language processing technology to analyze text data and detect specific patterns and phrases.

"A means for issuing an audio alert" is a device or software that issues an audio warning if it determines that there is a possibility of fraud.

"Means for transmitting text data entered by a user to a server" refers to software or hardware for transmitting text data entered by a user into a communication terminal to a server via a communication line such as the Internet.

"Means for the server to analyze text data using a generative artificial intelligence model and determine the possibility of fraud" refers to software or hardware that inputs the text data received by the server into a generative artificial intelligence model and evaluates the possibility of fraud from the analysis results.

"Means by which the server transmits the analysis results to the communications terminal" refers to software or hardware for transmitting the analysis results from the generative artificial intelligence model to the communications terminal.

"Means by which the communications terminal displays the analysis results to the user" refers to software or hardware that visually or audibly notifies the user of the analysis results received from the server.

This invention relates to a system that uses a generative artificial intelligence model to detect the possibility of special fraud. A specific embodiment of this system is described below.

System Configuration

Subject: Server

The server uses a generative artificial intelligence model to analyze text data and determine the likelihood of fraud. Specifically, the server uses the following hardware and software:

Hardware: High-performance processor, memory, and storage devices

Software: Generative AI models (e.g., AI models using natural language processing technology)

The server receives text data sent by the user and inputs it into a generative artificial intelligence model. The model detects specific phrases and patterns that indicate potential fraud and returns the results to the server. The server then sends the analysis results to the communication device.

Subject: Terminal

The device is responsible for sending text data entered by the user to the server and displaying the analysis results from the server to the user. Specifically, the device uses the following hardware and software:

Hardware: Communication devices such as smartphones, tablets, and PCs

Software: Dedicated application

The device sends the text data entered by the user to the server and displays the analysis results received from the server to the user. If the analysis results indicate the possibility of fraud, the device displays a warning message.

Subject: User

The user is responsible for entering text data through the device and checking the analysis results. Specifically, the user performs the following operations:

Enter message or email content into your device

Check the analysis results and take appropriate action as necessary.

Specific examples

Example 1

When a user receives a message saying, "Mom, please transfer the money now," the system works as follows:

1. The user enters a message into a dedicated application on the device.

2. The device sends this message to the server.

3. The server receives the message and inputs it as a prompt into the generative artificial intelligence model.

4. The generative artificial intelligence model analyzes the message and returns the result "Possible fraud."

5. The server sends the analysis results to the device.

6. The device displays the analysis results to the user, and the user confirms the warning message.

Example 2

When a user receives an email saying, "Your bank account has been compromised. Contact us immediately," the system behaves as follows:

1. The user enters the email content into a dedicated application on their device.

2. The device sends the contents of this email to the server.

3. The server receives the email content and inputs it as a prompt into the generative artificial intelligence model.

4. The generative artificial intelligence model analyzes the contents of the email and returns the result "Possible fraud."

5. The server sends the analysis results to the device.

6. The device displays the analysis results to the user, and the user confirms the warning message.

Example prompt

Below is an example of a prompt to input to a generative artificial intelligence model:

"Please determine if the following message is a potential scam: 'Mom, please transfer the money now.'"

"Determine whether the following email is a potential scam: 'Your bank account has been compromised. Contact us immediately.'"

This is how the specialized fraud detection system using a generative artificial intelligence model operates. The flow of the identification process in Example 3 will be explained using Figure 15.

Step 1:

The user enters text data.

The user enters the contents of the message or email they receive into a dedicated application on their device. For example, they might copy and paste the message "Mom, please transfer the money now" into the application. The input data is saved in text format on the device.

Step 2:

The device sends the text data to the server.

The terminal sends the text data entered by the user to the server. Specifically, the terminal application generates an HTTP request and sends a payload containing the text data to the server. At this time, the data is encrypted before being sent. The input is the text data entered by the user, and the output is the text data sent to the server.

Step 3:

The server receives the text data and inputs it into the generative artificial intelligence model.

The server receives text data sent from the terminal. The received data is first stored in a database. The server then inputs the text data as a prompt sentence into the generative AI model. An example of a prompt sentence is "Please determine whether the following message is likely to be fraudulent: 'Mom, please transfer the money now.'" The input is the text data received from the terminal, and the output is the prompt sentence input into the generative AI model.

Step 4:

A generative artificial intelligence model analyzes the text data and returns the results.

The generative artificial intelligence model analyzes the input prompt text and determines whether it is likely to be fraud. Specifically, the model analyzes it by comparing it with typical phrases and patterns of "bank transfer fraud" and "it's my son" fraud. If the analysis result indicates a high possibility of fraud, it returns a warning message stating "Possible fraud." The input is the prompt text, and the output is the analysis result.

Step 5:

The server sends the analysis results to the device.

The server sends the analysis results obtained from the generative AI model to the terminal. Specifically, the server generates an HTTP response and sends a payload containing the analysis results to the terminal. At this time, the data is encrypted before being sent. The input is the analysis results from the generative AI model, and the output is the analysis results sent to the terminal.

Step 6:

The device displays the analysis results to the user.

The device displays the analysis results received from the server to the user. For example, if the analysis result is a warning message stating "Possible fraud," the device application displays this message to the user as a pop-up window or notification. The input is the analysis result received from the server, and the output is the warning message displayed to the user. The user can check this warning message and take appropriate action.

(Application Example 3)

Next, we will explain Application Example 3 of Form Example 3. In the following explanation, the data processing device 12 will be referred to as the "server" and the headset-type terminal 314 will be referred to as the "terminal."

In recent years, special fraud methods have become more sophisticated, with many people falling victim to them. In particular, "bank transfer fraud" and "it's me" frauds targeting the elderly have become a social problem. In order to prevent these frauds, a system is needed that can detect possible fraud early and issue a warning to users. However, current systems have low fraud detection accuracy, making it difficult for users to recognize the possibility of fraud.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 3 is realized by the following means.

In this invention, the server includes means for detecting incoming fraudulent calls, means for listening to conversations from a voice input device installed in the communication terminal, means for converting the conversation data into text and sending it to the generative AI, means for issuing a voice alert from the voice input device if the generative AI determines that fraud is suspected from the conversation data, means for the generative AI to detect specific phrases or patterns that indicate the possibility of fraud, and means for issuing a warning to the user if the generative AI detects specific phrases or patterns that indicate the possibility of fraud. This makes it possible to detect the possibility of fraud with high accuracy and issue a warning to the user quickly.

"Means for detecting incoming fraudulent calls" refers to a function that identifies and notifies a communication terminal of incoming calls that may be fraudulent.

"A voice input device installed on a communication terminal" is a device for collecting voice that is attached to communication devices such as smartphones and home telephones.

"Means for converting conversation data into text and sending it to generative AI" refers to a function for converting voice data collected by a voice input device into text data and sending that text data to generative AI.

"Means for issuing a voice alert from the voice input device when the generative AI determines that fraud is suspected based on the conversation data" refers to a function that issues a warning sound through the voice input device when the generative AI determines that there is a possibility of fraud based on the conversation data analyzed.

"Means for the generative AI to detect specific phrases or patterns that indicate the possibility of fraud" refers to a function that allows the generative AI to detect typical fraudulent phrases and patterns that it has previously learned from conversation data.

"Means for issuing a warning to the user when the generative AI detects a specific phrase or pattern that indicates the possibility of fraud" refers to a function that issues a warning to the user when the generative AI detects a phrase or pattern that indicates the possibility of fraud.

As an embodiment of this invention, we will explain how to install a fraud detection assistant system on a smartphone.

First, the system is equipped with a means for detecting incoming fraudulent calls. This means has the function of identifying and notifying the communication terminal of incoming calls that may be fraudulent. Specifically, the content of the call is collected in real time using a voice input device installed on the communication terminal.

The collected voice data is then converted into text data through a means of transcribing the conversation data and sending it to the generative AI. This conversion is done using voice recognition software (e.g., Google's voice recognition API). The converted text data is then sent to the generative AI model.

Generative AI models are pre-trained to detect specific phrases and patterns that indicate potential fraud. These models are built using, for example, the Hugging Face transformers library. If the generative AI determines from the conversation data that fraud is suspected, it will warn the user by issuing an audio alert via a voice input device.

Furthermore, if the generative AI detects certain phrases or patterns that indicate potential fraud, a user alert mechanism will be activated. This mechanism will alert the user to the potential fraud by providing a visual or audio warning.

As a concrete example, consider the following audio file:

Voice: "Mom, please transfer the money right now. It's urgent."

Example prompt for parsing this audio file:

Judge whether the phrase "Mom, please transfer the money now. It's urgent." could be a scam.

By feeding this prompt into a generative AI model, it can analyze the possibility of fraud and issue a warning to the user.

The flow of the specific processing in Application Example 3 will be explained using Figure 16.

Step 1:

Detect incoming fraudulent calls

The server identifies and notifies the communication terminal of incoming calls that may be fraudulent. Specifically, it monitors the communication terminal's call log and detects possible fraud based on specific numbers or patterns. The input is call log data, and the output is a notification of a potentially fraudulent call.

Step 2:

Listen to conversations from a voice input device

The terminal collects call content in real time using an audio input device installed on the communication terminal. Specifically, audio data is acquired through a microphone and saved as an audio file. The input is the audio during the call, and the output is an audio file.

Step 3:

Conversation data is transcribed and sent to generative AI.

The device converts the collected voice data into text data and sends it to the generative AI. Specifically, it uses voice recognition software (e.g., Google's voice recognition API) to transcribe the voice data. The input is an audio file, and the output is text data.

Step 4:

Generative AI analyzes conversation data

The server uses a generative AI model to detect specific phrases and patterns in text data that indicate potential fraud. Specifically, it analyzes the text data using a generative AI model (e.g., Hugging Face's transformers library). The input is the text data, and the output is a determination of the likelihood of fraud.

Step 5:

Audible alerts when fraud is suspected

If the generative AI determines that there is a possibility of fraud, the device will issue an audio alert from the voice input device. Specifically, it will play a warning sound or message. The input is the result of the determination of the possibility of fraud, and the output is an audio alert.

Step 6:

Warn the user

If the generative AI detects certain phrases or patterns that indicate potential fraud, the device will alert the user by displaying a warning message on the screen or sending a notification. The input is the result of the determination of potential fraud, and the output is a warning to the user.

It is also possible to further combine an emotion engine that estimates the user's emotion. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

"Example 1"

One embodiment of the present invention is a system that combines a generative AI with an emotion engine. This system analyzes emotions from the user's voice tone, volume, speaking rate, etc. Specifically, if the user's voice tone or volume suddenly increases while they are receiving a scam call, the system senses that the user may be panicking. Based on this information, the generative AI determines that the possibility of a scam has increased further and issues an audio alert.

"Example 2"

In another embodiment, the system could issue an audio alert if the emotion engine exceeds a certain threshold. For example, if the user's tone of voice exceeds a certain threshold, or if the user's speaking speed exceeds a certain rate, the emotion engine could determine that the user is experiencing strong emotion. Based on this determination, the generative AI could issue an audio alert, warning the user of possible fraud.

"Example 3"

Furthermore, as another embodiment, a system in which the emotion engine and generative AI work in conjunction with each other is also possible. In this system, the emotion engine analyzes the user's emotions and sends the results to the generative AI. The generative AI combines information from the emotion engine with conversation data it has analyzed to determine the possibility of fraud. In this way, more accurate fraud detection is possible.

The processing flow for each example form is explained below.

"Example 1"

Step 1: The user receives a scam call.

Step 2: A smartphone app or a microphone installed on a home phone listens to the conversation.

Step 3: Convert the conversation data into text and send it to the generative AI.

Step 4: Generative AI analyzes the text data and determines the likelihood of fraud.

Step 5: The emotion engine analyzes the user's emotions based on their tone of voice, volume, speaking rate, etc.

Step 6: Based on the information from the emotion engine, the generative AI determines that the likelihood of fraud has increased and issues an audio alert.

"Example 2"

Step 1: The user receives a scam call.

Step 2: A smartphone app or a microphone installed on a home phone listens to the conversation.

Step 3: Convert the conversation data into text and send it to the generative AI.

Step 4: Generative AI analyzes the text data and determines the likelihood of fraud.

Step 5: The emotion engine analyzes the user's emotions based on their tone of voice, volume, speaking rate, etc.

Step 6: If the emotion engine exceeds a certain threshold, it notifies the generative AI.

Step 7: Based on the notification from the emotion engine, the generative AI determines that the likelihood of fraud has increased and issues an audio alert.

"Example 3"

Step 1: The user receives a scam call.

Step 2: A smartphone app or a microphone installed on a home phone listens to the conversation.

Step 3: Convert the conversation data into text and send it to the generative AI.

Step 4: Generative AI analyzes the text data and determines the likelihood of fraud.

Step 5: The emotion engine analyzes the user's emotions based on their tone of voice, volume, speaking rate, etc., and sends the results to the generative AI.

Step 6: The generative AI combines information from the emotion engine with the conversation data it has analyzed to determine the likelihood of fraud.

Step 7: If it determines there is a high possibility of fraud, the generative AI will issue an audio alert.

(Example 1)

Next, Example 1 of Form Example 1 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset-type terminal 314 will be referred to as the "terminal."

Conventional fraud call detection systems analyze only the content of conversations to determine the possibility of fraud, which means they are unable to take the user's emotional state into account, resulting in low fraud detection accuracy. Furthermore, they are unable to respond adequately when the user panics, making it difficult to prevent fraud from occurring.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

In this invention, the server includes means for detecting incoming fraudulent calls, means for listening to conversations from a voice input device installed in the communication terminal, means for converting the conversation data into text and transmitting it to the generative artificial intelligence, means for issuing a voice alert from the voice input device if the generative artificial intelligence determines that fraud is suspected from the conversation data, means for collecting parameters such as the user's voice tone, volume, and speaking rate and analyzing their emotions, and means for reassessing the possibility of fraud based on the results of the emotion analysis. This makes it possible to detect fraud while taking the user's emotional state into account, thereby preventing fraud damage before it occurs.

"Means for detecting incoming fraudulent calls" refers to equipment or software that detects whether an incoming call to a communications terminal is likely to be fraudulent.

"A voice input device installed on a communication terminal" refers to a device for inputting voice, such as a microphone attached to a communication device such as a smartphone or home telephone.

"Means for converting conversation data into text and sending it to generative AI" refers to a device or software that converts voice data into text data and sends that text data to generative AI.

"Generative AI" is an artificial intelligence system that analyzes received data and determines the likelihood of fraud by detecting specific patterns and phrases.

"A means for issuing an audio alert" is a device or software that plays an audio message to warn the user if it determines that there is a high possibility of fraud.

"Means for collecting parameters such as the user's voice tone, volume, and speaking rate, and analyzing emotions" refers to a device or software that collects information such as tone, volume, and speaking rate from the user's voice data, and analyzes the user's emotional state based on that information.

"Means for reassessing the possibility of fraud based on the results of sentiment analysis" refers to a device or software that inputs the results of sentiment analysis into generative artificial intelligence and reassess the possibility of fraud.

Form for implementing the invention

This invention is a system that detects fraudulent phone calls and warns users, and is implemented using an application and server installed on a communications terminal. This system detects incoming fraudulent calls, listens to the conversation, transcribes the conversation data, and sends it to a generative artificial intelligence. Furthermore, it aims to prevent fraud by analyzing the user's emotional state and reassessing the possibility of fraud.

Hardware and software used

1. Communication terminal

Communication devices such as smartphones and home phones.

A microphone as an audio input device.

2. Server

Speech recognition software (e.g., Google Cloud Speech-to-Text API) is used to transcribe audio data.

For sentiment analysis, we use an emotion engine (e.g., IBM Watson Tone Analyzer).

For generative artificial intelligence, we use AI models with natural language processing technology (e.g., OpenAI's GPT-4).

Specific system operation

1. Incoming phone call detection

The server detects incoming phone calls through an application installed on the communication device. The application uses the communication device's native API to catch incoming phone call events.

2. Listening to Conversations

The device uses a microphone to listen to phone conversations. The application captures the microphone's audio input in real time and saves it as audio data.

3. Transcription of audio data

The server converts the audio data it hears into text data. This process involves calling the Google Cloud Speech-to-Text API to convert the audio data into text.

4. Sending to generative AI

The server sends the transcribed conversation data to the generative AI. The server sends the text data to the generative AI using an HTTP request.

5. Fraud Detection

The generative AI analyzes the received conversation data and determines whether fraud is suspected. Using natural language processing technology, the generative AI analyzes the content of the conversation and evaluates whether it matches any fraudulent patterns.

6. Sentiment Analysis

The server collects parameters such as the user's voice tone, volume, and speaking rate and analyzes them using an emotion engine. The server uses voice analysis software to extract emotion parameters from the voice data.

7. Panic Detection

If the user's tone or volume of voice suddenly increases, the server determines that the user may be panicking. The server analyzes data from the emotion engine and evaluates whether there is a sudden change.

8. Reassessing the possibility of fraud

Based on information from the emotion engine, the generative AI determines that the likelihood of fraud has increased. The generative AI receives the emotion data as additional input and reassess the likelihood of fraud.

9. Audio alerts

If the server determines that there is a high possibility of fraud, it instructs the communication device to play an audio alert from its speaker. The server then sends a command to the communication device's application to play the audio alert.

Examples and prompts

As a concrete example, imagine a user is receiving a phone call on their smartphone. If the person on the other end of the line says, "Your bank account has been fraudulently used. Please tell us your account number and password immediately," the user's voice tone will suddenly rise and the volume will also increase. Based on this information, the generative artificial intelligence will determine that there is a high possibility of fraud, and will emit an audio alert from the communication device's speaker saying, "This may be a fraud. Please be careful."

Example prompt:

"Analyze the conversation data below to determine if it may be a scam. Also consider changes in the user's tone and volume of voice.

Conversation data: 'Your bank account has been compromised. Please provide your account number and password immediately.'

User's voice tone: Sudden rise

User volume: Increased"

The flow of the identification process in Example 1 will be explained using Figure 17.

Step 1:

Incoming phone call detection

The server detects incoming phone calls through an application installed on the communication terminal.

Input: Incoming call event from communication terminal.

Data processing/calculation: Capture incoming call events using the communication device's native API.

Output: Event information indicating that an incoming call was detected.

Specific behavior: The application uses native Android and iOS APIs to catch incoming phone calls and notify the server.

Step 2:

Listening to conversations

The device uses a microphone to listen to the conversation during the call.

Input: Audio data during the call.

Data processing/calculation: Captures microphone audio input in real time and saves it as audio data.

Output: Audio data.

Specific behavior: The application captures audio input from the microphone in real time and saves it as audio data.

Step 3:

Transcription of audio data

The server converts the audio data it hears into text data.

Input: Audio data.

Data processing/calculation: Call the Google Cloud Speech-to-Text API to convert voice data into text.

Output: Character data.

Specific operation: The server converts the audio data into text using the Google Cloud Speech-to-Text API.

Step 4:

Send to generative AI

The server sends the transcribed conversation data to the generative artificial intelligence.

Input: Character data.

Data processing/calculation: Send text data to generative artificial intelligence using an HTTP request.

Output: Result of request sent to generative artificial intelligence.

Specific operation: The server sends text data to the generative artificial intelligence using an HTTP request.

Step 5:

Fraud detection

The generative artificial intelligence analyzes the received conversation data and determines whether fraud is suspected.

Input: Character data.

Data processing/calculation: Using natural language processing technology, the content of the conversation is analyzed and evaluated to see if it matches a fraud pattern.

Output: Assessment result regarding likelihood of fraud.

Specific operation: The generative artificial intelligence uses natural language processing technology to analyze the content of the conversation and evaluate whether it matches any fraud patterns.

Step 6:

Emotion analysis

The server collects parameters such as the user's voice tone, volume, and speaking speed, and analyzes them using an emotion engine.

Input: Audio data.

Data processing/calculation: Use voice analysis software to extract emotional parameters from voice data.

Output: Emotion parameters.

Specific operation: The server uses voice analysis software (e.g., IBM Watson Tone Analyzer) to extract emotion parameters from the voice data.

Step 7:

Detecting panic states

If the user's voice tone or volume suddenly increases, the server will determine that the user may be panicking.

Input: Emotion parameters.

Data processing/calculation: Analyze data from the emotion engine and evaluate whether there are any sudden changes.

Output: Panic state assessment results.

Specific behavior: The server analyzes data from the emotion engine and evaluates whether there are any sudden changes.

Step 8:

Reassessing the possibility of fraud

Based on information from the emotion engine, the generative artificial intelligence determines that the likelihood of fraud has increased.

Input: Emotion parameters.

Data processing/calculation: Emotional data is taken as additional input and the likelihood of fraud is reassessed.

Output: Reassessed fraud probability.

Specific behavior: The generative AI receives emotional data as additional input and reassesss the likelihood of fraud.

Step 9:

Send audio alerts

If the server determines that there is a high possibility of fraud, it will instruct the communication device to emit an audio alert from its speaker.

Input: Reassessed potential fraud.

Data processing/calculation: Sends a command to the communication terminal application to play an audio alert.

Output: Plays an audio alert.

Specific operation: The server sends a command to the application on the communication device to play an audio alert, which is then played from the communication device's speaker.

(Application Example 1)

Next, we will explain Application Example 1 of Form Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the headset-type terminal 314 will be referred to as the "terminal."

In recent years, fraudulent phone scams have become more sophisticated, with many people falling victim to them. Elderly people and those unfamiliar with technology are particularly prone to falling victim to these scams, as they are less wary of them. Furthermore, when users receive a fraudulent call, they often panic, making it difficult for them to make a calm judgment. There is a need for effective measures to improve this situation and prevent fraudulent incidents.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

In this invention, the server includes means for detecting incoming fraudulent calls, means for listening to conversations from a voice input device installed in the communication terminal, means for converting the conversation data into text and sending it to the generative AI, means for issuing a voice alert from the voice input device if the generative AI determines that fraud is suspected from the conversation data, means including an emotion engine for analyzing emotions from the user's voice tone, volume, speaking speed, etc., and means for further increasing the possibility of fraud based on the analysis results of the emotion engine. In this way, by combining fraudulent call detection with user emotion analysis, it is possible to prevent fraud victims before they occur.

A "fraudulent call" is a call made with the intent of defrauding someone of money or personal information through fraudulent means.

"Incoming call" refers to an incoming phone call.

A "communications terminal" is an electronic device used to send and receive voice and data.

An "audio input device" is a device for converting audio into digital data.

"Conversation data" refers to data that has been converted into text from speech captured by a voice input device.

"Generative AI" is artificial intelligence that makes specific decisions and generates results based on input data.

"Audio alert" is a means of warning or notifying someone by voice.

An "emotion engine" is software or a system for analyzing emotions from voice data.

"Tone" refers to the pitch and texture of the voice.

"Volume" refers to the loudness of the sound.

"Speech rate" refers to the speed at which you speak.

The following system configuration is used to implement this invention.

System Configuration

1. Hardware

Communication terminal: A device capable of voice communication, such as a smartphone or home telephone.

Audio input device: A microphone built into the communication terminal.

Audio output device: Speaker built into the communication terminal.

2. Software

Speech Recognition Library: Use the speech_recognition library to convert speech to text.

Generative AI model: An AI model that uses the transformers library to determine the likelihood of fraud.

Emotion Engine: A model for analyzing user emotions using the transformers library.

Audio Alert Library: Uses the pyttsx3 library to emit audio alerts.

Processing flow

1. Incoming call detection

The server detects that a call has been received by the communication terminal.

The communication terminal's voice input device listens to the conversation and acquires voice data.

2. Voice Recognition

The server uses a voice recognition library to convert the acquired voice data into text data.

3. Fraud Detection Using Generative AI

The server inputs the text data into a generative AI model to determine the likelihood of fraud.

If certain phrases or patterns are detected, it is determined that there is a high possibility of fraud.

4. Sentiment analysis

The server uses an emotion engine to analyze the user's emotions based on their tone of voice, volume, speaking speed, etc.

If a user is perceived to be in a panic, the likelihood of fraud increases even more.

5. Audio alerts

If the server determines that there is a high possibility of fraud, it uses the audio alert library to issue an audio alert from the communication terminal's audio output device.

The alert is intended to alert users to potential fraud.

Specific examples

For example, if a user receives a phone call and it is determined that the call may be a scam, an audio alert will sound from the communication device's speaker saying, "Warning! Possible scam." This alert will make the user aware of the possibility of fraud and allow them to respond calmly.

Example prompt

"Could this call be a scam?"

In this way, this invention combines the detection of fraudulent calls with user sentiment analysis, making it possible to prevent fraudulent incidents.

The flow of the specific processing in Application Example 1 is explained using Figure 18.

Step 1:

The server detects that a call has been received by the communication terminal.

Input: Incoming signal from communication terminal.

Data processing: Receives incoming call signals and begins processing when the call starts.

Output: Call start flag.

Specific operation: When a communication terminal receives an incoming call, the server detects the signal and recognizes that a call has started.

Step 2:

The device's voice input device listens to the conversation and captures the voice data.

Input: Audio during a call.

Data processing: The voice input device collects voice data in real time.

Output: Captured audio data.

Specific operation: Audio during a call is collected through the microphone and saved as audio data.

Step 3:

The server uses a voice recognition library to convert the acquired voice data into text data.

Input: Acquired audio data.

Data processing: Use the speech recognition library (speech_recognition) to convert audio data into text data.

Output: Converted text data.

Specific operation: Voice data is input into the voice recognition library and output as text data.

Step 4:

The server inputs the text data into a generative AI model to determine the likelihood of fraud.

Input: Converted text data.

Data processing: Text data is fed into generative AI models (transformers) to assess the likelihood of fraud.

Output: Assessment results regarding the likelihood of fraud.

Specific operation: Text data is input into a generative AI model, which outputs an evaluation result indicating whether the likelihood of fraud is high.

Step 5:

The server uses an emotion engine to analyze the user's emotions based on their tone of voice, volume, speaking speed, etc.

Input: Converted text and audio data.

Data processing: Analyze user emotions using emotion engines (transformers).

Output: Analysis results on user sentiment.

Specific operation: Voice and text data are input into the emotion engine, and the user's emotional state is analyzed.

Step 6:

If the server determines that there is a high possibility of fraud, it uses the audio alert library to issue an audio alert from the communication terminal's audio output device.

Input: Fraud likelihood assessment results and user sentiment analysis results.

Data processing: If a high possibility of fraud is determined, an audio alert is generated using the audio alert library (pyttsx3).

Output: Audio alert.

Specific operation: If it is determined that there is a high possibility of fraud, an audio alert will be emitted from the communication device's speaker saying, "Warning! Possible fraud."

(Example 2)

Next, Example 2 of Form Example 2 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset-type terminal 314 will be referred to as the "terminal."

Conventional fraud prevention systems had low accuracy in detecting potential fraud, making it impossible to completely eliminate the risk of users falling victim to fraud. Furthermore, warnings were issued without taking the user's emotional state into consideration, which sometimes led to users being unable to respond appropriately. This made it difficult to prevent fraud from occurring.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

In this invention, the server includes means for detecting incoming fraudulent calls, means for listening to conversations from a voice input device installed in the communication device, means for converting the conversation data into text and sending it to the generative AI model, means for issuing an audio alert from the voice input device if the generative AI model determines that fraud is suspected from the conversation data, means for an emotion analysis engine to analyze the user's tone of voice and speaking rate, and means for the generative AI model to issue an audio alert if the emotion analysis engine exceeds a specific threshold. This makes it possible to detect the possibility of fraud with high accuracy and issue an appropriate warning according to the user's emotional state.

"Means for detecting incoming fraudulent calls" refers to a function that identifies and notifies a communication device of incoming calls that may be fraudulent.

A "voice input device" is a device that records conversations between a user and another person in real time and stores them in digital format.

"Means for converting conversation data into text and sending it to the generative AI model" refers to a function for converting voice data into text data and sending that text data to the generative AI model.

A "generative AI model" is an artificial intelligence system that analyzes input data, detects specific patterns and phrases, and determines the likelihood of fraud.

"Means for issuing audio alerts" refers to an audio notification function that warns users if a potential fraud is detected.

The "emotion analysis engine" is an engine that analyzes the user's tone of voice and speaking speed to determine their emotional state.

"When a specific threshold is exceeded" refers to when the emotion analysis engine analyzes the user's emotional state and it exceeds a pre-set reference value.

"Communication equipment" refers to devices used for making voice calls, such as home telephones and smartphones.

Form for implementing the invention

This invention is a system that detects incoming fraudulent calls and issues a warning to the user. A specific embodiment of this system is described below.

Hardware and software used

Communication devices: devices used for voice calls, such as home phones and smartphones.

Audio input device: A microphone installed in the communication device

Server: A computer system for processing voice data and running generative AI models and sentiment analysis engines.

Generative AI model: An artificial intelligence system that analyzes conversation data and determines the likelihood of fraud.

Emotion analysis engine: An engine that analyzes the user's tone of voice and speaking rate to determine their emotional state.

Speech recognition software: Software for converting voice data into text data (e.g., Google Speech-to-Text API)

System Operation

1. When a user starts a conversation on a communication device, the device's voice input device records the conversation in real time. The recorded voice data is saved in digital format.

2. The server uses speech recognition software to convert the collected voice data into text. For example, the speech "Hello, this is a bank representative" is converted into text "Hello, this is a bank representative."

3. The server sends the converted text data to the generative AI model. The generative AI model receives this data and prepares it for analysis.

4. The generative AI model analyzes the conversation data and detects specific keywords and phrases (e.g., "bank account" or "password"), which determines whether there is potential for fraud.

5. If the generative AI model determines that there is a high possibility of fraud, the server will send instructions to the terminal and an audio alert will be issued from the communication device's speaker saying, "There is a possibility of fraud. Please be careful."

6. The emotion analysis engine analyzes the user's tone of voice and speaking rate in real time. For example, if the user's voice suddenly gets higher in pitch or their speaking rate increases, the emotion analysis engine will detect this.

7. If the sentiment analysis engine determines that the user's emotions exceed a certain threshold, the generative AI model will issue an audio alert saying, "This may be a scam. Please remain calm."

Specific examples

Example 1: When a user says over the phone, "Please tell me your bank account information," the generative AI model detects the keyword "bank account" and determines that there is a possibility of fraud. The speaker on the communication device issues a voice alert saying, "There is a possibility of fraud. Please be careful."

Example 2: If a user says "Please tell me my password urgently" over the phone and their voice tone gets higher and faster, the emotion analysis engine determines that the user is experiencing strong emotions. The generative AI model issues a voice alert, warning, "This may be a scam. Please remain calm."

Example prompt

Prompt 1: "If someone calls and asks for your bank account information, determine whether it's a scam."

Prompt 2: "Please explain how the emotion engine determines if the user's voice tone gets higher and their speaking rate gets faster."

The system is designed to reduce the risk of users falling victim to telephone fraud. By combining a generative AI model with a sentiment analysis engine, it can detect potential fraud with greater accuracy and alert users.

The flow of the identification process in Example 2 will be explained using Figure 19.

Step 1:

The user begins a conversation on a communication device.

Specific operations involve a user initiating a call using a home phone or smartphone. The input is the user's voice, and the output is voice data collected by a voice input device.

Step 2:

The device's voice input device listens to the conversation and collects voice data.

Specifically, the device's microphone records the conversation between the user and the other party in real time. The input is the user's and the other party's voice, and the output is audio data stored in digital format.

Step 3:

The server converts the audio data into text data.

Specifically, the server uses speech recognition software (e.g., Google Speech-to-Text API) to convert the collected voice data into text data. The input is digital voice data, and the output is text conversation data.

Step 4:

The server sends the transcribed conversation data to the generative AI model.

Specific operations involve the server sending the converted text data to the generative AI model. The input is text-format conversation data, and the output is the data sent to the generative AI model.

Step 5:

A generative AI model analyzes conversation data to determine the likelihood of fraud.

Specifically, the generative AI model analyzes conversation data and detects specific keywords and phrases (e.g., "bank account" or "password"). The input is text-based conversation data, and the output is a judgment about the likelihood of fraud.

Step 6:

If there is a possibility of fraud, an audio alert will be issued via the device's voice input device.

Specifically, if the generative AI model determines that there is a high possibility of fraud, the server sends instructions to the terminal and an audio alert is emitted from the communication device's speaker saying, "There is a possibility of fraud. Please be careful." The input is the judgment result of the generative AI model, and the output is an audio alert.

Step 7:

The emotion analysis engine analyzes the user's tone of voice and speaking speed.

Specifically, the emotion analysis engine analyzes the user's tone of voice and speaking rate in real time. The input is the user's voice data, and the output is the analysis results regarding their emotional state.

Step 8:

If the sentiment analysis engine exceeds a certain threshold, the generative AI model will issue an audio alert.

Specifically, if the emotion analysis engine determines that the user's emotion exceeds a certain threshold, the generative AI model issues an audio alert saying, "This may be a scam. Please remain calm." The input is the analysis result of the emotion analysis engine, and the output is an audio alert.

(Application Example 2)

Next, we will explain Application Example 2 of Form Example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the headset-type terminal 314 will be referred to as the "terminal."

In recent years, fraudulent phone scams have become increasingly sophisticated, leaving many people vulnerable. Elderly people and those unfamiliar with technology are particularly vulnerable to these scams, as they are less vigilant against them. While systems for detecting fraudulent calls exist, few warning systems take the user's emotional state into account. This increases the risk of users becoming victims of fraudulent calls without realizing they are being scammed. Therefore, there is a need for a system that detects fraudulent calls and takes the user's emotional state into account.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

In this invention, the server includes means for detecting incoming fraudulent calls, means for listening to conversations from a voice input device installed in the smart device, means for converting the conversation data into text and sending it to the generative AI, means for issuing a voice alert from the voice input device if the generative AI determines that fraud is suspected from the conversation data, and means for analyzing the user's emotions using an emotion engine and issuing a warning if a specific threshold is exceeded. This makes it possible to detect fraudulent calls and issue warnings that take into account the user's emotional state.

A "fraudulent call" is a call made with the intent of defrauding someone of money or personal information through fraudulent means.

"Incoming call" refers to an incoming phone call.

A "smart device" is an electronic device with internet connectivity, such as a smartphone or tablet.

An "audio input device" is a device such as a microphone that converts audio into digital data.

"Conversation data" refers to data that has been converted into text from speech acquired through a voice input device.

"Generative AI" is a system that uses artificial intelligence to analyze data and make specific decisions or generate results.

"Audio alert" is a means of warning or notifying the user using audio.

An "emotion engine" is an algorithm or system that analyzes a user's emotional state from voice and text data.

A "specific threshold" is a numerical value or condition that serves as a benchmark when the emotion engine evaluates the user's emotional state.

A "warning" is a notification or alert that alerts the user.

A system for implementing this invention includes a series of means for detecting incoming fraudulent calls and issuing a warning to the user. A specific embodiment of the system is described below.

First, a voice input device (microphone) is installed on a smart device (e.g., a smartphone or tablet). This voice input device captures the user's call content in real time. The captured voice data is converted into text data using voice recognition software (e.g., the speech_recognition library).

The transcribed conversation data is then sent to a generative AI model (e.g., a model using the transformers library). This generative AI model analyzes the conversation data and determines whether there is a possibility of fraud. If a fraudulent activity is detected, software for issuing audio alerts (e.g., the pyttsx3 library) is activated to warn the user.

Furthermore, an emotion engine (e.g., an emotion analysis model using the transformers library) is used to analyze the user's emotional state. The emotion engine evaluates the user's tone of voice, speaking rate, etc., and issues a warning if certain thresholds are exceeded. This warning is also notified to the user as an audio alert.

For example, if a user calls and says, "Please tell me your bank account information," the generative AI model will detect the possibility of fraud and issue a voice alert. Also, if the user is feeling anger or anxiety, the emotion engine will detect this and issue a voice alert.

Examples of prompt statements include:

User's call: "Please tell me my bank account information."

Prompt for generative AI model: "Is this call potentially fraudulent?"

In this way, it is possible to create a system that can detect fraudulent calls and issue warnings that take into account the user's emotional state.

The flow of the specific processing in Application Example 2 is explained using Figure 20.

Step 1:

A user initiates a call on a smart device. The smart device's audio input device (microphone) captures the call content in real time. The input is the user's voice data, and the output is a stream of voice data.

Step 2:

The device uses speech recognition software (speech_recognition library) to convert the acquired voice data into text data. The input is a stream of voice data, and the output is transcribed conversation data.

Step 3:

The device sends the transcribed conversation data to a generative AI model (a model using the transformers library). The generative AI model analyzes the conversation data and determines whether there is a possibility of fraud. The input is the transcribed conversation data, and the output is a judgment regarding the possibility of fraud.

Step 4:

The server receives the judgment results of the generative AI model, and if it determines that there is a possibility of fraud, it launches software (pyttsx3 library) to issue an audio alert. The input is the judgment result regarding the possibility of fraud, and the output is an audio alert.

Step 5:

The device uses an emotion engine (an emotion analysis model using the Transformers library) to analyze the user's emotional state. The emotion engine evaluates the user's tone of voice, speaking rate, etc., and issues a warning if certain thresholds are exceeded. The input is transcribed conversation data, and the output is an evaluation result regarding the user's emotional state.

Step 6:

The server receives the emotion engine's evaluation results, and if a certain threshold is exceeded, it launches software (pyttsx3 library) to issue an audio alert. The input is the evaluation result regarding the emotional state, and the output is an audio alert.

Step 7:

The user receives an audio alert and acknowledges the warning about potential fraud or emotional state. The input is the audio alert and the output is the user's acknowledgment.

(Example 3)

Next, we will explain Example 3 of Form Example 3. In the following explanation, the data processing device 12 will be referred to as the "server" and the headset-type terminal 314 will be referred to as the "terminal."

Conventional fraud detection systems have the problem of low accuracy because they only analyze conversation data to determine the possibility of fraud. Furthermore, because they do not take the user's emotional state into account, there is a high risk of overlooking possible fraud. Furthermore, there are limited ways to notify users of possible fraud, which can lead to users not realizing the risk of fraud.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Example 3 is realized by the following means.

In this invention, the server includes means for detecting incoming fraudulent calls, means for listening to conversations from a voice input device installed in the communication terminal, means for converting the conversation data into text and transmitting it to the generative artificial intelligence, means for issuing a voice alert from the voice input device if the generative artificial intelligence determines that fraud is suspected from the conversation data, means for an emotion analysis engine to analyze the user's emotions, and means for the generative artificial intelligence to integrate information from the emotion analysis engine and the conversation data to determine the possibility of fraud. This makes it possible to detect the possibility of fraud with high accuracy and notify the user promptly.

"Means for detecting incoming fraudulent calls" refers to devices or software that monitor communication lines and identify incoming calls that may be fraudulent.

"A voice input device installed on a communication terminal" refers to a microphone attached to a communication device such as a smartphone or home telephone for collecting voice.

"Means for converting conversation data into text and sending it to generative AI" refers to software or a device for converting voice data into text data and sending that text data to generative AI.

"Generative AI" is an artificial intelligence model that uses natural language processing technology to analyze text data and detect specific patterns and phrases.

"Means for issuing audio alerts" refers to an audio output device that issues a warning to the user if the generative artificial intelligence detects the possibility of fraud.

An "emotion analysis engine" is software or a device that analyzes a user's voice and text data to identify their emotional state.

"Means for determining the likelihood of fraud" refers to an algorithm or software that uses generative artificial intelligence to integrate conversational data and information from a sentiment analysis engine to assess the likelihood of fraud.

This invention is a system that detects incoming fraudulent calls and notifies users of the possibility of fraud. A specific embodiment of this system is described below.

First, while a user is using a communication terminal (such as a smartphone or home phone), a means for detecting incoming fraudulent calls is activated. This means is a device or software that monitors communication lines and identifies incoming calls that may be fraudulent.

Next, a voice input device (microphone) installed on the communication terminal listens to the conversation between the user and the caller making the fraudulent call. This voice data is converted into text data by transcribing the conversation data and sending it to generative artificial intelligence. This conversion uses voice recognition technology. Specifically, voice recognition services such as Google Speech-to-Text API and IBM Watson Speech to Text can be used.

The server analyzes the text data using generative artificial intelligence (for example, natural language processing models such as BERT or GPT-3). The generative artificial intelligence detects specific phrases and patterns that may be fraudulent and evaluates the results. For example, if the phrase "Please transfer money" is included, it will determine that there is a high possibility of fraud.

Furthermore, the server uses an emotion analysis engine (for example, IBM Watson or Microsoft Azure's emotion analysis API) to analyze the user's emotions. It analyzes the user's voice and text data to identify emotional states such as anxiety or tension. The emotion data obtained by the emotion analysis engine is sent to the generative artificial intelligence.

Generative AI combines information from the sentiment analysis engine with conversational data to more accurately assess the likelihood of fraud. For example, if the message is "Please transfer money" and the user's emotions are anxious, it will conclude that there is a very high possibility of fraud.

Finally, if the server determines that the message is likely to be fraudulent, it will notify the user of this result. The notification is made by issuing an audio alert. Specifically, an audio alert will be issued from the communication device's speaker saying, "This message may be fraudulent. Please be careful."

As a concrete example, consider the case where a user receives a message saying, "Please transfer money." This message is picked up by the voice input device of the communication device and converted into text data. The server analyzes this text data using generative artificial intelligence and determines that there is a high possibility of fraud. At the same time, the emotion analysis engine detects the user's anxiety and sends this information to the generative artificial intelligence. The server combines this information and concludes that there is a very high possibility of fraud. Finally, the server sends an audio alert to the user's communication device saying, "This message may be fraudulent. Please be careful."

Example prompt:

"Please determine whether the following message is likely a scam. Message: 'Please transfer money.'"

In this way, the server, terminal, and user work together to detect the possibility of fraud with high accuracy and quickly notify the user. The flow of the identification process in Example 3 will be explained using Figure 21.

Step 1:

The user enters a message.

The user uses a communication device (such as a smartphone or home phone) to input the received message. For example, they might input a message such as "Please transfer money." The input message is saved on the device as voice data.

Step 2:

The device sends a message to the server.

The terminal sends the voice data entered by the user to the server. At this time, the terminal sends the voice data to the server via the Internet. The input is voice data, and the output is the transfer of voice data to the server.

Step 3:

The server converts the audio data into text data.

The server converts the received voice data into text data using voice recognition technology. Specifically, it uses voice recognition services such as Google Speech-to-Text API and IBM Watson Speech to Text. The input is voice data, and the output is text data.

Step 4:

The server analyzes the text data using generative artificial intelligence.

The server analyzes the text data using generative artificial intelligence (e.g., natural language processing models such as BERT or GPT-3). The generative artificial intelligence detects specific phrases and patterns that may be fraudulent and evaluates the results. The input is the text data, and the output is an evaluation result regarding the likelihood of fraud.

Step 5:

The server analyzes the user's emotions using an emotion analysis engine.

The server uses an emotion analysis engine (for example, IBM Watson or Microsoft Azure's emotion analysis API) to analyze the user's emotions. It analyzes the user's voice and text data to identify emotional states such as anxiety or tension. The input is voice or text data, and the output is emotion data.

Step 6:

The server combines the results of the generative artificial intelligence and the sentiment analysis engine to determine the likelihood of fraud.

The server integrates the analysis results from the generative AI with the emotional data from the emotion analysis engine. This allows for a more accurate assessment of the likelihood of fraud. For example, if the message is "Please transfer money" and the user's emotions are anxious, it will conclude that there is a very high likelihood of fraud. The input is the analysis results and emotional data, and the output is a final assessment of the likelihood of fraud.

Step 7:

The server will notify the user of possible fraud.

If the server determines that there is a high possibility of fraud, it notifies the user of the result. The notification is done by issuing an audio alert. Specifically, an audio alert saying "This message may be fraudulent. Please be careful" is issued from the speaker of the communication terminal. The input is the final assessment of the possibility of fraud, and the output is the audio alert.

(Application Example 3)

Next, we will explain Application Example 3 of Configuration Example 3. In the following explanation, the data processing device 12 will be referred to as the "server" and the headset-type terminal 314 will be referred to as the "terminal."

Traditional fraud detection systems only detect specific phrases and patterns and do not take the user's emotional state into account, making it difficult to accurately determine the likelihood of fraud. Furthermore, there was a lack of means to provide appropriate warnings to users when fraud was likely.

The identification processing by the identification processing unit 290 of the data processing device 12 in Application Example 3 is realized by the following means. In this invention, the server includes means for detecting incoming fraudulent calls, means for listening to conversations from a smartphone application or a microphone installed in a home phone, means for converting the conversation data into text and sending it to the generative AI, means for issuing an audio alert from the microphone if the generative AI determines that fraud is suspected based on the conversation data, means for analyzing the user's emotions using an emotion engine, means for determining the possibility of fraud by combining information from the emotion engine with the conversation data analyzed by the generative AI, and means for displaying a warning to the user if there is a high possibility of fraud. This makes it possible to more accurately determine the possibility of fraud and provide the user with an appropriate warning.

"Means for detecting incoming fraudulent calls" refers to devices or software that monitor telephone lines or communications networks and detect incoming calls that may be fraudulent.

"A microphone installed in a smartphone application or home phone" is a voice input device attached to a smartphone or home phone to collect audio.

"Means for listening to conversations" refers to devices or software that analyze audio data collected through a microphone and recognize the content of the conversation.

"Means for converting conversation data into text and sending it to the generative AI" refers to a device or software that converts voice data into text data and sends that text data to the generative AI.

"Generative AI" is an artificial intelligence model trained to perform a specific task, in this case to determine the likelihood of fraud.

"Means for issuing an audio alert from the microphone when fraud is suspected" refers to a device or software that issues an alarm or message through the microphone when the generative AI detects the possibility of fraud.

An "emotion engine" is software or a device that analyzes a user's emotional state from their voice or text data.

"Means for analyzing user emotions" refers to devices or software that use an emotion engine to analyze the user's emotional state.

"Means for determining the possibility of fraud by combining information from the emotion engine with conversation data analyzed by generative AI" refers to devices or software that integrate emotional information obtained from the emotion engine with the results of analysis of conversation data by generative AI, and make a comprehensive determination of the possibility of fraud.

"Means for displaying a warning to the user in the event of a high likelihood of fraud" refers to devices or software that provide a visual or audible warning to the user in the event that a high likelihood of fraud is determined.

A system for implementing this invention includes a series of means for detecting incoming fraudulent calls and providing a warning to the user. Specific embodiments are described below.

First, the server is equipped with a means for detecting incoming fraudulent calls. This means is a device or software that monitors telephone lines and communication networks to detect potentially fraudulent calls. For example, specific phone numbers or callers can be registered on a blacklist, and fraudulent calls can be detected based on this.

Next, the device (smartphone application or home phone) is equipped with a microphone to pick up on the conversation. This microphone collects the voice during the call and sends it to the server as conversation data. The conversation data is then converted into text using voice recognition technology and sent to the generative AI.

Generative AI analyzes conversation data to determine the likelihood of fraud. In doing so, it uses models trained to detect specific phrases and patterns. For example, it can detect typical phrases such as "bank transfer fraud" and "it's me" fraud.

Furthermore, an emotion engine analyzes the user's emotions. The emotion engine is software that analyzes the user's emotional state from voice and text data. The emotional information obtained from the emotion engine is integrated with the results of analysis of conversation data by generative AI to make a comprehensive judgment on the possibility of fraud.

If it determines that there is a high possibility of fraud, the device will display a warning to the user. This warning can be provided visually or audibly. For example, a warning message can be displayed on the smartphone screen or an audio alert can be issued.

For example, consider the following user input:

"Your account has been frozen. Please transfer the funds immediately."

"Hey, I need money right now."

By inputting these prompts into a generative AI model, it can analyze the possibility of fraud and, in conjunction with an emotion engine, display a warning to the user.

The flow of the specific processing in Application Example 3 will be explained using Figure 22.

Step 1:

The server monitors telephone lines and communication networks to detect incoming calls that may be fraudulent. The input is a signal from the telephone line or communication network, and the output is a flag indicating an incoming fraudulent call. Specifically, it registers specific phone numbers and callers on a blacklist and detects fraudulent calls based on that.

Step 2:

The device (smartphone application or home phone) collects audio during a call through a microphone. The input is the audio signal during the call, and the output is audio data. Specifically, the microphone converts the audio into a digital signal and sends that data to a server.

Step 3:

The server uses voice recognition technology to convert the voice data into text. The input is voice data and the output is text data. Specifically, the voice recognition software analyzes the voice data and converts it into the corresponding text.

Step 4:

The server sends text data to the generative AI, which analyzes it for the possibility of fraud. The input is text data, and the output is a score or flag indicating the possibility of fraud. Specifically, the generative AI model detects specific phrases and patterns and determines the possibility of fraud.

Step 5:

The server uses an emotion engine to analyze the user's emotions. The input is text data, and the output is data indicating the user's emotional state. Specifically, the emotion engine analyzes the text data and identifies the user's emotional state.

Step 6:

The server combines information from the emotion engine with conversation data analyzed by the generative AI to make a comprehensive judgment on the possibility of fraud. The input is emotional state data and a score or flag indicating the possibility of fraud, and the output is a final flag indicating the possibility of fraud. Specifically, it integrates emotional information with the analysis results of the generative AI to make a highly accurate judgment on the possibility of fraud.

Step 7:

If it determines that there is a high possibility of fraud, the device will display a warning to the user. The input is a flag indicating the final possibility of fraud, and the output is a warning message or audio alert. Specifically, a warning message is displayed on the smartphone screen or an audio alert is issued.

The specific processing unit 290 transmits the results of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the results of the specific processing. The microphone 238 acquires audio indicating the user input regarding the results of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

Data generation model 58 is what is known as generative AI (artificial intelligence). An example of data generation model 58 is generative AI such as ChatGPT (Internet search <URL: https://openai.com/blog/chatgpt>). Data generation model 58 is obtained by performing deep learning on a neural network. A prompt containing an instruction is input to data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. Data generation model 58 performs inference on the input inference data in accordance with the instructions indicated by the prompt, and outputs the inference results in the form of data such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and/or summarization.

Another example of generative AI is Gemini (Internet search <URL: https://gemini.google.com/?hl=ja>).

In the above embodiment, an example was given in which the specific processing was performed by the data processing device 12, but the technology disclosed herein is not limited to this, and the specific processing may also be performed by the headset-type terminal 314.

[Fourth embodiment]

Figure 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

As shown in FIG. 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

The data processing device 12 includes a computer 22, a database 24, and a communication I/F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and communication I/F 26 are also connected to the bus 34. The communication I/F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and/or a LAN (Local Area Network).

The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I/F 44, and a control target 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and control target 443 are also connected to the bus 52.

The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the user 20's surroundings (e.g., an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

The communication I/F 44 is connected to the network 54. The communication I/Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I/Fs 44 and 26 is carried out in a secure manner.

The control object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, in the data processing device 12, specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32.

The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as the specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

Storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

In the robot 414, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

Next, we will explain the identification process performed by the identification processing unit 290 of the data processing device 12.

"Example 1"

One embodiment of the present invention uses a smartphone application. In this case, the application installed on the smartphone detects an incoming phone call and listens to the conversation through the microphone. The listened-to conversation data is converted into text and sent to the generative AI. The generative AI determines from the conversation data that fraud is suspected, and if fraud is suspected, issues an audio alert from the smartphone's speaker.

"Example 2"

Another embodiment of the present invention involves installing a microphone on a home phone. In this case, the home phone's microphone listens to telephone conversations, and the listened conversation data is transcribed and sent to the generative AI. The generative AI determines from the conversation data that fraud is suspected, and if fraud is suspected, issues an audio alert from the home phone's speaker.

"Example 3"

A further embodiment of the present invention is one in which the generative AI detects specific phrases or patterns that indicate the possibility of specialized fraud. Specifically, the generative AI learns typical phrases for "bank transfer fraud" and typical patterns for "it's me" fraud, and by detecting these, determines the possibility of fraud.

The processing flow for each example form is explained below.

"Example 1"

Step 1: The application installed on the smartphone detects an incoming call.

Step 2: The application listens to your conversation through your smartphone's microphone.

Step 3: The conversation data heard is transcribed and sent to the generative AI.

Step 4: The generative AI determines that fraud is suspected based on the conversation data.

Step 5: If fraud is suspected, an audio alert will be issued through your smartphone's speaker.

"Example 2"

Step 1: A microphone installed on a home phone listens to the phone conversation.

Step 2: The conversation data that is heard is transcribed and sent to the generative AI.

Step 3: The generative AI determines that fraud is suspected based on the conversation data.

Step 4: If fraud is suspected, an audio alert will be issued over the home phone speaker.

"Example 3"

Step 1: The generative AI learns specific phrases or patterns that indicate potential fraud.

Step 2: The generative AI detects specific phrases or patterns it has learned to identify suspected fraud in the conversation data.

Step 3: If fraud is suspected, an audio alert will be issued.

(Example 1)

Next, Example 1 of Form Example 1 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 will be referred to as the "terminal."

In recent years, telephone fraud has been on the rise, with special frauds targeting the elderly becoming a particular social problem. Conventional countermeasures require users to identify fraudulent calls themselves, but as fraud methods become more sophisticated, this becomes increasingly difficult. Therefore, there is a need for a system that automatically detects potential fraud and issues a warning when users receive a fraudulent call.

The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means. In this invention, the server includes means for detecting incoming fraudulent calls, means for listening to the conversation from a microphone installed in the communication terminal, means for converting the conversation data into text and sending it to the generative AI model, and means for issuing an audio alert from the speaker of the communication terminal if the generative AI model determines that fraud is suspected based on the conversation data. This makes it possible to automatically detect the possibility of fraud when a user receives a fraudulent call and immediately issue a warning.

"Means for detecting incoming fraudulent calls" refers to functions or devices in a communications terminal for detecting incoming calls.

"Means for listening to conversations using a microphone installed in a communications terminal" refers to a function or device that uses a microphone built into or connected to a communications terminal to collect audio during a call in real time.

"Means for converting conversation data into text and sending it to the generative AI model" refers to functions or devices for converting collected voice data into text data and sending that text data to the generative AI model.

A "generative AI model" is an artificial intelligence model that analyzes input data and detects specific patterns and phrases.

"Means for issuing an audio alert from the speaker of the communication terminal when it is determined that fraud is suspected" refers to a function or device for playing an alert sound or message through the speaker of the communication terminal when the generative AI model detects the possibility of fraud.

"Specific phrases or patterns that indicate the possibility of specialized fraud" are specific words or sentence structures associated with fraudulent acts, and the generative AI model's detection of these serves as the basis for determining the possibility of fraud.

"Audio alert" is a message or sound that warns the user audibly.

This invention is a system that automatically detects fraudulent calls and issues a warning to the user using an application installed on a communication terminal. A specific embodiment of this system is described below.

First, the user installs a dedicated application on their communication device. This application has the function of detecting incoming phone calls. Specifically, it uses the communication device's phone API to catch incoming call events.

When a user answers a call, the device listens to the conversation in real time through the microphone. At this time, the application runs in the background and captures the conversation. The listened conversation data is transcribed using the Google Speech-to-Text API. The audio data is sent to the API and received as text data.

The transcribed conversation data is sent to a server via the internet. The HTTPS protocol is used for transmission to ensure data security. The server inputs the received conversation data into a generative AI model (e.g., OpenAI's GPT-4). The generative AI model uses prompt text to analyze the possibility of fraud. Specific examples of prompt text are as follows:

Analyze the conversation data below to determine if it may be a scam.

Conversation data: "Your bank account has been compromised. Please provide your account number for verification."

The generative AI model analyzes the conversation data and determines whether fraud is suspected. If fraud is suspected, the server sends that information back to the device, again using the HTTPS protocol.

The device analyzes the results received from the server, and if fraud is suspected, an audio alert is emitted from the communication device's speaker. Specifically, an audio message such as "There is a possibility of fraud. Please be careful" is played. This alert makes the user aware of the possibility of fraud and allows them to take appropriate action.

As a concrete example, consider the case where a user receives a phone call saying, "Your bank account has been fraudulently used. Please provide your account number to confirm." This conversation is picked up by a microphone and transcribed. The transcribed data becomes, "Your bank account has been fraudulently used. Please provide your account number to confirm." This data is sent to a generative AI model, which determines that there is a high possibility of fraud. The server sends the result back to the device, which then issues an audio alert. The user hears this alert and becomes aware of the possibility of fraud.

In this way, the invention can automatically detect possible fraud when a user receives a fraudulent call and immediately issue a warning.

The flow of the identification process in Example 1 will be explained using Figure 11.

Step 1: Detect incoming calls

The terminal detects incoming calls through an application installed on the communication terminal. Specifically, it uses the communication terminal's telephone API to catch incoming call events. The input is the incoming call signal, and the output is the incoming call detection event.

Step 2: Listen to the conversation

When a user answers a call, the device listens to the conversation in real time through the microphone. The application runs in the background and captures the conversation. The input is the audio data during the call, and the output is the collected audio data.

Step 3: Transcribe the conversation data

The device uses the Google Speech-to-Text API to transcribe the conversation data it hears. It sends the audio data to the API and receives it as text data. The input is the collected audio data, and the output is the transcribed text data.

Step 4: Send the transcribed data

The terminal sends the transcribed conversation data to the server. The HTTPS protocol is used for transmission to ensure data security. The input is transcribed text data, and the output is a data transmission event to the server.

Step 5: Fraud Detection

The server inputs the received conversation data into a generative AI model. The generative AI model uses prompts to analyze the possibility of fraud. Specific examples of prompts are as follows:

Analyze the conversation data below to determine if it may be a scam.

Conversation data: "Your bank account has been compromised. Please provide your account number for verification."

The input is transcribed text data, and the output is a judgment result regarding the possibility of fraud.

Step 6: Receive the results

The server receives the judgment results from the generative AI model and returns them to the device. The return is again done using the HTTPS protocol. The input is the judgment result from the generative AI model, and the output is a data transmission event to the device.

Step 7: Receive an audio alert

The device analyzes the judgment results received from the server, and if fraud is suspected, it issues an audio alert from the communication device's speaker. Specifically, it plays an audio message such as "There is a possibility of fraud. Please be careful." The input is the judgment result from the server, and the output is the generation of an audio alert.

(Application Example 1)

Next, we will explain Application Example 1 of Form Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 will be referred to as the "terminal."

In recent years, telephone fraud has been on the rise, with special fraud targeting the elderly becoming a particular social problem. Conventional fraud prevention measures make it difficult for users to detect potential fraud, making it difficult to prevent damage before it occurs. Therefore, there is a need for a system that can analyze telephone conversations in real time and immediately issue a warning if there is a possibility of fraud.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

In this invention, the server includes means for detecting incoming fraudulent calls, means for listening to conversations from a voice input device installed in the communication terminal, means for converting the conversation data into text and sending it to the generative AI, means for issuing an audio alert from the voice output device if the generative AI determines that fraud is suspected from the conversation data, means for the generative AI to detect specific phrases or patterns that indicate the possibility of fraud, and means for the audio alert to contain content that encourages the user to become aware of the possibility of fraud. This allows the user to immediately become aware of the possibility of fraud and prevent damage from occurring.

A "fraudulent call" is a call made with the intent of defrauding someone of money or personal information through fraudulent means.

"Incoming call" refers to an incoming phone call.

A "communications terminal" is a device for sending and receiving voice and data, including smartphones and home telephones.

An "audio input device" is a device that converts audio into digital data, such as a microphone.

"Conversation data" refers to data that has been converted into text from speech captured by a voice input device.

"Generative AI" is artificial intelligence that analyzes input data and detects specific patterns and phrases.

An "audio output device" is a device for outputting digital data as audio, and a speaker is an example of this.

"Audio alert" is a warning sound or message emitted from an audio output device.

"Specific phrases or patterns" are characteristic words or sentence structures that indicate possible fraud.

"User" refers to an individual or organization that uses this system.

A system for implementing this invention has the ability to detect incoming fraudulent calls and listen to the conversation from a voice input device installed on the communication terminal. The conversation data is transcribed in real time and sent to a generative AI. The generative AI detects specific phrases or patterns from the conversation data that indicate the possibility of fraud, and if fraud is suspected, issues a voice alert from a voice output device.

Hardware and software used

Hardware: Communication devices such as smartphones and home phones, microphones (audio input devices), speakers (audio output devices)

Software:

SpeechRecognition: A Python library for converting speech to text

requests: A Python library for sending HTTP requests

Generative AI: Artificial intelligence that analyzes text data using external APIs

System Operation

1. Incoming call detection: When the communication terminal detects an incoming call, it begins recording the conversation.

2. Conversation transcription: Recorded audio data is transcribed in real time using the SpeechRecognition library.

3. Generative AI Analysis: The transcribed conversation data is sent to a generative AI via an HTTP request. The generative AI detects specific phrases or patterns that indicate potential fraud.

4. Alert generation: If the generative AI detects the possibility of fraud, an audio alert will be emitted from the communication device's speaker. This audio alert will alert the user to the possibility of fraud.

Specific examples

For example, if a user receives a phone call and the caller says, "Your bank account has been fraudulently accessed. Please provide your account details immediately," the system will transcribe this conversation in real time and send it to the generative AI. The generative AI will detect that this phrase matches a specific pattern that indicates potential fraud and immediately issue an audio alert.

Example prompt

Examples of prompts for generative AI models include:

"Determine whether the following text is potentially fraudulent: 'Your bank account has been compromised. Please provide your account information immediately.'"

In this way, users can immediately become aware of potential fraud and prevent damage before it occurs.

The flow of the specific processing in Application Example 1 will be explained using Figure 12.

Step 1: Incoming call detection

The communication terminal detects an incoming call. The input is the incoming call signal, and the output is a trigger to start recording. Specifically, the communication terminal's phone application detects the incoming call and activates the recording module.

Step 2: Record the conversation

The communication terminal's audio input device (microphone) records the conversation. The input is the audio data during the call, and the output is the recorded audio file. Specifically, the recording module captures the audio during the call and saves it as an audio file.

Step 3: Transcribe the conversation

Recorded audio data is converted into text using the SpeechRecognition library. The input is an audio file, and the output is text data. Specifically, the speech recognition engine analyzes the audio file and generates the corresponding text.

Step 4: Analysis using generative AI

The transcribed text data is sent to the generative AI via an HTTP request. The input is text data, and the output is a flag indicating the possibility of fraud. Specifically, a request containing the text data is sent to the generative AI's API endpoint, and the analysis results are received.

Step 5: Determine the possibility of fraud

Generative AI analyzes text data to detect specific phrases or patterns that indicate potential fraud. The input is text data, and the output is a flag indicating potential fraud. Specifically, generative AI analyzes text data and flags cases where there is a high probability of fraud.

Step 6: Send an alert

If possible fraud is detected, an audio alert is emitted from the communication terminal's audio output device (speaker). The input is a flag indicating possible fraud, and the output is an audio alert. Specifically, the audio alert module receives a flag indicating possible fraud and plays an audio message to warn the user.

In this way, users can immediately become aware of potential fraud and prevent damage before it occurs.

(Example 2)

Next, Example 2 of Form Example 2 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 will be referred to as the "terminal."

In recent years, fraudulent phone scams have become more sophisticated, resulting in many people falling victim to them. Special frauds targeting the elderly in particular are on the rise, creating a need for effective countermeasures. Conventional countermeasures lack systems that can detect fraudulent phone calls in real time and issue warnings to users, making it difficult to prevent fraud before it occurs.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

In this invention, the server includes means for detecting incoming fraudulent calls, means for collecting conversation from a voice input device installed in the communication device, means for converting the conversation data into text data, means for transmitting the text data to the generative AI model, and means for issuing a voice alert from the voice input device if the generative AI model determines that fraud is suspected based on the conversation data. This makes it possible to detect fraudulent calls in real time and issue an immediate warning to the user.

"Means for detecting incoming fraudulent calls" refers to a function that identifies and detects incoming calls that may be fraudulent to a communications device.

An "audio input device" is a microphone or other audio capture device installed on a communications device to capture a user's speech.

"Means for converting speech data into text data" refers to speech recognition software or algorithms used to analyze collected voice data and convert it into text format.

A "generative AI model" is an artificial intelligence model that analyzes input text data and determines the possibility of fraud based on specific patterns and phrases.

"Means for issuing audio alerts" refers to a function that plays an audio message to warn the user if fraud is suspected.

This invention is a system that detects incoming fraudulent calls in real time and issues a warning to users. A specific embodiment of this system is described below.

1. System Configuration

The system consists of the following main components:

Methods for detecting incoming fraudulent calls

Voice input device

Method of converting conversation data into text data

Generative AI model

Method for issuing audio alerts

2. Hardware and Software Use

The voice input device uses a microphone installed in a home telephone. This microphone collects the user's speech in real time.

To convert conversation data into text data, we use voice recognition software (e.g., Google Speech-to-Text API). This software analyzes the collected voice data and converts it into text data.

The generative AI model uses advanced artificial intelligence models such as OpenAI GPT-4. This model analyzes the input text data and determines the likelihood of fraud.

The audio alert is generated using the speaker on a home phone, which will emit an audio alert if fraud is suspected.

3. System Operation

When a user starts a conversation on their home phone, a voice input device collects the conversation. The collected voice data is converted into text data using voice recognition software. This text data is sent to a server and input into a generative AI model. The generative AI model analyzes the conversation data and, if it determines that fraud is suspected, the server sends an alert signal to the home phone. The home phone's speaker emits an audio alert to warn the user.

4. Specific Examples

For example, suppose a user is speaking on their home phone, "Hello, your bank account has been compromised. Immediate action is required." A voice input device collects this speech, and speech recognition software converts it into text data. This text data is sent to a server, where it is analyzed by a generative AI model. If the generative AI model determines that there is a high possibility of fraud, the server sends an alert signal to the home phone. The speaker on the home phone will emit an audio alert saying, "Possible fraud. Please be careful."

5. Prompt Sentence Examples

"Analyze the conversation data below and determine whether or not you suspect fraud. If you suspect fraud, please explain why.

Conversation data: 'Hello, your bank account has been compromised. We need your immediate attention.'

In this way, users can be aware of their fraud risk in real time.

The flow of the identification process in Example 2 will be explained using Figure 13.

Step 1:

Audio collection

Device: A voice input device (microphone) installed on a home phone collects the user's conversation in real time.

Input: User's voice conversation

Output: Digital audio data

What it does: When a user starts a phone conversation, the microphone automatically captures the audio and stores it as digital audio data.

Step 2:

Transcription of audio data

Device: Built-in voice recognition software (e.g., Google Speech-to-Text API) in home phones converts collected voice data into text data.

Input: Digital audio data

Output: Text data

What it does: Speech recognition software analyzes the audio data and generates text data such as, "Hello, your bank account has been compromised. Immediate action is required."

Step 3:

Send text data

Device: The home phone sends the transcribed conversation data to the server.

Input: Text data

Output: Send data to the server

How it works: A home phone sends data over the internet to a server, which then passes the data to a generative AI model.

Step 4:

Fraud detection

Server: A generative AI model (e.g., OpenAI GPT-4) analyzes the received conversation data and determines whether fraud is suspected.

Input: Text data

Output: Judgment result regarding the likelihood of fraud

Specific behavior: The generative AI model analyzes the text "Hello, your bank account has been compromised. Immediate action is required" and determines that it is likely to be fraudulent.

Step 5:

Audio alert occurs

Server: If fraud is suspected, the server sends an alert signal to the home phone.

Device: The speaker on your home phone receives the alert signal from the server and plays an audio alert.

Input: Alert signal

Output: Audio alert

Specific behavior: The speaker on your home phone will play a voice alert saying, "This may be a scam. Be careful."

(Application Example 2)

Next, we will explain Application Example 2 of Form Example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 will be referred to as the "terminal."

In recent years, the number of victims of fraudulent phone calls has been increasing, and special frauds targeting the elderly in particular have become a social problem. Conventional fraud prevention systems have been inadequate in detecting fraudulent calls, making it difficult for users to recognize the possibility of fraud. Furthermore, there is a demand for systems that can detect fraud in real time and issue warnings to users on home phones and smartphones.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

In this invention, the server includes means for detecting incoming fraudulent calls, means for listening to conversations using a voice recognition device, means for converting the conversation data into text and sending it to the generative AI, means for issuing an audio alert from a voice output device if the generative AI determines that fraud is suspected based on the conversation data, and means including an application to be installed on a smartphone. This enables real-time detection of fraudulent calls and prompt warnings to users.

A "fraudulent call" is a call made with the intent of defrauding someone of money or personal information through fraudulent means.

"Incoming call" refers to an incoming phone call.

A "voice recognition device" is a device that converts voice into text data.

"Conversation data" refers to the content of a conversation converted into text by a voice recognition device.

"Generative AI" is artificial intelligence that generates new information based on input data.

An "audio output device" is a device for outputting audio.

"Voice alert" is a function that issues a warning via voice.

"Applications installed on smartphones" are software programs that run on smartphones.

The following hardware and software are required to implement this invention. Hardware requires a smartphone, a voice recognition device, and a voice output device. Software requires a voice recognition library (e.g., speech_recognition), a generative AI library (e.g., openai), and a text-to-speech library (e.g., pyttsx3).

First, the phone call is recorded in real time using the smartphone's microphone. The recorded voice data is converted into text data using a voice recognition device. The speech_recognition library is used for this voice recognition.

The converted text data is then sent to the generative AI. The generative AI analyzes the input text data and determines whether it is fraudulent. This analysis is performed using the OpenAI library. Specifically, the following prompt is sent to the generative AI:

Please determine if the following conversation is a scam:

"Hello, this is the bank's security department. Your account has been compromised. Please provide your account number and PIN."

If you suspect it may be a scam, please reply 'scam'.

If the generative AI determines that there is a possibility of fraud, it will issue an audio alert using the smartphone's audio output device. This audio alert uses the pyttsx3 library. The content of the audio alert is intended to alert the user to the possibility of fraud.

For example, if a user is on a call on their smartphone and a conversation that could be fraudulent takes place, the application automatically converts the conversation into text and sends it to a generative AI to determine whether it is fraudulent. If it determines that there is a possibility of fraud, an audio alert will sound from the smartphone speaker saying, "Possible fraud!"

In this way, the invention enables real-time detection of fraudulent calls and prompt warning to users.

The flow of the specific processing in Application Example 2 will be explained using Figure 14.

Step 1:

The user initiates a call on their smartphone. The smartphone's microphone records the call in real time.

Input: Call audio

Output: Recorded audio data

Specific operation: The smartphone's microphone captures the call audio and saves it as audio data.

Step 2:

The device sends the recorded voice data to a voice recognition device, which converts the voice data into text data.

Input: Recorded audio data

Output: Text data

Specific operation: The speech recognition library (e.g., speech_recognition) analyzes the audio data and generates corresponding text data.

Step 3:

The device sends the generated text data to the generative AI, which analyzes the text data and determines whether it is fraudulent.

Input: Text data

Output: Judgment result regarding the possibility of fraud

Specific operation: A generative AI library (e.g., openai) analyzes text data based on the prompt and determines whether it is likely to be fraudulent.

Step 4:

The server receives the judgment results from the generative AI and generates an audio alert if there is a possibility of fraud.

Input: Judgment result regarding the possibility of fraud

Output: Audio alert content

Specific behavior: The generative AI generates a warning message saying, "Possible fraud!"

Step 5:

The device sends the contents of the audio alert to the audio output device, which then issues the audio alert.

Input: Audio alert content

Output: Audio alert

Specific operation: A text-to-speech library (e.g., pyttsx3) converts the contents of the audio alert into speech, and the warning sound is output from the smartphone speaker.

(Example 3)

Next, Example 3 of Form Example 3 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 will be referred to as the "terminal."

Conventional fraud detection systems were limited to detecting incoming fraudulent phone calls and were inadequate at detecting fraud in text messages and emails received by users. Furthermore, there were limited ways to notify users of potential fraud, which could delay users' realization of the risk of fraud. This made it difficult to prevent fraud from occurring.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Example 3 is realized by the following means.

In this invention, the server includes means for transmitting text data entered by the user to the server, means for the server to analyze the text data using a generative artificial intelligence model and determine the possibility of fraud, and means for the server to transmit the analysis results to the communication terminal. This makes it possible to quickly detect the possibility of fraud even in text messages or emails received by the user and notify the user.

"Means for detecting incoming fraudulent calls" refers to equipment or software that identifies and detects potentially fraudulent calls from among calls received over a communication line.

"A voice input device installed on a communication terminal" refers to a microphone or related hardware for collecting voice that is attached to communication devices such as smartphones and home telephones.

"Means for converting conversation data into text and sending it to the generative AI" refers to software or hardware for converting voice data into text data and sending that text data to the generative AI.

"Generative AI" is an AI model that uses natural language processing technology to analyze text data and detect specific patterns and phrases.

"A means for issuing an audio alert" is a device or software that issues an audio warning if it determines that there is a possibility of fraud.

"Means for transmitting text data entered by a user to a server" refers to software or hardware for transmitting text data entered by a user into a communication terminal to a server via a communication line such as the Internet.

"Means for the server to analyze text data using a generative artificial intelligence model and determine the possibility of fraud" refers to software or hardware that inputs the text data received by the server into a generative artificial intelligence model and evaluates the possibility of fraud from the analysis results.

"Means by which the server transmits the analysis results to the communications terminal" refers to software or hardware for transmitting the analysis results from the generative artificial intelligence model to the communications terminal.

"Means by which the communications terminal displays the analysis results to the user" refers to software or hardware that visually or audibly notifies the user of the analysis results received from the server.

This invention relates to a system that uses a generative artificial intelligence model to detect the possibility of special fraud. A specific embodiment of this system is described below.

System Configuration

Subject: Server

The server uses a generative artificial intelligence model to analyze text data and determine the likelihood of fraud. Specifically, the server uses the following hardware and software:

Hardware: High-performance processor, memory, and storage devices

Software: Generative AI models (e.g., AI models using natural language processing technology)

The server receives text data sent by the user and inputs it into a generative artificial intelligence model. The model detects specific phrases and patterns that indicate potential fraud and returns the results to the server. The server then sends the analysis results to the communication device.

Subject: Terminal

The device is responsible for sending text data entered by the user to the server and displaying the analysis results from the server to the user. Specifically, the device uses the following hardware and software:

Hardware: Communication devices such as smartphones, tablets, and PCs

Software: Dedicated application

The device sends the text data entered by the user to the server and displays the analysis results received from the server to the user. If the analysis results indicate the possibility of fraud, the device displays a warning message.

Subject: User

The user is responsible for entering text data through the device and checking the analysis results. Specifically, the user performs the following operations:

Enter message or email content into your device

Check the analysis results and take appropriate action as necessary.

Specific examples

Example 1

When a user receives a message saying, "Mom, please transfer the money now," the system works as follows:

1. The user enters a message into a dedicated application on the device.

2. The device sends this message to the server.

3. The server receives the message and inputs it as a prompt into the generative artificial intelligence model.

4. The generative artificial intelligence model analyzes the message and returns the result "Possible fraud."

5. The server sends the analysis results to the device.

6. The device displays the analysis results to the user, and the user confirms the warning message.

Example 2

When a user receives an email saying, "Your bank account has been compromised. Contact us immediately," the system works as follows:

1. The user enters the email content into a dedicated application on their device.

2. The device sends the contents of this email to the server.

3. The server receives the email content and inputs it as a prompt into the generative artificial intelligence model.

4. The generative artificial intelligence model analyzes the contents of the email and returns the result "Possible fraud."

5. The server sends the analysis results to the device.

6. The device displays the analysis results to the user, and the user confirms the warning message.

Example prompt

Below is an example of a prompt to input to a generative artificial intelligence model:

"Please determine if the following message is a potential scam: 'Mom, please transfer the money now.'"

"Determine whether the following email is a potential scam: 'Your bank account has been compromised. Contact us immediately.'"

This is how the specialized fraud detection system using a generative artificial intelligence model operates. The flow of the identification process in Example 3 will be explained using Figure 15.

Step 1:

The user enters text data.

The user enters the contents of the message or email they receive into a dedicated application on their device. For example, they might copy and paste the message "Mom, please transfer the money now" into the application. The input data is saved in text format on the device.

Step 2:

The device sends the text data to the server.

The terminal sends the text data entered by the user to the server. Specifically, the terminal application generates an HTTP request and sends a payload containing the text data to the server. At this time, the data is encrypted before being sent. The input is the text data entered by the user, and the output is the text data sent to the server.

Step 3:

The server receives the text data and inputs it into the generative artificial intelligence model.

The server receives text data sent from the terminal. The received data is first stored in a database. The server then inputs the text data as a prompt sentence into the generative AI model. An example of a prompt sentence is "Please determine whether the following message is likely to be fraudulent: 'Mom, please transfer the money now.'" The input is the text data received from the terminal, and the output is the prompt sentence input into the generative AI model.

Step 4:

A generative artificial intelligence model analyzes the text data and returns the results.

The generative artificial intelligence model analyzes the input prompt text and determines whether it is likely to be fraud. Specifically, the model analyzes it by comparing it with typical phrases and patterns of "bank transfer fraud" and "it's my son" fraud. If the analysis result indicates a high possibility of fraud, it returns a warning message stating "Possible fraud." The input is the prompt text, and the output is the analysis result.

Step 5:

The server sends the analysis results to the device.

The server sends the analysis results obtained from the generative AI model to the terminal. Specifically, the server generates an HTTP response and sends a payload containing the analysis results to the terminal. At this time, the data is encrypted before being sent. The input is the analysis results from the generative AI model, and the output is the analysis results sent to the terminal.

Step 6:

The device displays the analysis results to the user.

The device displays the analysis results received from the server to the user. For example, if the analysis result is a warning message stating "Possible fraud," the device application displays this message to the user as a pop-up window or notification. The input is the analysis result received from the server, and the output is the warning message displayed to the user. The user can check this warning message and take appropriate action.

(Application Example 3)

Next, we will explain Application Example 3 of Form Example 3. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 will be referred to as the "terminal."

In recent years, special fraud methods have become more sophisticated, with many people falling victim to them. In particular, "bank transfer fraud" and "it's me" frauds targeting the elderly have become a social problem. In order to prevent these frauds, a system is needed that can detect possible fraud early and issue a warning to users. However, current systems have low fraud detection accuracy, making it difficult for users to recognize the possibility of fraud.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 3 is realized by the following means.

In this invention, the server includes means for detecting incoming fraudulent calls, means for listening to conversations from a voice input device installed in the communication terminal, means for converting the conversation data into text and sending it to the generative AI, means for issuing a voice alert from the voice input device if the generative AI determines that fraud is suspected from the conversation data, means for the generative AI to detect specific phrases or patterns that indicate the possibility of fraud, and means for issuing a warning to the user if the generative AI detects specific phrases or patterns that indicate the possibility of fraud. This makes it possible to detect the possibility of fraud with high accuracy and issue a warning to the user quickly.

"Means for detecting incoming fraudulent calls" refers to a function that identifies and notifies a communication terminal of incoming calls that may be fraudulent.

"A voice input device installed on a communication terminal" is a device for collecting voice that is attached to communication devices such as smartphones and home telephones.

"Means for converting conversation data into text and sending it to generative AI" refers to a function for converting voice data collected by a voice input device into text data and sending that text data to generative AI.

"Means for issuing a voice alert from the voice input device when the generative AI determines that fraud is suspected based on the conversation data" refers to a function that issues a warning sound through the voice input device when the generative AI determines that there is a possibility of fraud based on the conversation data analyzed.

"Means for the generative AI to detect specific phrases or patterns that indicate the possibility of fraud" refers to a function that enables the generative AI to detect typical fraudulent phrases and patterns that it has previously learned from conversation data.

"Means for issuing a warning to the user when the generative AI detects a specific phrase or pattern that indicates the possibility of fraud" refers to a function that issues a warning to the user when the generative AI detects a phrase or pattern that indicates the possibility of fraud.

As an embodiment of this invention, we will explain how to install a fraud detection assistant system on a smartphone.

First, the system is equipped with a means for detecting incoming fraudulent calls. This means has the function of identifying and notifying the communication terminal of incoming calls that may be fraudulent. Specifically, the content of the call is collected in real time using a voice input device installed on the communication terminal.

The collected voice data is then converted into text data through a means of transcribing the conversation data and sending it to the generative AI. This conversion is done using voice recognition software (e.g., Google's voice recognition API). The converted text data is then sent to the generative AI model.

Generative AI models are pre-trained to detect specific phrases and patterns that indicate potential fraud. These models are built using, for example, the Hugging Face transformers library. If the generative AI determines from the conversation data that fraud is suspected, it will warn the user by issuing an audio alert via a voice input device.

Furthermore, if the generative AI detects certain phrases or patterns that indicate potential fraud, a user alert mechanism will be activated. This mechanism will alert the user to the potential fraud by providing a visual or audio warning.

As a concrete example, consider the following audio file:

Voice: "Mom, please transfer the money right now. It's urgent."

Example prompt for parsing this audio file:

Judge whether the phrase "Mom, please transfer the money now. It's urgent." could be a scam.

By feeding this prompt into a generative AI model, it can analyze the possibility of fraud and issue a warning to the user.

The flow of the specific processing in Application Example 3 will be explained using Figure 16.

Step 1:

Detect incoming fraudulent calls

The server identifies and notifies the communication terminal of incoming calls that may be fraudulent. Specifically, it monitors the communication terminal's call log and detects possible fraud based on specific numbers or patterns. The input is call log data, and the output is a notification of a potentially fraudulent call.

Step 2:

Listen to conversations from a voice input device

The terminal collects call content in real time using an audio input device installed on the communication terminal. Specifically, audio data is acquired through a microphone and saved as an audio file. The input is the audio during the call, and the output is an audio file.

Step 3:

Conversation data is transcribed and sent to generative AI.

The device converts the collected voice data into text data and sends it to the generative AI. Specifically, it uses voice recognition software (e.g., Google's voice recognition API) to transcribe the voice data. The input is an audio file, and the output is text data.

Step 4:

Generative AI analyzes conversation data

The server uses a generative AI model to detect specific phrases and patterns in text data that indicate potential fraud. Specifically, it analyzes the text data using a generative AI model (e.g., Hugging Face's transformers library). The input is the text data, and the output is a determination of the likelihood of fraud.

Step 5:

Audible alerts when fraud is suspected

If the generative AI determines that there is a possibility of fraud, the device will issue an audio alert from the voice input device. Specifically, it will play a warning sound or message. The input is the result of the determination of the possibility of fraud, and the output is an audio alert.

Step 6:

Warn the user

If the generative AI detects certain phrases or patterns that indicate potential fraud, the device will alert the user by displaying a warning message on the screen or sending a notification. The input is the result of the determination of potential fraud, and the output is a warning to the user.

It is also possible to further combine an emotion engine that estimates the user's emotion. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

"Example 1"

One embodiment of the present invention is a system that combines a generative AI with an emotion engine. This system analyzes emotions from the user's voice tone, volume, speaking rate, etc. Specifically, if the user's voice tone or volume suddenly increases while they are receiving a scam call, the system senses that the user may be panicking. Based on this information, the generative AI determines that the possibility of a scam has increased further and issues an audio alert.

"Example 2"

In another embodiment, the system could issue an audio alert if the emotion engine exceeds a certain threshold. For example, if the user's tone of voice exceeds a certain threshold, or if the user's speaking speed exceeds a certain rate, the emotion engine could determine that the user is experiencing strong emotion. Based on this determination, the generative AI could issue an audio alert, warning the user of possible fraud.

"Example 3"

Furthermore, as another embodiment, a system in which the emotion engine and generative AI work in conjunction with each other is also possible. In this system, the emotion engine analyzes the user's emotions and sends the results to the generative AI. The generative AI combines information from the emotion engine with conversation data it has analyzed to determine the possibility of fraud. In this way, more accurate fraud detection is possible.

The processing flow for each example form is explained below.

"Example 1"

Step 1: The user receives a scam call.

Step 2: A smartphone app or a microphone installed on a home phone listens to the conversation.

Step 3: Convert the conversation data into text and send it to the generative AI.

Step 4: Generative AI analyzes the text data and determines the likelihood of fraud.

Step 5: The emotion engine analyzes the user's emotions based on their tone of voice, volume, speaking rate, etc.

Step 6: Based on the information from the emotion engine, the generative AI determines that the likelihood of fraud has increased and issues an audio alert.

"Example 2"

Step 1: The user receives a scam call.

Step 2: A smartphone app or a microphone installed on a home phone listens to the conversation.

Step 3: Convert the conversation data into text and send it to the generative AI.

Step 4: Generative AI analyzes the text data and determines the likelihood of fraud.

Step 5: The emotion engine analyzes the user's emotions based on their tone of voice, volume, speaking rate, etc.

Step 6: If the emotion engine exceeds a certain threshold, it notifies the generative AI.

Step 7: Based on the notification from the emotion engine, the generative AI determines that the likelihood of fraud has increased and issues an audio alert.

"Example 3"

Step 1: The user receives a scam call.

Step 2: A smartphone app or a microphone installed on a home phone listens to the conversation.

Step 3: Convert the conversation data into text and send it to the generative AI.

Step 4: Generative AI analyzes the text data and determines the likelihood of fraud.

Step 5: The emotion engine analyzes the user's emotions based on their tone of voice, volume, speaking rate, etc., and sends the results to the generative AI.

Step 6: The generative AI combines information from the emotion engine with the conversation data it has analyzed to determine the likelihood of fraud.

Step 7: If it determines there is a high possibility of fraud, the generative AI will issue an audio alert.

(Example 1)

Next, Example 1 of Form Example 1 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 will be referred to as the "terminal."

Conventional fraud call detection systems analyze only the content of conversations to determine the possibility of fraud, which means they are unable to take the user's emotional state into account, resulting in low fraud detection accuracy. Furthermore, they are unable to respond adequately when the user panics, making it difficult to prevent fraud from occurring.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

In this invention, the server includes means for detecting incoming fraudulent calls, means for listening to conversations from a voice input device installed in the communication terminal, means for converting the conversation data into text and transmitting it to the generative artificial intelligence, means for issuing a voice alert from the voice input device if the generative artificial intelligence determines that fraud is suspected from the conversation data, means for collecting parameters such as the user's voice tone, volume, and speaking rate and analyzing their emotions, and means for reassessing the possibility of fraud based on the results of the emotion analysis. This makes it possible to detect fraud while taking the user's emotional state into account, thereby preventing fraud damage before it occurs.

"Means for detecting incoming fraudulent calls" refers to equipment or software that detects whether an incoming call to a communications terminal is likely to be fraudulent.

"A voice input device installed on a communication terminal" refers to a device for inputting voice, such as a microphone attached to a communication device such as a smartphone or home telephone.

"Means for converting conversation data into text and sending it to generative AI" refers to a device or software that converts voice data into text data and sends that text data to generative AI.

"Generative AI" is an artificial intelligence system that analyzes received data and determines the likelihood of fraud by detecting specific patterns and phrases.

"A means for issuing an audio alert" is a device or software that plays an audio message to warn the user if it determines that there is a high possibility of fraud.

"Means for collecting parameters such as the user's voice tone, volume, and speaking rate, and analyzing emotions" refers to a device or software that collects information such as tone, volume, and speaking rate from the user's voice data, and analyzes the user's emotional state based on that information.

"Means for reassessing the possibility of fraud based on the results of sentiment analysis" refers to a device or software that inputs the results of sentiment analysis into generative artificial intelligence and reassess the possibility of fraud.

Form for implementing the invention

This invention is a system that detects fraudulent phone calls and warns users, and is implemented using an application and server installed on a communications terminal. This system detects incoming fraudulent calls, listens to the conversation, transcribes the conversation data, and sends it to a generative artificial intelligence. Furthermore, it aims to prevent fraud by analyzing the user's emotional state and reassessing the possibility of fraud.

Hardware and software used

1. Communication terminal

Communication devices such as smartphones and home phones.

A microphone as an audio input device.

2. Server

Speech recognition software (e.g., Google Cloud Speech-to-Text API) is used to transcribe audio data.

For sentiment analysis, we use an emotion engine (e.g., IBM Watson Tone Analyzer).

For generative artificial intelligence, we use AI models with natural language processing technology (e.g., OpenAI's GPT-4).

Specific system operation

1. Incoming phone call detection

The server detects incoming phone calls through an application installed on the communication device. The application uses the communication device's native API to catch incoming phone call events.

2. Listening to Conversations

The device uses a microphone to listen to phone conversations. The application captures the microphone's audio input in real time and saves it as audio data.

3. Transcription of audio data

The server converts the audio data it hears into text data. This process involves calling the Google Cloud Speech-to-Text API to convert the audio data into text.

4. Sending to generative AI

The server sends the transcribed conversation data to the generative AI. The server sends the text data to the generative AI using an HTTP request.

5. Fraud Detection

The generative AI analyzes the received conversation data and determines whether fraud is suspected. Using natural language processing technology, the generative AI analyzes the content of the conversation and evaluates whether it matches any fraudulent patterns.

6. Sentiment Analysis

The server collects parameters such as the user's voice tone, volume, and speaking rate and analyzes them using an emotion engine. The server uses voice analysis software to extract emotion parameters from the voice data.

7. Panic Detection

If the user's tone or volume of voice suddenly increases, the server determines that the user may be panicking. The server analyzes data from the emotion engine and evaluates whether there is a sudden change.

8. Reassessing the possibility of fraud

Based on information from the emotion engine, the generative AI determines that the likelihood of fraud has increased. The generative AI receives the emotion data as additional input and reassess the likelihood of fraud.

9. Audio alerts

If the server determines that there is a high possibility of fraud, it instructs the communication device to play an audio alert from its speaker. The server then sends a command to the communication device's application to play the audio alert.

Examples and prompts

As a concrete example, imagine a user is receiving a phone call on their smartphone. If the person on the other end of the line says, "Your bank account has been fraudulently used. Please tell us your account number and password immediately," the user's voice tone will suddenly rise and the volume will also increase. Based on this information, the generative artificial intelligence will determine that there is a high possibility of fraud, and will emit an audio alert from the communication device's speaker saying, "This may be a fraud. Please be careful."

Example prompt:

"Analyze the conversation data below to determine if it may be a scam. Also consider changes in the user's tone and volume of voice.

Conversation data: 'Your bank account has been compromised. Please provide your account number and password immediately.'

User's voice tone: Sudden rise

User volume: Increased"

The flow of the identification process in Example 1 will be explained using Figure 17.

Step 1:

Incoming phone call detection

The server detects incoming phone calls through an application installed on the communication terminal.

Input: Incoming call event from communication terminal.

Data processing/calculation: Capture incoming call events using the communication device's native API.

Output: Event information indicating that an incoming call was detected.

Specific behavior: The application uses native Android and iOS APIs to catch incoming phone calls and notify the server.

Step 2:

Listening to conversations

The device uses a microphone to listen to the conversation during the call.

Input: Audio data during the call.

Data processing/calculation: Captures microphone audio input in real time and saves it as audio data.

Output: Audio data.

Specific behavior: The application captures audio input from the microphone in real time and saves it as audio data.

Step 3:

Transcription of audio data

The server converts the audio data it hears into text data.

Input: Audio data.

Data processing/calculation: Call the Google Cloud Speech-to-Text API to convert voice data into text.

Output: Character data.

Specific operation: The server converts the audio data into text using the Google Cloud Speech-to-Text API.

Step 4:

Send to generative AI

The server sends the transcribed conversation data to the generative artificial intelligence.

Input: Character data.

Data processing/calculation: Send text data to generative artificial intelligence using an HTTP request.

Output: Result of request sent to generative artificial intelligence.

Specific operation: The server sends text data to the generative artificial intelligence using an HTTP request.

Step 5:

Fraud detection

The generative artificial intelligence analyzes the received conversation data and determines whether fraud is suspected.

Input: Character data.

Data processing/calculation: Using natural language processing technology, the content of the conversation is analyzed and evaluated to see if it matches a fraud pattern.

Output: Assessment result regarding likelihood of fraud.

Specific operation: The generative artificial intelligence uses natural language processing technology to analyze the content of the conversation and evaluate whether it matches any fraud patterns.

Step 6:

Emotion analysis

The server collects parameters such as the user's voice tone, volume, and speaking speed, and analyzes them using an emotion engine.

Input: Audio data.

Data processing/calculation: Use voice analysis software to extract emotional parameters from voice data.

Output: Emotion parameters.

Specific operation: The server uses voice analysis software (e.g., IBM Watson Tone Analyzer) to extract emotion parameters from the voice data.

Step 7:

Detecting panic states

If the user's voice tone or volume suddenly increases, the server will determine that the user may be panicking.

Input: Emotion parameters.

Data processing/calculation: Analyze data from the emotion engine and evaluate whether there are any sudden changes.

Output: Panic state assessment results.

Specific behavior: The server analyzes data from the emotion engine and evaluates whether there are any sudden changes.

Step 8:

Reassessing the possibility of fraud

Based on information from the emotion engine, the generative artificial intelligence determines that the likelihood of fraud has increased.

Input: Emotion parameters.

Data processing/calculation: Emotional data is taken as additional input and the likelihood of fraud is reassessed.

Output: Reassessed fraud probability.

Specific behavior: The generative AI receives emotional data as additional input and reassess the likelihood of fraud.

Step 9:

Send audio alerts

If the server determines that there is a high possibility of fraud, it will instruct the communication device to emit an audio alert from its speaker.

Input: Reassessed potential fraud.

Data processing/calculation: Sends a command to the communication terminal application to play an audio alert.

Output: Plays an audio alert.

Specific operation: The server sends a command to the application on the communication device to play an audio alert, which is then played from the communication device's speaker.

(Application Example 1)

Next, we will explain Application Example 1 of Form Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 will be referred to as the "terminal."

In recent years, fraudulent phone scams have become more sophisticated, with many people falling victim to them. Elderly people and those unfamiliar with technology are particularly prone to falling victim to these scams, as they are less wary of them. Furthermore, when users receive a fraudulent call, they often panic, making it difficult for them to make a calm judgment. There is a need for effective measures to improve this situation and prevent fraudulent incidents.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

In this invention, the server includes means for detecting incoming fraudulent calls, means for listening to conversations from a voice input device installed in the communication terminal, means for converting the conversation data into text and sending it to the generative AI, means for issuing a voice alert from the voice input device if the generative AI determines that fraud is suspected from the conversation data, means including an emotion engine for analyzing emotions from the user's voice tone, volume, speaking speed, etc., and means for further increasing the possibility of fraud based on the analysis results of the emotion engine. In this way, by combining fraudulent call detection with user emotion analysis, it is possible to prevent fraud victims before they occur.

A "fraudulent call" is a call made with the intent of defrauding someone of money or personal information through fraudulent means.

"Incoming call" refers to an incoming phone call.

A "communications terminal" is an electronic device used to send and receive voice and data.

An "audio input device" is a device for converting audio into digital data.

"Conversation data" refers to data that has been converted from speech captured by a voice input device.

"Generative AI" is artificial intelligence that makes specific decisions and generates results based on input data.

"Audio alert" is a means of warning or notifying someone by voice.

An "emotion engine" is software or a system for analyzing emotions from voice data.

"Tone" refers to the pitch and texture of the voice.

"Volume" refers to the loudness of the sound.

"Speech rate" refers to the speed at which you speak.

The following system configuration is used to implement this invention.

System Configuration

1. Hardware

Communication terminal: A device capable of voice communication, such as a smartphone or home telephone.

Audio input device: A microphone built into the communication device.

Audio output device: Speaker built into the communication terminal.

2. Software

Speech Recognition Library: Use the speech_recognition library to convert speech to text.

Generative AI model: An AI model that uses the transformers library to determine the likelihood of fraud.

Emotion Engine: A model for analyzing user emotions using the transformers library.

Audio Alert Library: Uses the pyttsx3 library to emit audio alerts.

Processing flow

1. Incoming call detection

The server detects that a call has been received by the communication terminal.

The communication terminal's voice input device listens to the conversation and acquires voice data.

2. Voice Recognition

The server uses a voice recognition library to convert the acquired voice data into text data.

3. Fraud Detection Using Generative AI

The server inputs the text data into a generative AI model to determine the likelihood of fraud.

If certain phrases or patterns are detected, it is determined that there is a high possibility of fraud.

4. Sentiment analysis

The server uses an emotion engine to analyze the user's emotions based on their tone of voice, volume, speaking speed, etc.

If a user is perceived to be in a panic, the likelihood of fraud increases even more.

5. Audio alerts

If the server determines that there is a high possibility of fraud, it uses the audio alert library to issue an audio alert from the communication terminal's audio output device.

The alert is intended to alert users to potential fraud.

Specific examples

For example, if a user receives a phone call and it is determined that the call may be fraudulent, an audio alert will sound from the communication device's speaker saying, "Warning! Possible fraud." This alert will make the user aware of the possibility of fraud and allow them to respond calmly.

Example prompt

"Could this call be a scam?"

In this way, this invention combines the detection of fraudulent calls with user sentiment analysis, making it possible to prevent fraudulent incidents.

The flow of the specific processing in Application Example 1 is explained using Figure 18.

Step 1:

The server detects that a call has been received by the communication terminal.

Input: Incoming signal from communication terminal.

Data processing: Receives incoming call signals and begins processing when the call starts.

Output: Call start flag.

Specific operation: When a communication terminal receives an incoming call, the server detects the signal and recognizes that a call has started.

Step 2:

The device's voice input device listens to the conversation and captures the voice data.

Input: Audio during a call.

Data processing: The voice input device collects voice data in real time.

Output: Captured audio data.

Specific operation: Audio during a call is collected through the microphone and saved as audio data.

Step 3:

The server uses a voice recognition library to convert the acquired voice data into text data.

Input: Acquired audio data.

Data processing: Use the speech recognition library (speech_recognition) to convert audio data into text data.

Output: Converted text data.

Specific operation: Voice data is input into the voice recognition library and output as text data.

Step 4:

The server inputs the text data into a generative AI model to determine the likelihood of fraud.

Input: Converted text data.

Data processing: Text data is fed into generative AI models (transformers) to assess the likelihood of fraud.

Output: Assessment results regarding the likelihood of fraud.

Specific operation: Text data is input into a generative AI model, which outputs an evaluation result indicating whether the likelihood of fraud is high.

Step 5:

The server uses an emotion engine to analyze the user's emotions based on their tone of voice, volume, speaking speed, etc.

Input: Converted text and audio data.

Data processing: Analyze user emotions using emotion engines (transformers).

Output: Analysis results on user sentiment.

Specific operation: Voice and text data are input into the emotion engine, and the user's emotional state is analyzed.

Step 6:

If the server determines that there is a high possibility of fraud, it uses the audio alert library to issue an audio alert from the communication terminal's audio output device.

Input: Fraud likelihood assessment results and user sentiment analysis results.

Data processing: If a high possibility of fraud is determined, an audio alert is generated using the audio alert library (pyttsx3).

Output: Audio alert.

Specific operation: If it is determined that there is a high possibility of fraud, an audio alert will be emitted from the communication device's speaker saying, "Warning! Possible fraud."

(Example 2)

Next, Example 2 of Form Example 2 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 will be referred to as the "terminal."

Conventional fraud prevention systems had low accuracy in detecting potential fraud, making it impossible to completely eliminate the risk of users falling victim to fraud. Furthermore, warnings were issued without taking the user's emotional state into consideration, which sometimes led to users being unable to respond appropriately. This made it difficult to prevent fraud from occurring.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

In this invention, the server includes means for detecting incoming fraudulent calls, means for listening to conversations from a voice input device installed in the communication device, means for converting the conversation data into text and sending it to the generative AI model, means for issuing an audio alert from the voice input device if the generative AI model determines that fraud is suspected from the conversation data, means for an emotion analysis engine to analyze the user's tone of voice and speaking rate, and means for the generative AI model to issue an audio alert if the emotion analysis engine exceeds a specific threshold. This makes it possible to detect the possibility of fraud with high accuracy and issue an appropriate warning according to the user's emotional state.

"Means for detecting incoming fraudulent calls" refers to a function that identifies and notifies a communication device of incoming calls that may be fraudulent.

A "voice input device" is a device that records conversations between a user and another person in real time and stores them in digital format.

"Means for converting conversation data into text and sending it to the generative AI model" refers to a function for converting voice data into text data and sending that text data to the generative AI model.

A "generative AI model" is an artificial intelligence system that analyzes input data, detects specific patterns and phrases, and determines the likelihood of fraud.

"Means for issuing audio alerts" refers to an audio notification function that warns users if a potential fraud is detected.

The "emotion analysis engine" is an engine that analyzes the user's tone of voice and speaking speed to determine their emotional state.

"When a specific threshold is exceeded" refers to when the emotion analysis engine analyzes the user's emotional state and it exceeds a pre-set reference value.

"Communication equipment" refers to devices used for making voice calls, such as home telephones and smartphones.

Form for implementing the invention

This invention is a system that detects incoming fraudulent calls and issues a warning to the user. A specific embodiment of this system is described below.

Hardware and software used

Communication devices: devices used for voice calls, such as home phones and smartphones.

Audio input device: A microphone installed in the communication device

Server: A computer system for processing voice data and running generative AI models and sentiment analysis engines.

Generative AI model: An artificial intelligence system that analyzes conversation data and determines the likelihood of fraud.

Emotion analysis engine: An engine that analyzes the user's tone of voice and speaking rate to determine their emotional state.

Speech recognition software: Software for converting voice data into text data (e.g., Google Speech-to-Text API)

System Operation

1. When a user starts a conversation on a communication device, the device's voice input device records the conversation in real time. The recorded voice data is saved in digital format.

2. The server uses speech recognition software to convert the collected voice data into text. For example, the speech "Hello, this is a bank representative" is converted into text "Hello, this is a bank representative."

3. The server sends the converted text data to the generative AI model. The generative AI model receives this data and prepares it for analysis.

4. The generative AI model analyzes the conversation data and detects specific keywords and phrases (e.g., "bank account" or "password"), which determines whether there is potential for fraud.

5. If the generative AI model determines that there is a high possibility of fraud, the server will send instructions to the terminal and an audio alert will be issued from the communication device's speaker saying, "This is likely fraud. Please be careful."

6. The emotion analysis engine analyzes the user's tone of voice and speaking rate in real time. For example, if the user's voice suddenly gets higher in pitch or their speaking rate increases, the emotion analysis engine will detect this.

7. If the sentiment analysis engine determines that the user's emotions exceed a certain threshold, the generative AI model will issue an audio alert saying, "This may be a scam. Please remain calm."

Specific examples

Example 1: When a user says over the phone, "Please tell me your bank account information," the generative AI model detects the keyword "bank account" and determines that there is a possibility of fraud. The speaker on the communication device issues a voice alert saying, "There is a possibility of fraud. Please be careful."

Example 2: If a user says "Please tell me my password urgently" over the phone and their voice tone gets higher and faster, the emotion analysis engine determines that the user is experiencing strong emotions. The generative AI model issues a voice alert, warning, "This may be a scam. Please remain calm."

Example prompt

Prompt 1: "If someone calls and asks for your bank account information, determine whether it's a scam."

Prompt 2: "Please explain how the emotion engine determines if the user's voice tone gets higher and their speaking rate gets faster."

The system is designed to reduce the risk of users falling victim to telephone fraud. By combining a generative AI model with a sentiment analysis engine, it can detect potential fraud with greater accuracy and alert users.

The flow of the identification process in Example 2 will be explained using Figure 19.

Step 1:

The user begins a conversation on a communication device.

Specific operations involve a user initiating a call using a home phone or smartphone. The input is the user's voice, and the output is voice data collected by a voice input device.

Step 2:

The device's voice input device listens to the conversation and collects voice data.

Specifically, the device's microphone records the conversation between the user and the other party in real time. The input is the user's and the other party's voice, and the output is audio data stored in digital format.

Step 3:

The server converts the audio data into text data.

Specifically, the server uses speech recognition software (e.g., Google Speech-to-Text API) to convert the collected voice data into text data. The input is digital voice data, and the output is text conversation data.

Step 4:

The server sends the transcribed conversation data to the generative AI model.

Specific operations involve the server sending the converted text data to the generative AI model. The input is text-format conversation data, and the output is the data sent to the generative AI model.

Step 5:

A generative AI model analyzes conversation data to determine the likelihood of fraud.

Specifically, the generative AI model analyzes conversation data and detects specific keywords and phrases (e.g., "bank account" or "password"). The input is text-based conversation data, and the output is a judgment about the likelihood of fraud.

Step 6:

If there is a possibility of fraud, an audio alert will be issued via the device's voice input device.

Specifically, if the generative AI model determines that there is a high possibility of fraud, the server sends instructions to the terminal and an audio alert is emitted from the communication device's speaker saying, "There is a possibility of fraud. Please be careful." The input is the judgment result of the generative AI model, and the output is an audio alert.

Step 7:

The emotion analysis engine analyzes the user's tone of voice and speaking speed.

Specifically, the emotion analysis engine analyzes the user's tone of voice and speaking rate in real time. The input is the user's voice data, and the output is the analysis results regarding their emotional state.

Step 8:

If the sentiment analysis engine exceeds a certain threshold, the generative AI model will issue an audio alert.

Specifically, if the emotion analysis engine determines that the user's emotion exceeds a certain threshold, the generative AI model issues an audio alert saying, "This may be a scam. Please remain calm." The input is the analysis result of the emotion analysis engine, and the output is an audio alert.

(Application Example 2)

Next, we will explain Application Example 2 of Form Example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 will be referred to as the "terminal."

In recent years, fraudulent phone scams have become increasingly sophisticated, leaving many people vulnerable. Elderly people and those unfamiliar with technology are particularly vulnerable to these scams, as they are less vigilant against them. While systems for detecting fraudulent calls exist, few warning systems take the user's emotional state into account. This increases the risk of users becoming victims of fraudulent calls without realizing they are being scammed. Therefore, there is a need for a system that detects fraudulent calls and takes the user's emotional state into account.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

In this invention, the server includes means for detecting incoming fraudulent calls, means for listening to conversations from a voice input device installed in the smart device, means for converting the conversation data into text and sending it to the generative AI, means for issuing a voice alert from the voice input device if the generative AI determines that fraud is suspected from the conversation data, and means for analyzing the user's emotions using an emotion engine and issuing a warning if a specific threshold is exceeded. This makes it possible to detect fraudulent calls and issue warnings that take into account the user's emotional state.

A "fraudulent call" is a call made with the intent of defrauding someone of money or personal information through fraudulent means.

"Incoming call" refers to an incoming phone call.

A "smart device" is an electronic device with internet connectivity, such as a smartphone or tablet.

An "audio input device" is a device such as a microphone that converts audio into digital data.

"Conversation data" refers to data that has been converted into text from speech acquired through a voice input device.

"Generative AI" is a system that uses artificial intelligence to analyze data and make specific decisions or generate results.

"Audio alert" is a means of warning or notifying the user using audio.

An "emotion engine" is an algorithm or system that analyzes a user's emotional state from voice and text data.

A "specific threshold" is a numerical value or condition that serves as a benchmark when the emotion engine evaluates the user's emotional state.

A "warning" is a notification or alert that alerts the user.

A system for implementing this invention includes a series of means for detecting incoming fraudulent calls and issuing a warning to the user. A specific embodiment of the system is described below.

First, a voice input device (microphone) is installed on a smart device (e.g., a smartphone or tablet). This voice input device captures the user's call content in real time. The captured voice data is converted into text data using voice recognition software (e.g., the speech_recognition library).

The transcribed conversation data is then sent to a generative AI model (e.g., a model using the transformers library). This generative AI model analyzes the conversation data and determines whether there is a possibility of fraud. If a fraudulent activity is detected, software for issuing audio alerts (e.g., the pyttsx3 library) is activated to warn the user.

Furthermore, an emotion engine (e.g., an emotion analysis model using the transformers library) is used to analyze the user's emotional state. The emotion engine evaluates the user's tone of voice, speaking rate, etc., and issues a warning if certain thresholds are exceeded. This warning is also notified to the user as an audio alert.

For example, if a user calls and says, "Please tell me your bank account information," the generative AI model will detect the possibility of fraud and issue a voice alert. Also, if the user is feeling anger or anxiety, the emotion engine will detect this and issue a voice alert.

Examples of prompt statements include:

User's call: "Please tell me my bank account information."

Prompt for generative AI model: "Is this call potentially fraudulent?"

In this way, it is possible to create a system that can detect fraudulent calls and issue warnings that take into account the user's emotional state.

The flow of the specific processing in Application Example 2 is explained using Figure 20.

Step 1:

A user initiates a call on a smart device. The smart device's audio input device (microphone) captures the call content in real time. The input is the user's voice data, and the output is a stream of voice data.

Step 2:

The device uses speech recognition software (speech_recognition library) to convert the acquired voice data into text data. The input is a stream of voice data, and the output is transcribed conversation data.

Step 3:

The device sends the transcribed conversation data to a generative AI model (a model using the transformers library). The generative AI model analyzes the conversation data and determines whether there is a possibility of fraud. The input is the transcribed conversation data, and the output is a judgment result regarding the possibility of fraud.

Step 4:

The server receives the judgment results of the generative AI model, and if it determines that there is a possibility of fraud, it launches software (pyttsx3 library) to issue an audio alert. The input is the judgment result regarding the possibility of fraud, and the output is an audio alert.

Step 5:

The device uses an emotion engine (an emotion analysis model using the Transformers library) to analyze the user's emotional state. The emotion engine evaluates the user's tone of voice, speaking rate, etc., and issues a warning if certain thresholds are exceeded. The input is transcribed conversation data, and the output is an evaluation result regarding the user's emotional state.

Step 6:

The server receives the emotion engine's evaluation results, and if a certain threshold is exceeded, it launches software (pyttsx3 library) to issue an audio alert. The input is the evaluation result regarding the emotional state, and the output is an audio alert.

Step 7:

The user receives an audio alert and acknowledges the warning about potential fraud or emotional state. The input is the audio alert and the output is the user's acknowledgment.

(Example 3)

Next, Example 3 of Form Example 3 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 will be referred to as the "terminal."

Conventional fraud detection systems have the problem of low accuracy because they only analyze conversation data to determine the possibility of fraud. Furthermore, because they do not take the user's emotional state into account, there is a high risk of overlooking possible fraud. Furthermore, there are limited ways to notify users of possible fraud, which can lead to users not realizing the risk of fraud.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Example 3 is realized by the following means.

In this invention, the server includes means for detecting incoming fraudulent calls, means for listening to conversations from a voice input device installed in the communication terminal, means for converting the conversation data into text and transmitting it to the generative artificial intelligence, means for issuing a voice alert from the voice input device if the generative artificial intelligence determines that fraud is suspected from the conversation data, means for an emotion analysis engine to analyze the user's emotions, and means for the generative artificial intelligence to integrate information from the emotion analysis engine and the conversation data to determine the possibility of fraud. This makes it possible to detect the possibility of fraud with high accuracy and notify the user promptly.

"Means for detecting incoming fraudulent calls" refers to devices or software that monitor communication lines and identify incoming calls that may be fraudulent.

"A voice input device installed on a communication terminal" refers to a microphone attached to a communication device such as a smartphone or home telephone for collecting voice.

"Means for converting conversation data into text and sending it to generative AI" refers to software or a device for converting voice data into text data and sending that text data to generative AI.

"Generative AI" is an artificial intelligence model that uses natural language processing technology to analyze text data and detect specific patterns and phrases.

"Means for issuing audio alerts" refers to an audio output device that issues a warning to the user if the generative artificial intelligence detects the possibility of fraud.

An "emotion analysis engine" is software or a device that analyzes a user's voice and text data to identify their emotional state.

"Means for determining the likelihood of fraud" refers to an algorithm or software that uses generative artificial intelligence to integrate conversational data and information from a sentiment analysis engine to assess the likelihood of fraud.

This invention is a system that detects incoming fraudulent calls and notifies users of the possibility of fraud. A specific embodiment of this system is described below.

First, while a user is using a communication terminal (such as a smartphone or home phone), a means for detecting incoming fraudulent calls is activated. This means is a device or software that monitors communication lines and identifies incoming calls that may be fraudulent.

Next, a voice input device (microphone) installed on the communication terminal listens to the conversation between the user and the caller making the fraudulent call. This voice data is converted into text data by transcribing the conversation data and sending it to generative artificial intelligence. This conversion uses voice recognition technology. Specifically, voice recognition services such as Google Speech-to-Text API and IBM Watson Speech to Text can be used.

The server analyzes the text data using generative artificial intelligence (for example, natural language processing models such as BERT or GPT-3). The generative artificial intelligence detects specific phrases and patterns that may be fraudulent and evaluates the results. For example, if the phrase "Please transfer money" is included, it will determine that there is a high possibility of fraud.

Furthermore, the server uses an emotion analysis engine (for example, IBM Watson or Microsoft Azure's emotion analysis API) to analyze the user's emotions. It analyzes the user's voice and text data to identify emotional states such as anxiety or tension. The emotion data obtained by the emotion analysis engine is sent to the generative artificial intelligence.

Generative AI combines information from the sentiment analysis engine with conversational data to more accurately assess the likelihood of fraud. For example, if the message is "Please transfer money" and the user's emotions are anxious, it will conclude that there is a very high possibility of fraud.

Finally, if the server determines that the message is likely to be fraudulent, it will notify the user of this result. The notification is made by issuing an audio alert. Specifically, an audio alert will be issued from the communication device's speaker saying, "This message may be fraudulent. Please be careful."

As a concrete example, consider the case where a user receives a message saying, "Please transfer money." This message is picked up by the voice input device of the communication device and converted into text data. The server analyzes this text data using generative artificial intelligence and determines that there is a high possibility of fraud. At the same time, the emotion analysis engine detects the user's anxiety and sends this information to the generative artificial intelligence. The server combines this information and concludes that there is a very high possibility of fraud. Finally, the server sends an audio alert to the user's communication device saying, "This message may be fraudulent. Please be careful."

Example prompt:

"Please determine whether the following message is likely a scam. Message: 'Please transfer money.'"

In this way, the server, terminal, and user work together to detect the possibility of fraud with high accuracy and quickly notify the user. The flow of the identification process in Example 3 will be explained using Figure 21.

Step 1:

The user enters a message.

The user uses a communication device (such as a smartphone or home phone) to input the received message. For example, they might input a message such as "Please transfer money." The input message is saved on the device as voice data.

Step 2:

The device sends a message to the server.

The terminal sends the voice data entered by the user to the server. At this time, the terminal sends the voice data to the server via the Internet. The input is voice data, and the output is the transfer of voice data to the server.

Step 3:

The server converts the audio data into text data.

The server converts the received voice data into text data using voice recognition technology. Specifically, it uses voice recognition services such as Google Speech-to-Text API and IBM Watson Speech to Text. The input is voice data, and the output is text data.

Step 4:

The server analyzes the text data using generative artificial intelligence.

The server analyzes the text data using generative artificial intelligence (e.g., natural language processing models such as BERT or GPT-3). The generative artificial intelligence detects specific phrases and patterns that may be fraudulent and evaluates the results. The input is the text data, and the output is an evaluation result regarding the likelihood of fraud.

Step 5:

The server analyzes the user's emotions using an emotion analysis engine.

The server uses an emotion analysis engine (for example, IBM Watson or Microsoft Azure's emotion analysis API) to analyze the user's emotions. It analyzes the user's voice and text data to identify emotional states such as anxiety or tension. The input is voice or text data, and the output is emotion data.

Step 6:

The server combines the results of the generative artificial intelligence and the sentiment analysis engine to determine the likelihood of fraud.

The server integrates the analysis results from the generative AI with the emotional data from the emotion analysis engine. This allows for a more accurate assessment of the likelihood of fraud. For example, if the message is "Please transfer money" and the user's emotions are anxious, it will conclude that there is a very high likelihood of fraud. The input is the analysis results and emotional data, and the output is a final assessment of the likelihood of fraud.

Step 7:

The server will notify the user of possible fraud.

If the server determines that there is a high possibility of fraud, it notifies the user of the result. The notification is done by issuing an audio alert. Specifically, an audio alert saying "This message may be fraudulent. Please be careful" is issued from the speaker of the communication terminal. The input is the final assessment of the possibility of fraud, and the output is the audio alert.

(Application Example 3)

Next, we will explain Application Example 3 of Form Example 3. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 will be referred to as the "terminal."

Traditional fraud detection systems only detect specific phrases and patterns and do not take the user's emotional state into account, making it difficult to accurately determine the likelihood of fraud. Furthermore, there was a lack of means to provide appropriate warnings to users when fraud was likely.

The identification processing by the identification processing unit 290 of the data processing device 12 in Application Example 3 is realized by the following means. In this invention, the server includes means for detecting incoming fraudulent calls, means for listening to conversations from a smartphone application or a microphone installed in a home phone, means for converting the conversation data into text and sending it to the generative AI, means for issuing an audio alert from the microphone if the generative AI determines that fraud is suspected based on the conversation data, means for analyzing the user's emotions using an emotion engine, means for determining the possibility of fraud by combining information from the emotion engine with the conversation data analyzed by the generative AI, and means for displaying a warning to the user if there is a high possibility of fraud. This makes it possible to more accurately determine the possibility of fraud and provide the user with an appropriate warning.

"Means for detecting incoming fraudulent calls" refers to devices or software that monitor telephone lines or communications networks and detect incoming calls that may be fraudulent.

"A microphone installed in a smartphone application or home phone" is a voice input device attached to a smartphone or home phone to collect audio.

"Means for listening to conversations" refers to devices or software that analyze audio data collected through a microphone and recognize the content of the conversation.

"Means for converting conversation data into text and sending it to the generative AI" refers to a device or software that converts voice data into text data and sends that text data to the generative AI.

"Generative AI" is an artificial intelligence model trained to perform a specific task, in this case to determine the likelihood of fraud.

"Means for issuing an audio alert from the microphone when fraud is suspected" refers to a device or software that issues an alarm or message through the microphone when the generative AI detects the possibility of fraud.

An "emotion engine" is software or a device that analyzes a user's emotional state from their voice or text data.

"Means for analyzing user emotions" refers to devices or software that use an emotion engine to analyze the user's emotional state.

"Means for determining the possibility of fraud by combining information from the emotion engine with conversation data analyzed by generative AI" refers to devices or software that integrate emotional information obtained from the emotion engine with the results of analysis of conversation data by generative AI, and make a comprehensive determination of the possibility of fraud.

"Means for displaying a warning to the user in the event of a high likelihood of fraud" refers to devices or software that provide a visual or audible warning to the user in the event that a high likelihood of fraud is determined.

A system for implementing this invention includes a series of means for detecting incoming fraudulent calls and providing a warning to the user. Specific embodiments are described below.

First, the server is equipped with a means for detecting incoming fraudulent calls. This means is a device or software that monitors telephone lines and communication networks to detect potentially fraudulent calls. For example, specific phone numbers or callers can be registered on a blacklist, and fraudulent calls can be detected based on this.

Next, the device (smartphone application or home phone) is equipped with a microphone to pick up on the conversation. This microphone collects the voice during the call and sends it to the server as conversation data. The conversation data is then converted into text using voice recognition technology and sent to the generative AI.

Generative AI analyzes conversation data to determine the likelihood of fraud. In doing so, it uses models trained to detect specific phrases and patterns. For example, it can detect typical phrases such as "bank transfer fraud" and "it's me" fraud.

Furthermore, an emotion engine analyzes the user's emotions. The emotion engine is software that analyzes the user's emotional state from voice and text data. The emotional information obtained from the emotion engine is integrated with the results of the analysis of conversation data by generative AI to make a comprehensive judgment on the possibility of fraud.

If it determines that there is a high possibility of fraud, the device will display a warning to the user. This warning can be provided visually or audibly. For example, a warning message can be displayed on the smartphone screen or an audio alert can be issued.

For example, consider the following user input:

"Your account has been frozen. Please transfer the funds immediately."

"Hey, I need money right now."

By inputting these prompts into a generative AI model, it can analyze the possibility of fraud and, in conjunction with an emotion engine, display a warning to the user.

The flow of the specific processing in Application Example 3 will be explained using Figure 22.

Step 1:

The server monitors telephone lines and communication networks to detect incoming calls that may be fraudulent. The input is a signal from the telephone line or communication network, and the output is a flag indicating an incoming fraudulent call. Specifically, it registers specific phone numbers and callers on a blacklist and detects fraudulent calls based on that.

Step 2:

The device (smartphone application or home phone) collects audio during a call through a microphone. The input is the audio signal during the call, and the output is audio data. Specifically, the microphone converts the audio into a digital signal and sends that data to a server.

Step 3:

The server uses voice recognition technology to convert the voice data into text. The input is voice data and the output is text data. Specifically, the voice recognition software analyzes the voice data and converts it into corresponding text.

Step 4:

The server sends text data to the generative AI, which analyzes it for the possibility of fraud. The input is text data, and the output is a score or flag indicating the possibility of fraud. Specifically, the generative AI model detects specific phrases and patterns and determines the possibility of fraud.

Step 5:

The server uses an emotion engine to analyze the user's emotions. The input is text data, and the output is data indicating the user's emotional state. Specifically, the emotion engine analyzes the text data and identifies the user's emotional state.

Step 6:

The server combines information from the emotion engine with conversation data analyzed by the generative AI to make a comprehensive judgment on the possibility of fraud. The input is emotional state data and a score or flag indicating the possibility of fraud, and the output is a final flag indicating the possibility of fraud. Specifically, it integrates emotional information with the analysis results of the generative AI to make a highly accurate judgment on the possibility of fraud.

Step 7:

If it determines that there is a high possibility of fraud, the device will display a warning to the user. The input is a flag indicating the final possibility of fraud, and the output is a warning message or audio alert. Specifically, a warning message is displayed on the smartphone screen or an audio alert is issued.

The specific processing unit 290 transmits the results of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the results of the specific processing. The microphone 238 acquires audio indicating the user input regarding the results of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

Data generation model 58 is what is known as generative AI (artificial intelligence). An example of data generation model 58 is generative AI such as ChatGPT (Internet search <URL: https://openai.com/blog/chatgpt>). Data generation model 58 is obtained by performing deep learning on a neural network. A prompt containing an instruction is input to data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. Data generation model 58 performs inference on the input inference data in accordance with the instructions indicated by the prompt, and outputs the inference results in the form of data such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and/or summarization.

Another example of generative AI is Gemini (Internet search <URL: https://gemini.google.com/?hl=ja>).

In the above embodiment, an example was given in which the specific processing was performed by the data processing device 12, but the technology disclosed herein is not limited to this, and the specific processing may also be performed by the robot 414.

The emotion identification model 59, which serves as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see Figure 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

Figure 9 shows an emotion map 400 on which multiple emotions are mapped. In emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both emotions and mental states. Emotions that are generally generated from reactions that occur in the brain are arranged on the left side of the concentric circles. Emotions that are generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions that are generally generated from reactions that occur in the brain and are induced by situational judgment are arranged above and below the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the top side of the concentric circles, and the emotion of "discomfort" is arranged on the bottom side. In this way, emotion map 400 maps multiple emotions based on the structure by which emotions are generated, with emotions that tend to occur simultaneously being mapped close together.

These emotions are distributed in the 3 o'clock direction on emotion map 400, and usually fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness takes precedence over internal sensations, resulting in a sense of calm.

The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further out you go on emotion map 400, the more visible (expressed in behavior) the emotion becomes.

Here, human emotions are based on various balances such as posture and blood sugar levels, and when these balances deviate from the ideal, it indicates discomfort, and when they approach the ideal, it indicates pleasure. Emotions can also be created for robots, cars, motorcycles, etc., based on various balances such as posture and remaining battery life, so that when these balances deviate from the ideal, it indicates discomfort, and when they approach the ideal, it indicates pleasure. Emotion maps may be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on speech emotion recognition and emotional brain physiological signal analysis systems, Tokushima University, doctoral dissertation: https://ci.nii.ac.jp/naid/500000375379). The left half of the emotion map is lined with emotions belonging to an area called "reaction," where sensation is dominant. The right half of the emotion map is lined with emotions belonging to an area called "situation," where situational awareness is dominant.

The emotion map defines two emotions that encourage learning. One is the negative emotion around the middle of "repentance" or "reflection" on the situation side. In other words, this is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the positive emotion around "desire" on the response side. In other words, this is when the robot experiences positive feelings such as "I want more" or "I want to know more."

The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are close to each other have similar values, as in the emotion map 900 shown in Figure 10. Figure 10 shows an example in which multiple emotions, such as "relieved," "calm," and "reassuring," have similar emotion values.

In the above embodiment, an example was given in which a specific process is performed by a single computer 22, but the technology disclosed herein is not limited to this, and distributed processing of the specific process may also be performed by multiple computers, including computer 22.

In the above embodiment, an example was described in which the specific processing program 56 is stored in the storage 32, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

It is not necessary to store the entire specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entire specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

The hardware resources that execute specific processes can include the following types of processors. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource that executes specific processes by executing software, i.e., a program. Processors also include FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), and ASICs (Application Specific Integrated Circuits), which are specially designed to execute specific processes.

Specialized electrical circuits are processors with specialized circuit configurations. All processors have built-in or connected memory, and all processors use memory to perform specific processes.

The hardware resource that executes the specific processing may be composed of one of these various processors, or may be composed of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). The hardware resource that executes the specific processing may also be a single processor.

As an example of a configuration using a single processor, first, there is a configuration in which one processor is configured using a combination of one or more CPUs and software, and this processor functions as a hardware resource that executes specific processing. Second, there is a configuration in which a processor is used to realize the functions of an entire system, including multiple hardware resources that execute specific processing, on a single IC chip, as typified by SoC (System-on-a-chip). In this way, specific processing is realized using one or more of the various processors listed above as hardware resources.

More specifically, the hardware structure of these various processors can be an electrical circuit that combines circuit elements such as semiconductor devices. Furthermore, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps can be added, or the processing order can be rearranged, all within the scope of the spirit of the invention.

The above-described written content and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or substitutions may be made to the above-described written content and illustrations, as long as they do not deviate from the spirit of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described written content and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

All publications, patent applications, and technical standards mentioned in this specification are incorporated by reference herein to the same extent as if each individual publication, patent application, and technical standard was specifically and individually indicated to be incorporated by reference.

The following is further disclosed regarding the above embodiments.

(Claim 1)

A system comprising: a means for detecting incoming fraudulent calls; a means for listening to conversations from a smartphone application or a microphone installed on a home phone; a means for converting the conversation data into text and sending it to a generative AI; and a means for issuing an audio alert from the microphone if the generative AI determines that fraud is suspected based on the conversation data.

(Claim 2)

The system of claim 1, wherein the generative AI determines that fraud is suspected by detecting specific phrases or patterns that indicate the possibility of specialized fraud.

(Claim 3)

The system of claim 1, wherein the audio alert alerts users to potential fraud.

(Claim 4)

The system of claim 1, wherein the generative AI includes an emotion engine that recognizes the user's emotions.

(Claim 5)

The system described in claim 4, wherein the emotion engine analyzes emotions from the user's tone of voice, volume, speaking rate, etc.

(Claim 6)

The system of claim 4, wherein the emotion engine issues an audio alert when the user's emotion exceeds a certain threshold.

"Example 1"

(Claim 1)
means for detecting incoming fraudulent calls using a predetermined list of incoming calls;
means for collecting and acquiring conversation data from a voice input device installed in the communication terminal;
A means for transcribing the conversation data and inputting instructions for analyzing the transcribed conversation data into a generative AI model;
and means for issuing an audio alert from the audio input/output device of the communication terminal if the generative AI model analyzes the conversation data and determines that fraud is suspected.

(Claim 2)
The system of claim 1 , wherein the generative AI model determines that fraud is suspected if it detects specific phrases or patterns that indicate the possibility of specialized fraud.

(Claim 3)
The system of claim 1 , wherein the audio alert emitted from the audio output device alerts a user to potential fraud.

"Application Example 1"

(Claim 1)
A means for detecting incoming fraudulent calls;
means for listening to a conversation from a voice input device installed in a communication terminal;
A means for converting conversation data into text and transmitting it to a generative AI;
means for issuing a voice alert from the voice output device when the generative AI determines that fraud is suspected from the conversation data;
means for the generative AI to detect specific phrases or patterns indicative of potential fraud;
said audio alert alerting the user to potential fraud;
A system including:

(Claim 2)
10. The system of claim 1, wherein the generative AI determines that fraud is suspected by detecting specific phrases or patterns indicative of possible fraud.

(Claim 3)
The system of claim 1 , wherein the audio alert alerts the user to potential fraud.

"Example 2"

(Claim 1)
A means for detecting incoming fraudulent calls;
means for collecting conversation from a voice input device installed in the communication device;
A means for converting conversation data into text data;
a means for transmitting the character data to the generative AI model;
The system includes a means for issuing an audio alert from the audio input device if the generative AI model determines that fraud is suspected from the conversation data.

(Claim 2)
The system of claim 1 , wherein the generative AI model determines that fraud is suspected by detecting specific phrases or patterns that indicate the possibility of specialized fraud.

(Claim 3)
The system of claim 1 , wherein the audio alert alerts the user to potential fraud.

"Application Example 2"

(Claim 1)
A means for detecting incoming fraudulent calls;
means for listening to a conversation from a speech recognition device;
A means for converting conversation data into text and transmitting it to a generative AI;
means for issuing a voice alert from a voice output device when the generative AI determines that fraud is suspected from the conversation data;
means including an application installed on a smartphone;
A system including:

(Claim 2)
The system of claim 1 , wherein the generative AI determines that fraud is suspected by detecting specific phrases or patterns that indicate the possibility of specialized fraud.

(Claim 3)
The system of claim 1 , wherein the audio alert alerts the user to potential fraud.

"Example 3"

(Claim 1)
A means for detecting incoming fraudulent calls;
means for listening to a conversation from a voice input device installed in a communication terminal;
A means for transcribing conversation data and transmitting it to a generative artificial intelligence;
means for issuing a voice alert from the voice input device when the generative artificial intelligence determines that fraud is suspected from the conversation data;
means for transmitting text data entered by a user to a server;
A means for the server to analyze the text data using a generative artificial intelligence model to determine the likelihood of fraud;
A means for the server to transmit the analysis result to the communication terminal;
a means for the communication terminal to display the analysis result to a user;
A system including:

(Claim 2)

The system described in claim 1, wherein the generative artificial intelligence determines that fraud is suspected by detecting specific phrases or patterns that indicate the possibility of specialized fraud.

(Claim 3)
The system of claim 1 , wherein the audio alert alerts a user to potential fraud.

"Application Example 3"

(Claim 1)
A means for detecting incoming fraudulent calls;
means for listening to a conversation from a voice input device installed in a communication terminal;
A means for converting conversation data into text and transmitting it to a generative AI;
means for issuing a voice alert from the voice input device when the generative AI determines that fraud is suspected from the conversation data;
means for the generative AI to detect specific phrases or patterns indicative of potential fraud;
means for alerting a user if the generative AI detects certain phrases or patterns that indicate potential fraud; and
A system including:

(Claim 2)
The system of claim 1 , wherein the generative AI determines that fraud is suspected by detecting specific phrases or patterns that indicate the possibility of specialized fraud.

(Claim 3)
The system of claim 1 , wherein the audio alert alerts the user to potential fraud.

"Example 1: Combining Emotion Engines"

(Claim 1)
means for detecting incoming fraudulent calls using a predetermined list of incoming calls;
means for collecting and acquiring conversation data from a voice input device installed in the communication terminal;
A means for transcribing the conversation data and inputting instructions for analyzing the transcribed conversation data to a generative AI;
means for issuing a voice alert from a voice output device of the communication terminal when the generative AI analyzes the conversation data and determines that fraud is suspected;
means for collecting parameters relating to the user's voice in the conversation data and performing sentiment analysis;
and means for reassessing the likelihood of fraud based on the results of the sentiment analysis.

(Claim 2)
The system of claim 1 , wherein the generative artificial intelligence determines that fraud is suspected by detecting specific phrases or patterns that indicate the possibility of specialized fraud.

(Claim 3)
The system of claim 1 , wherein the audio alert emitted from the audio output device alerts a user to potential fraud.

"Application example 1: Combining emotion engines"

(Claim 1)
A means for detecting incoming fraudulent calls;
means for listening to a conversation from a voice input device installed in a communication terminal;
A means for converting conversation data into text and transmitting it to a generative AI;
means for issuing a voice alert from the voice input device when the generative AI determines that fraud is suspected from the conversation data;
a means including an emotion engine for analyzing emotions based on the tone, volume, and speaking rate of a user's voice;
and measures to further increase the likelihood of fraud based on the results of the emotion engine analysis.
A system including:

(Claim 2)
2. The system of claim 1, wherein the generative AI determines that fraud is suspected by detecting specific phrases or patterns that indicate the possibility of specialized fraud.

(Claim 3)
The system of claim 1 , wherein the audio alert alerts the user to potential fraud.

"Example 2: Combining Emotion Engines"

(Claim 1)
A means for detecting incoming fraudulent calls;
means for listening to a conversation from a voice input device installed in the communication device;
A means of transcribing the conversation data and sending it to the generative AI model;
means for issuing a voice alert from the voice input device when the generative AI model determines that fraud is suspected from the conversation data;
The emotion analysis engine analyzes the tone and speed of the user's voice,
A system including a means for the generative AI model to issue an audio alert when the sentiment analysis engine exceeds a certain threshold.

(Claim 2)
10. The system of claim 1, wherein the generative AI model determines that fraud is suspected by detecting specific phrases or patterns indicative of the possibility of specialized fraud.

(Claim 3)
The system of claim 1 , wherein the audio alert alerts the user to potential fraud.

"Application Example 2: Combining Emotion Engines"

(Claim 1)
A means for detecting incoming fraudulent calls;
A means for listening to a conversation from a voice input device installed on a smart device;
A means of transcribing conversation data and sending it to generative AI,
means for issuing a voice alert from the voice input device when the generative AI determines that fraud is suspected from the conversation data;
A means for analyzing user emotions using an emotion engine and issuing a warning when certain thresholds are exceeded;
A system including:

(Claim 2)
The system of claim 1, wherein the generative AI determines that fraud is suspected by detecting specific phrases or patterns that indicate the possibility of specialized fraud.

(Claim 3)
The system of claim 1 , wherein the audio alert alerts the user to potential fraud.

"Example 3: Combining Emotion Engines"

(Claim 1)
A means for detecting incoming fraudulent calls;
means for listening to a conversation from a voice input device installed in a communication terminal;
A means for transcribing conversation data and transmitting it to a generative artificial intelligence;
means for issuing a voice alert from the voice input device when the generative artificial intelligence determines that fraud is suspected from the conversation data;
A means for the sentiment analysis engine to analyze the sentiment of the user;
A system that includes a means for a generative artificial intelligence to integrate information from a sentiment analysis engine with conversational data to determine likelihood of fraud.

(Claim 2)

The system described in claim 1, wherein the generative artificial intelligence determines that fraud is suspected by detecting specific phrases or patterns that indicate the possibility of specialized fraud.

(Claim 3)

The system of claim 1, wherein the audio alert alerts the user to potential fraud.

"Application Example 3: Combining Emotion Engines"

(Claim 1)
A means for detecting incoming fraudulent calls;
a means for listening to the conversation through a smartphone application or a microphone installed on a home phone;
A means for converting conversation data into text and transmitting it to a generative AI;
means for issuing an audio alert from the microphone when the generative AI determines that fraud is suspected from the conversation data;
A means for analyzing user emotions using an emotion engine;
A means to determine the possibility of fraud by combining information from the emotion engine with conversation data analyzed by generative AI; and
The system includes a means to warn users when fraud is likely.

(Claim 2)
The system of claim 1 , wherein the generative AI determines that fraud is suspected by detecting specific phrases or patterns that indicate the possibility of specialized fraud.

(Claim 3)
The system of claim 1 , wherein the audio alert alerts the user to potential fraud.

10, 210, 310, 410 Data processing system 12 Data processing device 14 Smart device 214 Smart glasses 314 Headset type terminal 414 Robot

Claims (3)

means for detecting an incoming fraudulent call that may be fraudulent using a predetermined list of incoming calls , and notifying a communication terminal of the incoming fraudulent call;
means for collecting and acquiring conversation data of the fraudulent call notified from a voice input device installed in the communication terminal;
A means for transcribing the conversation data and inputting instructions for analyzing the transcribed conversation data to a generative AI;
means for instructing the communication terminal to issue an audio alert by sending a command to play an audio alert to an audio output device of the communication terminal when the generative AI analyzes the conversation data and determines that fraud is suspected;
means for collecting parameters relating to the user's voice in the conversation data and performing sentiment analysis;
means for reassessing the likelihood of fraud based on the results of said sentiment analysis;
A system including: The system of claim 1, wherein the generative AI determines that fraud is suspected when it detects a specific phrase or pattern that indicates the possibility of specialized fraud. The system of claim 1, wherein the audio alert emitted from the audio output device alerts the user to possible fraud.