JP7808657B2 - system - Google Patents

Description

The technology disclosed herein relates to a system.

Patent document 1 discloses a persona chatbot control method executed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot's character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Japanese Patent Application Laid-Open No. 2022-180282

Creating a story involves many intertwining elements, such as story structure, character dialogue, scenic descriptions, plot twists, and creating moving scenes, and only by properly combining these elements can an interesting story emerge. However, creating all of these elements by hand takes an immense amount of time and effort. Additionally, creating the graphics and music to accompany a story requires specialized knowledge and skills, making it difficult for one person to do all of this alone.

This invention automatically creates character dialogue, scenery, and story structure when story settings such as the story's theme, setting, and characters are input. It also automatically generates plot twists, mystery elements, and moving scenes to make the story more interesting. It then automatically creates graphics and music that match each scene in the story. This enhances the dramatic effect of the story structure, character dialogue, images, and music, making it possible to effectively stir the viewer's emotions.

1 is a conceptual diagram illustrating an example of the configuration of a data processing system according to a first embodiment. 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. FIG. 10 is a conceptual diagram illustrating an example of the configuration of a data processing system according to a second embodiment. FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. FIG. 10 is a conceptual diagram illustrating an example of the configuration of a data processing system according to a third embodiment. FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. 1 shows an emotion map onto which multiple emotions are mapped. 1 shows an emotion map onto which multiple emotions are mapped. FIG. 10 is a sequence diagram showing a flow of processing in the data processing system according to the first embodiment of the first form example. FIG. 10 is a sequence diagram showing a processing flow of the data processing system in Application Example 1 of Form Example 1. FIG. 10 is a sequence diagram showing a processing flow of a data processing system in a second embodiment of the second form example. FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 of Form Example 2. FIG. 10 is a sequence diagram showing a processing flow of a data processing system in a third embodiment of the third form example. FIG. 13 is a sequence diagram showing the flow of processing in the data processing system in Application Example 3 of Form Example 3. FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in the first embodiment of the first form example when an emotion engine is combined. FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1 of Form Example 1 when an emotion engine is combined. FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in the second embodiment of the second form example when an emotion engine is combined. FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 of Form Example 2 when an emotion engine is combined. FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in the third embodiment of the third form example when an emotion engine is combined. FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 3 of Form Example 3 when an emotion engine is combined.

Below, an example of an embodiment of a system relating to the technology disclosed herein will be described with reference to the accompanying drawings.

First, let me explain the terminology used in the following explanation.

In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Furthermore, a processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose Computing on Graphics Processing Units), an APU (Accelerated Processing Unit), or a TPU (TENSOR PROCESSING UNIT (registered trademark)).

In the following embodiments, coded random access memory (RAM) is memory in which information is temporarily stored and is used by the processor as work memory.

In the following embodiments, the coded storage refers to one or more non-volatile storage devices that store various programs, parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

In the following embodiments, a communication I/F (Interface) with a symbol is an interface that includes a communication processor, an antenna, etc. The communication I/F controls communication between multiple computers. Examples of communication standards that can be applied to the communication I/F include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

In the following embodiments, "A and/or B" is synonymous with "at least one of A and B." In other words, "A and/or B" means that it may be just A, just B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and/or B" also applies when three or more things are expressed connected by "and/or."

[First embodiment]

Figure 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

As shown in FIG. 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

The data processing device 12 includes a computer 22, a database 24, and a communication I/F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and communication I/F 26 are also connected to the bus 34. The communication I/F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and/or a LAN (Local Area Network).

The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I/F 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

The reception device 38 is equipped with a touch panel 38A, a microphone 38B, etc., and receives user input. The touch panel 38A detects contact with an indicator (e.g., a pen or finger) to receive user input via the indicator. The microphone 38B detects the user's voice to receive user input via voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

The output device 40 is equipped with a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible by the user 20 (e.g., audio and/or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

The communication I/F 44 is connected to the network 54. The communication I/Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54.

Figure 2 shows an example of the main functions of the data processing device 12 and smart device 14.

As shown in FIG. 2, in the data processing device 12, specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

Storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

Next, we will explain the identification process performed by the identification processing unit 290 of the data processing device 12.

"Example 1"

The system of the present invention receives input from the user about the story setting, such as the story's theme, setting, and characters. This input is in text format, such as "A knight's adventure story set in medieval Europe." Next, based on this input, the system automatically creates character dialogue, scenes, and a story structure. This creation is carried out using existing story databases and artificial intelligence technology, generating dialogue and scenes that match the story flow, character personalities, and story theme.

"Example 2"

In addition, the system automatically generates hints, mystery elements, and moving scenes to make the story more interesting. This is done by laying hints at the right time, sprinkling mystery elements, and creating moving scenes in line with the story's theme, the characters' personalities, and the story's progression. For example, it automatically generates hints that people the knight meets during his adventure will play an important role later in the story, puzzles the knight faces, and moving scenes when the knight overcomes difficulties.

"Example 3"

The system then automatically creates graphics and music that match each scene in the story you've created. The graphics and music are created taking into account the atmosphere of the scene and the emotional state of the characters. For example, powerful graphics and music that heighten tension are created for battle scenes, while beautiful landscape graphics and moving music are created for moving scenes. This makes it possible to enhance the dramatic effect of the story's structure, the characters' conversations, images, and music, effectively stirring the viewer's emotions.

The processing flow for each example form is explained below.

"Example 1"

Step 1: The user provides textual input of the story setting, including the story's theme, setting, and characters. For example, it could be provided as "a knight's adventure story set in medieval Europe."

Step 2: Based on the input, the system automatically creates character dialogue, scenery, and story structure. This is done using existing story databases and artificial intelligence technology.

Step 3: Lines and scenes are generated that match the story flow, character personalities, and theme of the story.

"Example 2"

Step 1: The system automatically generates plot twists, mystery elements, moving scenes, and more to make the story more interesting.

Step 2: Lay out hints at the right time, sprinkle in mystery elements, and create moving scenes in line with the story's theme, the characters' personalities, and the story's progression.

Step 3: For example, foreshadowing, such as people the knight meets during his adventure playing important roles later in the story, puzzles the knight faces, and moving scenes when the knight overcomes difficulties, are automatically generated.

"Example 3"

Step 1: The system automatically creates graphics and music that match each scene in the story you've created.

Step 2: Graphics and music are created taking into account the atmosphere of the scene and the emotional state of the characters.

Step 3: For example, powerful graphics and music that heightens the tension are created for battle scenes, while beautiful landscape graphics and moving music are created for moving scenes. This enhances the dramatic effect of the story structure, character dialogue, images and music, making it possible to effectively stir the viewer's emotions.

(Example 1)

Next, we will explain Example 1 of Form Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the smart device 14 will be referred to as the "terminal."

When conventional story generation systems automatically generate a story based on a user-entered story setting, they struggle to maintain consistency in the story flow, character dialogue, and scenery. They also lack the ingenuity to ensure the generated story is moving and entertaining for the viewer. Furthermore, they are unable to automatically create graphics and music that match each scene in the story, which means they are unable to effectively move the viewer's emotions.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

In this invention, the server includes: means for inputting story settings such as the story's theme, setting, and characters; means for automatically creating character dialogue, scenery, and story structure based on the input story settings; means for automatically generating foreshadowing, mystery elements, and moving scenes to make the created story more interesting; means for automatically producing graphics and music that match each scene of the created story; means for enhancing the dramatic effect with the story structure, character dialogue, and images and music to effectively stir the viewer's emotions; means for receiving input from the user and analyzing it using natural language processing technology; means for generating prompt sentences to be input into the generative AI model based on the analysis results; means for automatically generating story dialogue, scenery, and structure using the generative AI model; and means for formatting the generated story and providing it to the user. This makes it possible to automatically generate a consistent story based on user input and provide the viewer with a moving and entertaining story.

A "story theme" is the central subject or theme of a story, and is the element that determines the overall direction and atmosphere of the story.

The "setting" refers to the place and historical background in which the story unfolds, and is an important element in shaping the setting and atmosphere of the story.

"Characters" are characters who act within a story and move the story forward, and are elements that have a major influence on the story's development and theme.

"Story setting" refers to the setting that includes basic elements such as the story's theme, setting, and characters, and is the element that forms the framework of the story.

"Dialogue" refers to the words or lines spoken by characters, and is an important element in expressing the progression of the story and the character's personality.

"Scenes" refer to the scenery and scenes depicted in a story, and are elements that visually express the atmosphere and emotions of the story.

"Story structure" refers to the overall flow and structure of a story, including the beginning, middle, and end, and is an element that makes the story unfold effectively.

"Foreshadowing" is a hint or sign that relates to later developments in a story, and is an element that adds consistency and depth to a story.

"Mystery elements" are elements that present mysteries or questions within a story and attract the interest of readers or viewers.

A "moving scene" is a scene in a story that strongly stirs emotions and is an impressive element that stays in the viewer's mind.

"Graphics" refers to images and illustrations that visually represent each scene in the story, and are elements that visually complement the atmosphere and emotion of the story.

"Songs" are melodies and sounds that are musically expressed to match each scene in a story, and are elements that musically complement the atmosphere and emotions of the story.

"Natural language processing technology" is a technology that enables computers to understand and analyze human language, and is used to analyze text data and extract information.

A "generative AI model" is a model that uses artificial intelligence technology to generate text and data, and is used to automatically generate dialogue, scenes, and structure for a story.

A "prompt" is an instruction to be input into a generative AI model, and is an element that guides the content generated by the model.

"Formatting" refers to the process of adding paragraph breaks and line breaks to make the generated narrative text easier to read, before it is presented to the user.

This invention is a system that automatically generates a story based on story settings such as the story's theme, setting, and characters, which are input by the user. Specific embodiments of this system are described below.

User input

The user uses a terminal to enter text information about the story's theme, setting, characters, etc. into the system's input form. For example, they might enter "A knight's adventure story set in medieval Europe."

Receive and analyze input data by the server

The server receives the text data sent by the user. It then analyzes the input data using natural language processing technology. Specifically, it uses the Python (registered trademark) NLTK library and spaCy to extract information such as theme, setting, and characters from the input text. For example, it extracts keywords such as "medieval Europe," "knights," and "adventure story."

Generating prompt statements

The server generates a prompt to be input into the generative AI model based on the extracted information. For example, if the user's input is "A knight's adventure story set in medieval Europe," the prompt would be "Please generate lines and scenes for the characters in a knight's adventure story set in medieval Europe."

Generating stories using generative AI models

The server inputs the generated prompt into a generative AI model (for example, OpenAI's (registered trademark) GPT-4 (registered trademark)). The generative AI model automatically generates the dialogue, scenery, and structure of the story based on the prompt. For example, it could generate text such as, "The brave knight Arthur met a wizard in a dark forest. The wizard said to Arthur, 'Deep in this forest lies an ancient dragon. Your mission is to slay the dragon.'"

Format and provide generated results

The server receives the story text returned by the generative AI model and formats it as needed, for example by adding paragraph breaks and line breaks to make it easier to read. The formatted story is then converted into a format that is easy for the user to read. Finally, the server provides the formatted story to the user, who can view the results through their device.

Specific examples

As an example of user input, consider "a knight's adventure story set in medieval Europe." Based on this input, the server generates the following prompt:

"Generate lines and scenes from characters in a knight's adventure story set in medieval Europe."

The generative AI model generates a story based on this prompt, outputting text such as the following:

"The brave knight Arthur met a wizard in a dark forest. The wizard told Arthur, 'Deep in this forest lies an ancient dragon. Your mission is to defeat the dragon.'"

In this way, it is possible to automatically generate a coherent story based on user input, providing an inspiring and entertaining story for the viewer.

The flow of the identification process in Example 1 will be explained using Figure 11.

Step 1:

The user uses their device to enter text information about the story's theme, setting, characters, etc. into the system's input form. For example, they might enter "A knight's adventure story set in medieval Europe." This input data is then sent to the server.

Step 2:

The server receives text data sent by the user. It analyzes the received data using natural language processing technology. Specifically, it uses Python's NLTK library and spaCy to extract information such as theme, setting, and characters from the input text. For example, it extracts keywords such as "medieval Europe," "knights," and "adventure story." The results of this analysis become the input for the next step.

Step 3:

Based on the analysis results, the server generates a prompt to be input into the generative AI model. For example, if the user's input is "A knight's adventure story set in medieval Europe," the prompt would be "Please generate lines and scenes for the characters in a knight's adventure story set in medieval Europe." This prompt becomes the input for the next step.

Step 4:

The server inputs the generated prompt into a generative AI model (for example, OpenAI's GPT-4). The generative AI model automatically generates the dialogue, scenery, and structure of a story based on the prompt. For example, it might generate text such as, "The brave knight Arthur met a wizard in a dark forest. The wizard said to Arthur, 'Deep in this forest lies an ancient dragon. Your mission is to slay the dragon.'" This generated text becomes the input for the next step.

Step 5:

The server receives the story text returned by the generative AI model and formats it as needed, for example by adding paragraph breaks and line breaks to make it easier to read. The formatted story is then converted into a format that is easier for the user to read. This formatted text becomes the input for the next step.

Step 6:

The server provides the formatted story to the user. The user can view the results through their device, for example, on a web page or application screen that displays the generated story. The user can then read and enjoy the generated story.

(Application Example 1)

Next, we will explain Application Example 1 of Form Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the smart device 14 will be referred to as the "terminal."

Existing story generation systems automatically generate stories based on user-entered themes and settings, but they often lack sufficient story structure, character dialogue, and scene generation, making it difficult to effectively sway viewers' emotions. Furthermore, the lack of functionality to preview, save, and share the generated stories creates a problem of inconvenient user experience.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

In this invention, the server includes: means for inputting story settings such as the story's theme, setting, and characters; means for automatically creating character dialogue, scenery, and story structure based on the input story settings; means for automatically generating foreshadowing, mystery elements, and moving scenes to make the created story more interesting; means for automatically producing graphics and music that match each scene of the created story; means for enhancing the dramatic effect with the story structure, character dialogue, images, and music to effectively stir the viewer's emotions; means for generating prompt sentences using a generative AI model and automatically generating a story; and means for previewing, saving, and sharing the generated story. This makes it possible to automatically generate high-quality stories based on the settings input by the user, effectively stirring the viewer's emotions.

A "story theme" is the central subject or theme of a story, and determines the overall direction and atmosphere of the story.

The "setting" refers to the place and historical background in which the story unfolds, and is an element that shapes the setting and atmosphere of the story.

"Characters" are characters that appear in a story and play an important role in the progression and development of the story.

"Story setting" refers to the setting that includes basic elements such as the story's theme, setting, and characters, and forms the framework of the story.

"Dialogue" refers to the words spoken by characters, and is an important element in expressing the progression of the story and the character's personality.

"Scenes" refer to the landscapes and scenes depicted in a story, and visually express the atmosphere and emotions of the story.

"Story structure" refers to the overall flow and structure of a story, including a beginning, middle, and end, and is the framework for effectively developing the story.

"Foreshadowing" is a hint or omen that relates to later developments in a story, and is an element that makes a story more interesting.

"Mystery elements" are elements that add mystery and suspense to a story, and are meant to attract the interest of readers and viewers.

A "moving scene" is a scene in a story that stirs strong emotions and is intended to leave a deep impression on the viewer.

"Graphics" refers to the visual images and illustrations that accompany each scene in the story, and are intended to visually express the atmosphere and emotions of the story.

"Songs" are pieces of music created to accompany each scene in a story, and are intended to musically express the atmosphere and emotions of the story.

A "generative AI model" is a model that uses artificial intelligence technology to automatically generate text, images, etc., and is used to automatically generate stories.

A "prompt" is an instruction to be input into a generative AI model, and is generated based on the theme and setting of the story.

"Preview" is a feature that allows users to preview the generated story in advance and check the content.

"Preservation" refers to recording the stories that are created in digital form so that they can be reused and shared later.

"Sharing" refers to sharing the story you create with other users and platforms, with the aim of promoting the spread and appreciation of the story.

A system for implementing this invention allows the user to input story settings such as the story's theme, setting, and characters, and automatically generates a story based on these settings. Specific embodiments of this system are described below.

System Configuration

This system is primarily composed of the following hardware and software:

Hardware: Smartphones, servers

Software: Python, OpenAI API

Program processing

User input

Users enter the story's theme, setting, and characters in text form through a smartphone application. For example, they might enter the theme "A knight's adventure story set in medieval Europe."

Prompt generation

The server generates a prompt based on the information entered by the user. This prompt is an instruction that is input into the generative AI model. As a specific example, the following prompt is generated:

Theme: A knight's adventure story set in medieval Europe

Setting: Medieval Europe

Characters: Brave knight, wizard, dragon

Generate a story.

Story generation

The server uses a generative AI model (e.g., GPT-4) to automatically generate a story based on the generated prompt. This generative AI model is called through the OpenAI API.

Preview, save, share

The generated story is displayed so that users can preview it through the smartphone application. Users can save the generated story and share it on social media, etc.

Specific examples

If a user sets the theme to "a knight's adventure story set in medieval Europe" and enters "a brave knight, a wizard, and a dragon" as the characters, the following story will be generated.

Theme: A knight's adventure story set in medieval Europe

Setting: Medieval Europe

Characters: Brave knight, wizard, dragon

Generate a story.

By inputting this prompt into a generative AI model, a story is automatically generated. Users can preview, save, and share the generated story.

Hardware and software used

Hardware: Smartphones, servers

Software: Python, OpenAI API

This makes it possible to automatically generate high-quality stories based on user-entered settings, effectively stirring the viewer's emotions.

The flow of the specific processing in Application Example 1 will be explained using Figure 12.

Step 1:

Users use their smartphone to input the story's theme, setting, and characters in text format. The input data is then sent from the device to the server. An example input includes the theme "A knight's adventure story set in medieval Europe."

Step 2:

The server generates a prompt based on the received user input data. Specifically, it combines information about the theme, setting, and characters to create a prompt that is suitable for the generative AI model. The generated prompt looks like this:

Theme: A knight's adventure story set in medieval Europe

Setting: Medieval Europe

Characters: Brave knight, wizard, dragon

Generate a story.

This prompt text is generated.

Step 3:

The server calls a generative AI model (e.g., GPT-4) with the generated prompt text. It sends the prompt text using the OpenAI API and requests the generation of a story. The input is the prompt text, and the output is the generated story text.

Step 4:

The generative AI model automatically generates a story based on the prompt text. The generated story text is returned to the server. For example, the following story might be generated:

Once upon a time, in a vast kingdom in medieval Europe, there lived a brave knight. His name was Arthur. One day, with the help of a wizard, he embarked on an adventure to battle a fearsome dragon.

Step 5:

The server sends the text of the generated story to the smartphone device. The user can preview the generated story through the smartphone device. The input is the text of the generated story, and the output is the preview screen displayed on the user's device.

Step 6:

Users can save the generated story or share it on social media. Saved stories are recorded in the device's storage, and shared stories are sent to the specified platform. The input is the user's selection, and the output is the saved file or shared content.

These steps make it possible to automatically generate a high-quality story based on the settings entered by the user, effectively stirring the viewer's emotions.

(Example 2)

Next, we will explain Example 2 of Form Example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the smart device 14 will be referred to as the "terminal."

While conventional story generation systems can automatically generate stories based on the story's theme and character settings, they have difficulty appropriately generating foreshadowing, mystery elements, and moving scenes to make the story more interesting. Furthermore, they lack the functionality to organize the generated story elements and provide them to users, making it difficult for users to effectively use the generated story. Furthermore, they lack the means to enhance the dramatic effect through story structure, character dialogue, visuals, and music, and to effectively stir the viewer's emotions.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

In this invention, the server includes means for inputting story settings such as the story's theme, setting, and characters; means for automatically creating character dialogue, scenery, and story structure based on the input story settings; means for automatically generating foreshadowing, mystery elements, and moving scenes to make the created story more interesting; means for organizing the generated story elements and providing them to the user; and means for enhancing the dramatic effect with the story structure, character dialogue, images, and music to effectively stir the viewer's emotions. This makes it possible to make the story more interesting and enable users to effectively use the generated story.

"Story theme" refers to the central subject or subject matter of a story.

"Setting" refers to the place or environment in which the story unfolds.

"Characters" refers to the characters and people who appear in the story.

"Story setting" refers to the setting, including basic elements such as the story's theme, setting, and characters.

"Dialogue" refers to the words or lines spoken by characters.

"Scene" refers to the scenery or scene depicted in the story.

"Story structure" refers to the overall flow and structure of a story, including its beginning, middle, and end.

"Foreshadowing" refers to elements or events placed in the first half of a story that will have important meaning later on.

"Mystery elements" refer to elements that add mystery or detective elements to a story, attracting the interest of readers and viewers.

A "moving scene" refers to a scene in a story that stirs strong emotions.

"Graphics" refers to images and illustrations that are displayed in accordance with story scenes.

"Song" refers to the music or background music that plays along with the story scenes.

"Directory effects" refers to techniques that use the visual and auditory elements of a story to elicit emotions and reactions from the viewer.

"Audience sentiment" refers to the emotions and feelings of the people who watch or read the story.

This invention is a system that automatically generates plot twists, mystery elements, and moving scenes to make a story more interesting, based on the story's theme, the characters' personalities, and the story's progression. A specific embodiment of this system is shown below.

System Configuration

Hardware

The server uses a cloud server equipped with a high-performance GPU.

Software

The server uses a generative AI model. Specifically, it uses a generative AI model such as OpenAI's GPT-4.

Program processing

User input

The user uses the device to input information about the story's theme, the characters' personalities, and the story's progression. For example, the user might input the following information:

Theme: Medieval Knight's Adventure

Characters: A brave knight, a mysterious old man, and a princess

Story progression: The knight meets a mysterious old man during his adventure, receives advice from him, and continues his adventure. Ultimately, the knight saves the princess and protects the kingdom.

Generating prompt statements

The server receives the information entered by the user and generates a prompt based on that information. The generated prompt looks like this:

Theme: Medieval Knight's Adventure

Characters: A brave knight, a mysterious old man, and a princess

Story progression: The knight meets a mysterious old man during his adventure, receives advice from him, and continues his adventure. Ultimately, the knight saves the princess and protects the kingdom.

Generating story elements

The server inputs the generated prompt text into the generative AI model to generate story elements. The generative AI model generates the following story elements based on the prompt text.

Foreshadowing: The mysterious old man the knight meets during his adventure actually knows the kingdom's hidden secrets.

Mystery element: The knight solves the mystery hidden in an old map he received from an old man.

Emotional scene: The knight saves the princess and fights a final battle to protect the kingdom.

Organizing and providing generated story elements

The server organizes the story elements output by the generative AI model and provides them to the user. The user can check the generated story elements via their device and make corrections or additions as needed.

Specific examples

Below is a concrete example of how a user can use the system to generate a story.

Theme: Medieval Knight's Adventure

Characters: A brave knight, a mysterious old man, and a princess

Story progression: The knight meets a mysterious old man during his adventure, receives advice from him, and continues his adventure. Ultimately, the knight saves the princess and protects the kingdom.

The server inputs this prompt into a generative AI model, which automatically generates plot twists, mystery elements, and moving scenes based on the story's theme, the characters' personalities, and the story's progression. The generated story elements are provided to the user, who can review them and make corrections or additions as needed.

In this way, it is possible to make the story more interesting and enable users to effectively use the generated story.

The flow of the identification process in Example 2 will be explained using Figure 13.

Step 1:

The user enters information about the story's theme and characters.

Input: Information about the story's theme, character personalities, and story progression.

Specific operation: The user enters the theme "Adventures of a Medieval Knight" and character information - "A Brave Knight, a Mysterious Old Man, and a Princess" - into the input form on their device. Once the input is complete, they click the "Submit" button.

Output: The entered information is sent to the server.

Step 2:

The server receives the entered information and generates a prompt.

Input: User-submitted information about the story's theme, character personalities, and story progression.

Specific operation: Based on the received information, the server generates a prompt sentence to input into the generative AI model.

Output: The generated prompt text.

Theme: Medieval Knight's Adventure

Characters: A brave knight, a mysterious old man, and a princess

Story progression: The knight meets a mysterious old man during his adventure, receives advice from him, and continues his adventure. Ultimately, the knight saves the princess and protects the kingdom.

Step 3:

The server inputs prompts into the generative AI model to generate story elements.

Input: The generated prompt text.

Specific operation: The server inputs prompt text into the generative AI model, generating story hints, mystery elements, and moving scenes.

Output: Generated story elements.

Foreshadowing: The mysterious old man the knight meets during his adventure actually knows the kingdom's hidden secrets.

Mystery element: The knight solves the mystery hidden in an old map he received from an old man.

Emotional scene: The knight saves the princess and fights a final battle to protect the kingdom.

Step 4:

The server organizes the generated story elements and provides them to the user.

Input: Story elements output from the generative AI model.

Specific operation: The server organizes the generated story elements and provides them to the user. The user can review the generated story elements through their device and make corrections or additions as necessary.

Output: The organized story elements are presented to the user.

(Application Example 2)

Next, we will explain Application Example 2 of Form Example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the smart device 14 will be referred to as the "terminal."

While conventional story generation systems can automatically create stories based on the story's theme and character settings, they have difficulty properly generating foreshadowing, mystery elements, and moving scenes to make the story more interesting. It is also difficult to automatically create graphics and music that match each scene in the generated story and effectively stir the viewer's emotions. Furthermore, when using generative AI models to generate story elements, there is a lack of a way to properly generate prompt sentences and input them into the model, which poses the challenge of not being able to provide a satisfying story to the user.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

In this invention, the server includes means for inputting story settings such as the story theme, setting, and characters; means for automatically creating character dialogue, scenery, and story structure based on the input story settings; means for automatically generating foreshadowing, mystery elements, and moving scenes to make the created story more interesting; means for automatically creating graphics and music that match each scene of the created story; means for enhancing the dramatic effect with the story structure, character dialogue, images, and music to effectively stir the viewer's emotions; means for generating story elements using a generative AI model; means for generating prompt sentences and inputting them into the generative AI model; and means for displaying the generated story elements to the user. This makes it possible to make the story more interesting and effectively stir the viewer's emotions.

A "story theme" is the central subject or subject of a story.

"Setting" refers to the place or environment in which the story unfolds.

"Characters" refers to the characters that appear in the story.

"Story setting" refers to the basic elements of the story, such as its theme, setting, and characters.

"Dialogue" refers to the words spoken by characters.

"Scene" refers to the scenery or scene depicted in the story.

"Story structure" refers to the overall flow and structure of a story, including its beginning, middle, and end.

"Foreshadowing" is the advance indication of important elements related to later developments.

"Mystery elements" refer to the inclusion of mysteries or mysterious events in a story.

A "moving scene" is one that strongly stirs the viewer's emotions.

"Graphics" refers to visual images and illustrations.

"Song" refers to music or melody.

"Producer effects" refer to visual and auditory techniques used to make a story more compelling.

"Viewers" are people who watch a story.

A "generative AI model" is an algorithm that uses artificial intelligence to generate text, images, etc.

A "prompt" is an instruction sentence to be input into a generative AI model.

"User" refers to a person who uses the system.

A system for implementing this invention includes means for inputting story settings such as the story's theme, setting, and characters; means for automatically creating character dialogue, scenery, and story structure based on the input story settings; means for automatically generating foreshadowing, mystery elements, and moving scenes to make the created story more interesting; means for automatically creating graphics and music that match each scene of the created story; means for enhancing the dramatic effect with the story structure, character dialogue, images, and music, and effectively stirring the viewer's emotions; means for generating story elements using a generative AI model; means for generating prompt sentences and inputting them into the generative AI model; and means for displaying the generated story elements to the user.

The server receives information about the story's theme and characters entered by the user and generates a prompt based on that information. The generated prompt is then input into a generative AI model (e.g., GPT-3 (registered trademark)) to generate story elements. The generated story elements are then displayed to the user.

Specifically, the server operates in the following steps: First, the user inputs information about the story's theme and characters. Next, the server generates a prompt based on this information. The generated prompt is input into a generative AI model, which generates story elements. The generated story elements are then displayed to the user.

The hardware used includes servers and user devices. The software used includes generative AI models (e.g., GPT-3) and prompt generation algorithms.

As a concrete example, the following prompt sentence is generated:

Theme: Knight's Adventure

Characters: Brave knight, mysterious old man, wizard

Generate plot twists, mystery elements, and moving scenes to make your story more interesting.

By inputting this prompt into a generative AI model, story elements are automatically generated. The generated story elements are displayed to the user, and the dramatic effect can be enhanced with story structure, character dialogue, visuals, and music.

The flow of the specific processing in Application Example 2 will be explained using Figure 14.

Step 1:

The user enters information about the story's theme and characters.

Input: Story theme, character information

Output: Input theme and character information

Specific actions: The user enters the story's theme (e.g., "The Knight's Adventure") and characters (e.g., "The Brave Knight, the Mysterious Old Man, and the Wizard") into an input form on a smartphone application.

Step 2:

The server receives the entered theme and character information and generates a prompt.

Input: Entered theme and character information

Output: Generated prompt text

Specific operation: Based on the received theme and character information, the server generates a prompt like the following: "Theme: A knight's adventure. Characters: A brave knight, a mysterious old man, a wizard. Please generate hints, mystery elements, and moving scenes to make the story more interesting."

Step 3:

The server inputs the generated prompt sentence into the generative AI model.

Input: Generated prompt text

Output: Story elements generated by the generative AI model

Specific operation: The server sends the generated prompt to a generative AI model such as GPT-3 to generate story elements (e.g., foreshadowing, mystery elements, emotional scenes).

Step 4:

The server receives the generated story elements and displays them to the user.

Input: Story elements generated by a generative AI model

Output: Story elements displayed to the user

Specific operation: The server sends the story elements received from the generative AI model to the user's smartphone application and displays them to the user. The user can then review the generated story elements.

Step 5:

The server automatically creates graphics and music that match each scene in the story.

Input: Generated story elements

Output: Graphics and music that match each scene

Specific operation: Based on the generated story elements, the server automatically creates appropriate graphics and music, taking into account the atmosphere of the scene and the emotional state of the characters.

Step 6:

The server enhances the dramatic effect through story structure, character dialogue, visuals, and music.

Input: Graphics, music, and story elements that match each scene

Output: A story with enhanced dramatic effects

Specific operation: The server combines the story structure, character dialogue, images, and music to enhance the dramatic effect, effectively stirring the viewer's emotions.

(Example 3)

Next, we will explain Example 3 of Form Example 3. In the following explanation, the data processing device 12 will be referred to as the "server" and the smart device 14 will be referred to as the "terminal."

Previous story generation systems struggled to automatically create graphics and music that matched each scene in the story, preventing them from effectively evoking the viewer's emotions. Furthermore, generating graphics and music that take into account the scene's atmosphere and the characters' emotional states was a time-consuming and labor-intensive process.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Example 3 is realized by the following means.

In this invention, the server includes: means for inputting story settings such as the story theme, setting, and characters; means for automatically creating character dialogue, scenery, and story structure based on the input story settings; means for automatically generating foreshadowing, mystery elements, and moving scenes to make the created story more interesting; means for automatically creating graphics and music that match each scene of the created story; means for analyzing scene data and extracting the atmosphere of the scene and the emotional state of the characters; means for generating prompt text to be input into a generative AI model based on the extracted information; means for inputting the prompt text into the generative AI model and generating graphics and music that match the scene; means for integrating the generated graphics and music with the scene data and generating final output; and means for enhancing the dramatic effect with the story structure, character dialogue, images, and music to effectively stir the viewer's emotions. This makes it possible to automatically create graphics and music that match each scene of the story, enhancing the dramatic effect that effectively stirs the viewer's emotions.

"Story settings such as the story's theme, setting, and characters" are the basic elements of a story, and refer to the background of the story, the characteristics of the characters, and the settings necessary for the story's progression.

"Means of input" refers to the interface that allows the user to provide the story setting, such as the story's theme, setting, and characters, to the system.

"Means for automatically creating character dialogue, scenes, and story structure" refers to a function that automatically generates character dialogue, scene descriptions, and the overall flow of a story based on the input story setting.

"Means for automatically generating foreshadowing, mystery elements, moving scenes, etc." refers to a function that automatically adds elements that cause surprises or emotions to a pre-set story in order to make the story more interesting.

"Means for automatically creating graphics and music" refers to a function that automatically generates visual and auditory elements to match each scene in a story.

"Means for analyzing scene data" refers to the function of analyzing the content of each scene in a story and extracting the atmosphere of the scene and the emotional state of the characters.

"Means for generating prompt sentences to be input to a generative AI model" refers to a function that creates text to provide appropriate instructions to a generative AI model based on analyzed scene data.

"A means of inputting a prompt sentence into a generative AI model and generating graphics and music that match the scene" refers to a function that inputs a prompt sentence into a generative AI model and, as a result, generates graphics and music that are appropriate for the scene.

"Means for integrating generated graphics and music into scene data and generating final output" refers to the functionality for incorporating generated visual and auditory elements into scene data to create the final narrative output.

"Means to effectively stir the viewer's emotions" refers to the function of having a strong impact on the viewer's emotions through the story structure, the characters' dialogue, images, and music.

This invention relates to a system that automatically creates graphics and music that match each scene in a story. This system inputs story settings such as the story's theme, setting, and characters, and automatically creates character dialogue, scenery, and story structure based on the input story settings. It also automatically generates foreshadowing, mystery elements, and moving scenes to make the story more interesting, and automatically creates graphics and music that match each scene.

Hardware and software used

This system uses the following hardware and software:

Server: Analyzes data and runs generative AI models.

Terminal: Provides an interface for users to enter the story setting for their story.

Generative AI models: For example, OpenAI's GPT-4, DALL-E, and music generation models are used.

Data processing and calculation

The server receives the story settings entered by the user via their device. The story settings include the story's theme, setting, and character traits. The server analyzes this data and automatically creates the characters' lines, scenery, and story structure.

Next, the server automatically generates foreshadowing, mystery elements, and moving scenes based on the story you've created. This makes the story more interesting and emotionally stirring for the viewer.

Furthermore, the server analyzes scene data to automatically create graphics and music that match each scene in the story. As a result of the analysis, it extracts the atmosphere of the scene and the emotional state of the characters. Based on the extracted information, the server generates prompt sentences to input into the generative AI model.

Specific examples

For example, in the case of an emotional scene, the server inputs the following prompt sentence into the generative AI model:

Example prompt for graphic generation: "Generate a graphic of a beautiful landscape. The scene is emotional, and the character is in tears."

Example music generation prompt: "Generate moving, poignant music. The scene is emotional, and the characters are in tears."

The generative AI model generates graphics and music based on these prompts. The server integrates the generated graphics and music with the scene data to generate the final output. The device displays the integrated results to the user.

In this way, the user can automatically create graphics and music that match each scene in the story, enhancing the dramatic effect that effectively stirs the viewer's emotions. The flow of the specific processing in Example 3 will be explained using Figure 15.

Step 1:

The user enters story settings such as the story's theme, setting, and characters into the device.

Specific operation: The user enters information such as the story's theme (e.g., fantasy), setting (e.g., medieval castle), and characters (e.g., hero, wizard) into the device's input form. The device then sends the entered story settings to the server.

Input: Story setting, including theme, setting, and characters.

Output: Story setting data sent to the server

Step 2:

The server analyzes the received story setting data and automatically creates the characters' lines, scenes, and story structure.

Specific operations: The server uses natural language processing (NLP) technology to analyze story setting data and generate character dialogue, scene descriptions, and the overall story flow.

Input: Story setting data

Output: Character dialogue, scenery, and story structure data

Step 3:

Based on the story you create, the server automatically generates foreshadowing, mystery elements, moving scenes, and more.

Specific operation: The server analyzes the story's composition data and adds hints, mystery elements, and moving scenes to make the story more interesting.

Input: Story composition data

Output: Story data including foreshadowing, mystery elements, and moving scenes

Step 4:

The server analyzes scene data to automatically create graphics and music that match each scene in the story.

Specific operation: The server divides the story data into scenes and extracts the atmosphere of each scene and the emotional state of the characters.

Input: Story data

Output: Scene-specific mood and character emotional state data

Step 5:

The server generates prompt sentences to input into the generative AI model based on the extracted information.

Specific operation: The server creates a prompt that corresponds to the atmosphere of the scene and the emotional state of the characters. For example, in the case of an emotional scene, the server generates the prompt, "Generate a graphic of a beautiful landscape. The scene is emotional, and the characters are crying."

Input: Scene atmosphere and character emotional state data

Output: Prompt text to be input to the generative AI model

Step 6:

The server inputs prompt text into the generative AI model, generating graphics and music that match the scene.

Specific operation: The server sends a prompt to the generative AI model (e.g., DALL-E), which generates graphics that match the scene. Similarly, it sends a prompt to the music generation model, which generates music that matches the scene.

Input: Prompt text to be input to the generative AI model

Output: Generated graphics and music

Step 7:

The server integrates the generated graphics and music with the scene data to generate the final output.

Specific operation: The server integrates the generated graphics and music with each scene data to create the final output of the entire story.

Input: Generated graphics, music, and scene data

Output: Final integrated narrative data

Step 8:

The terminal displays the final, integrated story data to the user.

Specific operation: The terminal receives the final story data sent from the server and displays it visually and audibly to the user.

Input: Final integrated narrative data

Output: The visual and auditory content of the story presented to the user.

(Application Example 3)

Next, we will explain Application Example 3 of Form Example 3. In the following explanation, the data processing device 12 will be referred to as the "server" and the smart device 14 will be referred to as the "terminal."

Previous story generation systems struggled to automatically create graphics and music that matched each scene in the story, limiting the visual and auditory effects they could produce. Furthermore, even when users input their own stories, there was a lack of means to generate graphics and music appropriate for that story, resulting in a decline in content quality. Furthermore, the lack of functionality to preview and export the generated content resulted in poor user convenience.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 3 is realized by the following means.

In this invention, the server includes means for inputting story settings such as the story theme, setting, and characters; means for automatically creating character dialogue, scenery, and story structure based on the input story settings; means for automatically generating foreshadowing, mystery elements, and moving scenes to make the created story more interesting; means for automatically creating graphics and music that match each scene of the created story; means for enhancing the dramatic effect with the story structure, character dialogue, video, and music, and effectively stirring the viewer's emotions; means for analyzing story scenes and extracting the scene atmosphere and the emotional state of the characters; means for generating graphics based on scene descriptions; means for generating music based on scene descriptions; means for previewing the generated content; and means for exporting the completed content. This makes it possible to automatically generate, preview, and export visually and aurally rich content based on a story entered by the user.

"Story theme" refers to the central subject or message of a story.

"Setting" refers to the place or environment in which the story unfolds.

"Characters" refers to the characters that appear in the story.

"Story setting" refers to basic elements such as the story's theme, setting, and characters.

"Dialogue" refers to the words spoken by characters.

"Scene" refers to the scenery or scene depicted in the story.

"Story structure" refers to the overall flow of a story, including its beginning, middle, and end.

"Foreshadowing" refers to elements that foreshadow information or events that will become important later in the story.

"Mystery elements" refer to elements that add mystery or suspense to a story.

A "moving scene" refers to a scene that strongly stirs the viewer's emotions.

"Graphics" refers to visual images and illustrations generated to match story scenes.

"Song" refers to music generated to match the story scenes.

'Scene atmosphere' refers to the overall feeling or mood of a particular scene in a story.

"A character's emotional state" refers to the emotions the character is feeling in the story.

A "scene description" refers to a detailed description or explanation of a particular scene in a story.

"Preview" refers to checking the generated content in advance.

"Export" refers to saving the generated content as an external file.

A system for implementing this invention includes means for inputting story settings such as the story theme, setting, and characters; means for automatically creating character dialogue, scenery, and story structure based on the input story settings; means for automatically generating foreshadowing, mystery elements, and moving scenes to make the created story more interesting; means for automatically creating graphics and music that match each scene of the created story; means for enhancing the story structure, character dialogue, and visuals and music to effectively stir the viewer's emotions; means for analyzing story scenes and extracting the atmosphere of the scene and the emotional state of the characters; means for generating graphics based on scene descriptions; means for generating music based on scene descriptions; means for previewing the generated content; and means for exporting the completed content.

Hardware and software configuration

Hardware:

Smartphone

Software:

Python

OpenAI API

PIL (Python Imaging Library)

Music generation library

Data processing and calculation

How to enter story settings such as theme, setting, and characters:

Users enter story settings such as the story's theme, setting, and characters in text format through a smartphone application.

Methods for automatically creating character dialogue, scenes, and story structure:

Based on the input story setting, the server uses a generative AI model (e.g., GPT-3) to automatically create character dialogue, scenes, and story structure.

Methods for automatically generating foreshadowing, mystery elements, moving scenes, etc.:

The server uses a generative AI model to automatically generate plot twists, mystery elements, and moving scenes to make the story more interesting.

Methods for automatically creating graphics and music that match each scene in a story:

The server analyzes each scene in the story and extracts the scene's atmosphere and the characters' emotional state. Based on this, it generates graphics using OpenAI's image generation API and music using OpenAI's music generation API.

How to preview generated content:

Users can preview each scene of the story, including generated graphics and music, through a smartphone application.

How to export your finished content:

Users can export the finished content as a video file through a smartphone application.

Specific examples

For example, if a user inputs the story "A hero climbed a mountain to fight a dragon," the system will analyze this scene and generate powerful graphics and music that heightens the tension to match the battle scene between the hero and the dragon.

Example prompt:

Analyze the following story scene and output the scene's atmosphere and the characters' emotional states:

The hero climbed the mountain to fight the dragon.

In this way, users can enjoy their stories as visually and aurally rich content.

The flow of the specific processing in Application Example 3 will be explained using Figure 16.

Step 1:

Users enter story settings such as the story's theme, setting, and characters in text format through a smartphone application.

Input: Story setting, including theme, setting, and characters.

Output: Story setting data in text format

Step 2:

Based on the input story setting, the server uses a generative AI model (e.g., GPT-3) to automatically create character dialogue, scenes, and story structure.

Input: Text-based story setting data

Output: Character dialogue, scenery, and story structure data

Specific operation: The server sends a prompt to the generative AI model and receives the generated text.

Step 3:

The server uses a generative AI model to automatically generate plot twists, mystery elements, and moving scenes to make the story more interesting.

Input: Character dialogue, scenery, and story structure data

Output: Foreshadowing, mystery elements, and moving scene data

Specific operation: The server sends a prompt to the generative AI model and receives the generated text.

Step 4:

The server analyzes each scene in the story and extracts the scene's atmosphere and the emotional state of the characters.

Input: Foreshadowing, mystery elements, and moving scene data

Output: Scene atmosphere, character emotional state data

Specific operation: The server sends a prompt to the generative AI model and receives the analysis results.

Step 5:

The server generates graphics based on the scene description.

Input: Scene atmosphere, character emotional state data

Output: Graphics data

Specific operation: The server sends a prompt to OpenAI's image generation API and receives the generated image.

Step 6:

The server generates music based on the scene description.

Input: Scene atmosphere, character emotional state data

Output: Music data

Specific operation: The server sends a prompt to OpenAI's music generation API and receives the generated music.

Step 7:

Users can preview each scene of the story, including generated graphics and music, through a smartphone application.

Input: Graphics data, music data

Output: Preview screen

Specific behavior: The smartphone application displays the generated data and provides a preview to the user.

Step 8:

Users can export the finished content as a video file through a smartphone application.

Input: Preview screen

Output: Video file

Specific operation: The smartphone application saves the generated data as a video file and provides it to the user.

Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

"Example 1"

One embodiment of the present invention is a story generation system that incorporates an emotion engine. This system receives story settings, such as the story's theme, setting, and characters, as input information from the user. Based on this story setting, the system then automatically creates character dialogue, scenery, and story structure. It also automatically generates foreshadowing, mystery elements, and moving scenes to make the story more interesting. It also automatically creates graphics and music that match each scene in the story. The story structure, character dialogue, video, and music enhance the dramatic effect, effectively stirring the viewer's emotions.

"Example 2"

Furthermore, the system includes an emotion engine. This emotion engine recognizes the user's emotions in real time and adjusts the story development according to those emotions. For example, if the user feels joyful about the story, the story will develop in a more cheerful manner. Conversely, if the user feels sad about the story, the story will develop in a more profound manner.

"Example 3"

The emotion engine uses information such as the user's facial expressions, tone of voice, and text input to recognize the user's emotions. For example, if the user is smiling, the emotion engine will recognize that the user is feeling happy. If the user speaks in a sad tone of voice, the emotion engine will recognize that the user is feeling sad. Furthermore, if the user inputs the text "fun," the emotion engine will recognize that the user is feeling happy. Based on this information, the emotion engine adjusts the development of the story and provides a story experience that matches the user's emotions.

The processing flow for each example form is explained below.

"Example 1"

Step 1: Receive story settings such as theme, setting, and characters as input from the user.

Step 2: Based on the received story setting, character dialogue, scenes, and story structure are automatically created.

Step 3: Automatically generate hints, mystery elements, moving scenes, and more to make your story more interesting.

Step 4: Automatically create graphics and music that match each scene in the story you've created. Step 5: Enhance the dramatic effect with the story structure, character dialogue, images, and music to effectively stir the viewer's emotions.

"Example 2"

Step 1: Launch the emotion engine and recognize the user's emotions in real time.

Step 2: Adjust the story development according to the recognized emotion. For example, if the user feels happy about the story, the story will develop in a more cheerful manner. Conversely, if the user feels sad about the story, the story will develop in a more profound manner.

"Example 3"

Step 1: The emotion engine uses information such as the user's facial expressions, tone of voice, and text input to recognize the user's emotions.

Step 2: For example, if the user is smiling, the emotion engine will recognize that the user is feeling happy. Also, if the user is speaking in a sad tone of voice, the emotion engine will recognize that the user is feeling sad.

Step 3: Furthermore, if the user enters the text "fun," the emotion engine recognizes that the user is feeling enjoyment. Based on this information, the emotion engine adjusts the development of the story and provides a narrative experience that matches the user's emotions.

(Example 1)

Next, we will explain Example 1 of Form Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the smart device 14 will be referred to as the "terminal."

In conventional story generation systems, when automatically generating a story based on a story setting entered by the user, the story flow, character dialogue, and scenery often lacked consistency. Furthermore, it was difficult to generate foreshadowing, mystery elements, and moving scenes to make the story more interesting, making it difficult to effectively move the viewer's emotions. Furthermore, it was difficult to automatically create graphics and music that matched each scene in the story. To address these issues, the present invention aims to provide a system that generates more consistent stories and effectively moves the viewer's emotions.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

In this invention, the server includes means for inputting story settings such as the story's theme, setting, and characters; means for analyzing the input story settings; means for searching an existing story database based on the analyzed story settings; means for generating and inputting prompt sentences to a generation AI model; means for automatically creating character dialogue, scenes, and story structure using the generation AI model; means for automatically generating foreshadowing, mystery elements, and moving scenes to make the created story more interesting; means for automatically creating graphics and music that match each scene of the created story; and means for enhancing the dramatic effect with the story structure, character dialogue, images, and music, and effectively stirring the viewer's emotions. This makes it possible to generate a consistent story based on the story settings input by the user, effectively stirring the viewer's emotions.

A "story theme" is the central subject or subject of a story.

"Setting" refers to the place and historical background in which the story unfolds.

"Characters" refers to the characters and people who appear in the story.

"Story setting" refers to the setting that includes basic elements such as the story's theme, setting, and characters.

"Input means" refers to the method or device by which the user provides the story setting, such as the story's theme, setting, and characters, to the system.

"Means for analyzing" refers to a method or device for analyzing the input story setting and extracting important keywords and phrases.

"Means for searching a database" refers to a method or device for searching relevant information from an existing narrative database based on the analyzed story setting.

A "generative AI model" is a model that uses artificial intelligence technology to generate text and stories.

A "prompt" is an instruction or question that is input into a generative AI model.

"Means for generating and inputting prompt sentences" refers to a method or device for creating prompt sentences suitable for a generative AI model and inputting them into the model.

"Dialogue" refers to the words or lines spoken by characters.

"Scene" refers to the scenery or scene depicted in the story.

"Narrative structure" refers to the structure and arrangement that determines the flow and development of a story.

"Foreshadowing" refers to elements or hints placed in the first half of a story that will have important meaning later on.

"Mystery elements" are elements that add mystery and suspense to a story.

A "moving scene" is one that strongly stirs the viewer's emotions.

"Graphics" refers to images and illustrations generated to match each scene in the story.

"Songs" refer to music and melodies generated to match each scene in the story.

"Directory effects" refer to effects that use story structure, character dialogue, images, and music to stir the viewer's emotions.

The present invention relates to a story generation system that generates a coherent story based on a story setting entered by the user, and aims to effectively stir the emotions of the viewer.

System Configuration

User input

The user enters the story's theme, setting, characters, and other story settings in text format. For example, they can provide a specific setting such as "a knight's adventure story set in medieval Europe."

Server input analysis

The server analyzes the story setting received from the user. This analysis uses natural language processing techniques, specifically Python's NLTK library and SpaCy. The server tokenizes the input text and extracts important keywords and phrases.

Search the database

The server searches an existing story database based on the extracted keywords and phrases. This database stores information on past stories and characters. The server finds similar stories and character settings and uses them as reference information.

Prompt generation for generative AI models

Based on user input and information obtained from the database, the server generates prompts to be input into the generative AI model. For example, it creates a prompt such as, "Create a knight's adventure story set in medieval Europe. The main character is the brave knight Arthur, whose mission is to defeat an evil dragon and save the kingdom. Please include the opening of the story, lines from the characters, and descriptions of the scenery."

Story generation using generative AI models

The server inputs prompt text into a generative AI model (for example, OpenAI's GPT-4), which generates the story flow, character dialogue, and scenery. Based on the user's input, the generative AI model generates text that matches the story's theme and the characters' personalities.

Applying the Emotion Engine

The server applies an emotional engine to the generated story, adding emotional elements to each scene in the story. For example, it automatically generates foreshadowing, mystery elements, and moving scenes. This emotional engine uses IBM Watson (registered trademark) Tone Analyzer.

Graphics and music generation

The server automatically creates graphics and music that match each scene in the story. Image generation AI is used to generate the graphics, and music generation AI is used to generate the music.

Merge the final output

The server integrates the generated story, graphics, and music and provides them to the user. The user can then view and enjoy the completed story. For example, the user can view the story through a web browser and enjoy it along with the graphics and music.

Specific examples and prompt sentence examples

Specific examples

If the user inputs the setting "A knight's adventure story set in medieval Europe," the server will generate a story like this:

The story begins: "In the vast plains of medieval Europe stood the brave knight Arthur. His mission was to slay the evil dragon and save the kingdom."

Character Quote: "Arthur: 'I swear on this sword that I will defeat the dragon!'"

Scene description: "Under the sunset, Arthur gripped his sword and renewed his resolve."

Example prompt

An example of a prompt to input to the generative AI model is as follows:

"Create a knight's adventure story set in medieval Europe. The main character is a brave knight named Arthur, whose mission is to defeat an evil dragon and save the kingdom. Please include an opening line, character dialogue, and a description of the scenery."

By inputting this prompt into a generative AI model, each element of the story is automatically generated.

The flow of the identification process in Example 1 will be explained using Figure 17.

Step 1:

The user enters the story's theme, setting, characters, and other story settings in text format. For example, they might enter "A knight's adventure story set in medieval Europe." This input is done through the system's user interface. The entered text is sent to the server.

Step 2:

The server analyzes the story setting received from the user. Specifically, it uses Python's NLTK library and SpaCy to tokenize the text and extract important keywords and phrases. For example, it extracts keywords such as "medieval Europe," "knights," and "adventure story." The results of this analysis are used in the next step.

Step 3:

The server searches an existing story database based on the extracted keywords and phrases. The database contains information on past stories and characters. The server finds similar stories and character settings and uses them as reference information. For example, it searches for past stories with a "medieval Europe" theme or stories with a "knight" as the main character. The search results are used in the next step.

Step 4:

Based on user input and information obtained from the database, the server generates a prompt to be input into the generative AI model. For example, it might create a prompt such as, "Create a knight's adventure story set in medieval Europe. The main character is the brave knight Arthur, whose mission is to defeat an evil dragon and save the kingdom. Please include the opening of the story, lines from the characters, and a description of the scenery." This prompt will be used in the next step.

Step 5:

The server inputs prompt text into a generative AI model (for example, OpenAI's GPT-4), which generates the story flow, character dialogue, and scenery. Based on the user's input, the generative AI model generates text that matches the story's theme and the characters' personalities. For example, the generative AI model generates the following story:

The story begins: "In the vast plains of medieval Europe stood the brave knight Arthur. His mission was to slay the evil dragon and save the kingdom."

Character Quote: "Arthur: 'I swear on this sword that I will defeat the dragon!'"

Scene description: "Under the sunset, Arthur gripped his sword and renewed his resolve."

Step 6:

The server applies an emotion engine to the generated story, adding emotional elements to each scene in the story. For example, it automatically generates foreshadowing, mystery elements, and moving scenes. This emotion engine uses IBM Watson's Tone Analyzer. The emotion engine analyzes the generated text and adds appropriate emotional elements.

Step 7:

The server automatically creates graphics and music that match each scene in the story. Image generation AI is used to generate the graphics, and music generation AI is used to generate the music. For example, to match the opening scene of the story, it generates graphics depicting the scenery of medieval Europe and music that evokes a sense of adventure.

Step 8:

The server integrates the generated story, graphics, and music and provides them to the user. The user can then view and enjoy the completed story. For example, the user can view the story through a web browser and enjoy it along with the graphics and music.

(Application Example 1)

Next, we will explain Application Example 1 of Form Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the smart device 14 will be referred to as the "terminal."

Conventional story generation systems could generate stories based on settings entered by the user, but they were unable to visualize and sound out the generated stories in real time and allow users to experience them in a virtual reality space. It was also difficult to automatically generate graphics and music that matched each scene in the story, enhancing the dramatic effects that effectively stirred the viewer's emotions. This limited the user experience and led to a lack of immersion in the story.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

In this invention, the server includes: means for inputting story settings such as the story theme, setting, and characters; means for automatically creating character dialogue, scenery, and story structure based on the input story settings; means for automatically generating foreshadowing, mystery elements, and moving scenes to make the created story more interesting; means for automatically producing graphics and music that match each scene of the created story; means for enhancing the dramatic effect with the story structure, character dialogue, images, and music, and effectively stirring the viewer's emotions; means for generating a story using a generative AI model based on the story settings input by the user; means for visualizing and audibly converting the generated story in real time; and means for the user to experience the generated story in a virtual reality space using a head-mounted display. This allows the user to visualize and audibly convert the generated story in real time and experience it in a virtual reality space.

A "story theme" is the central subject or theme of a story, and determines the overall direction and atmosphere of the story.

The "setting" refers to the place and historical background in which the story unfolds, and is an element that shapes the setting and atmosphere of the story.

"Characters" are characters who act in a story and play an important role in the progression and development of the story.

"Story setting" refers to the setting that includes basic elements such as the story's theme, setting, and characters, and forms the foundation of the story.

A "generative AI model" is a model that uses artificial intelligence technology to generate text and data, and is used to automatically create story structures and dialogue.

"Foreshadowing" refers to elements or hints in a story that give a glimpse of future developments, and is a technique used to make a story more interesting.

"Mystery elements" are elements in a story that present a mystery or question, and are intended to attract the interest of readers or viewers.

A "moving scene" is a scene in a story that stirs strong emotions and leaves a deep impression on the viewer's mind.

"Graphics" refers to visual images and footage that visually represent scenes and characters in a story.

"Song" refers to music, used to create scenes and atmosphere in a story.

"Visualization" refers to the visual representation of the story content, allowing users to enjoy the story visually.

"Audio" refers to the expression of the story content in audio, allowing users to enjoy the story by listening to it.

A "virtual reality space" is a virtual space created using computer technology that allows users to experience reality.

A "head-mounted display" is a display device worn on the head, used to allow users to experience virtual reality spaces.

A system for implementing this invention has the following configuration: First, a user uses a smartphone or head-mounted display to input story settings such as the story's theme, setting, and characters. This input is done in text format.

Next, the server uses a generative AI model (e.g., GPT-4) to automatically create character dialogue, scenes, and a story structure based on the story setting entered by the user. The generative AI model references an existing story database to generate dialogue and scenes that match the story flow, character personalities, and story theme.

In addition, the server uses an emotion engine to add hints, mystery elements, and moving scenes to the stories created, making them more interesting and moving.

The server then uses a graphics generation engine and a music generation engine to automatically create graphics and music that match each scene in the story. This enhances the visual and auditory effects of the story, effectively stirring the emotions of the viewer.

Finally, the generated story is visualized and audible in real time, and the user can experience the story in a virtual reality space using a head-mounted display. This allows the user to become immersed in the story and enjoy a deeper experience.

As a concrete example, if a user inputs "a knight's adventure story set in medieval Europe," the following prompt sentence will be input to the generative AI model:

Example prompt:

Theme: Medieval Europe

Setting: Knight's Adventure

Characters: Brave knight, wizard, dragon

Story setting: A brave knight and a wizard embark on an adventure to slay a dragon.

Based on this prompt, a generative AI model generates a story, an emotion engine adds emotional elements, and a graphics generation engine and music generation engine generate graphics and music. Users can experience this story in real time through a head-mounted display.

The flow of the specific processing in Application Example 1 is explained using Figure 18.

Step 1:

Users use a smartphone or head-mounted display to input story settings such as the story's theme, setting, and characters in text format. The input data is sent to the server.

Input: Story setting, including theme, setting, and characters.

Output: Story setting data sent to the server

Step 2:

Based on the received story setting data, the server uses a generative AI model (e.g., GPT-4) to automatically create the story structure, character dialogue, and scenery. The generative AI model references an existing story database to generate text that matches the story flow and character personalities.

Input: Story setting data

Output: Generated story structure, dialogue, and scenery

Step 3:

The server uses an emotion engine to add emotional elements such as foreshadowing, mystery elements, and moving scenes to the generated story. The emotion engine automatically inserts emotional elements appropriate to each scene in the story.

Input: Generated story structure, dialogue, and scenery

Output: A story with added emotional content

Step 4:

The server uses a graphics generation engine to automatically create graphics that match each scene in the story. The graphics generation engine takes into account the atmosphere of the scene and the emotional state of the characters to generate appropriate visuals.

Input: A story with an emotional element

Output: Graphics that match each scene

Step 5:

The server uses a music generation engine to automatically create music that matches each scene in the story. The music generation engine takes into account the atmosphere of the scene and the emotional state of the characters to generate appropriate music.

Input: A story with an emotional element

Output: Songs that match each scene

Step 6:

The server integrates the generated story, graphics, and music, visualizing and audiovisualizing them in real time, allowing users to experience the story visually and aurally.

Input: Generated story, graphics, music

Output: Real-time visual and audio narrative

Step 7:

Using a head-mounted display, users experience a story generated in a virtual reality space, immersing themselves in the story and providing a deeper experience.

Input: Real-time visual and audio narrative

Output: Narrative experience in a virtual reality space

(Example 2)

Next, we will explain Example 2 of Form Example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the smart device 14 will be referred to as the "terminal."

While conventional story generation systems can automatically generate a story based on the story's theme and character settings, they have difficulty automatically generating foreshadowing, mystery elements, and moving scenes that make the story more interesting. Furthermore, they lack the functionality to adjust the story's development in real time based on the user's emotions, making it impossible to provide a narrative experience that is in tune with the user's emotions. Furthermore, it is difficult to automatically create graphics and music that match each scene in the story. A new system that can solve these issues is needed.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

In this invention, the server includes means for inputting story settings such as the story theme, setting, and characters; means for automatically creating character dialogue, scenery, and story structure based on the input story settings; means for automatically generating foreshadowing, mystery elements, and moving scenes to make the created story more interesting; means for recognizing the user's emotions in real time and adjusting the story development in accordance with those emotions; means for automatically creating graphics and music that match each scene of the created story; and means for enhancing the dramatic effect with the story structure, character dialogue, images, and music to effectively stir the viewer's emotions. This makes it possible not only to automatically generate a story based on the story theme and character settings, but also to automatically generate foreshadowing, mystery elements, and moving scenes, and provide a story development that matches the user's emotions.

The "story theme" indicates the overall subject or direction of the story.

"Setting" refers to the place or environment in which the story unfolds.

"Characters" refers to the characters who take action in the story.

"Story setting" refers to the setting, including basic elements such as the story's theme, setting, and characters.

"Dialogue" refers to the words or lines spoken by characters.

"Scene" refers to the scenery or scene depicted in the story.

"Story structure" refers to the overall flow and structure of a story, including its beginning, middle, and end.

"Foreshadowing" refers to elements that are laid out in advance in a way that will have important meaning later in the story.

"Mystery elements" refer to elements that attract the interest of readers or viewers by including mysteries or unknown points in a story.

A "moving scene" is one that strongly stirs the emotions of the reader or viewer.

"User emotions" refers to feelings such as joy, sadness, and surprise felt by users as they experience the story.

"Real-time recognition" refers to detecting the user's emotions instantly and on the spot.

"Adjusting the story development" refers to changing the progress and content of the story based on the user's emotions.

"Graphics" refers to images and illustrations that are displayed in accordance with story scenes.

"Song" refers to the music that plays along with the story scenes.

"Directory effects" refer to the effects of using the visual and auditory elements of a story to stimulate the emotions and interest of readers or viewers.

This invention is a system that automatically generates a story based on the story's theme and character settings, and also automatically generates foreshadowing, mystery elements, and moving scenes. It also has the ability to recognize the user's emotions in real time and adjust the story's development accordingly.

Hardware and software used

The server generates story elements using a generative AI model (e.g., GPT-4). Based on information received from the user about the story's theme, the characters' personalities, and the story's progression, the server inputs prompt sentences into the generative AI model to generate story elements.

The device sends the user's input information to the server and displays the story elements received from the server to the user. The device also uses emotion recognition software to collect emotional data from the user's facial expressions and voice and send it to the server.

Users enter information about the story's theme, the characters' personalities, and the story's progression into their device and enjoy the story. The device recognizes the user's emotions in real time and reflects them in the development of the story.

Data processing and calculation

The server performs the following data processing and calculations:

Generate appropriate foreshadowing based on the story's theme and the characters' personalities.

Mystery elements are scattered throughout the story as it progresses.

Analyze the characters' emotions and situations to create moving scenes.

The device collects emotional data from the user's facial expressions and voice and sends it to the server. The server analyzes this emotional data and adjusts the story's development.

Specific examples

For example, to generate a hint that a person the knight meets during his adventure will play an important role later in the story, the following prompt sentence can be input into the generative AI model:

Example prompt:

"Please create a plot twist that will lead to the mysterious old man the knight meets during his adventure playing an important role later in the story."

Also, if the user is feeling happy about the story, a prompt to make the story even more upbeat might look like this:

Example prompt:

"Users are enjoying this, so please make the story even more upbeat. For example, add a scene where the knight finds a new companion."

In this way, the server utilizes a generative AI model and emotion engine to automatically generate a story development that corresponds to the user's emotions.

The flow of the identification process in Example 2 will be explained using Figure 19.

Step 1:

The user enters information about the story's theme, the characters' personalities, and the story's progression.

Input: Story theme (e.g., adventure), character traits (e.g., bravery), and information about the story's progression.

Output: User input information

Specific operation: The user enters the theme "adventure" into the device's input form and sets the character's personality to "brave."

Step 2:

The device sends the user's input information to the server.

Input: User input information

Output: User input information sent to the server

Specific operation: The device sends the information "Theme: Adventure, Personality: Brave" to the server.

Step 3:

The server generates story elements using generative AI models.

Input: User input information

Output: Generated story elements

Specific operation: The server inputs the prompt "Generate a story with an adventure theme and a brave knight as the protagonist" into the generation AI model, and generates the elements of the story.

Step 4:

The server sends the generated story elements to the device.

Input: Generated story elements

Output: Story elements sent to the device

Specific operation: The server sends story elements generated by the server (e.g., "The mysterious old man the knight meets during his adventure") to the device.

Step 5:

The device displays the story to the user.

Input: Story elements received from the server

Output: The story displayed to the user

Specific operation: The device displays a scene to the user in which a knight meets a mysterious old man during his adventure.

Step 6:

Recognize user emotions in real time.

Input: User's facial expressions and voice

Output: Recognized user emotion data

Specific operation: When the user smiles, the device captures the user's facial expression with its camera and recognizes that the user is feeling happy.

Step 7:

The device sends the user's emotional data to the server.

Input: Recognized user emotion data

Output: Emotion data sent to the server

Specific operation: The device sends emotional data indicating "the user is feeling happy" to the server.

Step 8:

The server analyzes the emotional data and adjusts the story development.

Input: User emotion data

Output: Coordinated story development

Specific operation: The server analyzes the emotional data and inputs prompt sentences into the generative AI model to "make the story more upbeat."

Step 9:

The server will send the adjusted story to your device.

Input: Adjusted story development

Output: The adjusted story sent to the terminal.

Specific operation: The server generates a story with an upbeat plot (e.g., "A scene where the knight finds a new companion") and sends it to the device.

Step 10:

The device will then display the adjusted story to the user.

Input: Adjusted story received from the server

Output: The adjusted story displayed to the user.

Specific operation: The device displays a scene to the user in which a knight finds a new companion.

(Application Example 2)

Next, we will explain Application Example 2 of Form Example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the smart device 14 will be referred to as the "terminal."

While conventional story generation systems could automatically generate a story based on the story's theme and character settings, they were unable to adjust the story's development in real time in response to the user's emotions. They also lacked the functionality to automatically generate plot twists, mystery elements, and moving scenes to make a story more interesting. This made it difficult to provide an interactive story experience that responded to the user's emotions.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

In this invention, the server includes: means for inputting story settings such as the story theme, setting, and characters; means for automatically creating character dialogue, scenery, and story structure based on the input story settings; means for automatically generating foreshadowing, mystery elements, and moving scenes to make the created story more interesting; means for automatically creating graphics and music that match each scene of the created story; means for enhancing the dramatic effect with the story structure, character dialogue, images, and music, and effectively stirring the viewer's emotions; means for recognizing the user's emotions in real time and adjusting the story development in response to those emotions; and means for generating story prompts using a generative AI model and automatically generating a story. This makes it possible to provide an interactive story experience that responds to the user's emotions.

"Story theme" refers to the central subject or message of a story.

"Setting" refers to the place or environment in which the story unfolds.

"Characters" refers to the characters or people who play a role in the story.

"Story setting" refers to the setting, including basic elements such as the story's theme, setting, and characters.

"Dialogue" refers to the words or lines spoken by characters.

"Scene" refers to the scenery or scene depicted in the story.

"Story structure" refers to the overall flow and structure of a story, including its beginning, middle, and end.

"Foreshadowing" refers to elements or hints placed in the first half of a story that will have important meaning later on.

"Mystery elements" refer to elements in a story that raise questions or raise doubts.

A "moving scene" is one that evokes strong emotions in the viewer or reader.

"Graphics" refers to images and illustrations that are displayed in accordance with story scenes.

"Song" refers to the music or soundtrack that plays along with the story scenes.

"Stage effects" refer to visual and auditory techniques used to make story scenes appear more effective.

"Viewer sentiment" refers to the emotions and feelings of the people watching the story.

"Recognizing emotions in real time" refers to analyzing the user's facial expressions, tone of voice, etc. to grasp their emotions at that moment.

"Adjusting the development of the story" refers to changing the progress and content of the story in response to the user's emotions.

"Generative AI model" refers to a model that uses artificial intelligence to generate text or stories.

A "prompt" refers to an instruction or keyword that is input into a generative AI model.

A system for implementing this invention has the following configuration: First, the server provides a means for inputting story settings such as the story's theme, setting, and characters. By inputting these settings, the user determines the basic elements of the story.

Next, the server has a means to automatically create character dialogue, scenery, and story structure based on the input story setting. This means uses a generative AI model to generate story prompts and automatically generate the story.

Furthermore, the server has the means to automatically generate hints, mystery elements, moving scenes, and more to make the stories you create more interesting. This helps to add depth and interest to the story.

The server also has the means to automatically create graphics and music that match each scene in the story. This enhances the visual and auditory effects of the story. Specifically, the graphics and music are created taking into account the atmosphere of the scene and the emotional state of the characters.

Furthermore, the server has the means to enhance the dramatic effect through the story structure, character dialogue, visuals, and music, effectively stirring the viewer's emotions. This allows users to be more emotionally drawn into the story.

What's particularly important is that the server has a means of recognizing the user's emotions in real time and adjusting the story development accordingly. It uses the smartphone's camera and microphone to analyze the user's facial expressions and tone of voice to understand their emotions. Based on this emotional data, it generates prompt sentences for the story and adjusts the story development accordingly.

For example, if a user is reading a story and the camera captures their facial expressions and recognizes that they are smiling, the story will develop in a lighthearted and cheerful way. Conversely, if the user is sad, the story will develop in a more profound way.

Examples of prompt statements include:

"The protagonist continues his adventure with a happy feeling."

"The protagonist faces difficulties, but does not lose hope."

"The protagonist takes on a new mystery."

In this way, it is possible to provide an interactive narrative experience that responds to the user's emotions.

The flow of the specific processing in Application Example 2 is explained using Figure 20.

Step 1:

The user enters story settings such as theme, setting, and characters.

Input: Setting information such as the story's theme, setting, and characters entered by the user.

Output: Story setting data.

Specific operation: The user uses a smartphone application to input the basic setting of the story, including information about the story's theme, location, and main characters. The server receives this setting information and stores it as story setting data.

Step 2:

The server automatically creates character dialogue, scenes, and story structure based on the story settings entered.

Input: Story setting data.

Output: Story dialogue, scenery, and composition data.

Specific operation: The server uses a generative AI model to generate character dialogue, scenery, and story structure based on story setting data. The generated data is saved as elements necessary for the story's progression.

Step 3:

The server automatically generates hints, mystery elements, moving scenes, and more to make the stories it creates more interesting.

Input: Story lines, scenery, and composition data.

Output: Foreshadowing, mystery elements, and moving scene data.

Specifically, how it works: The server uses a generative AI model to generate plot twists, mystery elements, and moving scenes based on the dialogue, scenery, and composition data of the story. This adds depth and intrigue to the story.

Step 4:

The server automatically creates graphics and music that match each scene in the story.

Input: Foreshadowing, mystery elements, and moving scene data.

Output: Graphics data, music data.

Specific operation: The server automatically creates graphics and music, taking into account the atmosphere of the scene and the emotional state of the characters. This is done using image generation models and music generation models. The generated graphics and music are then applied to each scene in the story.

Step 5:

The server recognizes the user's emotions in real time and adjusts the story development accordingly.

Input: User's facial expression data, tone of voice data.

Output: Tailored narrative prompt.

Specific operation: The smartphone's camera and microphone are used to capture the user's facial expressions and tone of voice. The server analyzes this data and recognizes the user's emotions in real time. Based on the recognized emotions, it generates story prompts and adjusts the story's development.

Step 6:

The server uses a generative AI model to generate story prompts and automatically generate the story.

Input: Tailored story prompt.

Output: The generated story.

Specific operation: The server uses a generative AI model to generate a story based on the adjusted prompt. The generated story is used to provide an interactive experience that responds to the user's emotions.

(Example 3)

Next, we will explain Example 3 of Form Example 3. In the following explanation, the data processing device 12 will be referred to as the "server" and the smart device 14 will be referred to as the "terminal."

With conventional story generation systems, it was difficult to automatically create graphics and music that matched each scene in the story, making it difficult to effectively sway the viewer's emotions. Furthermore, they lacked the ability to recognize the user's emotions in real time and adjust the story's development accordingly, making it difficult to provide the user with a consistent emotional experience. This resulted in a decline in the quality of the story experience and low user satisfaction.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Example 3 is realized by the following means.

In this invention, the server includes means for inputting story settings such as the story theme, setting, and characters; means for automatically creating character dialogue, scenery, and story structure based on the input story settings; means for automatically generating foreshadowing, mystery elements, and moving scenes to make the created story more interesting; means for automatically creating graphics and music that match each scene of the created story; means for using information such as the user's facial expressions, tone of voice, and text input to recognize the user's emotions; means for adjusting the story development in accordance with the user's emotions; and means for enhancing the dramatic effects of the story structure, character dialogue, images, and music to effectively stir the viewer's emotions. This makes it possible to automatically create graphics and music that match each scene of the story and provide a story development that matches the user's emotions.

"Story theme" refers to the central subject or subject of a story.

"Setting" refers to the place or environment in which the story unfolds.

"Characters" refers to the characters that appear in the story.

"Story setting" refers to the setting, including basic elements such as the story's theme, setting, and characters.

"Dialogue" refers to the words spoken by characters.

"Scene" refers to the scenery or scene depicted in the story.

"Story structure" refers to the overall flow of a story, including its beginning, middle, and end.

"Foreshadowing" refers to elements that provide advance hints at information or events that will become important later in the story.

"Mystery elements" refer to elements that add mystery or suspense to a story.

A "moving scene" refers to a scene that strongly stirs the viewer's emotions.

"Graphics" refers to images and illustrations generated to match story scenes.

"Song" refers to music generated to match the story scenes.

"User emotion" refers to the emotional state the user is feeling, such as joy, sadness, or surprise.

"Facial expression" refers to emotions expressed through the movement of the user's facial muscles.

"Tone of voice" refers to the pitch, strength, and intonation of the user's voice.

"Text input" refers to text information entered by a user using a keyboard or touch panel.

"Story development" refers to the process and flow of the story.

"Directory effects" refer to the impact that the visual and auditory elements of a story have on the viewer.

"Viewer sentiment" refers to the emotions and feelings of the people watching the story.

This invention is a system that inputs story settings such as the story's theme, setting, and characters, automatically generates a story based on the input story settings, and further adjusts the story's development according to the user's emotions. This system is implemented using the following hardware and software.

Hardware and software used

Hardware: High-performance servers (e.g., servers with GPUs) and user devices (e.g., PCs or smartphones with cameras and microphones)

Software: Generative AI models (e.g., GPT-4, image generation AI, music generation AI), emotion recognition software (e.g., OpenCV, voice analysis libraries)

Program processing explanation

1. Story input and analysis

The user uses their device to input story settings such as the story's theme, setting, and characters. For example, they might input "a scene where a hero fights a dragon." The server receives this input and uses a generative AI model (e.g., GPT-4) to analyze each scene in the story and extract the scene's atmosphere and the characters' emotional states.

2. Graphics and Music Generation

Based on the analysis results, the server uses image generation AI to generate graphics that match the scene. For example, it could generate an impressive graphic of a hero fighting a dragon. Similarly, it uses music generation AI to generate music that suits the scene. For example, it could generate music that heightens the tension in a battle scene.

3. User Emotion Recognition

The device uses a camera and microphone to capture the user's facial expressions and tone of voice. For example, if the user is smiling, the device sends that information to the server. The server then uses emotion recognition software (e.g., OpenCV, voice analysis library) to analyze the user's emotions.

4. Story Development Adjustments

The server uses an emotion engine to adjust the story development based on the user's emotional information. For example, if the user is smiling, the story development will be adjusted to a more exciting direction.

Specific examples and prompt sentence examples

Example:

The user enters "a scene in which a hero fights a dragon."

The server analyzes the scene as a battle scene and uses image generation AI to generate powerful graphics of the dragon and hero.

Music generation AI is used to generate music that heightens the tension.

If the user is smiling, the emotion engine recognizes that the user is having fun and adjusts the story development in a more exciting direction.

Example prompt:

"Generate graphics for a scene in which a hero fights a dragon."

"Generate music that matches an emotional scene."

"If the user is smiling, adjust the story development to make it more enjoyable."

This system makes it possible to automatically create graphics and music that match each scene in a story, and to provide a story development that responds to the user's emotions. This effectively stirs the viewer's emotions and improves the quality of the story experience. The flow of the specific processing in Example 3 is explained using Figure 21.

Step 1:

The user enters the text of the story.

The user enters "A scene in which a hero fights a dragon" into the text input field on the device and clicks the send button. The entered text is sent from the device to the server.

Step 2:

The server analyzes the story text.

The server inputs the received text into a generative AI model (e.g., GPT-4) and analyzes each scene in the story. As a result of the analysis, the atmosphere of the scene and the emotional state of the characters are extracted. For example, "battle scene" and "tension" are extracted. The analysis results are stored in an internal database.

Step 3:

The server generates graphics that match the scene.

Based on the analysis results, the server uses image generation AI (e.g., DALL-E) to generate graphics that match the scene. For example, enter "a scene of a hero fighting a dragon" as a prompt and obtain the generated graphic. The generated graphic is then sent to the user's device.

Step 4:

The server generates music that matches the scene.

The server uses music generation AI to generate music appropriate for the scene. For example, a user can enter "music that will increase tension in a battle scene" as a prompt and obtain the generated music. The generated music is then sent to the user's device.

Step 5:

The device recognizes the user's emotions.

The device uses a camera to capture the user's facial expressions and a microphone to record the tone of voice. This data is sent to the server in real time. For example, if the user is smiling, that information is sent to the server.

Step 6:

The server adjusts the story development based on the user's emotions.

The server analyzes the received emotional data using emotion recognition software (e.g., OpenCV, voice analysis library) to recognize the user's emotions. For example, if it recognizes that the user is smiling, it uses the emotion engine to adjust the story development in an exciting direction. A new scene is generated and sent to the user's device.

(Application Example 3)

Next, we will explain Application Example 3 of Form Example 3. In the following explanation, the data processing device 12 will be referred to as the "server" and the smart device 14 will be referred to as the "terminal."

Conventional story generation systems have the problem of not adjusting the story based on the user's emotions, resulting in a uniform user experience. Furthermore, because the story scenes and music are fixed, they are unable to provide dynamic changes that respond to the user's emotions. This makes it difficult to provide a story experience that deeply appeals to the user's emotions.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 3 is realized by the following means.

In this invention, the server includes means for inputting story settings such as the story theme, setting, and characters; means for automatically creating character dialogue, scenery, and story structure based on the input story settings; means for automatically generating foreshadowing, mystery elements, and moving scenes to make the created story more interesting; means for automatically creating graphics and music that match each scene of the created story; means for enhancing the dramatic effect with the story structure, character dialogue, images, and music, and effectively stirring the viewer's emotions; means for recognizing the user's emotions and adjusting story scenes and music in accordance with the recognized emotions; and means for recognizing emotions using the user's facial expressions, tone of voice, text input, etc. This enables a dynamic story experience that matches the user's emotions.

A "story theme" refers to the central idea or message of a story.

"Setting" refers to the place or environment in which the story unfolds.

"Characters" refers to the characters that appear in the story.

"Story setting" refers to the setting, including basic elements such as the story's theme, setting, and characters.

"Dialogue" refers to the words spoken by characters.

"Scene" refers to the scenery or scene depicted in the story.

"Story structure" refers to the overall flow of a story, including its beginning, middle, and end.

"Foreshadowing" refers to an element in the first half of a story that becomes important later in the story.

"Mystery elements" refer to elements that add mystery and suspense to a story.

A "moving scene" is one that strongly stirs the viewer's emotions.

"Graphics" refers to images and footage that visually represent scenes from a story.

"Song" refers to the music that plays along with the story scenes.

"Directory effects" refer to effects that influence the viewer's emotions through story scenes and music.

"User emotion" refers to the emotional state the user is feeling, such as joy, sadness, or surprise.

"Means for recognizing emotions" refers to technology that determines a user's emotions using their facial expressions, tone of voice, text input, etc.

"Means for adjusting story scenes and music" refers to technology that changes story scenes and music according to the recognized user emotions.

A system for implementing this invention can dynamically adjust story scenes and music according to the user's emotions. A specific embodiment of this system is described below.

System Configuration

The system includes the following major hardware and software components:

Hardware: Smartphone camera

Software: OpenCV (face detection), Keras (emotion recognition model), music playback software

Program processing

The server first captures the user's facial expression using the smartphone camera. The captured image is then subjected to face detection using OpenCV. Next, the Keras emotion recognition model is used to recognize the user's emotion from the detected face.

Based on the recognized emotion, the server adjusts the story scenes and music. For example, if the user is smiling, the story scenes will change to brighter, more cheerful ones, and the music will also become more upbeat. Conversely, if the user has a sad expression, the story will transition to a more moving scene, and the music will also change to something more moving.

Specific examples

As a concrete example, consider the case where a user smiles while enjoying a story. In this case, the server would process the following:

1. Capture the user's facial expressions using the smartphone camera.

2. Detect faces using OpenCV.

3. The user's emotion is recognized as "happiness" using the Keras emotion recognition model.

4. Based on the recognized emotion, the story scene is changed to a brighter, more cheerful one and upbeat music is played.

Example prompt

An example of a prompt to input to the generative AI model is as follows:

If the user smiles, change the story scene to something bright and cheerful and play upbeat music.

In this way, it is possible to provide a dynamic narrative experience that responds to the user's emotions. This system allows users to become more immersed in the story and have an experience that appeals to their emotions.

The flow of the specific processing in Application Example 3 will be explained using Figure 22.

Step 1:

The server uses the smartphone's camera to capture the user's facial expressions. The input is the camera image, and the output is the captured image data. Specifically, the camera acquires the image in real time and captures the image frame by frame.

Step 2:

The server uses OpenCV to detect faces from the captured image data. The input is the captured image data, and the output is the location information of the detected face. Specifically, OpenCV's face detection algorithm identifies the face in the image and obtains its coordinates.

Step 3:

The server uses a Keras emotion recognition model to recognize the user's emotion from the detected face. The input is face position information and image data, and the output is the recognized emotion label. Specifically, the face area is extracted, preprocessed, and then input into the emotion recognition model to classify the emotion.

Step 4:

The server adjusts the story scenes and music based on the recognized emotions. The input is the recognized emotion label, and the output is the adjusted scenes and music. Specific operations include selecting scenes and music that correspond to the emotion label and changing the progression of the story.

Step 5:

The server sends the adjusted scene and music to the user's device. The input is the adjusted scene and music, and the output is the story scene displayed on the user's device and the music played. Specifically, the server encodes the scene and music data and sends it to the user's device over the network.

Step 6:

The user's device plays the received scenes and music and provides them to the user. The input is the scene and music data received from the server, and the output is the video displayed to the user and the music played. Specifically, the device decodes the data, displays it on the screen, and plays the music from the speakers.

The specific processing unit 290 transmits the results of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the results of the specific processing. The microphone 38B acquires audio indicating the user input regarding the results of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

Data generation model 58 is what is known as generative AI (artificial intelligence). An example of data generation model 58 is generative AI such as ChatGPT (registered trademark) (Internet search <URL: https://openai.com/blog/chatgpt>). Data generation model 58 is obtained by performing deep learning on a neural network. A prompt containing an instruction is input to data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. Data generation model 58 performs inference on the input inference data in accordance with the instructions indicated by the prompt, and outputs the inference results in the form of data such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and/or summarization.

Another example of generative AI is Gemini (registered trademark) (Internet search <URL: https://gemini.google.com/?hl=ja>).

In the above embodiment, an example was given in which the specific processing was performed by the data processing device 12, but the technology disclosed herein is not limited to this, and the specific processing may also be performed by the smart device 14.

[Second embodiment]

Figure 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

As shown in FIG. 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

The data processing device 12 includes a computer 22, a database 24, and a communication I/F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and communication I/F 26 are also connected to the bus 34. The communication I/F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and/or a LAN (Local Area Network).

The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I/F 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the user 20's surroundings (e.g., an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

The communication I/F 44 is connected to the network 54. The communication I/Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I/Fs 44 and 26 is carried out in a secure manner.

Figure 4 shows an example of the main functions of the data processing device 12 and smart glasses 214. As shown in Figure 4, in the data processing device 12, specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32.

The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as the specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

Storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

Next, we will explain the identification process performed by the identification processing unit 290 of the data processing device 12.

"Example 1"

The system of the present invention receives input from the user about the story setting, such as the story's theme, setting, and characters. This input is in text format, such as "A knight's adventure story set in medieval Europe." Next, based on this input, the system automatically creates character dialogue, scenes, and a story structure. This creation is carried out using existing story databases and artificial intelligence technology, generating dialogue and scenes that match the story flow, character personalities, and story theme.

"Example 2"

In addition, the system automatically generates hints, mystery elements, and moving scenes to make the story more interesting. This is done by laying hints at the right time, sprinkling mystery elements, and creating moving scenes in line with the story's theme, the characters' personalities, and the story's progression. For example, it automatically generates hints that people the knight meets during his adventure will play an important role later in the story, puzzles the knight faces, and moving scenes when the knight overcomes difficulties.

"Example 3"

The system then automatically creates graphics and music that match each scene in the story you've created. The graphics and music are created taking into account the atmosphere of the scene and the emotional state of the characters. For example, powerful graphics and music that heighten tension are created for battle scenes, while beautiful landscape graphics and moving music are created for moving scenes. This makes it possible to enhance the dramatic effect of the story's structure, the characters' conversations, images, and music, effectively stirring the viewer's emotions.

The processing flow for each example form is explained below.

"Example 1"

Step 1: The user provides textual input of the story setting, including the story's theme, setting, and characters. For example, it could be provided as "a knight's adventure story set in medieval Europe."

Step 2: Based on the input, the system automatically creates character dialogue, scenery, and story structure. This is done using existing story databases and artificial intelligence technology.

Step 3: Lines and scenes are generated that match the story flow, character personalities, and theme of the story.

"Example 2"

Step 1: The system automatically generates plot twists, mystery elements, moving scenes, and more to make the story more interesting.

Step 2: Lay out hints at the right time, sprinkle in mystery elements, and create moving scenes in line with the story's theme, the characters' personalities, and the story's progression.

Step 3: For example, foreshadowing, such as people the knight meets during his adventure playing important roles later in the story, puzzles the knight faces, and moving scenes when the knight overcomes difficulties, are automatically generated.

"Example 3"

Step 1: The system automatically creates graphics and music that match each scene in the story you've created.

Step 2: Graphics and music are created taking into account the atmosphere of the scene and the emotional state of the characters.

Step 3: For example, powerful graphics and music that heightens the tension are created for battle scenes, while beautiful landscape graphics and moving music are created for moving scenes. This enhances the dramatic effect of the story structure, character dialogue, images and music, making it possible to effectively stir the viewer's emotions.

(Example 1)

Next, Example 1 of Form Example 1 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

When conventional story generation systems automatically generate a story based on a user-entered story setting, they struggle to maintain consistency in the story flow, character dialogue, and scenery. They also lack the ingenuity to ensure the generated story is moving and entertaining for the viewer. Furthermore, they are unable to automatically create graphics and music that match each scene in the story, which means they are unable to effectively move the viewer's emotions.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

In this invention, the server includes: means for inputting story settings such as the story's theme, setting, and characters; means for automatically creating character dialogue, scenery, and story structure based on the input story settings; means for automatically generating foreshadowing, mystery elements, and moving scenes to make the created story more interesting; means for automatically producing graphics and music that match each scene of the created story; means for enhancing the dramatic effect with the story structure, character dialogue, and images and music to effectively stir the viewer's emotions; means for receiving input from the user and analyzing it using natural language processing technology; means for generating prompt sentences to be input into the generative AI model based on the analysis results; means for automatically generating story dialogue, scenery, and structure using the generative AI model; and means for formatting the generated story and providing it to the user. This makes it possible to automatically generate a consistent story based on user input and provide the viewer with a moving and entertaining story.

A "story theme" is the central subject or theme of a story, and is the element that determines the overall direction and atmosphere of the story.

The "setting" refers to the place and historical background in which the story unfolds, and is an important element in shaping the setting and atmosphere of the story.

"Characters" are characters who act within a story and move the story forward, and are elements that have a major influence on the story's development and theme.

"Story setting" refers to the setting that includes basic elements such as the story's theme, setting, and characters, and is the element that forms the framework of the story.

"Dialogue" refers to the words or lines spoken by characters, and is an important element in expressing the progression of the story and the character's personality.

"Scenes" refer to the scenery and scenes depicted in a story, and are elements that visually express the atmosphere and emotions of the story.

"Story structure" refers to the overall flow and structure of a story, including the beginning, middle, and end, and is an element that makes the story unfold effectively.

"Foreshadowing" is a hint or sign that relates to later developments in a story, and is an element that adds consistency and depth to a story.

"Mystery elements" are elements that present mysteries or questions within a story and attract the interest of readers or viewers.

A "moving scene" is a scene in a story that strongly stirs emotions and is an impressive element that stays in the viewer's mind.

"Graphics" refers to images and illustrations that visually represent each scene in the story, and are elements that visually complement the atmosphere and emotion of the story.

"Songs" are melodies and sounds that are musically expressed to match each scene in a story, and are elements that musically complement the atmosphere and emotions of the story.

"Natural language processing technology" is a technology that enables computers to understand and analyze human language, and is used to analyze text data and extract information.

A "generative AI model" is a model that uses artificial intelligence technology to generate text and data, and is used to automatically generate dialogue, scenes, and structure for a story.

A "prompt" is an instruction to be input into a generative AI model, and is an element that guides the content generated by the model.

"Formatting" refers to the process of adding paragraph breaks and line breaks to make the generated narrative text easier to read before it is presented to the user.

This invention is a system that automatically generates a story based on story settings such as the story's theme, setting, and characters, which are input by the user. Specific embodiments of this system are described below.

User input

The user uses a terminal to enter text information about the story's theme, setting, characters, etc. into the system's input form. For example, they might enter "A knight's adventure story set in medieval Europe."

Receive and analyze input data by the server

The server receives the text data sent by the user. It then analyzes the input data using natural language processing technology. Specifically, it uses Python's NLTK library and spaCy to extract information such as theme, setting, and characters from the input text. For example, it extracts keywords such as "medieval Europe," "knights," and "adventure story."

Generating prompt statements

The server generates a prompt to be input into the generative AI model based on the extracted information. For example, if the user's input is "A knight's adventure story set in medieval Europe," the prompt would be "Please generate lines and scenes for the characters in a knight's adventure story set in medieval Europe."

Generating stories using generative AI models

The server inputs the generated prompt into a generative AI model (for example, OpenAI's GPT-4). The generative AI model automatically generates the dialogue, scenery, and structure of a story based on the prompt. For example, it could generate text such as, "The brave knight Arthur met a wizard in a dark forest. The wizard said to Arthur, 'Deep in this forest lies an ancient dragon. Your mission is to slay the dragon.'"

Format and provide generated results

The server receives the story text returned by the generative AI model and formats it as needed, for example by adding paragraph breaks and line breaks to make it easier to read. The formatted story is then converted into a format that is easy for the user to read. Finally, the server provides the formatted story to the user, who can view the results through their device.

Specific examples

As an example of user input, consider "a knight's adventure story set in medieval Europe." Based on this input, the server generates the following prompt:

"Generate lines and scenes from characters in a knight's adventure story set in medieval Europe."

The generative AI model generates a story based on this prompt, outputting text such as the following:

"The brave knight Arthur met a wizard in a dark forest. The wizard told Arthur, 'Deep in this forest lies an ancient dragon. Your mission is to defeat the dragon.'"

In this way, it is possible to automatically generate a coherent story based on user input, providing an inspiring and entertaining story for the viewer.

The flow of the identification process in Example 1 will be explained using Figure 11.

Step 1:

The user uses their device to enter text information about the story's theme, setting, characters, etc. into the system's input form. For example, they might enter "A knight's adventure story set in medieval Europe." This input data is then sent to the server.

Step 2:

The server receives text data sent by the user. It analyzes the received data using natural language processing technology. Specifically, it uses Python's NLTK library and spaCy to extract information such as theme, setting, and characters from the input text. For example, it extracts keywords such as "medieval Europe," "knights," and "adventure story." The results of this analysis become the input for the next step.

Step 3:

Based on the analysis results, the server generates a prompt to be input into the generative AI model. For example, if the user's input is "A knight's adventure story set in medieval Europe," the prompt would be "Please generate lines and scenes for the characters in a knight's adventure story set in medieval Europe." This prompt becomes the input for the next step.

Step 4:

The server inputs the generated prompt into a generative AI model (for example, OpenAI's GPT-4). The generative AI model automatically generates the dialogue, scenery, and structure of a story based on the prompt. For example, it might generate text such as, "The brave knight Arthur met a wizard in a dark forest. The wizard said to Arthur, 'Deep in this forest lies an ancient dragon. Your mission is to slay the dragon.'" This generated text becomes the input for the next step.

Step 5:

The server receives the story text returned by the generative AI model and formats it as needed, for example by adding paragraph breaks and line breaks to make it easier to read. The formatted story is then converted into a format that is easier for the user to read. This formatted text becomes the input for the next step.

Step 6:

The server provides the formatted story to the user. The user can view the results through their device, for example, on a web page or application screen that displays the generated story. The user can then read and enjoy the generated story.

(Application Example 1)

Next, we will explain Application Example 1 of Form Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

Existing story generation systems automatically generate stories based on user-entered themes and settings, but they often lack sufficient story structure, character dialogue, and scene generation, making it difficult to effectively sway viewers' emotions. Furthermore, the lack of functionality to preview, save, and share the generated stories creates a problem of inconvenient user experience.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

In this invention, the server includes: means for inputting story settings such as the story's theme, setting, and characters; means for automatically creating character dialogue, scenery, and story structure based on the input story settings; means for automatically generating foreshadowing, mystery elements, and moving scenes to make the created story more interesting; means for automatically producing graphics and music that match each scene of the created story; means for enhancing the dramatic effect with the story structure, character dialogue, images, and music to effectively stir the viewer's emotions; means for generating prompt sentences using a generative AI model and automatically generating a story; and means for previewing, saving, and sharing the generated story. This makes it possible to automatically generate high-quality stories based on the settings input by the user, effectively stirring the viewer's emotions.

A "story theme" is the central subject or theme of a story, and determines the overall direction and atmosphere of the story.

The "setting" refers to the place and historical background in which the story unfolds, and is an element that shapes the setting and atmosphere of the story.

"Characters" are characters that appear in a story and play an important role in the progression and development of the story.

"Story setting" refers to the setting that includes basic elements such as the story's theme, setting, and characters, and forms the framework of the story.

"Dialogue" refers to the words spoken by characters, and is an important element in expressing the progression of the story and the character's personality.

"Scenes" refer to the landscapes and scenes depicted in a story, and visually express the atmosphere and emotions of the story.

"Story structure" refers to the overall flow and structure of a story, including a beginning, middle, and end, and is the framework for effectively developing the story.

"Foreshadowing" is a hint or omen that relates to later developments in a story, and is an element that makes a story more interesting.

"Mystery elements" are elements that add mystery and suspense to a story, and are meant to attract the interest of readers and viewers.

A "moving scene" is a scene in a story that stirs strong emotions and is intended to leave a deep impression on the viewer.

"Graphics" refers to the visual images and illustrations that accompany each scene in the story, and are intended to visually express the atmosphere and emotions of the story.

"Songs" are pieces of music created to accompany each scene in a story, and are intended to musically express the atmosphere and emotions of the story.

A "generative AI model" is a model that uses artificial intelligence technology to automatically generate text, images, etc., and is used to automatically generate stories.

A "prompt" is an instruction to be input into a generative AI model, and is generated based on the theme and setting of the story.

"Preview" is a feature that allows users to preview the generated story in advance and check the content.

"Preservation" refers to recording the stories that are created in digital form so that they can be reused and shared later.

"Sharing" refers to sharing the story you create with other users and platforms, with the aim of promoting the spread and appreciation of the story.

A system for implementing this invention allows the user to input story settings such as the story's theme, setting, and characters, and automatically generates a story based on these settings. Specific embodiments of this system are described below.

System Configuration

This system is primarily composed of the following hardware and software:

Hardware: Smartphones, servers

Software: Python, OpenAI API

Program processing

User input

Users enter the story's theme, setting, and characters in text form through a smartphone application. For example, they might enter the theme "A knight's adventure story set in medieval Europe."

Prompt generation

The server generates a prompt based on the information entered by the user. This prompt is an instruction that is input into the generative AI model. As a specific example, the following prompt is generated:

Theme: A knight's adventure story set in medieval Europe

Setting: Medieval Europe

Characters: Brave knight, wizard, dragon

Generate a story.

Story generation

The server uses a generative AI model (e.g., GPT-4) to automatically generate a story based on the generated prompt. This generative AI model is called through the OpenAI API.

Preview, save, share

The generated story is displayed so that users can preview it through the smartphone application. Users can save the generated story and share it on social media, etc.

Specific examples

If a user sets the theme to "a knight's adventure story set in medieval Europe" and enters "a brave knight, a wizard, and a dragon" as the characters, the following story will be generated.

Theme: A knight's adventure story set in medieval Europe

Setting: Medieval Europe

Characters: Brave knight, wizard, dragon

Generate a story.

By inputting this prompt into a generative AI model, a story is automatically generated. Users can preview, save, and share the generated story.

Hardware and software used

Hardware: Smartphones, servers

Software: Python, OpenAI API

This makes it possible to automatically generate high-quality stories based on user-entered settings, effectively stirring the viewer's emotions.

The flow of the specific processing in Application Example 1 will be explained using Figure 12.

Step 1:

Users use their smartphone to input the story's theme, setting, and characters in text format. The input data is then sent from the device to the server. An example input includes the theme "A knight's adventure story set in medieval Europe."

Step 2:

The server generates a prompt based on the received user input data. Specifically, it combines information about the theme, setting, and characters to create a prompt that is suitable for the generative AI model. The generated prompt looks like this:

Theme: A knight's adventure story set in medieval Europe

Setting: Medieval Europe

Characters: Brave knight, wizard, dragon

Generate a story.

This prompt text is generated.

Step 3:

The server calls a generative AI model (e.g., GPT-4) with the generated prompt text. It sends the prompt text using the OpenAI API and requests the generation of a story. The input is the prompt text, and the output is the generated story text.

Step 4:

The generative AI model automatically generates a story based on the prompt text. The generated story text is returned to the server. For example, the following story might be generated:

Once upon a time, in a vast kingdom in medieval Europe, there lived a brave knight. His name was Arthur. One day, with the help of a wizard, he embarked on an adventure to battle a fearsome dragon.

Step 5:

The server sends the text of the generated story to the smartphone device. The user can preview the generated story through the smartphone device. The input is the text of the generated story, and the output is the preview screen displayed on the user's device.

Step 6:

Users can save the generated story or share it on social media. Saved stories are recorded in the device's storage, and shared stories are sent to the specified platform. The input is the user's selection, and the output is the saved file or shared content.

These steps make it possible to automatically generate a high-quality story based on the settings entered by the user, effectively stirring the viewer's emotions.

(Example 2)

Next, we will explain Example 2 of Form Example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

While conventional story generation systems can automatically generate stories based on the story's theme and character settings, they have difficulty appropriately generating foreshadowing, mystery elements, and moving scenes to make the story more interesting. Furthermore, they lack the functionality to organize the generated story elements and provide them to users, making it difficult for users to effectively use the generated story. Furthermore, they lack the means to enhance the dramatic effect through story structure, character dialogue, visuals, and music, and to effectively stir the viewer's emotions.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

In this invention, the server includes means for inputting story settings such as the story's theme, setting, and characters; means for automatically creating character dialogue, scenery, and story structure based on the input story settings; means for automatically generating foreshadowing, mystery elements, and moving scenes to make the created story more interesting; means for organizing the generated story elements and providing them to the user; and means for enhancing the dramatic effect with the story structure, character dialogue, images, and music to effectively stir the viewer's emotions. This makes it possible to make the story more interesting and enable users to effectively use the generated story.

"Story theme" refers to the central subject or subject matter of a story.

"Setting" refers to the place or environment in which the story unfolds.

"Characters" refers to the characters and people who appear in the story.

"Story setting" refers to the setting, including basic elements such as the story's theme, setting, and characters.

"Dialogue" refers to the words or lines spoken by characters.

"Scene" refers to the scenery or scene depicted in the story.

"Story structure" refers to the overall flow and structure of a story, including its beginning, middle, and end.

"Foreshadowing" refers to elements or events placed in the first half of a story that will have important meaning later on.

"Mystery elements" refer to elements that add mystery or detective elements to a story, attracting the interest of readers and viewers.

A "moving scene" refers to a scene in a story that stirs strong emotions.

"Graphics" refers to images and illustrations that are displayed in accordance with story scenes.

"Song" refers to the music or background music that plays along with the story scenes.

"Directory effects" refers to techniques that use the visual and auditory elements of a story to elicit emotions and reactions from the viewer.

"Audience sentiment" refers to the emotions and feelings of the people who watch or read the story.

This invention is a system that automatically generates plot twists, mystery elements, and moving scenes to make a story more interesting, based on the story's theme, the characters' personalities, and the story's progression. A specific embodiment of this system is shown below.

System Configuration

Hardware

The server uses a cloud server equipped with a high-performance GPU.

Software

The server uses a generative AI model. Specifically, it uses a generative AI model such as OpenAI's GPT-4.

Program processing

User input

The user uses the device to input information about the story's theme, the characters' personalities, and the story's progression. For example, the user might input the following information:

Theme: Medieval Knight's Adventure

Characters: A brave knight, a mysterious old man, and a princess

Story progression: The knight meets a mysterious old man during his adventure, receives advice from him, and continues his adventure. Ultimately, the knight saves the princess and protects the kingdom.

Generating prompt statements

The server receives the information entered by the user and generates a prompt based on that information. The generated prompt looks like this:

Theme: Medieval Knight's Adventure

Characters: A brave knight, a mysterious old man, and a princess

Story progression: The knight meets a mysterious old man during his adventure, receives advice from him, and continues his adventure. Ultimately, the knight saves the princess and protects the kingdom.

Generating story elements

The server inputs the generated prompt text into the generative AI model to generate story elements. The generative AI model generates the following story elements based on the prompt text.

Foreshadowing: The mysterious old man the knight meets during his adventure actually knows the kingdom's hidden secrets.

Mystery element: The knight solves the mystery hidden in an old map he received from an old man.

Emotional scene: The knight saves the princess and fights a final battle to protect the kingdom.

Organizing and providing generated story elements

The server organizes the story elements output by the generative AI model and provides them to the user. The user can check the generated story elements via their device and make corrections or additions as needed.

Specific examples

Below is a concrete example of how a user can use the system to generate a story.

Theme: Medieval Knight's Adventure

Characters: A brave knight, a mysterious old man, and a princess

Story progression: The knight meets a mysterious old man during his adventure, receives advice from him, and continues his adventure. Ultimately, the knight saves the princess and protects the kingdom.

The server inputs this prompt into a generative AI model, which automatically generates plot twists, mystery elements, and moving scenes based on the story's theme, the characters' personalities, and the story's progression. The generated story elements are provided to the user, who can review them and make corrections or additions as needed.

In this way, it is possible to make the story more interesting and enable users to effectively use the generated story.

The flow of the identification process in Example 2 will be explained using Figure 13.

Step 1:

The user enters information about the story's theme and characters.

Input: Information about the story's theme, character personalities, and story progression.

Specific operation: The user enters the theme "Adventures of a Medieval Knight" and character information - "A Brave Knight, a Mysterious Old Man, and a Princess" - into the input form on their device. Once the input is complete, they click the "Submit" button.

Output: The entered information is sent to the server.

Step 2:

The server receives the entered information and generates a prompt.

Input: User-submitted information about the story's theme, character personalities, and story progression.

Specific operation: Based on the received information, the server generates a prompt sentence to input into the generative AI model.

Output: The generated prompt text.

Theme: Medieval Knight's Adventure

Characters: A brave knight, a mysterious old man, and a princess

Story progression: The knight meets a mysterious old man during his adventure, receives advice from him, and continues his adventure. Ultimately, the knight saves the princess and protects the kingdom.

Step 3:

The server inputs prompts into the generative AI model to generate story elements.

Input: The generated prompt text.

Specific operation: The server inputs prompt text into the generative AI model, generating story hints, mystery elements, and moving scenes.

Output: Generated story elements.

Foreshadowing: The mysterious old man the knight meets during his adventure actually knows the kingdom's hidden secrets.

Mystery element: The knight solves the mystery hidden in an old map he received from an old man.

Emotional scene: The knight saves the princess and fights a final battle to protect the kingdom.

Step 4:

The server organizes the generated story elements and provides them to the user.

Input: Story elements output from the generative AI model.

Specific operation: The server organizes the generated story elements and provides them to the user. The user can review the generated story elements through their device and make corrections or additions as necessary.

Output: The organized story elements are presented to the user.

(Application Example 2)

Next, we will explain Application Example 2 of Form Example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

While conventional story generation systems can automatically create stories based on the story's theme and character settings, they have difficulty properly generating foreshadowing, mystery elements, and moving scenes to make the story more interesting. It is also difficult to automatically create graphics and music that match each scene in the generated story and effectively stir the viewer's emotions. Furthermore, when using generative AI models to generate story elements, there is a lack of a way to properly generate prompt sentences and input them into the model, which poses the challenge of not being able to provide a satisfying story to the user.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

In this invention, the server includes means for inputting story settings such as the story theme, setting, and characters; means for automatically creating character dialogue, scenery, and story structure based on the input story settings; means for automatically generating foreshadowing, mystery elements, and moving scenes to make the created story more interesting; means for automatically creating graphics and music that match each scene of the created story; means for enhancing the dramatic effect with the story structure, character dialogue, images, and music to effectively stir the viewer's emotions; means for generating story elements using a generative AI model; means for generating prompt sentences and inputting them into the generative AI model; and means for displaying the generated story elements to the user. This makes it possible to make the story more interesting and effectively stir the viewer's emotions.

A "story theme" is the central subject or subject of a story.

"Setting" refers to the place or environment in which the story unfolds.

"Characters" refers to the characters that appear in the story.

"Story setting" refers to the basic elements of the story, such as its theme, setting, and characters.

"Dialogue" refers to the words spoken by characters.

"Scene" refers to the scenery or scene depicted in the story.

"Story structure" refers to the overall flow and structure of a story, including its beginning, middle, and end.

"Foreshadowing" is the advance indication of important elements related to later developments.

"Mystery elements" refer to the inclusion of mysteries or mysterious events in a story.

A "moving scene" is one that strongly stirs the viewer's emotions.

"Graphics" refers to visual images and illustrations.

"Song" refers to music or melody.

"Producer effects" refer to visual and auditory techniques used to make a story more appealing.

"Viewers" are people who watch a story.

A "generative AI model" is an algorithm that uses artificial intelligence to generate text, images, etc.

A "prompt" is an instruction to be input into a generative AI model.

"User" refers to a person who uses the system.

A system for implementing this invention includes means for inputting story settings such as the story's theme, setting, and characters; means for automatically creating character dialogue, scenery, and story structure based on the input story settings; means for automatically generating foreshadowing, mystery elements, and moving scenes to make the created story more interesting; means for automatically creating graphics and music that match each scene of the created story; means for enhancing the dramatic effect with the story structure, character dialogue, images, and music, and effectively stirring the viewer's emotions; means for generating story elements using a generative AI model; means for generating prompt sentences and inputting them into the generative AI model; and means for displaying the generated story elements to the user.

The server receives information about the story's theme and characters entered by the user and generates a prompt based on that information. The generated prompt is then input into a generative AI model (e.g., GPT-3) to generate story elements. The generated story elements are then displayed to the user.

Specifically, the server operates in the following steps: First, the user inputs information about the story's theme and characters. Next, the server generates a prompt based on this information. The generated prompt is input into a generative AI model, which generates story elements. The generated story elements are then displayed to the user.

The hardware used includes servers and user devices. The software used includes generative AI models (e.g., GPT-3) and prompt generation algorithms.

As a concrete example, the following prompt sentence is generated:

Theme: Knight's Adventure

Characters: Brave knight, mysterious old man, wizard

Generate plot twists, mystery elements, and moving scenes to make your story more interesting.

By inputting this prompt into a generative AI model, story elements are automatically generated. The generated story elements are displayed to the user, and the dramatic effect can be enhanced with story structure, character dialogue, visuals, and music.

The flow of the specific processing in Application Example 2 will be explained using Figure 14.

Step 1:

The user enters information about the story's theme and characters.

Input: Story theme, character information

Output: Input theme and character information

Specific actions: The user enters the story's theme (e.g., "The Knight's Adventure") and characters (e.g., "The Brave Knight, the Mysterious Old Man, and the Wizard") into an input form on a smartphone application.

Step 2:

The server receives the entered theme and character information and generates a prompt.

Input: Entered theme and character information

Output: Generated prompt text

Specific operation: Based on the received theme and character information, the server generates a prompt like the following: "Theme: A knight's adventure. Characters: A brave knight, a mysterious old man, a wizard. Please generate hints, mystery elements, and moving scenes to make the story more interesting."

Step 3:

The server inputs the generated prompt sentence into the generative AI model.

Input: Generated prompt text

Output: Story elements generated by the generative AI model

Specific operation: The server sends the generated prompt to a generative AI model such as GPT-3 to generate story elements (e.g., foreshadowing, mystery elements, emotional scenes).

Step 4:

The server receives the generated story elements and displays them to the user.

Input: Story elements generated by a generative AI model

Output: Story elements displayed to the user

Specific operation: The server sends the story elements received from the generative AI model to the user's smartphone application and displays them to the user. The user can then review the generated story elements.

Step 5:

The server automatically creates graphics and music that match each scene in the story.

Input: Generated story elements

Output: Graphics and music that match each scene

Specific operation: Based on the generated story elements, the server automatically creates appropriate graphics and music, taking into account the atmosphere of the scene and the emotional state of the characters.

Step 6:

The server enhances the dramatic effect through story structure, character dialogue, visuals, and music.

Input: Graphics, music, and story elements that match each scene

Output: A story with enhanced dramatic effects

Specific operation: The server combines the story structure, character dialogue, images, and music to enhance the dramatic effect, effectively stirring the viewer's emotions.

(Example 3)

Next, we will explain Example 3 of Form Example 3. In the following explanation, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

Previous story generation systems struggled to automatically create graphics and music that matched each scene in the story, preventing them from effectively evoking the viewer's emotions. Furthermore, generating graphics and music that take into account the scene's atmosphere and the characters' emotional states was a time-consuming and labor-intensive process.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Example 3 is realized by the following means.

In this invention, the server includes: means for inputting story settings such as the story theme, setting, and characters; means for automatically creating character dialogue, scenery, and story structure based on the input story settings; means for automatically generating foreshadowing, mystery elements, and moving scenes to make the created story more interesting; means for automatically creating graphics and music that match each scene of the created story; means for analyzing scene data and extracting the atmosphere of the scene and the emotional state of the characters; means for generating prompt text to be input into a generative AI model based on the extracted information; means for inputting the prompt text into the generative AI model and generating graphics and music that match the scene; means for integrating the generated graphics and music with the scene data and generating final output; and means for enhancing the dramatic effect with the story structure, character dialogue, images, and music to effectively stir the viewer's emotions. This makes it possible to automatically create graphics and music that match each scene of the story, enhancing the dramatic effect that effectively stirs the viewer's emotions.

"Story settings such as the story's theme, setting, and characters" are the basic elements of a story, and refer to the background of the story, the characteristics of the characters, and the settings necessary for the story's progression.

"Means of input" refers to the interface that allows the user to provide the story setting, such as the story's theme, setting, and characters, to the system.

"Means for automatically creating character dialogue, scenes, and story structure" refers to a function that automatically generates character dialogue, scene descriptions, and the overall flow of a story based on the input story setting.

"Means for automatically generating foreshadowing, mystery elements, moving scenes, etc." refers to a function that automatically adds elements that cause surprises or emotions to a pre-set story in order to make the story more interesting.

"Means for automatically creating graphics and music" refers to a function that automatically generates visual and auditory elements to match each scene in a story.

"Means for analyzing scene data" refers to the function of analyzing the content of each scene in a story and extracting the atmosphere of the scene and the emotional state of the characters.

"Means for generating prompt sentences to be input to a generative AI model" refers to a function that creates text to provide appropriate instructions to a generative AI model based on analyzed scene data.

"A means of inputting a prompt sentence into a generative AI model and generating graphics and music that match the scene" refers to a function that inputs a prompt sentence into a generative AI model and, as a result, generates graphics and music that are appropriate for the scene.

"Means for integrating generated graphics and music into scene data and generating final output" refers to the functionality for incorporating generated visual and auditory elements into scene data to create the final narrative output.

"Means to effectively stir the viewer's emotions" refers to the function of having a strong impact on the viewer's emotions through the story structure, the characters' dialogue, images, and music.

This invention relates to a system that automatically creates graphics and music that match each scene in a story. This system inputs story settings such as the story's theme, setting, and characters, and automatically creates character dialogue, scenery, and story structure based on the input story settings. It also automatically generates foreshadowing, mystery elements, and moving scenes to make the story more interesting, and automatically creates graphics and music that match each scene.

Hardware and software used

This system uses the following hardware and software:

Server: Analyzes data and runs generative AI models.

Terminal: Provides an interface for users to enter the story setting for their story.

Generative AI models: For example, OpenAI's GPT-4, DALL-E, and music generation models are used.

Data processing and calculation

The server receives the story settings entered by the user via their device. The story settings include the story's theme, setting, and character traits. The server analyzes this data and automatically creates the characters' lines, scenery, and story structure.

Next, the server automatically generates foreshadowing, mystery elements, moving scenes, and more based on the story you've created. This makes the story more interesting and emotionally stirring for the viewer.

Furthermore, the server analyzes scene data to automatically create graphics and music that match each scene in the story. As a result of the analysis, it extracts the atmosphere of the scene and the emotional state of the characters. Based on the extracted information, the server generates prompt sentences to input into the generative AI model.

Specific examples

For example, in the case of an emotional scene, the server inputs the following prompt sentence into the generative AI model:

Example prompt for graphic generation: "Generate a graphic of a beautiful landscape. The scene is emotional, and the character is in tears."

Example music generation prompt: "Generate moving, poignant music. The scene is emotional, and the characters are in tears."

The generative AI model generates graphics and music based on these prompts. The server integrates the generated graphics and music with the scene data to generate the final output. The device displays the integrated results to the user.

In this way, the user can automatically create graphics and music that match each scene in the story, enhancing the dramatic effect that effectively stirs the viewer's emotions. The flow of the specific processing in Example 3 will be explained using Figure 15.

Step 1:

The user enters story settings such as the story's theme, setting, and characters into the device.

Specific operation: The user enters information such as the story's theme (e.g., fantasy), setting (e.g., medieval castle), and characters (e.g., hero, wizard) into the device's input form. The device then sends the entered story settings to the server.

Input: Story setting, including theme, setting, and characters.

Output: Story setting data sent to the server

Step 2:

The server analyzes the received story setting data and automatically creates the characters' lines, scenes, and story structure.

Specific operations: The server uses natural language processing (NLP) technology to analyze story setting data and generate character dialogue, scene descriptions, and the overall story flow.

Input: Story setting data

Output: Character dialogue, scenery, and story structure data

Step 3:

Based on the story you create, the server automatically generates foreshadowing, mystery elements, moving scenes, and more.

Specific operation: The server analyzes the story's composition data and adds hints, mystery elements, and moving scenes to make the story more interesting.

Input: Story composition data

Output: Story data including foreshadowing, mystery elements, and moving scenes

Step 4:

The server analyzes scene data to automatically create graphics and music that match each scene in the story.

Specific operation: The server divides the story data into scenes and extracts the atmosphere of each scene and the emotional state of the characters.

Input: Story data

Output: Scene-specific mood and character emotional state data

Step 5:

The server generates prompt sentences to input into the generative AI model based on the extracted information.

Specific operation: The server creates a prompt that corresponds to the atmosphere of the scene and the emotional state of the characters. For example, in the case of an emotional scene, the server generates the prompt, "Generate a graphic of a beautiful landscape. The scene is emotional, and the characters are crying."

Input: Scene atmosphere and character emotional state data

Output: Prompt text to be input to the generative AI model

Step 6:

The server inputs prompt text into the generative AI model, generating graphics and music that match the scene.

Specific operation: The server sends a prompt to the generative AI model (e.g., DALL-E), which generates graphics that match the scene. Similarly, it sends a prompt to the music generation model, which generates music that matches the scene.

Input: Prompt text to be input to the generative AI model

Output: Generated graphics and music

Step 7:

The server integrates the generated graphics and music with the scene data to generate the final output.

Specific operation: The server integrates the generated graphics and music with each scene data to create the final output of the entire story.

Input: Generated graphics, music, and scene data

Output: Final integrated narrative data

Step 8:

The terminal displays the final, integrated story data to the user.

Specific operation: The terminal receives the final story data sent from the server and displays it visually and audibly to the user.

Input: Final integrated narrative data

Output: The visual and auditory content of the story presented to the user.

(Application Example 3)

Next, we will explain Application Example 3 of Form Example 3. In the following explanation, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

Previous story generation systems struggled to automatically create graphics and music that matched each scene in the story, limiting the visual and auditory effects they could produce. Furthermore, even when users input their own stories, there was a lack of means to generate graphics and music appropriate for that story, resulting in a decline in content quality. Furthermore, the lack of functionality to preview and export the generated content resulted in poor user convenience.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 3 is realized by the following means.

In this invention, the server includes means for inputting story settings such as the story theme, setting, and characters; means for automatically creating character dialogue, scenery, and story structure based on the input story settings; means for automatically generating foreshadowing, mystery elements, and moving scenes to make the created story more interesting; means for automatically creating graphics and music that match each scene of the created story; means for enhancing the dramatic effect with the story structure, character dialogue, video, and music, and effectively stirring the viewer's emotions; means for analyzing story scenes and extracting the scene atmosphere and the emotional state of the characters; means for generating graphics based on scene descriptions; means for generating music based on scene descriptions; means for previewing the generated content; and means for exporting the completed content. This makes it possible to automatically generate, preview, and export visually and aurally rich content based on a story entered by the user.

"Story theme" refers to the central subject or message of a story.

"Setting" refers to the place or environment in which the story unfolds.

"Characters" refers to the characters that appear in the story.

"Story setting" refers to basic elements such as the story's theme, setting, and characters.

"Dialogue" refers to the words spoken by characters.

"Scene" refers to the scenery or scene depicted in the story.

"Story structure" refers to the overall flow of a story, including its beginning, middle, and end.

"Foreshadowing" refers to elements that foreshadow information or events that will become important later in the story.

"Mystery elements" refer to elements that add mystery or suspense to a story.

A "moving scene" refers to a scene that strongly stirs the viewer's emotions.

"Graphics" refers to visual images and illustrations generated to match story scenes.

"Song" refers to music generated to match the story scenes.

'Scene atmosphere' refers to the overall feeling or mood of a particular moment in a story.

"A character's emotional state" refers to the emotions the character is feeling in the story.

A "scene description" refers to a detailed description or explanation of a particular scene in a story.

"Preview" refers to checking the generated content in advance.

"Export" refers to saving the generated content as an external file.

A system for implementing this invention includes means for inputting story settings such as the story theme, setting, and characters; means for automatically creating character dialogue, scenery, and story structure based on the input story settings; means for automatically generating foreshadowing, mystery elements, and moving scenes to make the created story more interesting; means for automatically creating graphics and music that match each scene of the created story; means for enhancing the story structure, character dialogue, and visuals and music to effectively stir the viewer's emotions; means for analyzing story scenes and extracting the atmosphere of the scene and the emotional state of the characters; means for generating graphics based on scene descriptions; means for generating music based on scene descriptions; means for previewing the generated content; and means for exporting the completed content.

Hardware and software configuration

Hardware:

Smartphone

Software:

Python

OpenAI API

PIL (Python Imaging Library)

Music generation library

Data processing and calculation

How to enter story settings such as theme, setting, and characters:

Users enter story settings such as the story's theme, setting, and characters in text format through a smartphone application.

Methods for automatically creating character dialogue, scenes, and story structure:

Based on the input story setting, the server uses a generative AI model (e.g., GPT-3) to automatically create character dialogue, scenes, and story structure.

Methods for automatically generating foreshadowing, mystery elements, moving scenes, etc.:

The server uses a generative AI model to automatically generate plot twists, mystery elements, and moving scenes to make the story more interesting.

Methods for automatically creating graphics and music that match each scene in a story:

The server analyzes each scene in the story and extracts the scene's atmosphere and the characters' emotional state. Based on this, it generates graphics using OpenAI's image generation API and music using OpenAI's music generation API.

How to preview generated content:

Users can preview each scene of the story, including the generated graphics and music, through a smartphone application.

How to export your finished content:

Users can export the finished content as a video file through a smartphone application.

Specific examples

For example, if a user inputs the story "A hero climbed a mountain to fight a dragon," the system will analyze this scene and generate powerful graphics and music that heightens the tension to match the battle scene between the hero and the dragon.

Example prompt:

Analyze the following story scene and output the scene's atmosphere and the characters' emotional states:

The hero climbed the mountain to fight the dragon.

In this way, users can enjoy their stories as visually and aurally rich content.

The flow of the specific processing in Application Example 3 will be explained using Figure 16.

Step 1:

Users enter story settings such as the story's theme, setting, and characters in text format through a smartphone application.

Input: Story setting, including theme, setting, and characters.

Output: Story setting data in text format

Step 2:

Based on the input story setting, the server uses a generative AI model (e.g., GPT-3) to automatically create character dialogue, scenes, and story structure.

Input: Text-based story setting data

Output: Character dialogue, scenery, and story structure data

Specific operation: The server sends a prompt to the generative AI model and receives the generated text.

Step 3:

The server uses a generative AI model to automatically generate plot twists, mystery elements, and moving scenes to make the story more interesting.

Input: Character dialogue, scenery, and story structure data

Output: Foreshadowing, mystery elements, and moving scene data

Specific operation: The server sends a prompt to the generative AI model and receives the generated text.

Step 4:

The server analyzes each scene in the story and extracts the scene's atmosphere and the emotional state of the characters.

Input: Foreshadowing, mystery elements, and moving scene data

Output: Scene atmosphere, character emotional state data

Specific operation: The server sends a prompt to the generative AI model and receives the analysis results.

Step 5:

The server generates graphics based on the scene description.

Input: Scene atmosphere, character emotional state data

Output: Graphics data

Specific operation: The server sends a prompt to OpenAI's image generation API and receives the generated image.

Step 6:

The server generates music based on the scene description.

Input: Scene atmosphere, character emotional state data

Output: Music data

Specific operation: The server sends a prompt to OpenAI's music generation API and receives the generated music.

Step 7:

Users can preview each scene of the story, including the generated graphics and music, through a smartphone application.

Input: Graphics data, music data

Output: Preview screen

Specific behavior: The smartphone application displays the generated data and provides a preview to the user.

Step 8:

Users can export the finished content as a video file through a smartphone application.

Input: Preview screen

Output: Video file

Specific operation: The smartphone application saves the generated data as a video file and provides it to the user.

It is also possible to further combine an emotion engine that estimates the user's emotion. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

"Example 1"

One embodiment of the present invention is a story generation system that incorporates an emotion engine. This system receives story settings, such as the story's theme, setting, and characters, as input information from the user. Based on this story setting, the system then automatically creates character dialogue, scenery, and story structure. It also automatically generates foreshadowing, mystery elements, and moving scenes to make the story more interesting. It also automatically creates graphics and music that match each scene in the story. The story structure, character dialogue, video, and music enhance the dramatic effect, effectively stirring the viewer's emotions.

"Example 2"

Furthermore, the system includes an emotion engine. This emotion engine recognizes the user's emotions in real time and adjusts the story development according to those emotions. For example, if the user feels joyful about the story, the story will develop in a more cheerful manner. Conversely, if the user feels sad about the story, the story will develop in a more profound manner.

"Example 3"

The emotion engine uses information such as the user's facial expressions, tone of voice, and text input to recognize the user's emotions. For example, if the user is smiling, the emotion engine will recognize that the user is feeling happy. If the user speaks in a sad tone of voice, the emotion engine will recognize that the user is feeling sad. Furthermore, if the user inputs the text "fun," the emotion engine will recognize that the user is feeling happy. Based on this information, the emotion engine adjusts the development of the story and provides a story experience that matches the user's emotions.

The processing flow for each example is explained below.

"Example 1"

Step 1: Receive story settings such as theme, setting, and characters as input from the user.

Step 2: Based on the received story setting, character dialogue, scenes, and story structure are automatically created.

Step 3: Automatically generate hints, mystery elements, moving scenes, and more to make your story more interesting.

Step 4: Automatically create graphics and music that match each scene in the story you've created. Step 5: Enhance the dramatic effect with the story structure, character dialogue, images, and music to effectively stir the viewer's emotions.

"Example 2"

Step 1: Launch the emotion engine and recognize the user's emotions in real time.

Step 2: Adjust the story development according to the recognized emotion. For example, if the user feels happy about the story, the story will develop in a more cheerful manner. Conversely, if the user feels sad about the story, the story will develop in a more profound manner.

"Example 3"

Step 1: The emotion engine uses information such as the user's facial expressions, tone of voice, and text input to recognize the user's emotions.

Step 2: For example, if the user is smiling, the emotion engine will recognize that the user is feeling happy. Also, if the user is speaking in a sad tone of voice, the emotion engine will recognize that the user is feeling sad.

Step 3: Furthermore, if the user enters the text "fun," the emotion engine recognizes that the user is feeling enjoyment. Based on this information, the emotion engine adjusts the development of the story and provides a narrative experience that matches the user's emotions.

(Example 1)

Next, Example 1 of Form Example 1 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

In conventional story generation systems, when automatically generating a story based on a story setting entered by the user, the story flow, character dialogue, and scenery often lacked consistency. Furthermore, it was difficult to generate foreshadowing, mystery elements, and moving scenes to make the story more interesting, making it difficult to effectively move the viewer's emotions. Furthermore, it was difficult to automatically create graphics and music that matched each scene in the story. To address these issues, the present invention aims to provide a system that generates more consistent stories and effectively moves the viewer's emotions.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

In this invention, the server includes means for inputting story settings such as the story's theme, setting, and characters; means for analyzing the input story settings; means for searching an existing story database based on the analyzed story settings; means for generating and inputting prompt sentences to a generation AI model; means for automatically creating character dialogue, scenes, and story structure using the generation AI model; means for automatically generating foreshadowing, mystery elements, and moving scenes to make the created story more interesting; means for automatically creating graphics and music that match each scene of the created story; and means for enhancing the dramatic effect with the story structure, character dialogue, images, and music, and effectively stirring the viewer's emotions. This makes it possible to generate a consistent story based on the story settings input by the user, effectively stirring the viewer's emotions.

A "story theme" is the central subject or subject of a story.

"Setting" refers to the place and historical background in which the story unfolds.

"Characters" refers to the characters and people who appear in the story.

"Story setting" refers to the setting that includes basic elements such as the story's theme, setting, and characters.

"Input means" refers to the method or device by which the user provides the story setting, such as the story's theme, setting, and characters, to the system.

"Means for analyzing" refers to a method or device for analyzing the input story setting and extracting important keywords and phrases.

"Means for searching a database" refers to a method or device for searching relevant information from an existing narrative database based on the analyzed story setting.

A "generative AI model" is a model that uses artificial intelligence technology to generate text and stories.

A "prompt" is an instruction or question that is input into a generative AI model.

"Means for generating and inputting prompt sentences" refers to a method or device for creating prompt sentences suitable for a generative AI model and inputting them into the model.

"Dialogue" refers to the words or lines spoken by characters.

"Scene" refers to the scenery or scene depicted in the story.

"Narrative structure" refers to the structure and arrangement that determines the flow and development of a story.

"Foreshadowing" refers to elements or hints placed in the first half of a story that will have important meaning later on.

"Mystery elements" are elements that add mystery and suspense to a story.

A "moving scene" is one that strongly stirs the viewer's emotions.

"Graphics" refers to images and illustrations generated to match each scene in the story.

"Songs" refer to music and melodies generated to match each scene in the story.

"Directory effects" refer to effects that use story structure, character dialogue, images, and music to stir the viewer's emotions.

The present invention relates to a story generation system that generates a coherent story based on a story setting entered by the user, and aims to effectively stir the emotions of the viewer.

System Configuration

User input

The user enters the story's theme, setting, characters, and other story settings in text format. For example, they can provide a specific setting such as "a knight's adventure story set in medieval Europe."

Server input analysis

The server analyzes the story setting received from the user. This analysis uses natural language processing techniques, specifically Python's NLTK library and SpaCy. The server tokenizes the input text and extracts important keywords and phrases.

Search the database

The server searches an existing story database based on the extracted keywords and phrases. This database stores information on past stories and characters. The server finds similar stories and character settings and uses them as reference information.

Prompt generation for generative AI models

Based on user input and information obtained from the database, the server generates prompts to be input into the generative AI model. For example, it creates a prompt such as, "Create a knight's adventure story set in medieval Europe. The main character is the brave knight Arthur, whose mission is to defeat an evil dragon and save the kingdom. Please include the opening of the story, lines from the characters, and descriptions of the scenery."

Story generation using generative AI models

The server inputs prompt text into a generative AI model (for example, OpenAI's GPT-4), which generates the story flow, character dialogue, and scenery. Based on the user's input, the generative AI model generates text that matches the story's theme and the characters' personalities.

Applying the Emotion Engine

The server applies an emotion engine to the generated story, adding emotional elements to each scene in the story. For example, it automatically generates foreshadowing, mystery elements, and moving scenes. This emotion engine uses IBM Watson's Tone Analyzer.

Graphics and music generation

The server automatically creates graphics and music that match each scene in the story. Image generation AI is used to generate the graphics, and music generation AI is used to generate the music.

Merge the final output

The server integrates the generated story, graphics, and music and provides them to the user. The user can then view and enjoy the completed story. For example, the user can view the story through a web browser and enjoy it along with the graphics and music.

Specific examples and prompt sentence examples

Specific examples

If the user inputs the setting "A knight's adventure story set in medieval Europe," the server will generate a story like this:

The story begins: "In the vast plains of medieval Europe stood the brave knight Arthur. His mission was to slay the evil dragon and save the kingdom."

Character Quote: "Arthur: 'I swear on this sword that I will defeat the dragon!'"

Scene description: "Under the sunset, Arthur gripped his sword and renewed his resolve."

Example prompt

An example of a prompt to input to the generative AI model is as follows:

"Create a knight's adventure story set in medieval Europe. The main character is a brave knight named Arthur, whose mission is to defeat an evil dragon and save the kingdom. Please include an opening line, character dialogue, and a description of the scenery."

By inputting this prompt into a generative AI model, each element of the story is automatically generated.

The flow of the identification process in Example 1 will be explained using Figure 17.

Step 1:

The user enters the story's theme, setting, characters, and other story settings in text format. For example, they might enter "A knight's adventure story set in medieval Europe." This input is done through the system's user interface. The entered text is sent to the server.

Step 2:

The server analyzes the story setting received from the user. Specifically, it uses Python's NLTK library and SpaCy to tokenize the text and extract important keywords and phrases. For example, it extracts keywords such as "medieval Europe," "knights," and "adventure story." The results of this analysis are used in the next step.

Step 3:

The server searches an existing story database based on the extracted keywords and phrases. The database contains information on past stories and characters. The server finds similar stories and character settings and uses them as reference information. For example, it searches for past stories with a "medieval Europe" theme or stories with a "knight" as the main character. The search results are used in the next step.

Step 4:

Based on user input and information obtained from the database, the server generates a prompt to be input into the generative AI model. For example, it might create a prompt such as, "Create a knight's adventure story set in medieval Europe. The main character is the brave knight Arthur, whose mission is to defeat an evil dragon and save the kingdom. Please include the opening of the story, lines from the characters, and a description of the scenery." This prompt will be used in the next step.

Step 5:

The server inputs prompt text into a generative AI model (for example, OpenAI's GPT-4), which generates the story flow, character dialogue, and scenery. Based on the user's input, the generative AI model generates text that matches the story's theme and the characters' personalities. For example, the generative AI model generates the following story:

The story begins: "In the vast plains of medieval Europe stood the brave knight Arthur. His mission was to slay the evil dragon and save the kingdom."

Character Quote: "Arthur: 'I swear on this sword that I will defeat the dragon!'"

Scene description: "Under the sunset, Arthur gripped his sword and renewed his resolve."

Step 6:

The server applies an emotion engine to the generated story, adding emotional elements to each scene in the story. For example, it automatically generates foreshadowing, mystery elements, and moving scenes. This emotion engine uses IBM Watson's Tone Analyzer. The emotion engine analyzes the generated text and adds appropriate emotional elements.

Step 7:

The server automatically creates graphics and music that match each scene in the story. Image generation AI is used to generate the graphics, and music generation AI is used to generate the music. For example, to match the opening scene of the story, it generates graphics depicting the scenery of medieval Europe and music that evokes a sense of adventure.

Step 8:

The server integrates the generated story, graphics, and music and provides them to the user. The user can then view and enjoy the completed story. For example, the user can view the story through a web browser and enjoy it along with the graphics and music.

(Application Example 1)

Next, we will explain Application Example 1 of Form Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

Conventional story generation systems could generate stories based on settings entered by the user, but they were unable to visualize and sound out the generated stories in real time and allow users to experience them in a virtual reality space. It was also difficult to automatically generate graphics and music that matched each scene in the story, enhancing the dramatic effects that effectively stirred the viewer's emotions. This limited the user experience and led to a lack of immersion in the story.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

In this invention, the server includes: means for inputting story settings such as the story theme, setting, and characters; means for automatically creating character dialogue, scenery, and story structure based on the input story settings; means for automatically generating foreshadowing, mystery elements, and moving scenes to make the created story more interesting; means for automatically producing graphics and music that match each scene of the created story; means for enhancing the dramatic effect with the story structure, character dialogue, images, and music, and effectively stirring the viewer's emotions; means for generating a story using a generative AI model based on the story settings input by the user; means for visualizing and audibly converting the generated story in real time; and means for the user to experience the generated story in a virtual reality space using a head-mounted display. This allows the user to visualize and audibly convert the generated story in real time and experience it in a virtual reality space.

A "story theme" is the central subject or theme of a story, and determines the overall direction and atmosphere of the story.

The "setting" refers to the place and historical background in which the story unfolds, and is an element that shapes the setting and atmosphere of the story.

"Characters" are characters who act in a story and play an important role in the progression and development of the story.

"Story setting" refers to the setting that includes basic elements such as the story's theme, setting, and characters, and forms the foundation of the story.

A "generative AI model" is a model that uses artificial intelligence technology to generate text and data, and is used to automatically create story structures and dialogue.

"Foreshadowing" refers to elements or hints in a story that give a glimpse of future developments, and is a technique used to make a story more interesting.

"Mystery elements" are elements in a story that present a mystery or question, and are intended to attract the interest of readers or viewers.

A "moving scene" is a scene in a story that stirs strong emotions and leaves a deep impression on the viewer's mind.

"Graphics" refers to visual images and footage that visually represent scenes and characters in a story.

"Song" refers to music, used to create scenes and atmosphere in a story.

"Visualization" refers to the visual representation of the story content, allowing users to enjoy the story visually.

"Audio" refers to the expression of the story content in audio, allowing users to enjoy the story by listening to it.

A "virtual reality space" is a virtual space created using computer technology that allows users to experience reality.

A "head-mounted display" is a display device worn on the head, used to allow users to experience virtual reality spaces.

A system for implementing this invention has the following configuration: First, a user uses a smartphone or head-mounted display to input story settings such as the story's theme, setting, and characters. This input is done in text format.

Next, the server uses a generative AI model (e.g., GPT-4) to automatically create character dialogue, scenes, and a story structure based on the story setting entered by the user. The generative AI model references an existing story database to generate dialogue and scenes that match the story flow, character personalities, and story theme.

In addition, the server uses an emotion engine to add hints, mystery elements, and moving scenes to the stories created, making them more interesting and moving.

The server then uses a graphics generation engine and a music generation engine to automatically create graphics and music that match each scene in the story. This enhances the visual and auditory effects of the story, effectively stirring the emotions of the viewer.

Finally, the generated story is visualized and audible in real time, and the user can experience the story in a virtual reality space using a head-mounted display. This allows the user to become immersed in the story and enjoy a deeper experience.

As a concrete example, if a user inputs "a knight's adventure story set in medieval Europe," the following prompt sentence will be input to the generative AI model:

Example prompt:

Theme: Medieval Europe

Setting: Knight's Adventure

Characters: Brave knight, wizard, dragon

Story setting: A brave knight and a wizard embark on an adventure to slay a dragon.

Based on this prompt, a generative AI model generates a story, an emotion engine adds emotional elements, and a graphics generation engine and music generation engine generate graphics and music. Users can experience this story in real time through a head-mounted display.

The flow of the specific processing in Application Example 1 is explained using Figure 18.

Step 1:

Users use a smartphone or head-mounted display to input story settings such as the story's theme, setting, and characters in text format. The input data is sent to the server.

Input: Story setting, including theme, setting, and characters.

Output: Story setting data sent to the server

Step 2:

Based on the received story setting data, the server uses a generative AI model (e.g., GPT-4) to automatically create the story structure, character dialogue, and scenery. The generative AI model references an existing story database to generate text that matches the story flow and character personalities.

Input: Story setting data

Output: Generated story structure, dialogue, and scenery

Step 3:

The server uses an emotion engine to add emotional elements such as foreshadowing, mystery elements, and moving scenes to the generated story. The emotion engine automatically inserts emotional elements appropriate to each scene in the story.

Input: Generated story structure, dialogue, and scenery

Output: A story with added emotional content

Step 4:

The server uses a graphics generation engine to automatically create graphics that match each scene in the story. The graphics generation engine takes into account the atmosphere of the scene and the emotional state of the characters to generate appropriate visuals.

Input: A story with an emotional element

Output: Graphics that match each scene

Step 5:

The server uses a music generation engine to automatically create music that matches each scene in the story. The music generation engine takes into account the atmosphere of the scene and the emotional state of the characters to generate appropriate music.

Input: A story with an emotional element

Output: Songs that match each scene

Step 6:

The server integrates the generated story, graphics, and music, visualizing and audiovisualizing them in real time, allowing users to experience the story visually and aurally.

Input: Generated story, graphics, music

Output: Real-time visual and audio narrative

Step 7:

Using a head-mounted display, users experience a story generated in a virtual reality space, immersing themselves in the story and providing a deeper experience.

Input: Real-time visual and audio narrative

Output: Narrative experience in a virtual reality space

(Example 2)

Next, we will explain Example 2 of Form Example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

While conventional story generation systems can automatically generate a story based on the story's theme and character settings, they have difficulty automatically generating foreshadowing, mystery elements, and moving scenes that make the story more interesting. Furthermore, they lack the functionality to adjust the story's development in real time based on the user's emotions, making it impossible to provide a narrative experience that is in tune with the user's emotions. Furthermore, it is difficult to automatically create graphics and music that match each scene in the story. A new system that can solve these issues is needed.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

In this invention, the server includes means for inputting story settings such as the story theme, setting, and characters; means for automatically creating character dialogue, scenery, and story structure based on the input story settings; means for automatically generating foreshadowing, mystery elements, and moving scenes to make the created story more interesting; means for recognizing the user's emotions in real time and adjusting the story development in accordance with those emotions; means for automatically creating graphics and music that match each scene of the created story; and means for enhancing the dramatic effect with the story structure, character dialogue, images, and music to effectively stir the viewer's emotions. This makes it possible not only to automatically generate a story based on the story theme and character settings, but also to automatically generate foreshadowing, mystery elements, and moving scenes, and provide a story development that matches the user's emotions.

The "story theme" indicates the overall subject or direction of the story.

"Setting" refers to the place or environment in which the story unfolds.

"Characters" refers to the characters who take action in the story.

"Story setting" refers to the setting, including basic elements such as the story's theme, setting, and characters.

"Dialogue" refers to the words or lines spoken by characters.

"Scene" refers to the scenery or scene depicted in the story.

"Story structure" refers to the overall flow and structure of a story, including its beginning, middle, and end.

"Foreshadowing" refers to elements that are laid out in advance in a way that will have important meaning later in the story.

"Mystery elements" refer to elements that attract the interest of readers or viewers by including mysteries or unknown points in a story.

A "moving scene" is one that strongly stirs the emotions of the reader or viewer.

"User emotions" refers to feelings such as joy, sadness, and surprise felt by users as they experience the story.

"Real-time recognition" refers to detecting the user's emotions instantly and on the spot.

"Adjusting the story development" refers to changing the progress and content of the story based on the user's emotions.

"Graphics" refers to images and illustrations that are displayed in accordance with story scenes.

"Song" refers to the music that plays along with the story scenes.

"Directory effects" refer to the effects of using the visual and auditory elements of a story to stimulate the emotions and interest of readers or viewers.

This invention is a system that automatically generates a story based on the story's theme and character settings, and also automatically generates foreshadowing, mystery elements, and moving scenes. It also has the ability to recognize the user's emotions in real time and adjust the story's development accordingly.

Hardware and software used

The server generates story elements using a generative AI model (e.g., GPT-4). Based on information received from the user about the story's theme, the characters' personalities, and the story's progression, the server inputs prompt sentences into the generative AI model to generate story elements.

The device sends the user's input information to the server and displays the story elements received from the server to the user. The device also uses emotion recognition software to collect emotional data from the user's facial expressions and voice and send it to the server.

Users enter information about the story's theme, the characters' personalities, and the story's progression into their device and enjoy the story. The device recognizes the user's emotions in real time and reflects them in the development of the story.

Data processing and calculation

The server performs the following data processing and calculations:

Generate appropriate foreshadowing based on the story's theme and the characters' personalities.

Mystery elements are scattered throughout the story as it progresses.

Analyze the characters' emotions and situations to create moving scenes.

The device collects emotional data from the user's facial expressions and voice and sends it to the server. The server analyzes this emotional data and adjusts the story's development.

Specific examples

For example, to generate a hint that a person the knight meets during his adventure will play an important role later in the story, the following prompt sentence can be input into the generative AI model:

Example prompt:

"Please create a plot twist that will lead to the mysterious old man the knight meets during his adventure playing an important role later in the story."

Also, if the user is feeling happy about the story, a prompt to make the story even more upbeat might look like this:

Example prompt:

"Users are enjoying this, so please make the story even more upbeat. For example, add a scene where the knight finds a new companion."

In this way, the server utilizes a generative AI model and emotion engine to automatically generate a story development that corresponds to the user's emotions.

The flow of the identification process in Example 2 will be explained using Figure 19.

Step 1:

The user enters information about the story's theme, the characters' personalities, and the story's progression.

Input: Story theme (e.g., adventure), character traits (e.g., bravery), and information about the story's progression.

Output: User input information

Specific operation: The user enters the theme "adventure" into the device's input form and sets the character's personality to "brave."

Step 2:

The device sends the user's input information to the server.

Input: User input information

Output: User input information sent to the server

Specific operation: The device sends the information "Theme: Adventure, Personality: Brave" to the server.

Step 3:

The server generates story elements using generative AI models.

Input: User input information

Output: Generated story elements

Specific operation: The server inputs the prompt "Generate a story with an adventure theme and a brave knight as the protagonist" into the generation AI model, and generates the elements of the story.

Step 4:

The server sends the generated story elements to the device.

Input: Generated story elements

Output: Story elements sent to the device

Specific operation: The server sends story elements generated by the server (e.g., "The mysterious old man the knight meets during his adventure") to the device.

Step 5:

The device displays the story to the user.

Input: Story elements received from the server

Output: The story displayed to the user

Specific operation: The device displays a scene to the user in which a knight meets a mysterious old man during his adventure.

Step 6:

Recognize user emotions in real time.

Input: User's facial expressions and voice

Output: Recognized user emotion data

Specific operation: When the user smiles, the device captures the user's facial expression with its camera and recognizes that the user is feeling happy.

Step 7:

The device sends the user's emotional data to the server.

Input: Recognized user emotion data

Output: Emotion data sent to the server

Specific operation: The device sends emotional data indicating "the user is feeling happy" to the server.

Step 8:

The server analyzes the emotional data and adjusts the story development.

Input: User emotion data

Output: Coordinated story development

Specific operation: The server analyzes the emotional data and inputs prompt sentences into the generative AI model to "make the story more upbeat."

Step 9:

The server will send the adjusted story to your device.

Input: Adjusted story development

Output: The adjusted story sent to the terminal.

Specific operation: The server generates a story with an upbeat plot (e.g., "A scene where the knight finds a new companion") and sends it to the device.

Step 10:

The device will then display the adjusted story to the user.

Input: Adjusted story received from the server

Output: The adjusted story displayed to the user.

Specific operation: The device displays a scene to the user in which a knight finds a new companion.

(Application Example 2)

Next, we will explain Application Example 2 of Form Example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

While conventional story generation systems could automatically generate a story based on the story's theme and character settings, they were unable to adjust the story's development in real time in response to the user's emotions. They also lacked the functionality to automatically generate plot twists, mystery elements, and moving scenes to make a story more interesting. This made it difficult to provide an interactive story experience that responded to the user's emotions.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

In this invention, the server includes: means for inputting story settings such as the story theme, setting, and characters; means for automatically creating character dialogue, scenery, and story structure based on the input story settings; means for automatically generating foreshadowing, mystery elements, and moving scenes to make the created story more interesting; means for automatically creating graphics and music that match each scene of the created story; means for enhancing the dramatic effect with the story structure, character dialogue, images, and music, and effectively stirring the viewer's emotions; means for recognizing the user's emotions in real time and adjusting the story development in response to those emotions; and means for generating story prompts using a generative AI model and automatically generating a story. This makes it possible to provide an interactive story experience that responds to the user's emotions.

"Story theme" refers to the central subject or message of a story.

"Setting" refers to the place or environment in which the story unfolds.

"Characters" refers to the characters or people who play a role in the story.

"Story setting" refers to the setting, including basic elements such as the story's theme, setting, and characters.

"Dialogue" refers to the words or lines spoken by characters.

"Scene" refers to the scenery or scene depicted in the story.

"Story structure" refers to the overall flow and structure of a story, including its beginning, middle, and end.

"Foreshadowing" refers to elements or hints placed in the first half of a story that will have important meaning later on.

"Mystery elements" refer to elements in a story that raise questions or raise doubts.

A "moving scene" is one that evokes strong emotions in the viewer or reader.

"Graphics" refers to images and illustrations that are displayed in accordance with story scenes.

"Song" refers to the music or soundtrack that plays along with the story scenes.

"Stage effects" refer to visual and auditory techniques used to make story scenes appear more effective.

"Viewer sentiment" refers to the emotions and feelings of the people watching the story.

"Recognizing emotions in real time" refers to analyzing the user's facial expressions, tone of voice, etc. to grasp their emotions at that moment.

"Adjusting the development of the story" refers to changing the progress and content of the story in response to the user's emotions.

"Generative AI model" refers to a model that uses artificial intelligence to generate text or stories.

A "prompt" refers to an instruction or keyword that is input into a generative AI model.

A system for implementing this invention has the following configuration: First, the server provides a means for inputting story settings such as the story's theme, setting, and characters. By inputting these settings, the user determines the basic elements of the story.

Next, the server has a means to automatically create character dialogue, scenery, and story structure based on the input story setting. This means uses a generative AI model to generate story prompts and automatically generate the story.

Furthermore, the server has the means to automatically generate hints, mystery elements, moving scenes, and more to make the stories you create more interesting. This helps to add depth and interest to the story.

The server also has the means to automatically create graphics and music that match each scene in the story. This enhances the visual and auditory effects of the story. Specifically, the graphics and music are created taking into account the atmosphere of the scene and the emotional state of the characters.

Furthermore, the server has the means to enhance the dramatic effect through the story structure, character dialogue, visuals, and music, effectively stirring the viewer's emotions. This allows users to be more emotionally drawn into the story.

What's particularly important is that the server has a means of recognizing the user's emotions in real time and adjusting the story development accordingly. It uses the smartphone's camera and microphone to analyze the user's facial expressions and tone of voice to understand their emotions. Based on this emotional data, it generates prompt sentences for the story and adjusts the story development accordingly.

For example, if a user is reading a story and the camera captures their facial expressions and recognizes that they are smiling, the story will develop in a lighthearted and cheerful way. Conversely, if the user is sad, the story will develop in a more profound way.

Examples of prompt statements include:

"The protagonist continues his adventure with a happy feeling."

"The protagonist faces difficulties, but does not lose hope."

"The protagonist takes on a new mystery."

In this way, it is possible to provide an interactive narrative experience that responds to the user's emotions.

The flow of the specific processing in Application Example 2 is explained using Figure 20.

Step 1:

The user enters story settings such as theme, setting, and characters.

Input: Setting information such as the story's theme, setting, and characters entered by the user.

Output: Story setting data.

Specific operation: The user uses a smartphone application to input the basic setting of the story, including information about the story's theme, location, and main characters. The server receives this setting information and stores it as story setting data.

Step 2:

The server automatically creates character dialogue, scenes, and story structure based on the story settings entered.

Input: Story setting data.

Output: Story dialogue, scenery, and composition data.

Specific operation: The server uses the generative AI model to generate character dialogue, scenery, and story structure based on story setting data. The generated data is saved as elements necessary for the story's progression.

Step 3:

The server automatically generates hints, mystery elements, moving scenes, and more to make the stories it creates more interesting.

Input: Story lines, scenery, and composition data.

Output: Foreshadowing, mystery elements, and moving scene data.

Specifically, how it works: The server uses a generative AI model to generate plot twists, mystery elements, and moving scenes based on the dialogue, scenery, and composition data of the story. This adds depth and intrigue to the story.

Step 4:

The server automatically creates graphics and music that match each scene in the story.

Input: Foreshadowing, mystery elements, and moving scene data.

Output: Graphics data, music data.

Specific operation: The server automatically creates graphics and music, taking into account the atmosphere of the scene and the emotional state of the characters. This is done using image generation models and music generation models. The generated graphics and music are then applied to each scene in the story.

Step 5:

The server recognizes the user's emotions in real time and adjusts the story development accordingly.

Input: User's facial expression data, tone of voice data.

Output: Tailored narrative prompt.

Specific operation: The smartphone's camera and microphone are used to capture the user's facial expressions and tone of voice. The server analyzes this data and recognizes the user's emotions in real time. Based on the recognized emotions, it generates story prompts and adjusts the story's development.

Step 6:

The server uses a generative AI model to generate story prompts and automatically generate the story.

Input: Tailored story prompt.

Output: The generated story.

Specific operation: The server uses a generative AI model to generate a story based on the adjusted prompt. The generated story is used to provide an interactive experience that responds to the user's emotions.

(Example 3)

Next, we will explain Example 3 of Form Example 3. In the following explanation, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

With conventional story generation systems, it was difficult to automatically create graphics and music that matched each scene in the story, making it difficult to effectively sway the viewer's emotions. Furthermore, they lacked the ability to recognize the user's emotions in real time and adjust the story's development accordingly, making it difficult to provide the user with a consistent emotional experience. This resulted in a decline in the quality of the story experience and low user satisfaction.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Example 3 is realized by the following means.

In this invention, the server includes means for inputting story settings such as the story theme, setting, and characters; means for automatically creating character dialogue, scenery, and story structure based on the input story settings; means for automatically generating foreshadowing, mystery elements, and moving scenes to make the created story more interesting; means for automatically creating graphics and music that match each scene of the created story; means for using information such as the user's facial expressions, tone of voice, and text input to recognize the user's emotions; means for adjusting the story development in accordance with the user's emotions; and means for enhancing the dramatic effects of the story structure, character dialogue, images, and music to effectively stir the viewer's emotions. This makes it possible to automatically create graphics and music that match each scene of the story and provide a story development that matches the user's emotions.

"Story theme" refers to the central subject or subject of a story.

"Setting" refers to the place or environment in which the story unfolds.

"Characters" refers to the characters that appear in the story.

"Story setting" refers to the setting, including basic elements such as the story's theme, setting, and characters.

"Dialogue" refers to the words spoken by characters.

"Scene" refers to the scenery or scene depicted in the story.

"Story structure" refers to the overall flow of a story, including its beginning, middle, and end.

"Foreshadowing" refers to elements that provide advance hints at information or events that will become important later in the story.

"Mystery elements" refer to elements that add mystery or suspense to a story.

A "moving scene" refers to a scene that strongly stirs the viewer's emotions.

"Graphics" refers to images and illustrations generated to match story scenes.

"Song" refers to music generated to match the story scenes.

"User emotion" refers to the emotional state the user is feeling, such as joy, sadness, or surprise.

"Facial expression" refers to emotions expressed through the movement of the user's facial muscles.

"Tone of voice" refers to the pitch, strength, and intonation of the user's voice.

"Text input" refers to text information entered by a user using a keyboard or touch panel.

"Story development" refers to the process and flow of the story.

"Directory effects" refer to the impact that the visual and auditory elements of a story have on the viewer.

"Viewer sentiment" refers to the emotions and feelings of the people watching the story.

This invention is a system that inputs story settings such as the story's theme, setting, and characters, automatically generates a story based on the input story settings, and further adjusts the story's development according to the user's emotions. This system is implemented using the following hardware and software.

Hardware and software used

Hardware: High-performance servers (e.g., servers with GPUs) and user devices (e.g., PCs or smartphones with cameras and microphones)

Software: Generative AI models (e.g., GPT-4, image generation AI, music generation AI), emotion recognition software (e.g., OpenCV, voice analysis libraries)

Program processing explanation

1. Story input and analysis

The user uses their device to input story settings such as the story's theme, setting, and characters. For example, they might input "a scene where a hero fights a dragon." The server receives this input and uses a generative AI model (e.g., GPT-4) to analyze each scene in the story and extract the scene's atmosphere and the characters' emotional states.

2. Graphics and Music Generation

Based on the analysis results, the server uses image generation AI to generate graphics that match the scene. For example, it could generate an impressive graphic of a hero fighting a dragon. Similarly, it uses music generation AI to generate music that suits the scene. For example, it could generate music that heightens the tension in a battle scene.

3. User Emotion Recognition

The device uses a camera and microphone to capture the user's facial expressions and tone of voice. For example, if the user is smiling, the device sends that information to the server. The server then uses emotion recognition software (e.g., OpenCV, voice analysis library) to analyze the user's emotions.

4. Story Development Adjustments

The server uses an emotion engine to adjust the story development based on the user's emotional information. For example, if the user is smiling, the story development will be adjusted to a more exciting direction.

Specific examples and prompt sentence examples

Example:

The user enters "a scene in which a hero fights a dragon."

The server analyzes the scene as a battle scene and uses image generation AI to generate powerful graphics of the dragon and hero.

Music generation AI is used to generate music that heightens the tension.

If the user is smiling, the emotion engine recognizes that the user is having fun and adjusts the story development in a more exciting direction.

Example prompt:

"Generate graphics for a scene in which a hero fights a dragon."

"Generate music that matches an emotional scene."

"If the user is smiling, adjust the story development to make it more enjoyable."

This system makes it possible to automatically create graphics and music that match each scene in a story, and to provide a story development that responds to the user's emotions. This effectively stirs the viewer's emotions and improves the quality of the story experience. The flow of the specific processing in Example 3 is explained using Figure 21.

Step 1:

The user enters the text of the story.

The user enters "A scene in which a hero fights a dragon" into the text input field on the device and clicks the send button. The entered text is sent from the device to the server.

Step 2:

The server analyzes the story text.

The server inputs the received text into a generative AI model (e.g., GPT-4) and analyzes each scene in the story. As a result of the analysis, the atmosphere of the scene and the emotional state of the characters are extracted. For example, "battle scene" and "tension" are extracted. The analysis results are stored in an internal database.

Step 3:

The server generates graphics that match the scene.

Based on the analysis results, the server uses image generation AI (e.g., DALL-E) to generate graphics that match the scene. For example, enter "a scene of a hero fighting a dragon" as a prompt and obtain the generated graphic. The generated graphic is then sent to the user's device.

Step 4:

The server generates music that matches the scene.

The server uses music generation AI to generate music appropriate for the scene. For example, a user can enter "music that will increase tension in a battle scene" as a prompt and obtain the generated music. The generated music is then sent to the user's device.

Step 5:

The device recognizes the user's emotions.

The device uses a camera to capture the user's facial expressions and a microphone to record the tone of voice. This data is sent to the server in real time. For example, if the user is smiling, that information is sent to the server.

Step 6:

The server adjusts the story development based on the user's emotions.

The server analyzes the received emotional data using emotion recognition software (e.g., OpenCV, voice analysis library) to recognize the user's emotions. For example, if it recognizes that the user is smiling, it uses the emotion engine to adjust the story development in an exciting direction. A new scene is generated and sent to the user's device.

(Application Example 3)

Next, we will explain Application Example 3 of Form Example 3. In the following explanation, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

Conventional story generation systems have the problem of not adjusting the story based on the user's emotions, resulting in a uniform user experience. Furthermore, because the story scenes and music are fixed, they are unable to provide dynamic changes that respond to the user's emotions. This makes it difficult to provide a story experience that deeply appeals to the user's emotions.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 3 is realized by the following means.

In this invention, the server includes means for inputting story settings such as the story theme, setting, and characters; means for automatically creating character dialogue, scenery, and story structure based on the input story settings; means for automatically generating foreshadowing, mystery elements, and moving scenes to make the created story more interesting; means for automatically creating graphics and music that match each scene of the created story; means for enhancing the dramatic effect with the story structure, character dialogue, images, and music, and effectively stirring the viewer's emotions; means for recognizing the user's emotions and adjusting story scenes and music in accordance with the recognized emotions; and means for recognizing emotions using the user's facial expressions, tone of voice, text input, etc. This enables a dynamic story experience that matches the user's emotions.

A "story theme" refers to the central idea or message of a story.

"Setting" refers to the place or environment in which the story unfolds.

"Characters" refers to the characters that appear in the story.

"Story setting" refers to the setting, including basic elements such as the story's theme, setting, and characters.

"Dialogue" refers to the words spoken by characters.

"Scene" refers to the scenery or scene depicted in the story.

"Story structure" refers to the overall flow of a story, including its beginning, middle, and end.

"Foreshadowing" refers to an element in the first half of a story that becomes important later in the story.

"Mystery elements" refer to elements that add mystery and suspense to a story.

A "moving scene" is one that strongly stirs the viewer's emotions.

"Graphics" refers to images and footage that visually represent scenes from a story.

"Song" refers to the music that plays along with the story scenes.

"Directional effects" refer to effects that influence the viewer's emotions through story scenes and music.

"User emotion" refers to the emotional state the user is feeling, such as joy, sadness, or surprise.

"Means for recognizing emotions" refers to technology that determines a user's emotions using their facial expressions, tone of voice, text input, etc.

"Means for adjusting story scenes and music" refers to technology that changes story scenes and music according to the recognized user emotions.

A system for implementing this invention can dynamically adjust story scenes and music according to the user's emotions. A specific embodiment of this system is described below.

System Configuration

The system includes the following major hardware and software components:

Hardware: Smartphone camera

Software: OpenCV (face detection), Keras (emotion recognition model), music playback software

Program processing

The server first captures the user's facial expression using the smartphone camera. The captured video undergoes face detection using OpenCV. Next, it uses the Keras emotion recognition model to recognize the user's emotion from the detected face.

Based on the recognized emotion, the server adjusts the story scenes and music. For example, if the user is smiling, the story scenes will change to brighter, more cheerful ones, and the music will also become more upbeat. Conversely, if the user has a sad expression, the story will transition to a more moving scene, and the music will also change to something more moving.

Specific examples

As a concrete example, consider the case where a user smiles while enjoying a story. In this case, the server would process the following:

1. Capture the user's facial expressions using the smartphone camera.

2. Detect faces using OpenCV.

3. The user's emotion is recognized as "happiness" using the Keras emotion recognition model.

4. Based on the recognized emotion, the story scene is changed to a brighter, more cheerful one and upbeat music is played.

Example prompt

An example of a prompt to input to the generative AI model is as follows:

If the user smiles, change the story scene to something bright and cheerful and play upbeat music.

In this way, it is possible to provide a dynamic narrative experience that responds to the user's emotions. This system allows users to become more immersed in the story and have an experience that appeals to their emotions.

The flow of the specific processing in Application Example 3 will be explained using Figure 22.

Step 1:

The server uses the smartphone's camera to capture the user's facial expressions. The input is the camera image, and the output is the captured image data. Specifically, the camera acquires the image in real time and captures the image frame by frame.

Step 2:

The server uses OpenCV to detect faces from the captured image data. The input is the captured image data, and the output is the location information of the detected face. Specifically, OpenCV's face detection algorithm identifies the face in the image and obtains its coordinates.

Step 3:

The server uses a Keras emotion recognition model to recognize the user's emotion from the detected face. The input is face position information and image data, and the output is the recognized emotion label. Specifically, the face area is extracted, preprocessed, and then input into the emotion recognition model to classify the emotion.

Step 4:

The server adjusts the story scenes and music based on the recognized emotions. The input is the recognized emotion label, and the output is the adjusted scenes and music. Specific operations include selecting scenes and music that correspond to the emotion label and changing the progression of the story.

Step 5:

The server sends the adjusted scene and music to the user's device. The input is the adjusted scene and music, and the output is the story scene displayed on the user's device and the music played. Specifically, the server encodes the scene and music data and sends it to the user's device over the network.

Step 6:

The user's device plays the received scenes and music and provides them to the user. The input is the scene and music data received from the server, and the output is the video displayed to the user and the music played. Specifically, the device decodes the data, displays it on the screen, and plays the music from the speakers.

The specific processing unit 290 transmits the results of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the results of the specific processing. The microphone 238 acquires audio indicating the user input regarding the results of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

Data generation model 58 is what is known as generative AI (artificial intelligence). An example of data generation model 58 is generative AI such as ChatGPT (Internet search <URL: https://openai.com/blog/chatgpt>). Data generation model 58 is obtained by performing deep learning on a neural network. A prompt containing an instruction is input to data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. Data generation model 58 performs inference on the input inference data in accordance with the instructions indicated by the prompt, and outputs the inference results in the form of data such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and/or summarization.

Another example of generative AI is Gemini (Internet search <URL: https://gemini.google.com/?hl=ja>).

In the above embodiment, an example was given in which the specific processing was performed by the data processing device 12, but the technology disclosed herein is not limited to this, and the specific processing may also be performed by the smart glasses 214.

[Third embodiment]

Figure 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

As shown in FIG. 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

The data processing device 12 includes a computer 22, a database 24, and a communication I/F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and communication I/F 26 are also connected to the bus 34. The communication I/F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and/or a LAN (Local Area Network).

The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I/F 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the user 20's surroundings (e.g., an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

The communication I/F 44 is connected to the network 54. The communication I/Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I/Fs 44 and 26 is carried out in a secure manner.

Figure 6 shows an example of the main functions of the data processing device 12 and headset terminal 314. As shown in Figure 6, in the data processing device 12, specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32.

The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as the specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

Storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

In the headset terminal 314, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

Next, we will explain the identification process performed by the identification processing unit 290 of the data processing device 12.

"Example 1"

The system of the present invention receives input from the user about the story setting, such as the story's theme, setting, and characters. This input is in text format, such as "A knight's adventure story set in medieval Europe." Next, based on this input, the system automatically creates character dialogue, scenes, and a story structure. This creation is carried out using existing story databases and artificial intelligence technology, generating dialogue and scenes that match the story flow, character personalities, and story theme.

"Example 2"

In addition, the system automatically generates hints, mystery elements, and moving scenes to make the story more interesting. This is done by laying hints at the right time, sprinkling mystery elements, and creating moving scenes in line with the story's theme, the characters' personalities, and the story's progression. For example, it automatically generates hints that people the knight meets during his adventure will play an important role later in the story, puzzles the knight faces, and moving scenes when the knight overcomes difficulties.

"Example 3"

The system then automatically creates graphics and music that match each scene in the story you've created. The graphics and music are created taking into account the atmosphere of the scene and the emotional state of the characters. For example, powerful graphics and music that heighten tension are created for battle scenes, while beautiful landscape graphics and moving music are created for moving scenes. This makes it possible to enhance the dramatic effect of the story's structure, the characters' conversations, images, and music, effectively stirring the viewer's emotions.

The processing flow for each example is explained below.

"Example 1"

Step 1: The user provides textual input of the story setting, including the story's theme, setting, and characters. For example, it could be provided as "a knight's adventure story set in medieval Europe."

Step 2: Based on the input, the system automatically creates character dialogue, scenery, and story structure. This is done using existing story databases and artificial intelligence technology.

Step 3: Lines and scenes are generated that match the story flow, character personalities, and theme of the story.

"Example 2"

Step 1: The system automatically generates plot twists, mystery elements, moving scenes, and more to make the story more interesting.

Step 2: Lay out hints at the right time, sprinkle in mystery elements, and create moving scenes in line with the story's theme, the characters' personalities, and the story's progression.

Step 3: For example, foreshadowing, such as people the knight meets during his adventure playing important roles later in the story, puzzles the knight faces, and moving scenes when the knight overcomes difficulties, are automatically generated.

"Example 3"

Step 1: The system automatically creates graphics and music that match each scene in the story you've created.

Step 2: Graphics and music are created taking into account the atmosphere of the scene and the emotional state of the characters.

Step 3: For example, powerful graphics and music that heightens the tension are created for battle scenes, while beautiful landscape graphics and moving music are created for moving scenes. This enhances the dramatic effect of the story structure, character dialogue, images and music, making it possible to effectively stir the viewer's emotions.

(Example 1)

Next, Example 1 of Form Example 1 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset-type terminal 314 will be referred to as the "terminal."

When conventional story generation systems automatically generate a story based on a user-entered story setting, they struggle to maintain consistency in the story flow, character dialogue, and scenery. They also lack the ingenuity to ensure the generated story is moving and entertaining for the viewer. Furthermore, they are unable to automatically create graphics and music that match each scene in the story, which means they are unable to effectively move the viewer's emotions.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

In this invention, the server includes: means for inputting story settings such as the story's theme, setting, and characters; means for automatically creating character dialogue, scenery, and story structure based on the input story settings; means for automatically generating foreshadowing, mystery elements, and moving scenes to make the created story more interesting; means for automatically producing graphics and music that match each scene of the created story; means for enhancing the dramatic effect with the story structure, character dialogue, and images and music to effectively stir the viewer's emotions; means for receiving input from the user and analyzing it using natural language processing technology; means for generating prompt sentences to be input into the generative AI model based on the analysis results; means for automatically generating story dialogue, scenery, and structure using the generative AI model; and means for formatting the generated story and providing it to the user. This makes it possible to automatically generate a consistent story based on user input and provide the viewer with a moving and entertaining story.

A "story theme" is the central subject or theme of a story, and is the element that determines the overall direction and atmosphere of the story.

The "setting" refers to the place and historical background in which the story unfolds, and is an important element in shaping the setting and atmosphere of the story.

"Characters" are characters who act within a story and move the story forward, and are elements that have a major influence on the story's development and theme.

"Story setting" refers to the setting that includes basic elements such as the story's theme, setting, and characters, and is the element that forms the framework of the story.

"Dialogue" refers to the words or lines spoken by characters, and is an important element in expressing the progression of the story and the characters' personalities.

"Scenes" refer to the scenery and scenes depicted in a story, and are elements that visually express the atmosphere and emotions of the story.

"Story structure" refers to the overall flow and structure of a story, including the beginning, middle, and end, and is an element that makes the story unfold effectively.

"Foreshadowing" is a hint or sign that relates to later developments in a story, and is an element that adds consistency and depth to a story.

"Mystery elements" are elements that present mysteries or questions within a story and attract the interest of readers or viewers.

A "moving scene" is a scene in a story that strongly stirs emotions and is an impressive element that stays in the viewer's mind.

"Graphics" refers to images and illustrations that visually represent each scene in the story, and are elements that visually complement the atmosphere and emotion of the story.

"Songs" are melodies and sounds that are musically expressed to match each scene in a story, and are elements that musically complement the atmosphere and emotions of the story.

"Natural language processing technology" is a technology that enables computers to understand and analyze human language, and is used to analyze text data and extract information.

A "generative AI model" is a model that uses artificial intelligence technology to generate text and data, and is used to automatically generate dialogue, scenes, and structure for a story.

A "prompt" is an instruction to be input into a generative AI model, and is an element that guides the content generated by the model.

"Formatting" refers to the process of adding paragraph breaks and line breaks to make the generated narrative text easier to read before it is presented to the user.

This invention is a system that automatically generates a story based on story settings such as the story's theme, setting, and characters, which are input by the user. Specific embodiments of this system are described below.

User input

The user uses a terminal to enter text information about the story's theme, setting, characters, etc. into the system's input form. For example, they might enter "A knight's adventure story set in medieval Europe."

Receive and analyze input data by the server

The server receives the text data sent by the user. It then analyzes the input data using natural language processing technology. Specifically, it uses Python's NLTK library and spaCy to extract information such as theme, setting, and characters from the input text. For example, it extracts keywords such as "medieval Europe," "knights," and "adventure story."

Generating prompt statements

The server generates a prompt to be input into the generative AI model based on the extracted information. For example, if the user's input is "A knight's adventure story set in medieval Europe," the prompt would be "Please generate lines and scenes for the characters in a knight's adventure story set in medieval Europe."

Generating stories using generative AI models

The server inputs the generated prompt into a generative AI model (for example, OpenAI's GPT-4). The generative AI model automatically generates the dialogue, scenery, and structure of a story based on the prompt. For example, it could generate text such as, "The brave knight Arthur met a wizard in a dark forest. The wizard said to Arthur, 'Deep in this forest lies an ancient dragon. Your mission is to slay the dragon.'"

Format and provide generated results

The server receives the story text returned by the generative AI model and formats it as needed, for example by adding paragraph breaks and line breaks to make it easier to read. The formatted story is then converted into a format that is easy for the user to read. Finally, the server provides the formatted story to the user, who can view the results through their device.

Specific examples

As an example of user input, consider "a knight's adventure story set in medieval Europe." Based on this input, the server generates the following prompt:

"Generate lines and scenes from characters in a knight's adventure story set in medieval Europe."

The generative AI model generates a story based on this prompt, outputting text such as the following:

"The brave knight Arthur met a wizard in a dark forest. The wizard told Arthur, 'Deep in this forest lies an ancient dragon. Your mission is to defeat the dragon.'"

In this way, it is possible to automatically generate a coherent story based on user input, providing an inspiring and entertaining story for the viewer.

The flow of the identification process in Example 1 will be explained using Figure 11.

Step 1:

The user uses their device to enter text information about the story's theme, setting, characters, etc. into the system's input form. For example, they might enter "A knight's adventure story set in medieval Europe." This input data is then sent to the server.

Step 2:

The server receives text data sent by the user. It analyzes the received data using natural language processing technology. Specifically, it uses Python's NLTK library and spaCy to extract information such as theme, setting, and characters from the input text. For example, it extracts keywords such as "medieval Europe," "knights," and "adventure story." The results of this analysis become the input for the next step.

Step 3:

Based on the analysis results, the server generates a prompt to be input into the generative AI model. For example, if the user's input is "A knight's adventure story set in medieval Europe," the prompt would be "Please generate lines and scenes for the characters in a knight's adventure story set in medieval Europe." This prompt becomes the input for the next step.

Step 4:

The server inputs the generated prompt into a generative AI model (for example, OpenAI's GPT-4). The generative AI model automatically generates the dialogue, scenery, and structure of a story based on the prompt. For example, it might generate text such as, "The brave knight Arthur met a wizard in a dark forest. The wizard said to Arthur, 'Deep in this forest lies an ancient dragon. Your mission is to slay the dragon.'" This generated text becomes the input for the next step.

Step 5:

The server receives the story text returned by the generative AI model and formats it as needed, for example by adding paragraph breaks and line breaks to make it easier to read. The formatted story is then converted into a format that is easier for the user to read. This formatted text becomes the input for the next step.

Step 6:

The server provides the formatted story to the user. The user can view the results through their device, for example, on a web page or application screen that displays the generated story. The user can then read and enjoy the generated story.

(Application Example 1)

Next, we will explain Application Example 1 of Form Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the headset-type terminal 314 will be referred to as the "terminal."

Existing story generation systems automatically generate stories based on user-entered themes and settings, but they often lack sufficient story structure, character dialogue, and scene generation, making it difficult to effectively sway viewers' emotions. Furthermore, the lack of functionality to preview, save, and share the generated stories creates a problem of inconvenient user experience.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

In this invention, the server includes: means for inputting story settings such as the story's theme, setting, and characters; means for automatically creating character dialogue, scenery, and story structure based on the input story settings; means for automatically generating foreshadowing, mystery elements, and moving scenes to make the created story more interesting; means for automatically producing graphics and music that match each scene of the created story; means for enhancing the dramatic effect with the story structure, character dialogue, images, and music to effectively stir the viewer's emotions; means for generating prompt sentences using a generative AI model and automatically generating a story; and means for previewing, saving, and sharing the generated story. This makes it possible to automatically generate high-quality stories based on the settings input by the user, effectively stirring the viewer's emotions.

A "story theme" is the central subject or theme of a story, and determines the overall direction and atmosphere of the story.

The "setting" refers to the place and historical background in which the story unfolds, and is an element that shapes the setting and atmosphere of the story.

"Characters" are characters that appear in a story and play an important role in the progression and development of the story.

"Story setting" refers to the setting that includes basic elements such as the story's theme, setting, and characters, and forms the framework of the story.

"Dialogue" refers to the words spoken by characters, and is an important element in expressing the progression of the story and the character's personality.

"Scenes" refer to the landscapes and scenes depicted in a story, and visually express the atmosphere and emotions of the story.

"Story structure" refers to the overall flow and structure of a story, including a beginning, middle, and end, and is the framework for effectively developing the story.

"Foreshadowing" is a hint or omen that relates to later developments in a story, and is an element that makes a story more interesting.

"Mystery elements" are elements that add mystery and suspense to a story, and are meant to attract the interest of readers and viewers.

A "moving scene" is a scene in a story that stirs strong emotions and is intended to leave a deep impression on the viewer.

"Graphics" refers to the visual images and illustrations that accompany each scene in the story, and are intended to visually express the atmosphere and emotions of the story.

"Songs" are pieces of music created to accompany each scene in a story, and are intended to musically express the atmosphere and emotions of the story.

A "generative AI model" is a model that uses artificial intelligence technology to automatically generate text, images, etc., and is used to automatically generate stories.

A "prompt" is an instruction to be input into a generative AI model, and is generated based on the theme and setting of the story.

"Preview" is a feature that allows users to preview the generated story in advance and check the content.

"Preservation" refers to recording the stories that are created in digital form so that they can be reused and shared later.

"Sharing" refers to sharing the story you create with other users and platforms, with the aim of promoting the spread and appreciation of the story.

A system for implementing this invention allows the user to input story settings such as the story's theme, setting, and characters, and automatically generates a story based on these settings. Specific embodiments of this system are described below.

System Configuration

This system is primarily composed of the following hardware and software:

Hardware: Smartphones, servers

Software: Python, OpenAI API

Program processing

User input

Users enter the story's theme, setting, and characters in text form through a smartphone application. For example, they might enter the theme "A knight's adventure story set in medieval Europe."

Prompt generation

The server generates a prompt based on the information entered by the user. This prompt is an instruction that is input into the generative AI model. As a specific example, the following prompt is generated:

Theme: A knight's adventure story set in medieval Europe

Setting: Medieval Europe

Characters: Brave knight, wizard, dragon

Generate a story.

Story generation

The server uses a generative AI model (e.g., GPT-4) to automatically generate a story based on the generated prompt. This generative AI model is called through the OpenAI API.

Preview, save, share

The generated story is displayed so that users can preview it through the smartphone application. Users can save the generated story and share it on social media, etc.

Specific examples

If a user sets the theme to "a knight's adventure story set in medieval Europe" and enters "a brave knight, a wizard, and a dragon" as the characters, the following story will be generated.

Theme: A knight's adventure story set in medieval Europe

Setting: Medieval Europe

Characters: Brave knight, wizard, dragon

Generate a story.

By inputting this prompt into a generative AI model, a story is automatically generated. Users can preview, save, and share the generated story.

Hardware and software used

Hardware: Smartphones, servers

Software: Python, OpenAI API

This makes it possible to automatically generate high-quality stories based on user-entered settings, effectively stirring the emotions of viewers.

The flow of the specific processing in Application Example 1 will be explained using Figure 12.

Step 1:

Users use their smartphone to input the story's theme, setting, and characters in text format. The input data is then sent from the device to the server. An example input includes the theme "A knight's adventure story set in medieval Europe."

Step 2:

The server generates a prompt based on the received user input data. Specifically, it combines information about the theme, setting, and characters to create a prompt that is suitable for the generative AI model. The generated prompt looks like this:

Theme: A knight's adventure story set in medieval Europe

Setting: Medieval Europe

Characters: Brave knight, wizard, dragon

Generate a story.

This prompt text is generated.

Step 3:

The server calls a generative AI model (e.g., GPT-4) with the generated prompt text. It sends the prompt text using the OpenAI API and requests the generation of a story. The input is the prompt text, and the output is the generated story text.

Step 4:

The generative AI model automatically generates a story based on the prompt text. The generated story text is returned to the server. For example, the following story might be generated:

Once upon a time, in a vast kingdom in medieval Europe, there lived a brave knight. His name was Arthur. One day, with the help of a wizard, he embarked on an adventure to battle a fearsome dragon.

Step 5:

The server sends the text of the generated story to the smartphone device. The user can preview the generated story through the smartphone device. The input is the text of the generated story, and the output is the preview screen displayed on the user's device.

Step 6:

Users can save the generated story or share it on social media. Saved stories are recorded in the device's storage, and shared stories are sent to the specified platform. The input is the user's selection, and the output is the saved file or shared content.

These steps make it possible to automatically generate a high-quality story based on the settings entered by the user, effectively stirring the viewer's emotions.

(Example 2)

Next, Example 2 of Form Example 2 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset-type terminal 314 will be referred to as the "terminal."

While conventional story generation systems can automatically generate stories based on the story's theme and character settings, they have difficulty appropriately generating foreshadowing, mystery elements, and moving scenes to make the story more interesting. Furthermore, they lack the functionality to organize the generated story elements and provide them to users, making it difficult for users to effectively use the generated story. Furthermore, they lack the means to enhance the dramatic effect through story structure, character dialogue, visuals, and music, and to effectively stir the viewer's emotions.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

In this invention, the server includes means for inputting story settings such as the story's theme, setting, and characters; means for automatically creating character dialogue, scenery, and story structure based on the input story settings; means for automatically generating foreshadowing, mystery elements, and moving scenes to make the created story more interesting; means for organizing the generated story elements and providing them to the user; and means for enhancing the dramatic effect with the story structure, character dialogue, images, and music to effectively stir the viewer's emotions. This makes it possible to make the story more interesting and enable users to effectively use the generated story.

"Story theme" refers to the central subject or subject matter of a story.

"Setting" refers to the place or environment in which the story unfolds.

"Characters" refers to the characters and people who appear in the story.

"Story setting" refers to the setting, including basic elements such as the story's theme, setting, and characters.

"Dialogue" refers to the words or lines spoken by characters.

"Scene" refers to the scenery or scene depicted in the story.

"Story structure" refers to the overall flow and structure of a story, including its beginning, middle, and end.

"Foreshadowing" refers to elements or events placed in the first half of a story that will have important meaning later on.

"Mystery elements" refer to elements that add mystery or detective elements to a story, attracting the interest of readers and viewers.

A "moving scene" refers to a scene in a story that stirs strong emotions.

"Graphics" refers to images and illustrations that are displayed in accordance with story scenes.

"Song" refers to the music or background music that plays along with the story scenes.

"Directory effects" refers to techniques that use the visual and auditory elements of a story to elicit emotions and reactions from the viewer.

"Audience sentiment" refers to the emotions and feelings of the people who watch or read the story.

This invention is a system that automatically generates plot twists, mystery elements, and moving scenes to make a story more interesting, based on the story's theme, the characters' personalities, and the story's progression. A specific embodiment of this system is shown below.

System Configuration

Hardware

The server uses a cloud server equipped with a high-performance GPU.

Software

The server uses a generative AI model. Specifically, it uses a generative AI model such as OpenAI's GPT-4.

Program processing

User input

The user uses the device to input information about the story's theme, the characters' personalities, and the story's progression. For example, the user might input the following information:

Theme: Medieval Knight's Adventure

Characters: A brave knight, a mysterious old man, and a princess

Story progression: The knight meets a mysterious old man during his adventure, receives advice from him, and continues his adventure. Ultimately, the knight saves the princess and protects the kingdom.

Generating prompt statements

The server receives the information entered by the user and generates a prompt based on that information. The generated prompt looks like this:

Theme: Medieval Knight's Adventure

Characters: A brave knight, a mysterious old man, and a princess

Story progression: The knight meets a mysterious old man during his adventure, receives advice from him, and continues his adventure. Ultimately, the knight saves the princess and protects the kingdom.

Generating story elements

The server inputs the generated prompt text into the generative AI model to generate story elements. The generative AI model generates the following story elements based on the prompt text.

Foreshadowing: The mysterious old man the knight meets during his adventure actually knows the kingdom's hidden secrets.

Mystery element: The knight solves the mystery hidden in an old map he received from an old man.

Emotional scene: The knight saves the princess and fights a final battle to protect the kingdom.

Organizing and providing generated story elements

The server organizes the story elements output by the generative AI model and provides them to the user. The user can check the generated story elements via their device and make corrections or additions as needed.

Specific examples

Below is a concrete example of how a user can use the system to generate a story.

Theme: Medieval Knight's Adventure

Characters: A brave knight, a mysterious old man, and a princess

Story progression: The knight meets a mysterious old man during his adventure, receives advice from him, and continues his adventure. Ultimately, the knight saves the princess and protects the kingdom.

The server inputs this prompt into a generative AI model, which automatically generates plot twists, mystery elements, and moving scenes based on the story's theme, the characters' personalities, and the story's progression. The generated story elements are provided to the user, who can review them and make corrections or additions as needed.

In this way, it is possible to make the story more interesting and enable users to effectively use the generated story.

The flow of the identification process in Example 2 will be explained using Figure 13.

Step 1:

The user enters information about the story's theme and characters.

Input: Information about the story's theme, character personalities, and story progression.

Specific operation: The user enters the theme "Adventures of a Medieval Knight" and character information - "A Brave Knight, a Mysterious Old Man, and a Princess" - into the input form on their device. Once the input is complete, they click the "Submit" button.

Output: The entered information is sent to the server.

Step 2:

The server receives the entered information and generates a prompt.

Input: User-submitted information about the story's theme, character personalities, and story progression.

Specific operation: Based on the received information, the server generates a prompt sentence to input into the generative AI model.

Output: The generated prompt text.

Theme: Medieval Knight's Adventure

Characters: A brave knight, a mysterious old man, and a princess

Story progression: The knight meets a mysterious old man during his adventure, receives advice from him, and continues his adventure. Ultimately, the knight saves the princess and protects the kingdom.

Step 3:

The server inputs prompts into the generative AI model to generate story elements.

Input: The generated prompt text.

Specific operation: The server inputs prompt text into the generative AI model, generating story hints, mystery elements, and moving scenes.

Output: Generated story elements.

Foreshadowing: The mysterious old man the knight meets during his adventure actually knows the kingdom's hidden secrets.

Mystery element: The knight solves the mystery hidden in an old map he received from an old man.

Emotional scene: The knight saves the princess and fights a final battle to protect the kingdom.

Step 4:

The server organizes the generated story elements and provides them to the user.

Input: Story elements output from a generative AI model.

Specific operation: The server organizes the generated story elements and provides them to the user. The user can review the generated story elements through their device and make corrections or additions as necessary.

Output: The organized story elements are presented to the user.

(Application Example 2)

Next, we will explain Application Example 2 of Form Example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the headset-type terminal 314 will be referred to as the "terminal."

While conventional story generation systems can automatically create stories based on the story's theme and character settings, they have difficulty properly generating foreshadowing, mystery elements, and moving scenes to make the story more interesting. It is also difficult to automatically create graphics and music that match each scene in the generated story and effectively stir the viewer's emotions. Furthermore, when using generative AI models to generate story elements, there is a lack of a way to properly generate prompt sentences and input them into the model, which poses the challenge of not being able to provide a satisfying story to the user.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

In this invention, the server includes means for inputting story settings such as the story theme, setting, and characters; means for automatically creating character dialogue, scenery, and story structure based on the input story settings; means for automatically generating foreshadowing, mystery elements, and moving scenes to make the created story more interesting; means for automatically creating graphics and music that match each scene of the created story; means for enhancing the dramatic effect with the story structure, character dialogue, images, and music to effectively stir the viewer's emotions; means for generating story elements using a generative AI model; means for generating prompt sentences and inputting them into the generative AI model; and means for displaying the generated story elements to the user. This makes it possible to make the story more interesting and effectively stir the viewer's emotions.

A "story theme" is the central subject or subject of a story.

"Setting" refers to the place or environment in which the story unfolds.

"Characters" refers to the characters that appear in the story.

"Story setting" refers to the basic elements of the story, such as its theme, setting, and characters.

"Dialogue" refers to the words spoken by characters.

"Scene" refers to the scenery or scene depicted in the story.

"Story structure" refers to the overall flow and structure of a story, including its beginning, middle, and end.

"Foreshadowing" is the advance indication of important elements related to later developments.

"Mystery elements" refer to the inclusion of mysteries or mysterious events in a story.

A "moving scene" is one that strongly stirs the viewer's emotions.

"Graphics" refers to visual images and illustrations.

"Song" refers to music or melody.

"Producer effects" refer to visual and auditory techniques used to make a story more appealing.

"Viewers" are people who watch a story.

A "generative AI model" is an algorithm that uses artificial intelligence to generate text, images, etc.

A "prompt" is an instruction sentence to be input into a generative AI model.

"User" refers to a person who uses the system.

A system for implementing this invention includes means for inputting story settings such as the story's theme, setting, and characters; means for automatically creating character dialogue, scenery, and story structure based on the input story settings; means for automatically generating foreshadowing, mystery elements, and moving scenes to make the created story more interesting; means for automatically creating graphics and music that match each scene of the created story; means for enhancing the dramatic effect with the story structure, character dialogue, images, and music, and effectively stirring the viewer's emotions; means for generating story elements using a generative AI model; means for generating prompt sentences and inputting them into the generative AI model; and means for displaying the generated story elements to the user.

The server receives information about the story's theme and characters entered by the user and generates a prompt based on that information. The generated prompt is then input into a generative AI model (e.g., GPT-3) to generate story elements. The generated story elements are then displayed to the user.

Specifically, the server operates in the following steps: First, the user inputs information about the story's theme and characters. Next, the server generates a prompt based on this information. The generated prompt is input into a generative AI model, which generates story elements. The generated story elements are then displayed to the user.

The hardware used includes servers and user devices. The software used includes generative AI models (e.g., GPT-3) and prompt generation algorithms.

As a concrete example, the following prompt sentence is generated:

Theme: Knight's Adventure

Characters: Brave knight, mysterious old man, wizard

Generate plot twists, mystery elements, and moving scenes to make your story more interesting.

By inputting this prompt into a generative AI model, story elements are automatically generated. The generated story elements are displayed to the user, and the dramatic effect can be enhanced with story structure, character dialogue, visuals, and music.

The flow of the specific processing in Application Example 2 will be explained using Figure 14.

Step 1:

The user enters information about the story's theme and characters.

Input: Story theme, character information

Output: Input theme and character information

Specific actions: The user enters the story's theme (e.g., "The Knight's Adventure") and characters (e.g., "The Brave Knight, the Mysterious Old Man, and the Wizard") into an input form on a smartphone application.

Step 2:

The server receives the entered theme and character information and generates a prompt.

Input: Entered theme and character information

Output: Generated prompt text

Specific operation: Based on the received theme and character information, the server generates a prompt like the following: "Theme: A knight's adventure. Characters: A brave knight, a mysterious old man, a wizard. Please generate hints, mystery elements, and moving scenes to make the story more interesting."

Step 3:

The server inputs the generated prompt sentence into the generative AI model.

Input: Generated prompt text

Output: Story elements generated by the generative AI model

Specific operation: The server sends the generated prompt to a generative AI model such as GPT-3 to generate story elements (e.g., foreshadowing, mystery elements, emotional scenes).

Step 4:

The server receives the generated story elements and displays them to the user.

Input: Story elements generated by a generative AI model

Output: Story elements displayed to the user

Specific operation: The server sends the story elements received from the generative AI model to the user's smartphone application and displays them to the user. The user can then review the generated story elements.

Step 5:

The server automatically creates graphics and music that match each scene in the story.

Input: Generated story elements

Output: Graphics and music that match each scene

Specific operation: Based on the generated story elements, the server automatically creates appropriate graphics and music, taking into account the atmosphere of the scene and the emotional state of the characters.

Step 6:

The server enhances the dramatic effect through story structure, character dialogue, visuals, and music.

Input: Graphics, music, and story elements that match each scene

Output: A story with enhanced dramatic effects

Specific operation: The server combines the story structure, character dialogue, images, and music to enhance the dramatic effect and effectively stir the viewer's emotions.

(Example 3)

Next, we will explain Example 3 of Form Example 3. In the following explanation, the data processing device 12 will be referred to as the "server" and the headset-type terminal 314 will be referred to as the "terminal."

Previous story generation systems struggled to automatically create graphics and music that matched each scene in the story, preventing them from effectively evoking the viewer's emotions. Furthermore, generating graphics and music that take into account the scene's atmosphere and the characters' emotional states was a time-consuming and labor-intensive process.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Example 3 is realized by the following means.

In this invention, the server includes: means for inputting story settings such as the story theme, setting, and characters; means for automatically creating character dialogue, scenery, and story structure based on the input story settings; means for automatically generating foreshadowing, mystery elements, and moving scenes to make the created story more interesting; means for automatically creating graphics and music that match each scene of the created story; means for analyzing scene data and extracting the atmosphere of the scene and the emotional state of the characters; means for generating prompt text to be input into a generative AI model based on the extracted information; means for inputting the prompt text into the generative AI model and generating graphics and music that match the scene; means for integrating the generated graphics and music with the scene data and generating final output; and means for enhancing the dramatic effect with the story structure, character dialogue, images, and music to effectively stir the viewer's emotions. This makes it possible to automatically create graphics and music that match each scene of the story, enhancing the dramatic effect that effectively stirs the viewer's emotions.

"Story settings such as the story's theme, setting, and characters" are the basic elements of a story, and refer to the background of the story, the characteristics of the characters, and the settings necessary for the story's progression.

"Means of input" refers to the interface that allows the user to provide the story setting, such as the story's theme, setting, and characters, to the system.

"Means for automatically creating character dialogue, scenes, and story structure" refers to a function that automatically generates character dialogue, scene descriptions, and the overall flow of a story based on the input story setting.

"Means for automatically generating foreshadowing, mystery elements, moving scenes, etc." refers to a function that automatically adds elements that cause surprises or emotions to a pre-set story in order to make the story more interesting.

"Means for automatically creating graphics and music" refers to a function that automatically generates visual and auditory elements to match each scene in a story.

"Means for analyzing scene data" refers to the function of analyzing the content of each scene in a story and extracting the atmosphere of the scene and the emotional state of the characters.

"Means for generating prompt sentences to be input to a generative AI model" refers to a function that creates text to provide appropriate instructions to a generative AI model based on analyzed scene data.

"A means of inputting a prompt sentence into a generative AI model and generating graphics and music that match the scene" refers to a function that inputs a prompt sentence into a generative AI model and, as a result, generates graphics and music that are appropriate for the scene.

"Means for integrating generated graphics and music into scene data and generating final output" refers to the functionality for incorporating generated visual and auditory elements into scene data to create the final narrative output.

"Means to effectively stir the viewer's emotions" refers to the function of having a strong impact on the viewer's emotions through the story structure, the characters' dialogue, images, and music.

This invention relates to a system that automatically creates graphics and music that match each scene in a story. This system inputs story settings such as the story's theme, setting, and characters, and automatically creates character dialogue, scenery, and story structure based on the input story settings. It also automatically generates foreshadowing, mystery elements, and moving scenes to make the story more interesting, and automatically creates graphics and music that match each scene.

Hardware and software used

This system uses the following hardware and software:

Server: Analyzes data and runs generative AI models.

Terminal: Provides an interface for users to enter the story setting for their story.

Generative AI models: For example, OpenAI's GPT-4, DALL-E, and music generation models are used.

Data processing and calculation

The server receives the story settings entered by the user via their device. The story settings include the story's theme, setting, and character traits. The server analyzes this data and automatically creates the characters' lines, scenery, and story structure.

Next, the server automatically generates foreshadowing, mystery elements, moving scenes, and more based on the story you've created. This makes the story more interesting and emotionally stirring for the viewer.

Furthermore, the server analyzes scene data to automatically create graphics and music that match each scene in the story. As a result of the analysis, it extracts the atmosphere of the scene and the emotional state of the characters. Based on the extracted information, the server generates prompt sentences to input into the generative AI model.

Specific examples

For example, in the case of an emotional scene, the server inputs the following prompt sentence into the generative AI model:

Example prompt for graphic generation: "Generate a graphic of a beautiful landscape. The scene is emotional, and the character is in tears."

Example music generation prompt: "Generate moving, poignant music. The scene is emotional, and the characters are in tears."

The generative AI model generates graphics and music based on these prompts. The server integrates the generated graphics and music with the scene data to generate the final output. The device displays the integrated results to the user.

In this way, the user can automatically create graphics and music that match each scene in the story, enhancing the dramatic effect that effectively stirs the viewer's emotions. The flow of the specific processing in Example 3 will be explained using Figure 15.

Step 1:

The user enters story settings such as the story's theme, setting, and characters into the device.

Specific operation: The user enters information such as the story's theme (e.g., fantasy), setting (e.g., medieval castle), and characters (e.g., hero, wizard) into the device's input form. The device then sends the entered story settings to the server.

Input: Story setting, including theme, setting, and characters.

Output: Story setting data sent to the server

Step 2:

The server analyzes the received story setting data and automatically creates the characters' lines, scenes, and story structure.

Specific operations: The server uses natural language processing (NLP) technology to analyze story setting data and generate character dialogue, scene descriptions, and the overall story flow.

Input: Story setting data

Output: Character dialogue, scenery, and story structure data

Step 3:

Based on the story you create, the server automatically generates foreshadowing, mystery elements, moving scenes, and more.

Specific operation: The server analyzes the story's composition data and adds hints, mystery elements, and moving scenes to make the story more interesting.

Input: Story composition data

Output: Story data including foreshadowing, mystery elements, and moving scenes

Step 4:

The server analyzes scene data to automatically create graphics and music that match each scene in the story.

Specific operation: The server divides the story data into scenes and extracts the atmosphere of each scene and the emotional state of the characters.

Input: Story data

Output: Scene-specific mood and character emotional state data

Step 5:

The server generates prompt sentences to input into the generative AI model based on the extracted information.

Specific operation: The server creates a prompt that corresponds to the atmosphere of the scene and the emotional state of the characters. For example, in the case of an emotional scene, the server generates the prompt, "Generate a graphic of a beautiful landscape. The scene is emotional, and the characters are crying."

Input: Scene atmosphere and character emotional state data

Output: Prompt text to be input to the generative AI model

Step 6:

The server inputs prompt text into the generative AI model, generating graphics and music that match the scene.

Specific operation: The server sends a prompt to the generative AI model (e.g., DALL-E), which generates graphics that match the scene. Similarly, it sends a prompt to the music generation model, which generates music that matches the scene.

Input: Prompt text to be input to the generative AI model

Output: Generated graphics and music

Step 7:

The server integrates the generated graphics and music with the scene data to generate the final output.

Specific operation: The server integrates the generated graphics and music with each scene data to create the final output of the entire story.

Input: Generated graphics, music, and scene data

Output: Final integrated narrative data

Step 8:

The terminal displays the final, integrated story data to the user.

Specific operation: The terminal receives the final story data sent from the server and displays it visually and audibly to the user.

Input: Final integrated narrative data

Output: The visual and auditory content of the story presented to the user.

(Application Example 3)

Next, we will explain Application Example 3 of Form Example 3. In the following explanation, the data processing device 12 will be referred to as the "server" and the headset-type terminal 314 will be referred to as the "terminal."

Previous story generation systems struggled to automatically create graphics and music that matched each scene in the story, limiting the visual and auditory effects they could produce. Furthermore, even when users input their own stories, there was a lack of means to generate graphics and music appropriate for that story, resulting in a decline in content quality. Furthermore, the lack of functionality to preview and export the generated content resulted in poor user convenience.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 3 is realized by the following means.

In this invention, the server includes means for inputting story settings such as the story theme, setting, and characters; means for automatically creating character dialogue, scenery, and story structure based on the input story settings; means for automatically generating foreshadowing, mystery elements, and moving scenes to make the created story more interesting; means for automatically creating graphics and music that match each scene of the created story; means for enhancing the dramatic effect with the story structure, character dialogue, video, and music, and effectively stirring the viewer's emotions; means for analyzing story scenes and extracting the scene atmosphere and the emotional state of the characters; means for generating graphics based on scene descriptions; means for generating music based on scene descriptions; means for previewing the generated content; and means for exporting the completed content. This makes it possible to automatically generate, preview, and export visually and aurally rich content based on a story entered by the user.

"Story theme" refers to the central subject or message of a story.

"Setting" refers to the place or environment in which the story unfolds.

"Characters" refers to the characters that appear in the story.

"Story setting" refers to basic elements such as the story's theme, setting, and characters.

"Dialogue" refers to the words spoken by characters.

"Scene" refers to the scenery or scene depicted in the story.

"Story structure" refers to the overall flow of a story, including its beginning, middle, and end.

"Foreshadowing" refers to elements that foreshadow information or events that will become important later in the story.

"Mystery elements" refer to elements that add mystery or suspense to a story.

A "moving scene" refers to a scene that strongly stirs the viewer's emotions.

"Graphics" refers to visual images and illustrations generated to match story scenes.

"Song" refers to music generated to match the story scenes.

'Scene atmosphere' refers to the overall feeling or mood of a particular scene in a story.

"A character's emotional state" refers to the emotions the character is feeling in the story.

A "scene description" refers to a detailed description or explanation of a particular moment in a story.

"Preview" refers to checking the generated content in advance.

"Export" refers to saving the generated content as an external file.

A system for implementing this invention includes means for inputting story settings such as the story theme, setting, and characters; means for automatically creating character dialogue, scenery, and story structure based on the input story settings; means for automatically generating foreshadowing, mystery elements, and moving scenes to make the created story more interesting; means for automatically creating graphics and music that match each scene of the created story; means for enhancing the story structure, character dialogue, and visuals and music to effectively stir the viewer's emotions; means for analyzing story scenes and extracting the atmosphere of the scene and the emotional state of the characters; means for generating graphics based on scene descriptions; means for generating music based on scene descriptions; means for previewing the generated content; and means for exporting the completed content.

Hardware and software configuration

Hardware:

Smartphone

Software:

Python

OpenAI API

PIL (Python Imaging Library)

Music generation library

Data processing and calculation

How to enter story settings such as theme, setting, and characters:

Users enter story settings such as the story's theme, setting, and characters in text format through a smartphone application.

Methods for automatically creating character dialogue, scenes, and story structure:

Based on the input story setting, the server uses a generative AI model (e.g., GPT-3) to automatically create character dialogue, scenes, and story structure.

Methods for automatically generating foreshadowing, mystery elements, moving scenes, etc.:

The server uses a generative AI model to automatically generate plot twists, mystery elements, and moving scenes to make the story more interesting.

Methods for automatically creating graphics and music that match each scene in a story:

The server analyzes each scene in the story and extracts the scene's atmosphere and the characters' emotional state. Based on this, it generates graphics using OpenAI's image generation API and music using OpenAI's music generation API.

How to preview generated content:

Users can preview each scene of the story, including generated graphics and music, through a smartphone application.

How to export your finished content:

Users can export the finished content as a video file through a smartphone application.

Specific examples

For example, if a user inputs the story "A hero climbed a mountain to fight a dragon," the system will analyze this scene and generate powerful graphics and music that heightens the tension to match the battle scene between the hero and the dragon.

Example prompt:

Analyze the following story scene and output the scene's atmosphere and the characters' emotional states:

The hero climbed the mountain to fight the dragon.

In this way, users can enjoy their stories as visually and aurally rich content.

The flow of the specific processing in Application Example 3 will be explained using Figure 16.

Step 1:

Users enter story settings such as the story's theme, setting, and characters in text format through a smartphone application.

Input: Story setting, including theme, setting, and characters.

Output: Story setting data in text format

Step 2:

Based on the input story setting, the server uses a generative AI model (e.g., GPT-3) to automatically create character dialogue, scenes, and story structure.

Input: Text-based story setting data

Output: Character dialogue, scenery, and story structure data

Specific operation: The server sends a prompt to the generative AI model and receives the generated text.

Step 3:

The server uses a generative AI model to automatically generate plot twists, mystery elements, and moving scenes to make the story more interesting.

Input: Character dialogue, scenery, and story structure data

Output: Foreshadowing, mystery elements, and moving scene data

Specific operation: The server sends a prompt to the generative AI model and receives the generated text.

Step 4:

The server analyzes each scene in the story and extracts the scene's atmosphere and the emotional state of the characters.

Input: Foreshadowing, mystery elements, and moving scene data

Output: Scene atmosphere, character emotional state data

Specific operation: The server sends a prompt to the generative AI model and receives the analysis results.

Step 5:

The server generates graphics based on the scene description.

Input: Scene atmosphere, character emotional state data

Output: Graphics data

Specific operation: The server sends a prompt to OpenAI's image generation API and receives the generated image.

Step 6:

The server generates music based on the scene description.

Input: Scene atmosphere, character emotional state data

Output: Music data

Specific operation: The server sends a prompt to OpenAI's music generation API and receives the generated music.

Step 7:

Users can preview each scene of the story, including generated graphics and music, through a smartphone application.

Input: Graphics data, music data

Output: Preview screen

Specific behavior: The smartphone application displays the generated data and provides a preview to the user.

Step 8:

Users can export the finished content as a video file through a smartphone application.

Input: Preview screen

Output: Video file

Specific operation: The smartphone application saves the generated data as a video file and provides it to the user.

It is also possible to further combine an emotion engine that estimates the user's emotion. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

"Example 1"

One embodiment of the present invention is a story generation system that incorporates an emotion engine. This system receives story settings, such as the story's theme, setting, and characters, as input information from the user. Based on this story setting, the system then automatically creates character dialogue, scenery, and story structure. It also automatically generates foreshadowing, mystery elements, and moving scenes to make the story more interesting. It also automatically creates graphics and music that match each scene in the story. The story structure, character dialogue, video, and music enhance the dramatic effect, effectively stirring the viewer's emotions.

"Example 2"

Furthermore, the system includes an emotion engine. This emotion engine recognizes the user's emotions in real time and adjusts the story development according to those emotions. For example, if the user feels joyful about the story, the story will develop in a more cheerful manner. Conversely, if the user feels sad about the story, the story will develop in a more profound manner.

"Example 3"

The emotion engine uses information such as the user's facial expressions, tone of voice, and text input to recognize the user's emotions. For example, if the user is smiling, the emotion engine will recognize that the user is feeling happy. If the user speaks in a sad tone of voice, the emotion engine will recognize that the user is feeling sad. Furthermore, if the user inputs the text "fun," the emotion engine will recognize that the user is feeling happy. Based on this information, the emotion engine adjusts the development of the story and provides a story experience that matches the user's emotions.

The processing flow for each example form is explained below.

"Example 1"

Step 1: Receive story settings such as theme, setting, and characters as input from the user.

Step 2: Based on the received story setting, character dialogue, scenes, and story structure are automatically created.

Step 3: Automatically generate hints, mystery elements, moving scenes, and more to make your story more interesting.

Step 4: Automatically create graphics and music that match each scene in the story you've created. Step 5: Enhance the dramatic effect with the story structure, character dialogue, images, and music to effectively stir the viewer's emotions.

"Example 2"

Step 1: Launch the emotion engine and recognize the user's emotions in real time.

Step 2: Adjust the story development according to the recognized emotion. For example, if the user feels happy about the story, the story will develop in a more cheerful manner. Conversely, if the user feels sad about the story, the story will develop in a more profound manner.

"Example 3"

Step 1: The emotion engine uses information such as the user's facial expressions, tone of voice, and text input to recognize the user's emotions.

Step 2: For example, if the user is smiling, the emotion engine will recognize that the user is feeling happy. Also, if the user is speaking in a sad tone of voice, the emotion engine will recognize that the user is feeling sad.

Step 3: Furthermore, if the user enters the text "fun," the emotion engine recognizes that the user is feeling enjoyment. Based on this information, the emotion engine adjusts the development of the story and provides a narrative experience that matches the user's emotions.

(Example 1)

Next, Example 1 of Form Example 1 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset-type terminal 314 will be referred to as the "terminal."

In conventional story generation systems, when automatically generating a story based on a story setting entered by the user, the story flow, character dialogue, and scenery often lacked consistency. Furthermore, it was difficult to generate foreshadowing, mystery elements, and moving scenes to make the story more interesting, making it difficult to effectively move the viewer's emotions. Furthermore, it was difficult to automatically create graphics and music that matched each scene in the story. To address these issues, the present invention aims to provide a system that generates more consistent stories and effectively moves the viewer's emotions.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

In this invention, the server includes means for inputting story settings such as the story's theme, setting, and characters; means for analyzing the input story settings; means for searching an existing story database based on the analyzed story settings; means for generating and inputting prompt sentences to a generation AI model; means for automatically creating character dialogue, scenes, and story structure using the generation AI model; means for automatically generating foreshadowing, mystery elements, and moving scenes to make the created story more interesting; means for automatically creating graphics and music that match each scene of the created story; and means for enhancing the dramatic effect with the story structure, character dialogue, images, and music, and effectively stirring the viewer's emotions. This makes it possible to generate a consistent story based on the story settings input by the user, effectively stirring the viewer's emotions.

A "story theme" is the central subject or subject of a story.

"Setting" refers to the place and historical background in which the story unfolds.

"Characters" refers to the characters and people who appear in the story.

"Story setting" refers to the setting that includes basic elements such as the story's theme, setting, and characters.

"Input means" refers to the method or device by which the user provides the story setting, such as the story's theme, setting, and characters, to the system.

"Means for analyzing" refers to a method or device for analyzing the input story setting and extracting important keywords and phrases.

"Means for searching a database" refers to a method or device for searching relevant information from an existing narrative database based on the analyzed story setting.

A "generative AI model" is a model that uses artificial intelligence technology to generate text and stories.

A "prompt" is an instruction or question that is input into a generative AI model.

"Means for generating and inputting prompt sentences" refers to a method or device for creating prompt sentences suitable for a generative AI model and inputting them into the model.

"Dialogue" refers to the words or lines spoken by characters.

"Scene" refers to the scenery or scene depicted in the story.

"Narrative structure" refers to the structure and arrangement that determines the flow and development of a story.

"Foreshadowing" refers to elements or hints placed in the first half of a story that will have important meaning later on.

"Mystery elements" are elements that add mystery and suspense to a story.

A "moving scene" is one that strongly stirs the viewer's emotions.

"Graphics" refers to images and illustrations generated to match each scene in the story.

"Songs" refer to music and melodies generated to match each scene in the story.

"Directory effects" refer to effects that use story structure, character dialogue, images, and music to stir the viewer's emotions.

The present invention relates to a story generation system that generates a coherent story based on a story setting entered by the user, and aims to effectively stir the emotions of the viewer.

System Configuration

User input

The user enters the story's theme, setting, characters, and other story settings in text format. For example, they can provide a specific setting such as "a knight's adventure story set in medieval Europe."

Server input analysis

The server analyzes the story setting received from the user. This analysis uses natural language processing techniques, specifically Python's NLTK library and SpaCy. The server tokenizes the input text and extracts important keywords and phrases.

Search the database

The server searches an existing story database based on the extracted keywords and phrases. This database stores information on past stories and characters. The server finds similar stories and character settings and uses them as reference information.

Prompt generation for generative AI models

Based on user input and information obtained from the database, the server generates prompts to be input into the generative AI model. For example, it creates a prompt such as, "Create a knight's adventure story set in medieval Europe. The main character is the brave knight Arthur, whose mission is to defeat an evil dragon and save the kingdom. Please include the opening of the story, lines from the characters, and descriptions of the scenery."

Story generation using generative AI models

The server inputs prompt text into a generative AI model (for example, OpenAI's GPT-4), which generates the story flow, character dialogue, and scenery. Based on the user's input, the generative AI model generates text that matches the story's theme and the characters' personalities.

Applying the Emotion Engine

The server applies an emotion engine to the generated story, adding emotional elements to each scene in the story. For example, it automatically generates foreshadowing, mystery elements, and moving scenes. This emotion engine uses IBM Watson's Tone Analyzer.

Graphics and music generation

The server automatically creates graphics and music that match each scene in the story. Image generation AI is used to generate the graphics, and music generation AI is used to generate the music.

Merge the final output

The server integrates the generated story, graphics, and music and provides them to the user. The user can then view and enjoy the completed story. For example, the user can view the story through a web browser and enjoy it along with the graphics and music.

Specific examples and prompt sentence examples

Specific examples

If the user inputs the setting "A knight's adventure story set in medieval Europe," the server will generate a story like this:

The story begins: "In the vast plains of medieval Europe stood the brave knight Arthur. His mission was to slay the evil dragon and save the kingdom."

Character Quote: "Arthur: 'I swear on this sword that I will defeat the dragon!'"

Scene description: "Under the sunset, Arthur gripped his sword and renewed his resolve."

Example prompt

An example of a prompt to input to the generative AI model is as follows:

"Create a knight's adventure story set in medieval Europe. The main character is a brave knight named Arthur, whose mission is to defeat an evil dragon and save the kingdom. Please include an opening line, character dialogue, and a description of the scenery."

By inputting this prompt into a generative AI model, each element of the story is automatically generated.

The flow of the identification process in Example 1 will be explained using Figure 17.

Step 1:

The user enters the story's theme, setting, characters, and other story settings in text format. For example, they might enter "A knight's adventure story set in medieval Europe." This input is done through the system's user interface. The entered text is sent to the server.

Step 2:

The server analyzes the story setting received from the user. Specifically, it uses Python's NLTK library and SpaCy to tokenize the text and extract important keywords and phrases. For example, it extracts keywords such as "medieval Europe," "knights," and "adventure story." The results of this analysis are used in the next step.

Step 3:

The server searches an existing story database based on the extracted keywords and phrases. The database contains information on past stories and characters. The server finds similar stories and character settings and uses them as reference information. For example, it searches for past stories with a "medieval Europe" theme or stories with a "knight" as the main character. The search results are used in the next step.

Step 4:

Based on user input and information obtained from the database, the server generates a prompt to be input into the generative AI model. For example, it might create a prompt such as, "Create a knight's adventure story set in medieval Europe. The main character is the brave knight Arthur, whose mission is to defeat an evil dragon and save the kingdom. Please include the opening of the story, lines from the characters, and descriptions of the scenery." This prompt will be used in the next step.

Step 5:

The server inputs prompt text into a generative AI model (for example, OpenAI's GPT-4), which generates the story flow, character dialogue, and scenery. Based on the user's input, the generative AI model generates text that matches the story's theme and the characters' personalities. For example, the generative AI model generates the following story:

The story begins: "In the vast plains of medieval Europe stood the brave knight Arthur. His mission was to slay the evil dragon and save the kingdom."

Character Quote: "Arthur: 'I swear on this sword that I will defeat the dragon!'"

Scene description: "Under the sunset, Arthur gripped his sword and renewed his resolve."

Step 6:

The server applies an emotion engine to the generated story, adding emotional elements to each scene in the story. For example, it automatically generates foreshadowing, mystery elements, and moving scenes. This emotion engine uses IBM Watson's Tone Analyzer. The emotion engine analyzes the generated text and adds appropriate emotional elements.

Step 7:

The server automatically creates graphics and music that match each scene in the story. Image generation AI is used to generate the graphics, and music generation AI is used to generate the music. For example, to match the opening scene of the story, it generates graphics depicting the scenery of medieval Europe and music that evokes a sense of adventure.

Step 8:

The server integrates the generated story, graphics, and music and provides them to the user. The user can then view and enjoy the completed story. For example, the user can view the story through a web browser and enjoy it along with the graphics and music.

(Application Example 1)

Next, we will explain Application Example 1 of Form Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the headset-type terminal 314 will be referred to as the "terminal."

Conventional story generation systems could generate stories based on settings entered by the user, but they were unable to visualize and sound out the generated stories in real time and allow users to experience them in a virtual reality space. It was also difficult to automatically generate graphics and music that matched each scene in the story, enhancing the dramatic effects that effectively stirred the viewer's emotions. This limited the user experience and led to a lack of immersion in the story.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

In this invention, the server includes: means for inputting story settings such as the story theme, setting, and characters; means for automatically creating character dialogue, scenery, and story structure based on the input story settings; means for automatically generating foreshadowing, mystery elements, and moving scenes to make the created story more interesting; means for automatically producing graphics and music that match each scene of the created story; means for enhancing the dramatic effect with the story structure, character dialogue, images, and music, and effectively stirring the viewer's emotions; means for generating a story using a generative AI model based on the story settings input by the user; means for visualizing and audibly converting the generated story in real time; and means for the user to experience the generated story in a virtual reality space using a head-mounted display. This allows the user to visualize and audibly convert the generated story in real time and experience it in a virtual reality space.

A "story theme" is the central subject or theme of a story, and determines the overall direction and atmosphere of the story.

The "setting" refers to the place and historical background in which the story unfolds, and is an element that shapes the setting and atmosphere of the story.

"Characters" are characters who act in a story and play an important role in the progression and development of the story.

"Story setting" refers to the setting that includes basic elements such as the story's theme, setting, and characters, and forms the foundation of the story.

A "generative AI model" is a model that uses artificial intelligence technology to generate text and data, and is used to automatically create story structures and dialogue.

"Foreshadowing" refers to elements or hints in a story that give a glimpse of future developments, and is a technique used to make a story more interesting.

"Mystery elements" are elements in a story that present a mystery or question, and are intended to attract the interest of readers or viewers.

A "moving scene" is a scene in a story that stirs strong emotions and leaves a deep impression on the viewer's mind.

"Graphics" refers to visual images and footage that visually represent scenes and characters in a story.

"Song" refers to music, used to create scenes and atmosphere in a story.

"Visualization" refers to the visual representation of the story content, allowing users to enjoy the story visually.

"Audio" refers to the expression of the story content in audio, allowing users to enjoy the story by listening to it.

A "virtual reality space" is a virtual space created using computer technology that allows users to experience reality.

A "head-mounted display" is a display device worn on the head, used to allow users to experience virtual reality spaces.

A system for implementing this invention has the following configuration: First, a user uses a smartphone or head-mounted display to input story settings such as the story's theme, setting, and characters. This input is done in text format.

Next, the server uses a generative AI model (e.g., GPT-4) to automatically create character dialogue, scenes, and a story structure based on the story setting entered by the user. The generative AI model references an existing story database to generate dialogue and scenes that match the story flow, character personalities, and story theme.

In addition, the server uses an emotion engine to add hints, mystery elements, and moving scenes to the stories created, making them more interesting and moving.

The server then uses a graphics generation engine and a music generation engine to automatically create graphics and music that match each scene in the story. This enhances the visual and auditory effects of the story, effectively stirring the emotions of the viewer.

Finally, the generated story is visualized and audible in real time, and the user can experience the story in a virtual reality space using a head-mounted display. This allows the user to become immersed in the story and enjoy a deeper experience.

As a concrete example, if a user inputs "a knight's adventure story set in medieval Europe," the following prompt sentence will be input to the generative AI model:

Example prompt:

Theme: Medieval Europe

Setting: Knight's Adventure

Characters: Brave knight, wizard, dragon

Story setting: A brave knight and a wizard embark on an adventure to slay a dragon.

Based on this prompt, a generative AI model generates a story, an emotion engine adds emotional elements, and a graphics generation engine and music generation engine generate graphics and music. Users can experience this story in real time through a head-mounted display.

The flow of the specific processing in Application Example 1 is explained using Figure 18.

Step 1:

Users use a smartphone or head-mounted display to input story settings such as the story's theme, setting, and characters in text format. The input data is sent to the server.

Input: Story setting, including theme, setting, and characters.

Output: Story setting data sent to the server

Step 2:

Based on the received story setting data, the server uses a generative AI model (e.g., GPT-4) to automatically create the story structure, character dialogue, and scenery. The generative AI model references an existing story database to generate text that matches the story flow and character personalities.

Input: Story setting data

Output: Generated story structure, dialogue, and scenery

Step 3:

The server uses an emotion engine to add emotional elements such as foreshadowing, mystery elements, and moving scenes to the generated story. The emotion engine automatically inserts emotional elements appropriate to each scene in the story.

Input: Generated story structure, dialogue, and scenery

Output: A story with added emotional content

Step 4:

The server uses a graphics generation engine to automatically create graphics that match each scene in the story. The graphics generation engine takes into account the atmosphere of the scene and the emotional state of the characters to generate appropriate visuals.

Input: A story with an emotional element

Output: Graphics that match each scene

Step 5:

The server uses a music generation engine to automatically create music that matches each scene in the story. The music generation engine takes into account the atmosphere of the scene and the emotional state of the characters to generate appropriate music.

Input: A story with an emotional element

Output: Songs that match each scene

Step 6:

The server integrates the generated story, graphics, and music, visualizing and audiovisualizing them in real time, allowing users to experience the story visually and aurally.

Input: Generated story, graphics, music

Output: Real-time visual and audio narrative

Step 7:

Using a head-mounted display, users experience a story generated in a virtual reality space, immersing themselves in the story and providing a deeper experience.

Input: Real-time visual and audio narrative

Output: Narrative experience in a virtual reality space

(Example 2)

Next, Example 2 of Form Example 2 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset-type terminal 314 will be referred to as the "terminal."

While conventional story generation systems can automatically generate a story based on the story's theme and character settings, they have difficulty automatically generating foreshadowing, mystery elements, and moving scenes that make the story more interesting. Furthermore, they lack the functionality to adjust the story's development in real time based on the user's emotions, making it impossible to provide a narrative experience that is in tune with the user's emotions. Furthermore, it is difficult to automatically create graphics and music that match each scene in the story. A new system is needed to solve these issues.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

In this invention, the server includes means for inputting story settings such as the story theme, setting, and characters; means for automatically creating character dialogue, scenery, and story structure based on the input story settings; means for automatically generating foreshadowing, mystery elements, and moving scenes to make the created story more interesting; means for recognizing the user's emotions in real time and adjusting the story development in accordance with those emotions; means for automatically creating graphics and music that match each scene of the created story; and means for enhancing the dramatic effect with the story structure, character dialogue, images, and music to effectively stir the viewer's emotions. This makes it possible not only to automatically generate a story based on the story theme and character settings, but also to automatically generate foreshadowing, mystery elements, and moving scenes, and provide a story development that matches the user's emotions.

The "story theme" indicates the overall subject or direction of the story.

"Setting" refers to the place or environment in which the story unfolds.

"Characters" refers to the characters who take action in the story.

"Story setting" refers to the setting, including basic elements such as the story's theme, setting, and characters.

"Dialogue" refers to the words or lines spoken by characters.

"Scene" refers to the scenery or scene depicted in the story.

"Story structure" refers to the overall flow and structure of a story, including its beginning, middle, and end.

"Foreshadowing" refers to elements that are laid out in advance in a way that will have important meaning later in the story.

"Mystery elements" refer to elements that attract the interest of readers or viewers by including mysteries or unknown points in a story.

A "moving scene" is one that strongly stirs the emotions of the reader or viewer.

"User emotions" refers to feelings such as joy, sadness, and surprise felt by users as they experience the story.

"Real-time recognition" refers to detecting the user's emotions instantly and on the spot.

"Adjusting the story development" refers to changing the progress and content of the story based on the user's emotions.

"Graphics" refers to images and illustrations that are displayed in accordance with story scenes.

"Song" refers to the music that plays along with the story scenes.

"Directory effects" refer to the effects of using the visual and auditory elements of a story to stimulate the emotions and interest of readers or viewers.

This invention is a system that automatically generates a story based on the story's theme and character settings, and also automatically generates foreshadowing, mystery elements, and moving scenes. It also has the ability to recognize the user's emotions in real time and adjust the story's development accordingly.

Hardware and software used

The server generates story elements using a generative AI model (e.g., GPT-4). Based on information received from the user about the story's theme, the characters' personalities, and the story's progression, the server inputs prompt sentences into the generative AI model to generate story elements.

The device sends the user's input information to the server and displays the story elements received from the server to the user. The device also uses emotion recognition software to collect emotional data from the user's facial expressions and voice and send it to the server.

Users enter information about the story's theme, the characters' personalities, and the story's progression into their device and enjoy the story. The device recognizes the user's emotions in real time and reflects them in the development of the story.

Data processing and calculation

The server performs the following data processing and calculations:

Generate appropriate foreshadowing based on the story's theme and the characters' personalities.

Mystery elements are scattered throughout the story as it progresses.

Analyze the characters' emotions and situations to create moving scenes.

The device collects emotional data from the user's facial expressions and voice and sends it to the server. The server analyzes this emotional data and adjusts the story's development.

Specific examples

For example, to generate a hint that a person the knight meets during his adventure will play an important role later in the story, the following prompt sentence can be input into the generative AI model:

Example prompt:

"Please create a plot twist that will lead to the mysterious old man the knight meets during his adventure playing an important role later in the story."

Also, if the user is feeling happy about the story, a prompt to make the story even more upbeat might look like this:

Example prompt:

"Users are enjoying this, so please make the story even more upbeat. For example, add a scene where the knight finds a new companion."

In this way, the server utilizes a generative AI model and emotion engine to automatically generate a story development that corresponds to the user's emotions.

The flow of the identification process in Example 2 will be explained using Figure 19.

Step 1:

The user enters information about the story's theme, the characters' personalities, and the story's progression.

Input: Story theme (e.g., adventure), character traits (e.g., bravery), and information about the story's progression.

Output: User input information

Specific operation: The user enters the theme "adventure" into the device's input form and sets the character's personality to "brave."

Step 2:

The device sends the user's input information to the server.

Input: User input information

Output: User input information sent to the server

Specific operation: The device sends the information "Theme: Adventure, Personality: Brave" to the server.

Step 3:

The server generates story elements using generative AI models.

Input: User input information

Output: Generated story elements

Specific operation: The server inputs the prompt "Generate a story with an adventure theme and a brave knight as the protagonist" into the generation AI model, and generates the elements of the story.

Step 4:

The server sends the generated story elements to the device.

Input: Generated story elements

Output: Story elements sent to the device

Specific operation: The server sends story elements generated by the server (e.g., "The mysterious old man the knight meets during his adventure") to the device.

Step 5:

The device displays the story to the user.

Input: Story elements received from the server

Output: The story displayed to the user

Specific operation: The device displays a scene to the user in which a knight meets a mysterious old man during his adventure.

Step 6:

Recognize user emotions in real time.

Input: User's facial expressions and voice

Output: Recognized user emotion data

Specific operation: When the user smiles, the device captures the user's facial expression with its camera and recognizes that the user is feeling happy.

Step 7:

The device sends the user's emotional data to the server.

Input: Recognized user emotion data

Output: Emotion data sent to the server

Specific operation: The device sends emotional data indicating "the user is feeling happy" to the server.

Step 8:

The server analyzes the emotional data and adjusts the story development.

Input: User emotion data

Output: Coordinated story development

Specific operation: The server analyzes the emotional data and inputs prompt sentences into the generative AI model to "make the story more upbeat."

Step 9:

The server will send the adjusted story to your device.

Input: Adjusted story development

Output: The adjusted story sent to the terminal.

Specific operation: The server generates a story with an upbeat plot (e.g., "A scene where the knight finds a new companion") and sends it to the device.

Step 10:

The device will then display the adjusted story to the user.

Input: Adjusted story received from the server

Output: The adjusted story displayed to the user.

Specific operation: The device displays a scene to the user in which a knight finds a new companion.

(Application Example 2)

Next, we will explain Application Example 2 of Form Example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the headset-type terminal 314 will be referred to as the "terminal."

While conventional story generation systems could automatically generate a story based on the story's theme and character settings, they were unable to adjust the story's development in real time in response to the user's emotions. They also lacked the functionality to automatically generate plot twists, mystery elements, and moving scenes to make a story more interesting. This made it difficult to provide an interactive story experience that responded to the user's emotions.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

In this invention, the server includes: means for inputting story settings such as the story theme, setting, and characters; means for automatically creating character dialogue, scenery, and story structure based on the input story settings; means for automatically generating foreshadowing, mystery elements, and moving scenes to make the created story more interesting; means for automatically creating graphics and music that match each scene of the created story; means for enhancing the dramatic effect with the story structure, character dialogue, images, and music, and effectively stirring the viewer's emotions; means for recognizing the user's emotions in real time and adjusting the story development in response to those emotions; and means for generating story prompts using a generative AI model and automatically generating a story. This makes it possible to provide an interactive story experience that responds to the user's emotions.

"Story theme" refers to the central subject or message of a story.

"Setting" refers to the place or environment in which the story unfolds.

"Characters" refers to the characters or people who play a role in the story.

"Story setting" refers to the setting, including basic elements such as the story's theme, setting, and characters.

"Dialogue" refers to the words or lines spoken by characters.

"Scene" refers to the scenery or scene depicted in the story.

"Story structure" refers to the overall flow and structure of a story, including its beginning, middle, and end.

"Foreshadowing" refers to elements or hints placed in the first half of a story that will have important meaning later on.

"Mystery elements" refer to elements in a story that raise questions or raise doubts.

A "moving scene" is one that evokes strong emotions in the viewer or reader.

"Graphics" refers to images and illustrations that are displayed in accordance with story scenes.

"Song" refers to the music or soundtrack that plays along with the story scenes.

"Stage effects" refer to visual and auditory techniques used to make story scenes appear more effective.

"Viewer sentiment" refers to the emotions and feelings of the people watching the story.

"Recognizing emotions in real time" refers to analyzing the user's facial expressions, tone of voice, etc. to grasp their emotions at that moment.

"Adjusting the development of the story" refers to changing the progress and content of the story in response to the user's emotions.

"Generative AI model" refers to a model that uses artificial intelligence to generate text or stories.

A "prompt" refers to an instruction or keyword that is input into a generative AI model.

A system for implementing this invention has the following configuration: First, the server provides a means for inputting story settings such as the story's theme, setting, and characters. By inputting these settings, the user determines the basic elements of the story.

Next, the server has a means to automatically create character dialogue, scenery, and story structure based on the input story setting. This means uses a generative AI model to generate story prompts and automatically generate the story.

Furthermore, the server has the means to automatically generate hints, mystery elements, moving scenes, and more to make the stories you create more interesting. This helps to add depth and interest to the story.

The server also has the means to automatically create graphics and music that match each scene in the story. This enhances the visual and auditory effects of the story. Specifically, the graphics and music are created taking into account the atmosphere of the scene and the emotional state of the characters.

Furthermore, the server has the means to enhance the dramatic effect through the story structure, character dialogue, visuals, and music, effectively stirring the viewer's emotions. This allows users to be more emotionally drawn into the story.

What's particularly important is that the server has a means of recognizing the user's emotions in real time and adjusting the story development accordingly. It uses the smartphone's camera and microphone to analyze the user's facial expressions and tone of voice to understand their emotions. Based on this emotional data, it generates prompt sentences for the story and adjusts the story development accordingly.

For example, if a user is reading a story and the camera captures their facial expressions and recognizes that they are smiling, the story will develop in a lighthearted and cheerful way. Conversely, if the user is sad, the story will develop in a more profound way.

Examples of prompt statements include:

"The protagonist continues his adventure with a happy feeling."

"The protagonist faces difficulties, but does not lose hope."

"The protagonist takes on a new mystery."

In this way, it is possible to provide an interactive narrative experience that responds to the user's emotions.

The flow of the specific processing in Application Example 2 is explained using Figure 20.

Step 1:

The user enters story settings such as theme, setting, and characters.

Input: Setting information such as the story's theme, setting, and characters entered by the user.

Output: Story setting data.

Specific operation: The user uses a smartphone application to input the basic setting of the story, including information about the story's theme, location, and main characters. The server receives this setting information and stores it as story setting data.

Step 2:

The server automatically creates character dialogue, scenes, and story structure based on the story settings entered.

Input: Story setting data.

Output: Story dialogue, scenery, and composition data.

Specific operation: The server uses the generative AI model to generate character dialogue, scenery, and story structure based on story setting data. The generated data is saved as elements necessary for the story's progression.

Step 3:

The server automatically generates hints, mystery elements, moving scenes, and more to make the stories it creates more interesting.

Input: Story lines, scenery, and composition data.

Output: Foreshadowing, mystery elements, and moving scene data.

Specifically, how it works: The server uses a generative AI model to generate plot twists, mystery elements, and moving scenes based on the dialogue, scenery, and composition data of the story. This adds depth and intrigue to the story.

Step 4:

The server automatically creates graphics and music that match each scene in the story.

Input: Foreshadowing, mystery elements, and moving scene data.

Output: Graphics data, music data.

Specific operation: The server automatically creates graphics and music, taking into account the atmosphere of the scene and the emotional state of the characters. This is done using image generation models and music generation models. The generated graphics and music are then applied to each scene in the story.

Step 5:

The server recognizes the user's emotions in real time and adjusts the story development accordingly.

Input: User's facial expression data, tone of voice data.

Output: Tailored narrative prompt.

Specific operation: The smartphone's camera and microphone are used to capture the user's facial expressions and tone of voice. The server analyzes this data and recognizes the user's emotions in real time. Based on the recognized emotions, it generates story prompts and adjusts the story's development.

Step 6:

The server uses a generative AI model to generate story prompts and automatically generate the story.

Input: Tailored story prompt.

Output: The generated story.

Specific operation: The server uses a generative AI model to generate a story based on the adjusted prompt. The generated story is used to provide an interactive experience that responds to the user's emotions.

(Example 3)

Next, we will explain Example 3 of Form Example 3. In the following explanation, the data processing device 12 will be referred to as the "server" and the headset-type terminal 314 will be referred to as the "terminal."

With conventional story generation systems, it was difficult to automatically create graphics and music that matched each scene in the story, making it difficult to effectively sway the viewer's emotions. Furthermore, they lacked the ability to recognize the user's emotions in real time and adjust the story's development accordingly, making it difficult to provide the user with a consistent emotional experience. This resulted in a decline in the quality of the story experience and low user satisfaction.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Example 3 is realized by the following means.

In this invention, the server includes means for inputting story settings such as the story theme, setting, and characters; means for automatically creating character dialogue, scenery, and story structure based on the input story settings; means for automatically generating foreshadowing, mystery elements, and moving scenes to make the created story more interesting; means for automatically creating graphics and music that match each scene of the created story; means for using information such as the user's facial expressions, tone of voice, and text input to recognize the user's emotions; means for adjusting the story development in accordance with the user's emotions; and means for enhancing the dramatic effects of the story structure, character dialogue, images, and music to effectively stir the viewer's emotions. This makes it possible to automatically create graphics and music that match each scene of the story and provide a story development that matches the user's emotions.

"Story theme" refers to the central subject or subject of a story.

"Setting" refers to the place or environment in which the story unfolds.

"Characters" refers to the characters that appear in the story.

"Story setting" refers to the setting, including basic elements such as the story's theme, setting, and characters.

"Dialogue" refers to the words spoken by characters.

"Scene" refers to the scenery or scene depicted in the story.

"Story structure" refers to the overall flow of a story, including its beginning, middle, and end.

"Foreshadowing" refers to elements that provide advance hints at information or events that will become important later in the story.

"Mystery elements" refer to elements that add mystery or suspense to a story.

A "moving scene" refers to a scene that strongly stirs the viewer's emotions.

"Graphics" refers to images and illustrations generated to match story scenes.

"Song" refers to music generated to match the story scenes.

"User emotion" refers to the emotional state the user is feeling, such as joy, sadness, or surprise.

"Facial expression" refers to emotions expressed through the movement of the user's facial muscles.

"Tone of voice" refers to the pitch, strength, and intonation of the user's voice.

"Text input" refers to text information entered by a user using a keyboard or touch panel.

"Story development" refers to the process and flow of the story.

"Directory effects" refer to the impact that the visual and auditory elements of a story have on the viewer.

"Viewer sentiment" refers to the emotions and feelings of the people watching the story.

This invention is a system that inputs story settings such as the story's theme, setting, and characters, automatically generates a story based on the input story settings, and further adjusts the story's development according to the user's emotions. This system is implemented using the following hardware and software.

Hardware and software used

Hardware: High-performance servers (e.g., servers with GPUs) and user devices (e.g., PCs or smartphones with cameras and microphones)

Software: Generative AI models (e.g., GPT-4, image generation AI, music generation AI), emotion recognition software (e.g., OpenCV, voice analysis libraries)

Program processing explanation

1. Story input and analysis

The user uses their device to input story settings such as the story's theme, setting, and characters. For example, they might input "a scene where a hero fights a dragon." The server receives this input and uses a generative AI model (e.g., GPT-4) to analyze each scene in the story and extract the scene's atmosphere and the characters' emotional states.

2. Graphics and Music Generation

Based on the analysis results, the server uses image generation AI to generate graphics that match the scene. For example, it could generate an impressive graphic of a hero fighting a dragon. Similarly, it uses music generation AI to generate music that suits the scene. For example, it could generate music that heightens the tension in a battle scene.

3. User Emotion Recognition

The device uses a camera and microphone to capture the user's facial expressions and tone of voice. For example, if the user is smiling, the device sends that information to the server. The server then uses emotion recognition software (e.g., OpenCV, voice analysis library) to analyze the user's emotions.

4. Story Development Adjustments

The server uses an emotion engine to adjust the story development based on the user's emotional information. For example, if the user is smiling, the story development will be adjusted to a more exciting direction.

Specific examples and prompt sentence examples

Example:

The user enters "a scene in which a hero fights a dragon."

The server analyzes the scene as a battle scene and uses image generation AI to generate powerful graphics of the dragon and hero.

Music generation AI is used to generate music that heightens the tension.

If the user is smiling, the emotion engine recognizes that the user is having fun and adjusts the story development in a more exciting direction.

Example prompt:

"Generate graphics for a scene in which a hero fights a dragon."

"Generate music that matches an emotional scene."

"If the user is smiling, adjust the story development to make it more enjoyable."

This system makes it possible to automatically create graphics and music that match each scene in a story, and to provide a story development that responds to the user's emotions. This effectively stirs the viewer's emotions and improves the quality of the story experience. The flow of the specific processing in Example 3 is explained using Figure 21.

Step 1:

The user enters the text of the story.

The user enters "A scene in which a hero fights a dragon" into the text input field on the device and clicks the send button. The entered text is sent from the device to the server.

Step 2:

The server analyzes the story text.

The server inputs the received text into a generative AI model (e.g., GPT-4) and analyzes each scene in the story. As a result of the analysis, the atmosphere of the scene and the emotional state of the characters are extracted. For example, "battle scene" and "tension" are extracted. The analysis results are stored in an internal database.

Step 3:

The server generates graphics that match the scene.

Based on the analysis results, the server uses image generation AI (e.g., DALL-E) to generate graphics that match the scene. For example, enter "a scene of a hero fighting a dragon" as a prompt and obtain the generated graphic. The generated graphic is then sent to the user's device.

Step 4:

The server generates music that matches the scene.

The server uses music generation AI to generate music appropriate for the scene. For example, a user can enter "music that will increase tension in a battle scene" as a prompt and obtain the generated music. The generated music is then sent to the user's device.

Step 5:

The device recognizes the user's emotions.

The device uses a camera to capture the user's facial expressions and a microphone to record the tone of voice. This data is sent to the server in real time. For example, if the user is smiling, that information is sent to the server.

Step 6:

The server adjusts the story development based on the user's emotions.

The server analyzes the received emotional data using emotion recognition software (e.g., OpenCV, voice analysis library) to recognize the user's emotions. For example, if it recognizes that the user is smiling, it uses the emotion engine to adjust the story development in an exciting direction. A new scene is generated and sent to the user's device.

(Application Example 3)

Next, we will explain Application Example 3 of Configuration Example 3. In the following explanation, the data processing device 12 will be referred to as the "server" and the headset-type terminal 314 will be referred to as the "terminal."

Conventional story generation systems have the problem of not adjusting the story based on the user's emotions, resulting in a uniform user experience. Furthermore, because the story scenes and music are fixed, they are unable to provide dynamic changes that respond to the user's emotions. This makes it difficult to provide a story experience that deeply appeals to the user's emotions.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 3 is realized by the following means.

In this invention, the server includes means for inputting story settings such as the story theme, setting, and characters; means for automatically creating character dialogue, scenery, and story structure based on the input story settings; means for automatically generating foreshadowing, mystery elements, and moving scenes to make the created story more interesting; means for automatically creating graphics and music that match each scene of the created story; means for enhancing the dramatic effect with the story structure, character dialogue, images, and music, and effectively stirring the viewer's emotions; means for recognizing the user's emotions and adjusting story scenes and music in accordance with the recognized emotions; and means for recognizing emotions using the user's facial expressions, tone of voice, text input, etc. This enables a dynamic story experience that matches the user's emotions.

A "story theme" refers to the central idea or message of a story.

"Setting" refers to the place or environment in which the story unfolds.

"Characters" refers to the characters that appear in the story.

"Story setting" refers to the setting, including basic elements such as the story's theme, setting, and characters.

"Dialogue" refers to the words spoken by characters.

"Scene" refers to the scenery or scene depicted in the story.

"Story structure" refers to the overall flow of a story, including its beginning, middle, and end.

"Foreshadowing" refers to an element in the first half of a story that becomes important later in the story.

"Mystery elements" refer to elements that add mystery and suspense to a story.

A "moving scene" is one that strongly stirs the viewer's emotions.

"Graphics" refers to images and footage that visually represent scenes from a story.

"Song" refers to the music that plays along with the story scenes.

"Directional effects" refer to effects that influence the viewer's emotions through story scenes and music.

"User emotion" refers to the emotional state the user is feeling, such as joy, sadness, or surprise.

"Means for recognizing emotions" refers to technology that determines a user's emotions using their facial expressions, tone of voice, text input, etc.

"Means for adjusting story scenes and music" refers to technology that changes story scenes and music according to the recognized user emotions.

A system for implementing this invention can dynamically adjust story scenes and music according to the user's emotions. A specific embodiment of this system is described below.

System Configuration

The system includes the following major hardware and software components:

Hardware: Smartphone camera

Software: OpenCV (face detection), Keras (emotion recognition model), music playback software

Program processing

The server first captures the user's facial expression using the smartphone camera. The captured video undergoes face detection using OpenCV. Next, it uses the Keras emotion recognition model to recognize the user's emotion from the detected face.

Based on the recognized emotion, the server adjusts the story scenes and music. For example, if the user is smiling, the story scenes will change to brighter, more cheerful ones, and the music will also become more upbeat. Conversely, if the user has a sad expression, the story will transition to a more moving scene, and the music will also change to something more moving.

Specific examples

As a concrete example, consider the case where a user smiles while enjoying a story. In this case, the server would process the following:

1. Capture the user's facial expressions using the smartphone camera.

2. Detect faces using OpenCV.

3. The user's emotion is recognized as "happiness" using the Keras emotion recognition model.

4. Based on the recognized emotion, the story scene is changed to a brighter, more cheerful one and upbeat music is played.

Example prompt

An example of a prompt to input to the generative AI model is as follows:

If the user smiles, change the story scene to something bright and cheerful and play upbeat music.

In this way, it is possible to provide a dynamic narrative experience that responds to the user's emotions. This system allows users to become more immersed in the story and have an experience that appeals to their emotions.

The flow of the specific processing in Application Example 3 will be explained using Figure 22.

Step 1:

The server uses the smartphone's camera to capture the user's facial expressions. The input is the camera image, and the output is the captured image data. Specifically, the camera acquires the image in real time and captures the image frame by frame.

Step 2:

The server uses OpenCV to detect faces from captured image data. The input is the captured image data, and the output is the location information of the detected face. Specifically, OpenCV's face detection algorithm identifies the face in the image and obtains its coordinates.

Step 3:

The server uses a Keras emotion recognition model to recognize the user's emotion from the detected face. The input is face position information and image data, and the output is the recognized emotion label. Specifically, the face area is extracted, preprocessed, and then input into the emotion recognition model to classify the emotion.

Step 4:

The server adjusts the story scenes and music based on the recognized emotions. The input is the recognized emotion label, and the output is the adjusted scenes and music. Specific operations include selecting scenes and music that correspond to the emotion label and changing the progression of the story.

Step 5:

The server sends the adjusted scene and music to the user's device. The input is the adjusted scene and music, and the output is the story scene displayed on the user's device and the music played. Specifically, the server encodes the scene and music data and sends it to the user's device over the network.

Step 6:

The user's device plays the received scenes and music and provides them to the user. The input is the scene and music data received from the server, and the output is the video displayed to the user and the music played. Specifically, the device decodes the data, displays it on the screen, and plays the music from the speakers.

The specific processing unit 290 transmits the results of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the results of the specific processing. The microphone 238 acquires audio indicating the user input regarding the results of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

Data generation model 58 is what is known as generative AI (artificial intelligence). An example of data generation model 58 is generative AI such as ChatGPT (Internet search <URL: https://openai.com/blog/chatgpt>). Data generation model 58 is obtained by performing deep learning on a neural network. A prompt containing an instruction is input to data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. Data generation model 58 performs inference on the input inference data in accordance with the instructions indicated by the prompt, and outputs the inference results in the form of data such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and/or summarization.

Another example of generative AI is Gemini (Internet search <URL: https://gemini.google.com/?hl=ja>).

In the above embodiment, an example was given in which the specific processing was performed by the data processing device 12, but the technology disclosed herein is not limited to this, and the specific processing may also be performed by the headset-type terminal 314.

[Fourth embodiment]

Figure 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

As shown in FIG. 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

The data processing device 12 includes a computer 22, a database 24, and a communication I/F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and communication I/F 26 are also connected to the bus 34. The communication I/F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and/or a LAN (Local Area Network).

The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I/F 44, and a control target 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and control target 443 are also connected to the bus 52.

The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the user 20's surroundings (e.g., an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

The communication I/F 44 is connected to the network 54. The communication I/Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I/Fs 44 and 26 is carried out in a secure manner.

The control object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, in the data processing device 12, specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32.

The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as the specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

Storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

In the robot 414, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

Next, we will explain the identification process performed by the identification processing unit 290 of the data processing device 12.

"Example 1"

The system of the present invention receives input from the user about the story setting, such as the story's theme, setting, and characters. This input is in text format, such as "A knight's adventure story set in medieval Europe." Next, based on this input, the system automatically creates character dialogue, scenes, and a story structure. This creation is carried out using existing story databases and artificial intelligence technology, generating dialogue and scenes that match the story flow, character personalities, and story theme.

"Example 2"

In addition, the system automatically generates hints, mystery elements, and moving scenes to make the story more interesting. This is done by laying hints at the right time, sprinkling mystery elements, and creating moving scenes in line with the story's theme, the characters' personalities, and the story's progression. For example, it automatically generates hints that people the knight meets during his adventure will play an important role later in the story, puzzles the knight faces, and moving scenes when the knight overcomes difficulties.

"Example 3"

The system then automatically creates graphics and music that match each scene in the story you've created. The graphics and music are created taking into account the atmosphere of the scene and the emotional state of the characters. For example, powerful graphics and music that heighten tension are created for battle scenes, while beautiful landscape graphics and moving music are created for moving scenes. This makes it possible to enhance the dramatic effect of the story's structure, the characters' conversations, images, and music, effectively stirring the viewer's emotions.

The processing flow for each example form is explained below.

"Example 1"

Step 1: The user provides textual input of the story setting, including the story's theme, setting, and characters. For example, it could be provided as "a knight's adventure story set in medieval Europe."

Step 2: Based on the input, the system automatically creates character dialogue, scenery, and story structure. This is done using existing story databases and artificial intelligence technology.

Step 3: Lines and scenes are generated that match the story flow, character personalities, and theme of the story.

"Example 2"

Step 1: The system automatically generates plot twists, mystery elements, moving scenes, and more to make the story more interesting.

Step 2: Lay out hints at the right time, sprinkle in mystery elements, and create moving scenes in line with the story's theme, the characters' personalities, and the story's progression.

Step 3: For example, foreshadowing, such as people the knight meets during his adventure playing important roles later in the story, puzzles the knight faces, and moving scenes when the knight overcomes difficulties, are automatically generated.

"Example 3"

Step 1: The system automatically creates graphics and music that match each scene in the story you've created.

Step 2: Graphics and music are created taking into account the atmosphere of the scene and the emotional state of the characters.

Step 3: For example, powerful graphics and music that heightens the tension are created for battle scenes, while beautiful landscape graphics and moving music are created for moving scenes. This enhances the dramatic effect of the story structure, character dialogue, images and music, making it possible to effectively stir the viewer's emotions.

(Example 1)

Next, Example 1 of Form Example 1 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 will be referred to as the "terminal."

When conventional story generation systems automatically generate a story based on a user-entered story setting, they struggle to maintain consistency in the story flow, character dialogue, and scenery. They also lack the ingenuity to ensure the generated story is moving and entertaining for the viewer. Furthermore, they are unable to automatically create graphics and music that match each scene in the story, which means they are unable to effectively move the viewer's emotions.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

In this invention, the server includes: means for inputting story settings such as the story's theme, setting, and characters; means for automatically creating character dialogue, scenery, and story structure based on the input story settings; means for automatically generating foreshadowing, mystery elements, and moving scenes to make the created story more interesting; means for automatically producing graphics and music that match each scene of the created story; means for enhancing the dramatic effect with the story structure, character dialogue, and images and music to effectively stir the viewer's emotions; means for receiving input from the user and analyzing it using natural language processing technology; means for generating prompt sentences to be input into the generative AI model based on the analysis results; means for automatically generating story dialogue, scenery, and structure using the generative AI model; and means for formatting the generated story and providing it to the user. This makes it possible to automatically generate a consistent story based on user input and provide the viewer with a moving and entertaining story.

A "story theme" is the central subject or theme of a story, and is the element that determines the overall direction and atmosphere of the story.

The "setting" refers to the place and historical background in which the story unfolds, and is an important element in shaping the setting and atmosphere of the story.

"Characters" are characters who act within a story and move the story forward, and are elements that have a major influence on the story's development and theme.

"Story setting" refers to the setting that includes basic elements such as the story's theme, setting, and characters, and is the element that forms the framework of the story.

"Dialogue" refers to the words or lines spoken by characters, and is an important element in expressing the progression of the story and the characters' personalities.

"Scenes" refer to the scenery and scenes depicted in a story, and are elements that visually express the atmosphere and emotions of the story.

"Story structure" refers to the overall flow and structure of a story, including the beginning, middle, and end, and is an element that makes the story unfold effectively.

"Foreshadowing" is a hint or sign that relates to later developments in a story, and is an element that adds consistency and depth to a story.

"Mystery elements" are elements that present mysteries or questions within a story and attract the interest of readers or viewers.

A "moving scene" is a scene in a story that strongly stirs emotions and is an impressive element that stays in the viewer's mind.

"Graphics" refers to images and illustrations that visually represent each scene in the story, and are elements that visually complement the atmosphere and emotion of the story.

"Songs" are melodies and sounds that are musically expressed to match each scene in a story, and are elements that musically complement the atmosphere and emotions of the story.

"Natural language processing technology" is a technology that enables computers to understand and analyze human language, and is used to analyze text data and extract information.

A "generative AI model" is a model that uses artificial intelligence technology to generate text and data, and is used to automatically generate dialogue, scenes, and structure for a story.

A "prompt" is an instruction to be input into a generative AI model, and is an element that guides the content generated by the model.

"Formatting" refers to the process of adding paragraph breaks and line breaks to make the generated narrative text easier to read, before it is presented to the user.

This invention is a system that automatically generates a story based on story settings such as the story's theme, setting, and characters, which are input by the user. Specific embodiments of this system are described below.

User input

The user uses a terminal to enter text information about the story's theme, setting, characters, etc. into the system's input form. For example, they might enter "A knight's adventure story set in medieval Europe."

Receive and analyze input data by the server

The server receives the text data sent by the user. It then analyzes the input data using natural language processing technology. Specifically, it uses Python's NLTK library and spaCy to extract information such as theme, setting, and characters from the input text. For example, it extracts keywords such as "medieval Europe," "knights," and "adventure story."

Generating prompt statements

The server generates a prompt to be input into the generative AI model based on the extracted information. For example, if the user's input is "A knight's adventure story set in medieval Europe," the prompt would be "Please generate lines and scenes for the characters in a knight's adventure story set in medieval Europe."

Generating stories using generative AI models

The server inputs the generated prompt into a generative AI model (for example, OpenAI's GPT-4). The generative AI model automatically generates the dialogue, scenery, and structure of a story based on the prompt. For example, it could generate text such as, "The brave knight Arthur met a wizard in a dark forest. The wizard said to Arthur, 'Deep in this forest lies an ancient dragon. Your mission is to slay the dragon.'"

Format and provide generated results

The server receives the story text returned by the generative AI model and formats it as needed, for example by adding paragraph breaks and line breaks to make it easier to read. The formatted story is then converted into a format that is easy for the user to read. Finally, the server provides the formatted story to the user, who can view the results through their device.

Specific examples

As an example of user input, consider "a knight's adventure story set in medieval Europe." Based on this input, the server generates the following prompt:

"Generate lines and scenes from characters in a knight's adventure story set in medieval Europe."

The generative AI model generates a story based on this prompt, outputting text such as the following:

"The brave knight Arthur met a wizard in a dark forest. The wizard told Arthur, 'Deep in this forest lies an ancient dragon. Your mission is to defeat the dragon.'"

In this way, it is possible to automatically generate a coherent story based on user input, providing an inspiring and entertaining story for the viewer.

The flow of the identification process in Example 1 will be explained using Figure 11.

Step 1:

The user uses their device to enter text information about the story's theme, setting, characters, etc. into the system's input form. For example, they might enter "A knight's adventure story set in medieval Europe." This input data is then sent to the server.

Step 2:

The server receives text data sent by the user. It analyzes the received data using natural language processing technology. Specifically, it uses Python's NLTK library and spaCy to extract information such as theme, setting, and characters from the input text. For example, it extracts keywords such as "medieval Europe," "knights," and "adventure story." The results of this analysis become the input for the next step.

Step 3:

Based on the analysis results, the server generates a prompt to be input into the generative AI model. For example, if the user's input is "A knight's adventure story set in medieval Europe," the prompt would be "Please generate lines and scenes for the characters in a knight's adventure story set in medieval Europe." This prompt becomes the input for the next step.

Step 4:

The server inputs the generated prompt into a generative AI model (for example, OpenAI's GPT-4). The generative AI model automatically generates the dialogue, scenery, and structure of a story based on the prompt. For example, it might generate text such as, "The brave knight Arthur met a wizard in a dark forest. The wizard said to Arthur, 'Deep in this forest lies an ancient dragon. Your mission is to slay the dragon.'" This generated text becomes the input for the next step.

Step 5:

The server receives the story text returned by the generative AI model and formats it as needed, for example by adding paragraph breaks and line breaks to make it easier to read. The formatted story is then converted into a format that is easier for the user to read. This formatted text becomes the input for the next step.

Step 6:

The server provides the formatted story to the user. The user can view the results through their device, for example, on a web page or application screen that displays the generated story. The user can then read and enjoy the generated story.

(Application Example 1)

Next, we will explain Application Example 1 of Form Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 will be referred to as the "terminal."

Existing story generation systems automatically generate stories based on user-entered themes and settings, but they often lack sufficient story structure, character dialogue, and scene generation, making it difficult to effectively sway viewers' emotions. Furthermore, the lack of functionality to preview, save, and share the generated stories creates a problem of inconvenient user experience.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

In this invention, the server includes: means for inputting story settings such as the story's theme, setting, and characters; means for automatically creating character dialogue, scenery, and story structure based on the input story settings; means for automatically generating foreshadowing, mystery elements, and moving scenes to make the created story more interesting; means for automatically producing graphics and music that match each scene of the created story; means for enhancing the dramatic effect with the story structure, character dialogue, images, and music to effectively stir the viewer's emotions; means for generating prompt sentences using a generative AI model and automatically generating a story; and means for previewing, saving, and sharing the generated story. This makes it possible to automatically generate high-quality stories based on the settings input by the user, effectively stirring the viewer's emotions.

A "story theme" is the central subject or theme of a story, and determines the overall direction and atmosphere of the story.

The "setting" refers to the place and historical background in which the story unfolds, and is an element that shapes the setting and atmosphere of the story.

"Characters" are characters that appear in a story and play an important role in the progression and development of the story.

"Story setting" refers to the setting that includes basic elements such as the story's theme, setting, and characters, and forms the framework of the story.

"Dialogue" refers to the words spoken by characters, and is an important element in expressing the progression of the story and the character's personality.

"Scenes" refer to the landscapes and scenes depicted in a story, and visually express the atmosphere and emotions of the story.

"Story structure" refers to the overall flow and structure of a story, including a beginning, middle, and end, and is the framework for effectively developing the story.

"Foreshadowing" is a hint or omen that relates to later developments in a story, and is an element that makes a story more interesting.

"Mystery elements" are elements that add mystery and suspense to a story, and are meant to attract the interest of readers and viewers.

A "moving scene" is a scene in a story that stirs strong emotions and is intended to leave a deep impression on the viewer.

"Graphics" refers to the visual images and illustrations that accompany each scene in the story, and are intended to visually express the atmosphere and emotions of the story.

"Songs" are pieces of music created to accompany each scene in a story, and are intended to musically express the atmosphere and emotions of the story.

A "generative AI model" is a model that uses artificial intelligence technology to automatically generate text, images, etc., and is used to automatically generate stories.

A "prompt" is an instruction to be input into a generative AI model, and is generated based on the theme and setting of the story.

"Preview" is a feature that allows users to preview the generated story in advance and check the content.

"Preservation" refers to recording the stories that are created in digital form so that they can be reused and shared later.

"Sharing" refers to sharing the story you create with other users and platforms, with the aim of promoting the spread and appreciation of the story.

A system for implementing this invention allows the user to input story settings such as the story's theme, setting, and characters, and automatically generates a story based on these settings. Specific embodiments of this system are described below.

System Configuration

This system is primarily composed of the following hardware and software:

Hardware: Smartphones, servers

Software: Python, OpenAI API

Program processing

User input

Users enter the story's theme, setting, and characters in text form through a smartphone application. For example, they might enter the theme "A knight's adventure story set in medieval Europe."

Prompt generation

The server generates a prompt based on the information entered by the user. This prompt is an instruction that is input into the generative AI model. As a specific example, the following prompt is generated:

Theme: A knight's adventure story set in medieval Europe

Setting: Medieval Europe

Characters: Brave knight, wizard, dragon

Generate a story.

Story generation

The server uses a generative AI model (e.g., GPT-4) to automatically generate a story based on the generated prompt. This generative AI model is called through the OpenAI API.

Preview, save, share

The generated story is displayed so that users can preview it through the smartphone application. Users can save the generated story and share it on social media, etc.

Specific examples

If a user sets the theme to "a knight's adventure story set in medieval Europe" and enters "a brave knight, a wizard, and a dragon" as the characters, the following story will be generated.

Theme: A knight's adventure story set in medieval Europe

Setting: Medieval Europe

Characters: Brave knight, wizard, dragon

Generate a story.

By inputting this prompt into a generative AI model, a story is automatically generated. Users can preview, save, and share the generated story.

Hardware and software used

Hardware: Smartphones, servers

Software: Python, OpenAI API

This makes it possible to automatically generate high-quality stories based on user-entered settings, effectively stirring the viewer's emotions.

The flow of the specific processing in Application Example 1 will be explained using Figure 12.

Step 1:

Users use their smartphone to input the story's theme, setting, and characters in text format. The input data is then sent from the device to the server. An example input includes the theme "A knight's adventure story set in medieval Europe."

Step 2:

The server generates a prompt based on the received user input data. Specifically, it combines information about the theme, setting, and characters to create a prompt that is suitable for the generative AI model. The generated prompt looks like this:

Theme: A knight's adventure story set in medieval Europe

Setting: Medieval Europe

Characters: Brave knight, wizard, dragon

Generate a story.

This prompt text is generated.

Step 3:

The server calls a generative AI model (e.g., GPT-4) with the generated prompt text. It sends the prompt text using the OpenAI API and requests the generation of a story. The input is the prompt text, and the output is the generated story text.

Step 4:

The generative AI model automatically generates a story based on the prompt text. The generated story text is returned to the server. For example, the following story might be generated:

Once upon a time, in a vast kingdom in medieval Europe, there lived a brave knight. His name was Arthur. One day, with the help of a wizard, he embarked on an adventure to battle a fearsome dragon.

Step 5:

The server sends the text of the generated story to the smartphone device. The user can preview the generated story through the smartphone device. The input is the text of the generated story, and the output is the preview screen displayed on the user's device.

Step 6:

Users can save the generated story or share it on social media. Saved stories are recorded in the device's storage, and shared stories are sent to the specified platform. The input is the user's selection, and the output is the saved file or shared content.

These steps make it possible to automatically generate a high-quality story based on the settings entered by the user, effectively stirring the viewer's emotions.

(Example 2)

Next, Example 2 of Form Example 2 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 will be referred to as the "terminal."

While conventional story generation systems can automatically generate stories based on the story's theme and character settings, they have difficulty appropriately generating foreshadowing, mystery elements, and moving scenes to make the story more interesting. Furthermore, they lack the functionality to organize the generated story elements and provide them to users, making it difficult for users to effectively use the generated story. Furthermore, they lack the means to enhance the dramatic effect through story structure, character dialogue, visuals, and music, and to effectively stir the viewer's emotions.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

In this invention, the server includes means for inputting story settings such as the story's theme, setting, and characters; means for automatically creating character dialogue, scenery, and story structure based on the input story settings; means for automatically generating foreshadowing, mystery elements, and moving scenes to make the created story more interesting; means for organizing the generated story elements and providing them to the user; and means for enhancing the dramatic effect with the story structure, character dialogue, images, and music to effectively stir the viewer's emotions. This makes it possible to make the story more interesting and enable users to effectively use the generated story.

"Story theme" refers to the central subject or subject matter of a story.

"Setting" refers to the place or environment in which the story unfolds.

"Characters" refers to the characters and people who appear in the story.

"Story setting" refers to the setting, including basic elements such as the story's theme, setting, and characters.

"Dialogue" refers to the words or lines spoken by characters.

"Scene" refers to the scenery or scene depicted in the story.

"Story structure" refers to the overall flow and structure of a story, including its beginning, middle, and end.

"Foreshadowing" refers to elements or events placed in the first half of a story that will have important meaning later on.

"Mystery elements" refer to elements that add mystery or detective elements to a story, attracting the interest of readers and viewers.

A "moving scene" refers to a scene in a story that stirs strong emotions.

"Graphics" refers to images and illustrations that are displayed in accordance with story scenes.

"Song" refers to the music or background music that plays along with the story scenes.

"Directory effects" refers to techniques that use the visual and auditory elements of a story to elicit emotions and reactions from the viewer.

"Audience sentiment" refers to the emotions and feelings of the people who watch or read the story.

This invention is a system that automatically generates plot twists, mystery elements, and moving scenes to make a story more interesting, based on the story's theme, the characters' personalities, and the story's progression. A specific embodiment of this system is shown below.

System Configuration

Hardware

The server uses a cloud server equipped with a high-performance GPU.

Software

The server uses a generative AI model. Specifically, it uses a generative AI model such as OpenAI's GPT-4.

Program processing

User input

The user uses the device to input information about the story's theme, the characters' personalities, and the story's progression. For example, the user might input the following information:

Theme: Medieval Knight's Adventure

Characters: A brave knight, a mysterious old man, and a princess

Story progression: The knight meets a mysterious old man during his adventure, receives advice from him, and continues his adventure. Ultimately, the knight saves the princess and protects the kingdom.

Generating prompt statements

The server receives the information entered by the user and generates a prompt based on that information. The generated prompt looks like this:

Theme: Medieval Knight's Adventure

Characters: A brave knight, a mysterious old man, and a princess

Story progression: The knight meets a mysterious old man during his adventure, receives advice from him, and continues his adventure. Ultimately, the knight saves the princess and protects the kingdom.

Generating story elements

The server inputs the generated prompt text into the generative AI model to generate story elements. The generative AI model generates the following story elements based on the prompt text.

Foreshadowing: The mysterious old man the knight meets during his adventure actually knows the kingdom's hidden secrets.

Mystery element: The knight solves the mystery hidden in an old map he received from an old man.

Emotional scene: The knight saves the princess and fights a final battle to protect the kingdom.

Organizing and providing generated story elements

The server organizes the story elements output by the generative AI model and provides them to the user. The user can check the generated story elements via their device and make corrections or additions as needed.

Specific examples

Below is a concrete example of how a user can use the system to generate a story.

Theme: Medieval Knight's Adventure

Characters: A brave knight, a mysterious old man, and a princess

Story progression: The knight meets a mysterious old man during his adventure, receives advice from him, and continues his adventure. Ultimately, the knight saves the princess and protects the kingdom.

The server inputs this prompt into a generative AI model, which automatically generates plot twists, mystery elements, and moving scenes based on the story's theme, the characters' personalities, and the story's progression. The generated story elements are provided to the user, who can review them and make corrections or additions as needed.

In this way, it is possible to make the story more interesting and enable users to effectively use the generated story.

The flow of the identification process in Example 2 will be explained using Figure 13.

Step 1:

The user enters information about the story's theme and characters.

Input: Information about the story's theme, character personalities, and story progression.

Specific operation: The user enters the theme "Adventures of a Medieval Knight" and character information - "A Brave Knight, a Mysterious Old Man, and a Princess" - into the input form on their device. Once the input is complete, they click the "Submit" button.

Output: The entered information is sent to the server.

Step 2:

The server receives the entered information and generates a prompt.

Input: User-submitted information about the story's theme, character personalities, and story progression.

Specific operation: Based on the received information, the server generates a prompt sentence to input into the generative AI model.

Output: The generated prompt text.

Theme: Medieval Knight's Adventure

Characters: A brave knight, a mysterious old man, and a princess

Story progression: The knight meets a mysterious old man during his adventure, receives advice from him, and continues his adventure. Ultimately, the knight saves the princess and protects the kingdom.

Step 3:

The server inputs prompts into the generative AI model to generate story elements.

Input: The generated prompt text.

Specific operation: The server inputs prompt text into the generative AI model, generating story hints, mystery elements, and moving scenes.

Output: Generated story elements.

Foreshadowing: The mysterious old man the knight meets during his adventure actually knows the kingdom's hidden secrets.

Mystery element: The knight solves the mystery hidden in an old map he received from an old man.

Emotional scene: The knight saves the princess and fights a final battle to protect the kingdom.

Step 4:

The server organizes the generated story elements and provides them to the user.

Input: Story elements output from the generative AI model.

Specific operation: The server organizes the generated story elements and provides them to the user. The user can review the generated story elements through their device and make corrections or additions as necessary.

Output: The organized story elements are presented to the user.

(Application Example 2)

Next, we will explain Application Example 2 of Form Example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 will be referred to as the "terminal."

While conventional story generation systems can automatically create stories based on the story's theme and character settings, they have difficulty properly generating foreshadowing, mystery elements, and moving scenes to make the story more interesting. It is also difficult to automatically create graphics and music that match each scene in the generated story and effectively stir the viewer's emotions. Furthermore, when using generative AI models to generate story elements, there is a lack of a way to properly generate prompt sentences and input them into the model, which poses the challenge of not being able to provide a satisfying story to the user.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

In this invention, the server includes means for inputting story settings such as the story theme, setting, and characters; means for automatically creating character dialogue, scenery, and story structure based on the input story settings; means for automatically generating foreshadowing, mystery elements, and moving scenes to make the created story more interesting; means for automatically creating graphics and music that match each scene of the created story; means for enhancing the dramatic effect with the story structure, character dialogue, images, and music to effectively stir the viewer's emotions; means for generating story elements using a generative AI model; means for generating prompt sentences and inputting them into the generative AI model; and means for displaying the generated story elements to the user. This makes it possible to make the story more interesting and effectively stir the viewer's emotions.

A "story theme" is the central subject or subject of a story.

"Setting" refers to the place or environment in which the story unfolds.

"Characters" refers to the characters that appear in the story.

"Story setting" refers to the basic elements of the story, such as its theme, setting, and characters.

"Dialogue" refers to the words spoken by characters.

"Scene" refers to the scenery or scene depicted in the story.

"Story structure" refers to the overall flow and structure of a story, including its beginning, middle, and end.

"Foreshadowing" is the advance indication of important elements related to later developments.

"Mystery elements" refer to the inclusion of mysteries or mysterious events in a story.

A "moving scene" is one that strongly stirs the viewer's emotions.

"Graphics" refers to visual images and illustrations.

"Song" refers to music or melody.

"Producer effects" refer to visual and auditory techniques used to make a story more compelling.

"Viewers" are people who watch a story.

A "generative AI model" is an algorithm that uses artificial intelligence to generate text, images, etc.

A "prompt" is an instruction sentence to be input into a generative AI model.

"User" refers to a person who uses the system.

A system for implementing this invention includes means for inputting story settings such as the story's theme, setting, and characters; means for automatically creating character dialogue, scenery, and story structure based on the input story settings; means for automatically generating foreshadowing, mystery elements, and moving scenes to make the created story more interesting; means for automatically creating graphics and music that match each scene of the created story; means for enhancing the dramatic effect with the story structure, character dialogue, images, and music, and effectively stirring the viewer's emotions; means for generating story elements using a generative AI model; means for generating prompt sentences and inputting them into the generative AI model; and means for displaying the generated story elements to the user.

The server receives information about the story's theme and characters entered by the user and generates a prompt based on that information. The generated prompt is then input into a generative AI model (e.g., GPT-3) to generate story elements. The generated story elements are then displayed to the user.

Specifically, the server operates in the following steps: First, the user inputs information about the story's theme and characters. Next, the server generates a prompt based on this information. The generated prompt is input into a generative AI model, which generates story elements. The generated story elements are then displayed to the user.

The hardware used includes servers and user devices. The software used includes generative AI models (e.g., GPT-3) and prompt generation algorithms.

As a concrete example, the following prompt sentence is generated:

Theme: Knight's Adventure

Characters: Brave knight, mysterious old man, wizard

Generate plot twists, mystery elements, and moving scenes to make your story more interesting.

By inputting this prompt into a generative AI model, story elements are automatically generated. The generated story elements are displayed to the user, and the dramatic effect can be enhanced with story structure, character dialogue, visuals, and music.

The flow of the specific processing in Application Example 2 will be explained using Figure 14.

Step 1:

The user enters information about the story's theme and characters.

Input: Story theme, character information

Output: Input theme and character information

Specific actions: The user enters the story's theme (e.g., "The Knight's Adventure") and characters (e.g., "The Brave Knight, the Mysterious Old Man, and the Wizard") into an input form on a smartphone application.

Step 2:

The server receives the entered theme and character information and generates a prompt.

Input: Entered theme and character information

Output: Generated prompt text

Specific operation: Based on the received theme and character information, the server generates a prompt like the following: "Theme: A knight's adventure. Characters: A brave knight, a mysterious old man, a wizard. Please generate hints, mystery elements, and moving scenes to make the story more interesting."

Step 3:

The server inputs the generated prompt sentence into the generative AI model.

Input: Generated prompt text

Output: Story elements generated by the generative AI model

Specific operation: The server sends the generated prompt to a generative AI model such as GPT-3 to generate story elements (e.g., foreshadowing, mystery elements, emotional scenes).

Step 4:

The server receives the generated story elements and displays them to the user.

Input: Story elements generated by a generative AI model

Output: Story elements displayed to the user

Specific operation: The server sends the story elements received from the generative AI model to the user's smartphone application and displays them to the user. The user can then review the generated story elements.

Step 5:

The server automatically creates graphics and music that match each scene in the story.

Input: Generated story elements

Output: Graphics and music that match each scene

Specific operation: Based on the generated story elements, the server automatically creates appropriate graphics and music, taking into account the atmosphere of the scene and the emotional state of the characters.

Step 6:

The server enhances the dramatic effect through story structure, character dialogue, visuals, and music.

Input: Graphics, music, and story elements that match each scene

Output: A story with enhanced dramatic effects

Specific operation: The server combines the story structure, character dialogue, images, and music to enhance the dramatic effect, effectively stirring the viewer's emotions.

(Example 3)

Next, Example 3 of Form Example 3 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 will be referred to as the "terminal."

Previous story generation systems struggled to automatically create graphics and music that matched each scene in the story, preventing them from effectively evoking the viewer's emotions. Furthermore, generating graphics and music that take into account the scene's atmosphere and the characters' emotional states was a time-consuming and labor-intensive process.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Example 3 is realized by the following means.

In this invention, the server includes: means for inputting story settings such as the story theme, setting, and characters; means for automatically creating character dialogue, scenery, and story structure based on the input story settings; means for automatically generating foreshadowing, mystery elements, and moving scenes to make the created story more interesting; means for automatically creating graphics and music that match each scene of the created story; means for analyzing scene data and extracting the atmosphere of the scene and the emotional state of the characters; means for generating prompt text to be input into a generative AI model based on the extracted information; means for inputting the prompt text into the generative AI model and generating graphics and music that match the scene; means for integrating the generated graphics and music with the scene data and generating final output; and means for enhancing the dramatic effect with the story structure, character dialogue, images, and music to effectively stir the viewer's emotions. This makes it possible to automatically create graphics and music that match each scene of the story, enhancing the dramatic effect that effectively stirs the viewer's emotions.

"Story settings such as the story's theme, setting, and characters" are the basic elements of a story, and refer to the background of the story, the characteristics of the characters, and the settings necessary for the story's progression.

"Means of input" refers to the interface that allows the user to provide the story setting, such as the story's theme, setting, and characters, to the system.

"Means for automatically creating character dialogue, scenes, and story structure" refers to a function that automatically generates character dialogue, scene descriptions, and the overall flow of a story based on the input story setting.

"Means for automatically generating foreshadowing, mystery elements, moving scenes, etc." refers to a function that automatically adds elements that cause surprises or emotions to a pre-set story in order to make the story more interesting.

"Means for automatically creating graphics and music" refers to a function that automatically generates visual and auditory elements to match each scene in a story.

"Means for analyzing scene data" refers to the function of analyzing the content of each scene in a story and extracting the atmosphere of the scene and the emotional state of the characters.

"Means for generating prompt sentences to be input to a generative AI model" refers to a function that creates text to provide appropriate instructions to a generative AI model based on analyzed scene data.

"A means of inputting a prompt sentence into a generative AI model and generating graphics and music that match the scene" refers to a function that inputs a prompt sentence into a generative AI model and, as a result, generates graphics and music that are appropriate for the scene.

"Means for integrating generated graphics and music into scene data and generating final output" refers to the functionality for incorporating generated visual and auditory elements into scene data to create the final narrative output.

"Means of effectively stirring the viewer's emotions" refers to the function of having a strong impact on the viewer's emotions through the story structure, the characters' dialogue, images, and music.

This invention relates to a system that automatically creates graphics and music that match each scene in a story. This system inputs story settings such as the story's theme, setting, and characters, and automatically creates character dialogue, scenery, and story structure based on the input story settings. It also automatically generates foreshadowing, mystery elements, and moving scenes to make the story more interesting, and automatically creates graphics and music that match each scene.

Hardware and software used

This system uses the following hardware and software:

Server: Analyzes data and runs generative AI models.

Terminal: Provides an interface for users to enter the story setting for their story.

Generative AI models: For example, OpenAI's GPT-4, DALL-E, and music generation models are used.

Data processing and calculation

The server receives the story settings entered by the user via their device. The story settings include the story's theme, setting, and character traits. The server analyzes this data and automatically creates the characters' lines, scenery, and story structure.

Next, the server automatically generates foreshadowing, mystery elements, moving scenes, and more based on the story you've created. This makes the story more interesting and emotionally stirring for the viewer.

Furthermore, the server analyzes scene data to automatically create graphics and music that match each scene in the story. As a result of the analysis, it extracts the atmosphere of the scene and the emotional state of the characters. Based on the extracted information, the server generates prompt sentences to input into the generative AI model.

Specific examples

For example, in the case of an emotional scene, the server inputs the following prompt sentence into the generative AI model:

Example prompt for graphic generation: "Generate a graphic of a beautiful landscape. The scene is emotional, and the character is in tears."

Example music generation prompt: "Generate moving, poignant music. The scene is emotional, and the characters are in tears."

The generative AI model generates graphics and music based on these prompts. The server integrates the generated graphics and music with the scene data to generate the final output. The device displays the integrated results to the user.

In this way, the user can automatically create graphics and music that match each scene in the story, enhancing the dramatic effect that effectively stirs the viewer's emotions. The flow of the specific processing in Example 3 will be explained using Figure 15.

Step 1:

The user enters story settings such as the story's theme, setting, and characters into the device.

Specific operation: The user enters information such as the story's theme (e.g., fantasy), setting (e.g., medieval castle), and characters (e.g., hero, wizard) into the device's input form. The device then sends the entered story settings to the server.

Input: Story setting, including theme, setting, and characters.

Output: Story setting data sent to the server

Step 2:

The server analyzes the received story setting data and automatically creates the characters' lines, scenes, and story structure.

Specific operations: The server uses natural language processing (NLP) technology to analyze story setting data and generate character dialogue, scene descriptions, and the overall story flow.

Input: Story setting data

Output: Character dialogue, scenery, and story structure data

Step 3:

Based on the story you create, the server automatically generates foreshadowing, mystery elements, moving scenes, and more.

Specific operation: The server analyzes the story's composition data and adds hints, mystery elements, and moving scenes to make the story more interesting.

Input: Story composition data

Output: Story data including foreshadowing, mystery elements, and moving scenes

Step 4:

The server analyzes scene data to automatically create graphics and music that match each scene in the story.

Specific operation: The server divides the story data into scenes and extracts the atmosphere of each scene and the emotional state of the characters.

Input: Story data

Output: Scene-specific mood and character emotional state data

Step 5:

The server generates prompt sentences to input into the generative AI model based on the extracted information.

Specific operation: The server creates a prompt that corresponds to the atmosphere of the scene and the emotional state of the characters. For example, in the case of an emotional scene, the server generates the prompt, "Generate a graphic of a beautiful landscape. The scene is emotional, and the characters are crying."

Input: Scene atmosphere and character emotional state data

Output: Prompt text to be input to the generative AI model

Step 6:

The server inputs prompt text into the generative AI model, generating graphics and music that match the scene.

Specific operation: The server sends a prompt to the generative AI model (e.g., DALL-E), which generates graphics that match the scene. Similarly, it sends a prompt to the music generation model, which generates music that matches the scene.

Input: Prompt text to be input to the generative AI model

Output: Generated graphics and music

Step 7:

The server integrates the generated graphics and music with the scene data to generate the final output.

Specific operation: The server integrates the generated graphics and music with each scene data to create the final output of the entire story.

Input: Generated graphics, music, and scene data

Output: Final integrated narrative data

Step 8:

The terminal displays the final, integrated story data to the user.

Specific operation: The terminal receives the final story data sent from the server and displays it visually and audibly to the user.

Input: Final integrated narrative data

Output: The visual and auditory content of the story presented to the user.

(Application Example 3)

Next, we will explain Application Example 3 of Form Example 3. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 will be referred to as the "terminal."

Previous story generation systems struggled to automatically create graphics and music that matched each scene in the story, limiting the visual and auditory effects they could produce. Furthermore, even when users input their own stories, there was a lack of means to generate graphics and music appropriate for that story, resulting in a decline in content quality. Furthermore, the lack of functionality to preview and export the generated content resulted in poor user convenience.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 3 is realized by the following means.

In this invention, the server includes means for inputting story settings such as the story theme, setting, and characters; means for automatically creating character dialogue, scenery, and story structure based on the input story settings; means for automatically generating foreshadowing, mystery elements, and moving scenes to make the created story more interesting; means for automatically creating graphics and music that match each scene of the created story; means for enhancing the dramatic effect with the story structure, character dialogue, video, and music, and effectively stirring the viewer's emotions; means for analyzing story scenes and extracting the scene atmosphere and the emotional state of the characters; means for generating graphics based on scene descriptions; means for generating music based on scene descriptions; means for previewing the generated content; and means for exporting the completed content. This makes it possible to automatically generate, preview, and export visually and aurally rich content based on a story entered by the user.

"Story theme" refers to the central subject or message of a story.

"Setting" refers to the place or environment in which the story unfolds.

"Characters" refers to the characters that appear in the story.

"Story setting" refers to basic elements such as the story's theme, setting, and characters.

"Dialogue" refers to the words spoken by characters.

"Scene" refers to the scenery or scene depicted in the story.

"Story structure" refers to the overall flow of a story, including its beginning, middle, and end.

"Foreshadowing" refers to elements that foreshadow information or events that will become important later in the story.

"Mystery elements" refer to elements that add mystery or suspense to a story.

A "moving scene" refers to a scene that strongly stirs the viewer's emotions.

"Graphics" refers to visual images and illustrations generated to match story scenes.

"Song" refers to music generated to match the story scenes.

'Scene atmosphere' refers to the overall feeling or mood of a particular moment in a story.

"A character's emotional state" refers to the emotions the character is feeling in the story.

A "scene description" refers to a detailed description or explanation of a particular scene in a story.

"Preview" refers to checking the generated content in advance.

"Export" refers to saving the generated content as an external file.

A system for implementing this invention includes means for inputting story settings such as the story theme, setting, and characters; means for automatically creating character dialogue, scenery, and story structure based on the input story settings; means for automatically generating foreshadowing, mystery elements, and moving scenes to make the created story more interesting; means for automatically creating graphics and music that match each scene of the created story; means for enhancing the story structure, character dialogue, and visuals and music to effectively stir the viewer's emotions; means for analyzing story scenes and extracting the atmosphere of the scene and the emotional state of the characters; means for generating graphics based on scene descriptions; means for generating music based on scene descriptions; means for previewing the generated content; and means for exporting the completed content.

Hardware and software configuration

Hardware:

Smartphone

Software:

Python

OpenAI API

PIL (Python Imaging Library)

Music generation library

Data processing and calculation

How to enter story settings such as theme, setting, and characters:

Users enter story settings such as the story's theme, setting, and characters in text format through a smartphone application.

Methods for automatically creating character dialogue, scenes, and story structure:

Based on the input story setting, the server uses a generative AI model (e.g., GPT-3) to automatically create character dialogue, scenes, and story structure.

Methods for automatically generating foreshadowing, mystery elements, moving scenes, etc.:

The server uses a generative AI model to automatically generate plot twists, mystery elements, and moving scenes to make the story more interesting.

Methods for automatically creating graphics and music that match each scene in a story:

The server analyzes each scene in the story and extracts the scene's atmosphere and the characters' emotional state. Based on this, it generates graphics using OpenAI's image generation API and music using OpenAI's music generation API.

How to preview generated content:

Users can preview each scene of the story, including the generated graphics and music, through a smartphone application.

How to export your finished content:

Users can export the finished content as a video file through a smartphone application.

Specific examples

For example, if a user inputs the story "A hero climbed a mountain to fight a dragon," the system will analyze this scene and generate powerful graphics and music that heightens the tension to match the battle scene between the hero and the dragon.

Example prompt:

Analyze the following story scene and output the scene's atmosphere and the characters' emotional states:

The hero climbed the mountain to fight the dragon.

In this way, users can enjoy their stories as visually and aurally rich content.

The flow of the specific processing in Application Example 3 will be explained using Figure 16.

Step 1:

Users enter story settings such as the story's theme, setting, and characters in text format through a smartphone application.

Input: Story setting, including theme, setting, and characters.

Output: Story setting data in text format

Step 2:

Based on the input story setting, the server uses a generative AI model (e.g., GPT-3) to automatically create character dialogue, scenes, and story structure.

Input: Text-based story setting data

Output: Character dialogue, scenery, and story structure data

Specific operation: The server sends a prompt to the generative AI model and receives the generated text.

Step 3:

The server uses a generative AI model to automatically generate plot twists, mystery elements, and moving scenes to make the story more interesting.

Input: Character dialogue, scenery, and story structure data

Output: Foreshadowing, mystery elements, and moving scene data

Specific operation: The server sends a prompt to the generative AI model and receives the generated text.

Step 4:

The server analyzes each scene in the story and extracts the scene's atmosphere and the emotional state of the characters.

Input: Foreshadowing, mystery elements, and moving scene data

Output: Scene atmosphere, character emotional state data

Specific operation: The server sends a prompt to the generative AI model and receives the analysis results.

Step 5:

The server generates graphics based on the scene description.

Input: Scene atmosphere, character emotional state data

Output: Graphics data

Specific operation: The server sends a prompt to OpenAI's image generation API and receives the generated image.

Step 6:

The server generates music based on the scene description.

Input: Scene atmosphere, character emotional state data

Output: Music data

Specific operation: The server sends a prompt to OpenAI's music generation API and receives the generated music.

Step 7:

Users can preview each scene of the story, including the generated graphics and music, through a smartphone application.

Input: Graphics data, music data

Output: Preview screen

Specific behavior: The smartphone application displays the generated data and provides a preview to the user.

Step 8:

Users can export the finished content as a video file through a smartphone application.

Input: Preview screen

Output: Video file

Specific operation: The smartphone application saves the generated data as a video file and provides it to the user.

It is also possible to further combine an emotion engine that estimates the user's emotion. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

"Example 1"

One embodiment of the present invention is a story generation system that incorporates an emotion engine. This system receives story settings, such as the story's theme, setting, and characters, as input information from the user. Based on this story setting, the system then automatically creates character dialogue, scenery, and story structure. It also automatically generates foreshadowing, mystery elements, and moving scenes to make the story more interesting. It also automatically creates graphics and music that match each scene in the story. The story structure, character dialogue, video, and music enhance the dramatic effect, effectively stirring the viewer's emotions.

"Example 2"

Furthermore, the system includes an emotion engine. This emotion engine recognizes the user's emotions in real time and adjusts the story development according to those emotions. For example, if the user feels joyful about the story, the story will develop in a more cheerful manner. Conversely, if the user feels sad about the story, the story will develop in a more profound manner.

"Example 3"

The emotion engine uses information such as the user's facial expressions, tone of voice, and text input to recognize the user's emotions. For example, if the user is smiling, the emotion engine will recognize that the user is feeling happy. If the user speaks in a sad tone of voice, the emotion engine will recognize that the user is feeling sad. Furthermore, if the user inputs the text "fun," the emotion engine will recognize that the user is feeling happy. Based on this information, the emotion engine adjusts the development of the story and provides a story experience that matches the user's emotions.

The processing flow for each example form is explained below.

"Example 1"

Step 1: Receive story settings such as theme, setting, and characters as input from the user.

Step 2: Based on the received story setting, character dialogue, scenes, and story structure are automatically created.

Step 3: Automatically generate hints, mystery elements, moving scenes, and more to make your story more interesting.

Step 4: Automatically create graphics and music that match each scene in the story you've created. Step 5: Enhance the dramatic effect with the story structure, character dialogue, images, and music to effectively stir the viewer's emotions.

"Example 2"

Step 1: Launch the emotion engine and recognize the user's emotions in real time.

Step 2: Adjust the story development according to the recognized emotion. For example, if the user feels happy about the story, the story will develop in a more cheerful manner. Conversely, if the user feels sad about the story, the story will develop in a more profound manner.

"Example 3"

Step 1: The emotion engine uses information such as the user's facial expressions, tone of voice, and text input to recognize the user's emotions.

Step 2: For example, if the user is smiling, the emotion engine will recognize that the user is feeling happy. Also, if the user is speaking in a sad tone of voice, the emotion engine will recognize that the user is feeling sad.

Step 3: Furthermore, if the user enters the text "fun," the emotion engine recognizes that the user is feeling enjoyment. Based on this information, the emotion engine adjusts the development of the story and provides a narrative experience that matches the user's emotions.

(Example 1)

Next, Example 1 of Form Example 1 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 will be referred to as the "terminal."

In conventional story generation systems, when automatically generating a story based on a story setting entered by the user, the story flow, character dialogue, and scenery often lacked consistency. Furthermore, it was difficult to generate foreshadowing, mystery elements, and moving scenes to make the story more interesting, making it difficult to effectively move the viewer's emotions. Furthermore, it was difficult to automatically create graphics and music that matched each scene in the story. To address these issues, the present invention aims to provide a system that generates more consistent stories and effectively moves the viewer's emotions.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

In this invention, the server includes means for inputting story settings such as the story's theme, setting, and characters; means for analyzing the input story settings; means for searching an existing story database based on the analyzed story settings; means for generating and inputting prompt sentences to a generation AI model; means for automatically creating character dialogue, scenes, and story structure using the generation AI model; means for automatically generating foreshadowing, mystery elements, and moving scenes to make the created story more interesting; means for automatically creating graphics and music that match each scene of the created story; and means for enhancing the dramatic effect with the story structure, character dialogue, images, and music, and effectively stirring the viewer's emotions. This makes it possible to generate a consistent story based on the story settings input by the user, effectively stirring the viewer's emotions.

A "story theme" is the central subject or subject of a story.

"Setting" refers to the place and historical background in which the story unfolds.

"Characters" refers to the characters and people who appear in the story.

"Story setting" refers to the setting that includes basic elements such as the story's theme, setting, and characters.

"Input means" refers to the method or device by which the user provides the story setting, such as the story's theme, setting, and characters, to the system.

"Means for analyzing" refers to a method or device for analyzing the input story setting and extracting important keywords and phrases.

"Means for searching a database" refers to a method or device for searching relevant information from an existing narrative database based on the analyzed story setting.

A "generative AI model" is a model that uses artificial intelligence technology to generate text and stories.

A "prompt" is an instruction or question that is input into a generative AI model.

"Means for generating and inputting prompt sentences" refers to a method or device for creating prompt sentences suitable for a generative AI model and inputting them into the model.

"Dialogue" refers to the words or lines spoken by characters.

"Scene" refers to the scenery or scene depicted in the story.

"Narrative structure" refers to the structure and arrangement that determines the flow and development of a story.

"Foreshadowing" refers to elements or hints placed in the first half of a story that will have important meaning later on.

"Mystery elements" are elements that add mystery and suspense to a story.

A "moving scene" is one that strongly stirs the viewer's emotions.

"Graphics" refers to images and illustrations generated to match each scene in the story.

"Songs" refer to music and melodies generated to match each scene in the story.

"Directory effects" refer to effects that use story structure, character dialogue, images, and music to stir the viewer's emotions.

The present invention relates to a story generation system that generates a coherent story based on a story setting entered by the user, and aims to effectively stir the emotions of the viewer.

System Configuration

User input

The user enters the story's theme, setting, characters, and other story settings in text format. For example, they can provide a specific setting such as "a knight's adventure story set in medieval Europe."

Server input analysis

The server analyzes the story setting received from the user. This analysis uses natural language processing techniques, specifically Python's NLTK library and SpaCy. The server tokenizes the input text and extracts important keywords and phrases.

Search the database

The server searches an existing story database based on the extracted keywords and phrases. This database stores information on past stories and characters. The server finds similar stories and character settings and uses them as reference information.

Prompt generation for generative AI models

Based on user input and information obtained from the database, the server generates prompts to be input into the generative AI model. For example, it creates a prompt such as, "Create a knight's adventure story set in medieval Europe. The main character is the brave knight Arthur, whose mission is to defeat an evil dragon and save the kingdom. Please include the opening of the story, lines from the characters, and descriptions of the scenery."

Story generation using generative AI models

The server inputs prompt text into a generative AI model (for example, OpenAI's GPT-4), which generates the story flow, character dialogue, and scenery. Based on the user's input, the generative AI model generates text that matches the story's theme and the characters' personalities.

Applying the Emotion Engine

The server applies an emotion engine to the generated story, adding emotional elements to each scene in the story. For example, it automatically generates foreshadowing, mystery elements, and moving scenes. This emotion engine uses IBM Watson's Tone Analyzer.

Graphics and music generation

The server automatically creates graphics and music that match each scene in the story. Image generation AI is used to generate the graphics, and music generation AI is used to generate the music.

Merge the final output

The server integrates the generated story, graphics, and music and provides them to the user. The user can then view and enjoy the completed story. For example, the user can view the story through a web browser and enjoy it along with the graphics and music.

Specific examples and prompt sentence examples

Specific examples

If the user inputs the setting "A knight's adventure story set in medieval Europe," the server will generate a story like this:

The story begins: "In the vast plains of medieval Europe stood the brave knight Arthur. His mission was to slay the evil dragon and save the kingdom."

Character Quote: "Arthur: 'I swear on this sword that I will defeat the dragon!'"

Scene description: "Under the sunset, Arthur gripped his sword and renewed his resolve."

Example prompt

An example of a prompt to input to the generative AI model is as follows:

"Create a knight's adventure story set in medieval Europe. The main character is a brave knight named Arthur, whose mission is to defeat an evil dragon and save the kingdom. Please include an opening line, character dialogue, and a description of the scenery."

By inputting this prompt into a generative AI model, each element of the story is automatically generated.

The flow of the identification process in Example 1 will be explained using Figure 17.

Step 1:

The user enters the story's theme, setting, characters, and other story settings in text format. For example, they might enter "A knight's adventure story set in medieval Europe." This input is done through the system's user interface. The entered text is sent to the server.

Step 2:

The server analyzes the story setting received from the user. Specifically, it uses Python's NLTK library and SpaCy to tokenize the text and extract important keywords and phrases. For example, it extracts keywords such as "medieval Europe," "knights," and "adventure story." The results of this analysis are used in the next step.

Step 3:

The server searches an existing story database based on the extracted keywords and phrases. The database contains information on past stories and characters. The server finds similar stories and character settings and uses them as reference information. For example, it searches for past stories with a "medieval Europe" theme or stories with a "knight" as the main character. The search results are used in the next step.

Step 4:

Based on user input and information obtained from the database, the server generates a prompt to be input into the generative AI model. For example, it might create a prompt such as, "Create a knight's adventure story set in medieval Europe. The main character is the brave knight Arthur, whose mission is to defeat an evil dragon and save the kingdom. Please include the opening of the story, lines from the characters, and a description of the scenery." This prompt will be used in the next step.

Step 5:

The server inputs prompt text into a generative AI model (for example, OpenAI's GPT-4), which generates the story flow, character dialogue, and scenery. Based on the user's input, the generative AI model generates text that matches the story's theme and the characters' personalities. For example, the generative AI model generates the following story:

The story begins: "In the vast plains of medieval Europe stood the brave knight Arthur. His mission was to slay the evil dragon and save the kingdom."

Character Quote: "Arthur: 'I swear on this sword that I will defeat the dragon!'"

Scene description: "Under the sunset, Arthur gripped his sword and renewed his resolve."

Step 6:

The server applies an emotion engine to the generated story, adding emotional elements to each scene in the story. For example, it automatically generates foreshadowing, mystery elements, and moving scenes. This emotion engine uses IBM Watson's Tone Analyzer. The emotion engine analyzes the generated text and adds appropriate emotional elements.

Step 7:

The server automatically creates graphics and music that match each scene in the story. Image generation AI is used to generate the graphics, and music generation AI is used to generate the music. For example, to match the opening scene of the story, it generates graphics depicting the scenery of medieval Europe and music that evokes a sense of adventure.

Step 8:

The server integrates the generated story, graphics, and music and provides them to the user. The user can then view and enjoy the completed story. For example, the user can view the story through a web browser and enjoy it along with the graphics and music.

(Application Example 1)

Next, we will explain Application Example 1 of Form Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 will be referred to as the "terminal."

Conventional story generation systems could generate stories based on settings entered by the user, but they were unable to visualize and sound out the generated stories in real time and allow users to experience them in a virtual reality space. It was also difficult to automatically generate graphics and music that matched each scene in the story, enhancing the dramatic effects that effectively stirred the viewer's emotions. This limited the user experience and led to a lack of immersion in the story.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

In this invention, the server includes: means for inputting story settings such as the story theme, setting, and characters; means for automatically creating character dialogue, scenery, and story structure based on the input story settings; means for automatically generating foreshadowing, mystery elements, and moving scenes to make the created story more interesting; means for automatically producing graphics and music that match each scene of the created story; means for enhancing the dramatic effect with the story structure, character dialogue, images, and music, and effectively stirring the viewer's emotions; means for generating a story using a generative AI model based on the story settings input by the user; means for visualizing and audibly converting the generated story in real time; and means for the user to experience the generated story in a virtual reality space using a head-mounted display. This allows the user to visualize and audibly convert the generated story in real time and experience it in a virtual reality space.

A "story theme" is the central subject or theme of a story, and determines the overall direction and atmosphere of the story.

The "setting" refers to the place and historical background in which the story unfolds, and is an element that shapes the setting and atmosphere of the story.

"Characters" are characters who act in a story and play an important role in the progression and development of the story.

"Story setting" refers to the setting that includes basic elements such as the story's theme, setting, and characters, and forms the foundation of the story.

A "generative AI model" is a model that uses artificial intelligence technology to generate text and data, and is used to automatically create story structures and dialogue.

"Foreshadowing" refers to elements or hints in a story that give a glimpse of future developments, and is a technique used to make a story more interesting.

"Mystery elements" are elements in a story that present a mystery or question, intended to attract the interest of readers or viewers.

A "moving scene" is a scene in a story that stirs strong emotions and leaves a deep impression on the viewer's mind.

"Graphics" refers to visual images and footage that visually represent scenes and characters in a story.

"Song" refers to music, used to create scenes and atmosphere in a story.

"Visualization" refers to the visual representation of the story content, allowing users to enjoy the story visually.

"Audio" refers to the expression of the story content in audio, allowing users to enjoy the story by listening to it.

A "virtual reality space" is a virtual space created using computer technology that allows users to experience reality.

A "head-mounted display" is a display device worn on the head, used to allow users to experience virtual reality spaces.

A system for implementing this invention has the following configuration: First, a user uses a smartphone or head-mounted display to input story settings such as the story's theme, setting, and characters. This input is done in text format.

Next, the server uses a generative AI model (e.g., GPT-4) to automatically create character dialogue, scenes, and a story structure based on the story setting entered by the user. The generative AI model references an existing story database to generate dialogue and scenes that match the story flow, character personalities, and story theme.

In addition, the server uses an emotion engine to add hints, mystery elements, and moving scenes to the stories created, making them more interesting and moving.

The server then uses a graphics generation engine and a music generation engine to automatically create graphics and music that match each scene in the story. This enhances the visual and auditory effects of the story, effectively stirring the emotions of the viewer.

Finally, the generated story is visualized and audible in real time, and the user can experience the story in a virtual reality space using a head-mounted display. This allows the user to become immersed in the story and enjoy a deeper experience.

As a concrete example, if a user inputs "a knight's adventure story set in medieval Europe," the following prompt sentence will be input to the generative AI model:

Example prompt:

Theme: Medieval Europe

Setting: Knight's Adventure

Characters: Brave knight, wizard, dragon

Story setting: A brave knight and a wizard embark on an adventure to slay a dragon.

Based on this prompt, a generative AI model generates a story, an emotion engine adds emotional elements, and a graphics generation engine and music generation engine generate graphics and music. Users can experience this story in real time through a head-mounted display.

The flow of the specific processing in Application Example 1 is explained using Figure 18.

Step 1:

Users use a smartphone or head-mounted display to input story settings such as the story's theme, setting, and characters in text format. The input data is sent to the server.

Input: Story setting, including theme, setting, and characters.

Output: Story setting data sent to the server

Step 2:

Based on the received story setting data, the server uses a generative AI model (e.g., GPT-4) to automatically create the story structure, character dialogue, and scenery. The generative AI model references an existing story database to generate text that matches the story flow and character personalities.

Input: Story setting data

Output: Generated story structure, dialogue, and scenery

Step 3:

The server uses an emotion engine to add emotional elements such as foreshadowing, mystery elements, and moving scenes to the generated story. The emotion engine automatically inserts emotional elements appropriate to each scene in the story.

Input: Generated story structure, dialogue, and scenery

Output: A story with added emotional content

Step 4:

The server uses a graphics generation engine to automatically create graphics that match each scene in the story. The graphics generation engine takes into account the atmosphere of the scene and the emotional state of the characters to generate appropriate visuals.

Input: A story with an emotional element

Output: Graphics that match each scene

Step 5:

The server uses a music generation engine to automatically create music that matches each scene in the story. The music generation engine takes into account the atmosphere of the scene and the emotional state of the characters to generate appropriate music.

Input: A story with an emotional element

Output: Songs that match each scene

Step 6:

The server integrates the generated story, graphics, and music, visualizing and audiovisualizing them in real time, allowing users to experience the story visually and aurally.

Input: Generated story, graphics, music

Output: Real-time visual and audio narrative

Step 7:

Using a head-mounted display, users experience a story generated in a virtual reality space, immersing themselves in the story and providing a deeper experience.

Input: Real-time visual and audio narrative

Output: Narrative experience in a virtual reality space

(Example 2)

Next, Example 2 of Form Example 2 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 will be referred to as the "terminal."

While conventional story generation systems can automatically generate a story based on the story's theme and character settings, they have difficulty automatically generating foreshadowing, mystery elements, and moving scenes that make the story more interesting. Furthermore, they lack the functionality to adjust the story's development in real time based on the user's emotions, making it impossible to provide a narrative experience that is in tune with the user's emotions. Furthermore, it is difficult to automatically create graphics and music that match each scene in the story. A new system that can solve these issues is needed.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

In this invention, the server includes means for inputting story settings such as the story theme, setting, and characters; means for automatically creating character dialogue, scenery, and story structure based on the input story settings; means for automatically generating foreshadowing, mystery elements, and moving scenes to make the created story more interesting; means for recognizing the user's emotions in real time and adjusting the story development in accordance with those emotions; means for automatically creating graphics and music that match each scene of the created story; and means for enhancing the dramatic effect with the story structure, character dialogue, images, and music to effectively stir the viewer's emotions. This makes it possible not only to automatically generate a story based on the story theme and character settings, but also to automatically generate foreshadowing, mystery elements, and moving scenes, and provide a story development that matches the user's emotions.

The "story theme" indicates the overall subject or direction of the story.

"Setting" refers to the place or environment in which the story unfolds.

"Characters" refers to the characters who take action in the story.

"Story setting" refers to the setting, including basic elements such as the story's theme, setting, and characters.

"Dialogue" refers to the words or lines spoken by characters.

"Scene" refers to the scenery or scene depicted in the story.

"Story structure" refers to the overall flow and structure of a story, including its beginning, middle, and end.

"Foreshadowing" refers to elements that are laid out in advance in a way that will have important meaning later in the story.

"Mystery elements" refer to elements that attract the interest of readers or viewers by including mysteries or unknown points in a story.

A "moving scene" is one that strongly stirs the emotions of the reader or viewer.

"User emotions" refers to feelings such as joy, sadness, and surprise felt by users as they experience the story.

"Real-time recognition" refers to detecting the user's emotions instantly and on the spot.

"Adjusting the story development" refers to changing the progress and content of the story in response to the user's emotions.

"Graphics" refers to images and illustrations that are displayed in accordance with story scenes.

"Song" refers to the music that plays along with the story scenes.

"Directory effects" refer to the effects of using the visual and auditory elements of a story to stimulate the emotions and interest of readers or viewers.

This invention is a system that automatically generates a story based on the story's theme and character settings, and also automatically generates foreshadowing, mystery elements, and moving scenes. It also has the ability to recognize the user's emotions in real time and adjust the story's development accordingly.

Hardware and software used

The server generates story elements using a generative AI model (e.g., GPT-4). Based on information received from the user about the story's theme, the characters' personalities, and the story's progression, the server inputs prompt sentences into the generative AI model to generate story elements.

The device sends the user's input information to the server and displays the story elements received from the server to the user. The device also uses emotion recognition software to collect emotional data from the user's facial expressions and voice and send it to the server.

Users enter information about the story's theme, the characters' personalities, and the story's progression into their device and enjoy the story. The device recognizes the user's emotions in real time and reflects them in the development of the story.

Data processing and calculation

The server performs the following data processing and calculations:

Generate appropriate foreshadowing based on the story's theme and the characters' personalities.

Mystery elements are scattered throughout the story as it progresses.

Analyze the characters' emotions and situations to create moving scenes.

The device collects emotional data from the user's facial expressions and voice and sends it to the server. The server analyzes this emotional data and adjusts the story's development.

Specific examples

For example, to generate a hint that a person the knight meets during his adventure will play an important role later in the story, the following prompt sentence can be input into the generative AI model:

Example prompt:

"Please create a plot twist that will lead to the mysterious old man the knight meets during his adventure playing an important role later in the story."

Also, if the user is feeling happy about the story, a prompt to make the story even more upbeat might look like this:

Example prompt:

"Users are enjoying this, so please make the story even more upbeat. For example, add a scene where the knight finds a new companion."

In this way, the server utilizes a generative AI model and emotion engine to automatically generate a story development that corresponds to the user's emotions.

The flow of the identification process in Example 2 will be explained using Figure 19.

Step 1:

The user enters information about the story's theme, the characters' personalities, and the story's progression.

Input: Story theme (e.g., adventure), character traits (e.g., bravery), and information about the story's progression.

Output: User input information

Specific operation: The user enters the theme "adventure" into the device's input form and sets the character's personality to "brave."

Step 2:

The device sends the user's input information to the server.

Input: User input information

Output: User input information sent to the server

Specific operation: The device sends the information "Theme: Adventure, Personality: Brave" to the server.

Step 3:

The server generates story elements using generative AI models.

Input: User input information

Output: Generated story elements

Specific operation: The server inputs the prompt "Generate a story with an adventure theme and a brave knight as the protagonist" into the generation AI model, and generates the elements of the story.

Step 4:

The server sends the generated story elements to the device.

Input: Generated story elements

Output: Story elements sent to the device

Specific operation: The server sends story elements generated by the server (e.g., "The mysterious old man the knight meets during his adventure") to the device.

Step 5:

The device displays the story to the user.

Input: Story elements received from the server

Output: The story displayed to the user

Specific operation: The device displays a scene to the user in which a knight meets a mysterious old man during his adventure.

Step 6:

Recognize user emotions in real time.

Input: User's facial expressions and voice

Output: Recognized user emotion data

Specific operation: When the user smiles, the device captures the user's facial expression with its camera and recognizes that the user is feeling happy.

Step 7:

The device sends the user's emotional data to the server.

Input: Recognized user emotion data

Output: Emotion data sent to the server

Specific operation: The device sends emotional data indicating "the user is feeling happy" to the server.

Step 8:

The server analyzes the emotional data and adjusts the story development.

Input: User emotion data

Output: Coordinated story development

Specific operation: The server analyzes the emotional data and inputs prompt sentences into the generative AI model to "make the story more upbeat."

Step 9:

The server will send the adjusted story to your device.

Input: Adjusted story development

Output: The adjusted story sent to the terminal.

Specific operation: The server generates a story with an upbeat plot (e.g., "A scene where the knight finds a new companion") and sends it to the device.

Step 10:

The device will then display the adjusted story to the user.

Input: Adjusted story received from the server

Output: The adjusted story displayed to the user.

Specific operation: The device displays a scene to the user in which a knight finds a new companion.

(Application Example 2)

Next, we will explain Application Example 2 of Form Example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 will be referred to as the "terminal."

While conventional story generation systems could automatically generate a story based on the story's theme and character settings, they were unable to adjust the story's development in real time in response to the user's emotions. They also lacked the functionality to automatically generate plot twists, mystery elements, and moving scenes to make a story more interesting. This made it difficult to provide an interactive story experience that responded to the user's emotions.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

In this invention, the server includes: means for inputting story settings such as the story theme, setting, and characters; means for automatically creating character dialogue, scenery, and story structure based on the input story settings; means for automatically generating foreshadowing, mystery elements, and moving scenes to make the created story more interesting; means for automatically creating graphics and music that match each scene of the created story; means for enhancing the dramatic effect with the story structure, character dialogue, images, and music, and effectively stirring the viewer's emotions; means for recognizing the user's emotions in real time and adjusting the story development in response to those emotions; and means for generating story prompts using a generative AI model and automatically generating a story. This makes it possible to provide an interactive story experience that responds to the user's emotions.

"Story theme" refers to the central subject or message of a story.

"Setting" refers to the place or environment in which the story unfolds.

"Characters" refers to the characters or people who play a role in the story.

"Story setting" refers to the setting, including basic elements such as the story's theme, setting, and characters.

"Dialogue" refers to the words or lines spoken by characters.

"Scene" refers to the scenery or scene depicted in the story.

"Story structure" refers to the overall flow and structure of a story, including its beginning, middle, and end.

"Foreshadowing" refers to elements or hints placed in the first half of a story that will have important meaning later on.

"Mystery elements" refer to elements in a story that raise questions or raise doubts.

A "moving scene" is one that evokes strong emotions in the viewer or reader.

"Graphics" refers to images and illustrations that are displayed in accordance with story scenes.

"Song" refers to the music or soundtrack that plays along with the story scenes.

"Stage effects" refer to visual and auditory techniques used to make story scenes appear more effective.

"Viewer sentiment" refers to the emotions and feelings of the people watching the story.

"Recognizing emotions in real time" refers to analyzing the user's facial expressions, tone of voice, etc. to grasp their emotions at that moment.

"Adjusting the development of the story" refers to changing the progress and content of the story in response to the user's emotions.

"Generative AI model" refers to a model that uses artificial intelligence to generate text or stories.

A "prompt" refers to an instruction or keyword that is input into a generative AI model.

A system for implementing this invention has the following configuration: First, the server provides a means for inputting story settings such as the story's theme, setting, and characters. By inputting these settings, the user determines the basic elements of the story.

Next, the server has a means to automatically create character dialogue, scenery, and story structure based on the input story setting. This means uses a generative AI model to generate story prompts and automatically generate the story.

Furthermore, the server has the means to automatically generate hints, mystery elements, moving scenes, and more to make the stories you create more interesting. This helps to add depth and interest to the story.

The server also has the means to automatically create graphics and music that match each scene in the story. This enhances the visual and auditory effects of the story. Specifically, the graphics and music are created taking into account the atmosphere of the scene and the emotional state of the characters.

Furthermore, the server has the means to enhance the dramatic effect through the story structure, character dialogue, visuals, and music, effectively stirring the viewer's emotions. This allows users to be more emotionally drawn into the story.

What's particularly important is that the server has a means of recognizing the user's emotions in real time and adjusting the story development accordingly. It uses the smartphone's camera and microphone to analyze the user's facial expressions and tone of voice to understand their emotions. Based on this emotional data, it generates prompt sentences for the story and adjusts the story development accordingly.

For example, if a user is reading a story and the camera captures their facial expressions and recognizes that they are smiling, the story will develop in a lighthearted and cheerful way. Conversely, if the user is sad, the story will develop in a more profound way.

Examples of prompt statements include:

"The protagonist continues his adventure with a happy feeling."

"The protagonist faces difficulties, but does not lose hope."

"The protagonist takes on a new mystery."

In this way, it is possible to provide an interactive narrative experience that responds to the user's emotions.

The flow of the specific processing in Application Example 2 is explained using Figure 20.

Step 1:

The user enters story settings such as theme, setting, and characters.

Input: Setting information such as the story's theme, setting, and characters entered by the user.

Output: Story setting data.

Specific operation: The user uses a smartphone application to input the basic setting of the story, including information about the story's theme, location, and main characters. The server receives this setting information and stores it as story setting data.

Step 2:

The server automatically creates character dialogue, scenes, and story structure based on the story settings entered.

Input: Story setting data.

Output: Story dialogue, scenery, and composition data.

Specific operation: The server uses the generative AI model to generate character dialogue, scenery, and story structure based on story setting data. The generated data is saved as elements necessary for the story's progression.

Step 3:

The server automatically generates hints, mystery elements, moving scenes, and more to make the stories it creates more interesting.

Input: Story lines, scenery, and composition data.

Output: Foreshadowing, mystery elements, and moving scene data.

Specifically, how it works: The server uses a generative AI model to generate plot twists, mystery elements, and moving scenes based on the dialogue, scenery, and composition data of the story. This adds depth and intrigue to the story.

Step 4:

The server automatically creates graphics and music that match each scene in the story.

Input: Foreshadowing, mystery elements, and moving scene data.

Output: Graphics data, music data.

Specific operation: The server automatically creates graphics and music, taking into account the atmosphere of the scene and the emotional state of the characters. This is done using image generation models and music generation models. The generated graphics and music are then applied to each scene in the story.

Step 5:

The server recognizes the user's emotions in real time and adjusts the story development accordingly.

Input: User's facial expression data, tone of voice data.

Output: Tailored narrative prompt.

Specific operation: The smartphone's camera and microphone are used to capture the user's facial expressions and tone of voice. The server analyzes this data and recognizes the user's emotions in real time. Based on the recognized emotions, it generates story prompts and adjusts the story's development.

Step 6:

The server uses a generative AI model to generate story prompts and automatically generate the story.

Input: Tailored story prompt.

Output: The generated story.

Specific operation: The server uses a generative AI model to generate a story based on the adjusted prompt. The generated story is used to provide an interactive experience that responds to the user's emotions.

(Example 3)

Next, Example 3 of Form Example 3 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 will be referred to as the "terminal."

With conventional story generation systems, it was difficult to automatically create graphics and music that matched each scene in the story, making it difficult to effectively sway the viewer's emotions. Furthermore, they lacked the ability to recognize the user's emotions in real time and adjust the story's development accordingly, making it difficult to provide the user with a consistent emotional experience. This resulted in a decline in the quality of the story experience and low user satisfaction.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Example 3 is realized by the following means.

In this invention, the server includes means for inputting story settings such as the story theme, setting, and characters; means for automatically creating character dialogue, scenery, and story structure based on the input story settings; means for automatically generating foreshadowing, mystery elements, and moving scenes to make the created story more interesting; means for automatically creating graphics and music that match each scene of the created story; means for using information such as the user's facial expressions, tone of voice, and text input to recognize the user's emotions; means for adjusting the story development in accordance with the user's emotions; and means for enhancing the dramatic effects of the story structure, character dialogue, images, and music to effectively stir the viewer's emotions. This makes it possible to automatically create graphics and music that match each scene of the story and provide a story development that matches the user's emotions.

"Story theme" refers to the central subject or subject of a story.

"Setting" refers to the place or environment in which the story unfolds.

"Characters" refers to the characters that appear in the story.

"Story setting" refers to the setting, including basic elements such as the story's theme, setting, and characters.

"Dialogue" refers to the words spoken by characters.

"Scene" refers to the scenery or scene depicted in the story.

"Story structure" refers to the overall flow of a story, including its beginning, middle, and end.

"Foreshadowing" refers to elements that provide advance hints at information or events that will become important later in the story.

"Mystery elements" refer to elements that add mystery or suspense to a story.

A "moving scene" refers to a scene that strongly stirs the viewer's emotions.

"Graphics" refers to images and illustrations generated to match story scenes.

"Song" refers to music generated to match the story scenes.

"User emotion" refers to the emotional state the user is feeling, such as joy, sadness, or surprise.

"Facial expression" refers to emotions expressed through the movement of the user's facial muscles.

"Tone of voice" refers to the pitch, strength, and intonation of the user's voice.

"Text input" refers to text information entered by a user using a keyboard or touch panel.

"Story development" refers to the process and flow of the story.

"Directory effects" refer to the impact that the visual and auditory elements of a story have on the viewer.

"Viewer sentiment" refers to the emotions and feelings of the people watching the story.

This invention is a system that inputs story settings such as the story's theme, setting, and characters, automatically generates a story based on the input story settings, and further adjusts the story's development according to the user's emotions. This system is implemented using the following hardware and software.

Hardware and software used

Hardware: High-performance servers (e.g., servers with GPUs) and user devices (e.g., PCs or smartphones with cameras and microphones)

Software: Generative AI models (e.g., GPT-4, image generation AI, music generation AI), emotion recognition software (e.g., OpenCV, voice analysis libraries)

Program processing explanation

1. Story input and analysis

The user uses their device to input story settings such as the story's theme, setting, and characters. For example, they might input "a scene where a hero fights a dragon." The server receives this input and uses a generative AI model (e.g., GPT-4) to analyze each scene in the story and extract the scene's atmosphere and the characters' emotional states.

2. Graphics and Music Generation

Based on the analysis results, the server uses image generation AI to generate graphics that match the scene. For example, it could generate an impressive graphic of a hero fighting a dragon. Similarly, it uses music generation AI to generate music that suits the scene. For example, it could generate music that heightens the tension in a battle scene.

3. User Emotion Recognition

The device uses a camera and microphone to capture the user's facial expressions and tone of voice. For example, if the user is smiling, the device sends that information to the server. The server then uses emotion recognition software (e.g., OpenCV, voice analysis library) to analyze the user's emotions.

4. Story Development Adjustments

The server uses an emotion engine to adjust the story development based on the user's emotional information. For example, if the user is smiling, the story development will be adjusted to a more exciting direction.

Specific examples and prompt sentence examples

Example:

The user enters "a scene in which a hero fights a dragon."

The server analyzes the scene as a battle scene and uses image generation AI to generate powerful graphics of the dragon and hero.

Music generation AI is used to generate music that heightens tension.

If the user is smiling, the emotion engine recognizes that the user is having fun and adjusts the story development in a more exciting direction.

Example prompt:

"Generate graphics for a scene in which a hero fights a dragon."

"Generate music that matches an emotional scene."

"If the user is smiling, adjust the story development to make it more enjoyable."

This system makes it possible to automatically create graphics and music that match each scene in a story, and to provide a story development that responds to the user's emotions. This effectively stirs the viewer's emotions and improves the quality of the story experience. The flow of the specific processing in Example 3 is explained using Figure 21.

Step 1:

The user enters the text of the story.

The user enters "A scene in which a hero fights a dragon" into the text input field on the device and clicks the send button. The entered text is sent from the device to the server.

Step 2:

The server analyzes the story text.

The server inputs the received text into a generative AI model (e.g., GPT-4) and analyzes each scene in the story. As a result of the analysis, the atmosphere of the scene and the emotional state of the characters are extracted. For example, "battle scene" and "tension" are extracted. The analysis results are stored in an internal database.

Step 3:

The server generates graphics that match the scene.

Based on the analysis results, the server uses image generation AI (e.g., DALL-E) to generate graphics that match the scene. For example, enter "a scene of a hero fighting a dragon" as a prompt and obtain the generated graphic. The generated graphic is then sent to the user's device.

Step 4:

The server generates music that matches the scene.

The server uses music generation AI to generate music appropriate for the scene. For example, a user can enter "music that will increase tension in a battle scene" as a prompt and obtain the generated music. The generated music is then sent to the user's device.

Step 5:

The device recognizes the user's emotions.

The device uses a camera to capture the user's facial expressions and a microphone to record the tone of voice. This data is sent to the server in real time. For example, if the user is smiling, that information is sent to the server.

Step 6:

The server adjusts the story development based on the user's emotions.

The server analyzes the received emotional data using emotion recognition software (e.g., OpenCV, voice analysis library) to recognize the user's emotions. For example, if it recognizes that the user is smiling, it uses the emotion engine to adjust the story development in an exciting direction. A new scene is generated and sent to the user's device.

(Application Example 3)

Next, we will explain Application Example 3 of Form Example 3. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 will be referred to as the "terminal."

Conventional story generation systems have the problem of not adjusting the story based on the user's emotions, resulting in a uniform user experience. Furthermore, because the story scenes and music are fixed, they are unable to provide dynamic changes that respond to the user's emotions. This makes it difficult to provide a story experience that deeply appeals to the user's emotions.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 3 is realized by the following means.

In this invention, the server includes means for inputting story settings such as the story theme, setting, and characters; means for automatically creating character dialogue, scenery, and story structure based on the input story settings; means for automatically generating foreshadowing, mystery elements, and moving scenes to make the created story more interesting; means for automatically creating graphics and music that match each scene of the created story; means for enhancing the dramatic effect with the story structure, character dialogue, images, and music, and effectively stirring the viewer's emotions; means for recognizing the user's emotions and adjusting story scenes and music in accordance with the recognized emotions; and means for recognizing emotions using the user's facial expressions, tone of voice, text input, etc. This enables a dynamic story experience that matches the user's emotions.

A "story theme" refers to the central idea or message of a story.

"Setting" refers to the place or environment in which the story unfolds.

"Characters" refers to the characters that appear in the story.

"Story setting" refers to the setting, including basic elements such as the story's theme, setting, and characters.

"Dialogue" refers to the words spoken by characters.

"Scene" refers to the scenery or scene depicted in the story.

"Story structure" refers to the overall flow of a story, including its beginning, middle, and end.

"Foreshadowing" refers to an element in the first half of a story that becomes important later in the story.

"Mystery elements" refer to elements that add mystery and suspense to a story.

A "moving scene" is one that strongly stirs the viewer's emotions.

"Graphics" refers to images and footage that visually represent scenes from a story.

"Song" refers to the music that plays along with the story scenes.

"Directional effects" refer to effects that influence the viewer's emotions through story scenes and music.

"User emotion" refers to the emotional state the user is feeling, such as joy, sadness, or surprise.

"Means for recognizing emotions" refers to technology that determines a user's emotions using their facial expressions, tone of voice, text input, etc.

"Means for adjusting story scenes and music" refers to technology that changes story scenes and music according to the recognized user emotions.

A system for implementing this invention can dynamically adjust story scenes and music according to the user's emotions. A specific embodiment of this system is described below.

System Configuration

The system includes the following major hardware and software components:

Hardware: Smartphone camera

Software: OpenCV (face detection), Keras (emotion recognition model), music playback software

Program processing

The server first captures the user's facial expression using the smartphone camera. The captured video undergoes face detection using OpenCV. Next, it uses the Keras emotion recognition model to recognize the user's emotion from the detected face.

Based on the recognized emotion, the server adjusts the story scenes and music. For example, if the user is smiling, the story scenes will change to brighter, more cheerful ones, and the music will also become more upbeat. Conversely, if the user has a sad expression, the story will transition to a more moving scene, and the music will also change to something more moving.

Specific examples

As a concrete example, consider the case where a user smiles while enjoying a story. In this case, the server would process the following:

1. Capture the user's facial expressions using the smartphone camera.

2. Detect faces using OpenCV.

3. The user's emotion is recognized as "happiness" using the Keras emotion recognition model.

4. Based on the recognized emotion, the story scene is changed to a brighter, more cheerful one and upbeat music is played.

Example prompt

An example of a prompt to input to the generative AI model is as follows:

If the user smiles, change the story scene to something bright and cheerful and play upbeat music.

In this way, it is possible to provide a dynamic narrative experience that responds to the user's emotions. This system allows users to become more immersed in the story and have an experience that appeals to their emotions.

The flow of the specific processing in Application Example 3 will be explained using Figure 22.

Step 1:

The server uses the smartphone's camera to capture the user's facial expressions. The input is the camera image, and the output is the captured image data. Specifically, the camera acquires the image in real time and captures the image frame by frame.

Step 2:

The server uses OpenCV to detect faces from the captured image data. The input is the captured image data, and the output is the location information of the detected face. Specifically, OpenCV's face detection algorithm identifies the face in the image and obtains its coordinates.

Step 3:

The server uses a Keras emotion recognition model to recognize the user's emotion from the detected face. The input is face position information and image data, and the output is the recognized emotion label. Specifically, the face area is extracted, preprocessed, and then input into the emotion recognition model to classify the emotion.

Step 4:

The server adjusts the story scenes and music based on the recognized emotions. The input is the recognized emotion label, and the output is the adjusted scenes and music. Specific operations include selecting scenes and music that correspond to the emotion label and changing the progression of the story.

Step 5:

The server sends the adjusted scene and music to the user's device. The input is the adjusted scene and music, and the output is the story scene displayed on the user's device and the music played. Specifically, the server encodes the scene and music data and sends it to the user's device over the network.

Step 6:

The user's device plays the received scenes and music and provides them to the user. The input is the scene and music data received from the server, and the output is the video displayed to the user and the music played. Specifically, the device decodes the data, displays it on the screen, and plays the music from the speakers.

The specific processing unit 290 transmits the results of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the results of the specific processing. The microphone 238 acquires audio indicating the user input regarding the results of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

Data generation model 58 is what is known as generative AI (artificial intelligence). An example of data generation model 58 is generative AI such as ChatGPT (Internet search <URL: https://openai.com/blog/chatgpt>). Data generation model 58 is obtained by performing deep learning on a neural network. A prompt containing an instruction is input to data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. Data generation model 58 performs inference on the input inference data in accordance with the instructions indicated by the prompt, and outputs the inference results in the form of data such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and/or summarization.

Another example of generative AI is Gemini (Internet search <URL: https://gemini.google.com/?hl=ja>).

In the above embodiment, an example was given in which the specific processing was performed by the data processing device 12, but the technology disclosed herein is not limited to this, and the specific processing may also be performed by the robot 414.

The emotion identification model 59, which serves as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see Figure 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

Figure 9 shows an emotion map 400 on which multiple emotions are mapped. In emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both emotions and mental states. Emotions that are generally generated from reactions that occur in the brain are arranged on the left side of the concentric circles. Emotions that are generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions that are generally generated from reactions that occur in the brain and are induced by situational judgment are arranged above and below the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the top side of the concentric circles, and the emotion of "discomfort" is arranged on the bottom side. In this way, emotion map 400 maps multiple emotions based on the structure by which emotions are generated, with emotions that tend to occur simultaneously being mapped close together.

These emotions are distributed in the 3 o'clock direction on emotion map 400, and usually fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness takes precedence over internal sensations, resulting in a calm impression.

The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further out you go on emotion map 400, the more visible (expressed in behavior) the emotion becomes.

Here, human emotions are based on various balances such as posture and blood sugar levels, and when these balances deviate from the ideal, it indicates discomfort, and when they approach the ideal, it indicates pleasure. Emotions can also be created for robots, cars, motorcycles, etc., based on various balances such as posture and remaining battery life, so that when these balances deviate from the ideal, it indicates discomfort, and when they approach the ideal, it indicates pleasure. Emotion maps may be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on speech emotion recognition and emotional brain physiological signal analysis systems, Tokushima University, doctoral dissertation: https://ci.nii.ac.jp/naid/500000375379). The left half of the emotion map is lined with emotions belonging to an area called "reaction," where sensation is dominant. The right half of the emotion map is lined with emotions belonging to an area called "situation," where situational awareness is dominant.

The emotion map defines two emotions that encourage learning. One is the negative emotion around the middle of "repentance" or "reflection" on the situation side. In other words, this is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the positive emotion around "desire" on the response side. In other words, this is when the robot experiences positive feelings such as "I want more" or "I want to know more."

The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are close to each other have similar values, as in the emotion map 900 shown in Figure 10. Figure 10 shows an example in which multiple emotions, such as "relieved," "calm," and "reassuring," have similar emotion values.

In the above embodiment, an example was given in which a specific process is performed by a single computer 22, but the technology disclosed herein is not limited to this, and distributed processing of the specific process may also be performed by multiple computers, including computer 22.

In the above embodiment, an example was described in which the specific processing program 56 is stored in the storage 32, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

It is not necessary to store the entire specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entire specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

The following types of processors can be used as hardware resources for executing specific processes. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource for executing specific processes by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which are processors with a circuit configuration designed specifically for executing specific processes. All processors have built-in or connected memory, and all use the memory to execute specific processes.

The hardware resource that executes the specific processing may be composed of one of these various processors, or may be composed of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). The hardware resource that executes the specific processing may also be a single processor.

As an example of a configuration using a single processor, first, there is a configuration in which one processor is configured using a combination of one or more CPUs and software, and this processor functions as a hardware resource that executes specific processing. Second, there is a configuration in which a processor is used to realize the functions of an entire system, including multiple hardware resources that execute specific processing, on a single IC chip, as typified by SoC (System-on-a-chip). In this way, specific processing is realized using one or more of the various processors listed above as hardware resources.

More specifically, the hardware structure of these various processors can be an electrical circuit that combines circuit elements such as semiconductor devices. Furthermore, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps can be added, or the processing order can be rearranged, all within the scope of the spirit of the invention.

The above-described written content and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above explanation of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or substitutions may be made to the above-described written content and illustrations, as long as they do not deviate from the spirit of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described written content and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

All publications, patent applications, and technical standards mentioned in this specification are incorporated by reference herein to the same extent as if each individual publication, patent application, and technical standard was specifically and individually indicated to be incorporated by reference.

The following is further disclosed regarding the above embodiments.

(Claim 1)
It is a system that allows you to input story settings such as the theme, setting, and characters of a story, and automatically creates lines for characters, scenes, and a story structure based on the input story settings. It also automatically generates plot twists, mystery elements, and moving scenes to make the story more interesting.

The system includes a means to automatically create graphics and music that match each scene in the story, and a means to enhance the dramatic effect through the story structure, character dialogue, images and music, effectively stirring the emotions of the viewer.

(Claim 2)
2. The system according to claim 1, wherein the means for automatically creating a story structure creates a story based on a three-act structure consisting of a beginning, middle, and end.

(Claim 3)
2. The system according to claim 1, wherein the means for automatically creating graphics and music that match each scene of the story creates the graphics and music taking into consideration the atmosphere of the scene and the emotional state of the characters.

(Claim 4)
A system including: means for inputting story settings such as the theme, setting, and characters of the story; means for automatically creating lines for characters, scenes, and a story structure based on the input story settings; means for automatically generating hints, mystery elements, and moving scenes to make the created story more interesting; means for automatically creating graphics and music that match each scene of the created story; means for enhancing the production effect with the story structure, character conversations, images, and music, and effectively stirring the emotions of viewers; and an emotion engine that recognizes the user's emotions.

(Claim 5)
The system of claim 4 , wherein the emotion engine recognizes a user's emotions in real time and adjusts the story development according to the emotions.

(Claim 6)
The system of claim 4 , wherein the emotion engine uses information such as a user's facial expression, tone of voice, and text input to recognize the user's emotion.

"Example 1"
(Claim 1)
A way to input story settings such as the theme, setting, and characters of the story,
Based on the input story setting, it automatically creates the characters' lines, scenes, and story structure.
It is a way to automatically generate plot twists, mystery elements, and moving scenes to make the story more interesting,
A way to automatically create graphics and music that match each scene in the story you've created,
The story structure, the characters' conversations, the images and music enhance the production effect and effectively shake the viewers' emotions.
means for receiving and analyzing input from a user using natural language processing techniques;
A means for generating a prompt sentence to be input to the generative AI model based on the analysis result;
A means to automatically generate story lines, scenes, and structures using generative AI models,
A system that includes a means for formatting the generated narrative and providing it to the user.

(Claim 2)
2. The system according to claim 1, wherein the means for automatically creating a story structure creates a story based on a three-act structure consisting of a beginning, middle, and end.

(Claim 3)
2. The system according to claim 1, wherein the means for automatically creating graphics and music that match each scene of the story creates the graphics and music taking into consideration the atmosphere of the scene and the emotional state of the characters.

"Application Example 1"
(Claim 1)
A way to input story settings such as the theme, setting, and characters of the story,
Based on the input story setting, it automatically creates the characters' lines, scenes, and story structure.
It is a way to automatically generate plot twists, mystery elements, and moving scenes to make the story more interesting,
A way to automatically create graphics and music that match each scene in the story you've created,
The story structure, the characters' conversations, the images and music enhance the production effect and effectively shake the viewers' emotions.
A means to generate prompt sentences using a generative AI model and automatically generate a story;
The system includes means to preview, save and share the generated narratives.

(Claim 2)
2. The system according to claim 1, wherein the means for automatically creating a story structure creates a story based on a three-act structure consisting of a beginning, middle, and end.

(Claim 3)
2. The system according to claim 1, wherein the means for automatically creating graphics and music that match each scene of the story creates the graphics and music taking into consideration the atmosphere of the scene and the emotional state of the characters.

"Example 2"
(Claim 1)
A way to input story settings such as the theme, setting, and characters of the story,
Based on the input story setting, it automatically creates the characters' lines, scenes, and story structure.
It is a way to automatically generate plot twists, mystery elements, and moving scenes to make the story more interesting,
A means for organizing the generated story elements and providing them to the user;
A system that includes means to enhance the production effect through story structure, character dialogue, images and music, and effectively stir the viewer's emotions.

(Claim 2)
2. The system according to claim 1, wherein the means for automatically creating a story structure creates a story based on a three-act structure consisting of a beginning, middle, and end.

(Claim 3)
2. The system according to claim 1, wherein the means for automatically creating graphics and music that match each scene of the story creates the graphics and music taking into consideration the atmosphere of the scene and the emotional state of the characters.

"Application Example 2"
(Claim 1)
A way to input story settings such as the theme, setting, and characters of the story,
Based on the input story setting, it automatically creates the characters' lines, scenes, and story structure.
It is a way to automatically generate plot twists, mystery elements, and moving scenes to make the story more interesting,
A way to automatically create graphics and music that match each scene in the story you've created,
The story structure, the characters' conversations, the images and music enhance the production effect and effectively shake the viewers' emotions.
a means of generating narrative elements using a generative AI model; and
A means for generating prompt sentences and inputting them into a generative AI model;
The system includes a means for displaying the generated narrative elements to a user.

(Claim 2)
2. The system according to claim 1, wherein the means for automatically creating a story structure creates a story based on a three-act structure with a beginning, middle, and end.

(Claim 3)
2. The system according to claim 1, wherein the means for automatically creating graphics and music that match each scene of the story creates the graphics and music taking into consideration the atmosphere of the scene and the emotional state of the characters.

"Example 3"
(Claim 1)
A way to input story settings such as the theme, setting, and characters of the story,
Based on the input story setting, it automatically creates the characters' lines, scenes, and story structure.
It is a way to automatically generate plot twists, mystery elements, and moving scenes to make the story more interesting,
A way to automatically create graphics and music that match each scene in the story you've created,
A means for analyzing scene data and extracting the atmosphere of the scene and the emotional state of the characters;
A means for generating a prompt sentence to be input to the generative AI model based on the extracted information;
A means to input a prompt sentence into the generative AI model and generate graphics and music that match the scene;
a means for integrating the generated graphics and music with the scene data to generate the final output;
A system that includes means to enhance the production effect through story structure, character dialogue, images and music, and effectively stir the viewer's emotions.

(Claim 2)
2. The system according to claim 1, wherein the means for automatically creating a story structure creates a story based on a three-act structure consisting of a beginning, middle, and end.

(Claim 3)
2. The system according to claim 1, wherein the means for automatically creating graphics and music that match each scene of the story creates the graphics and music taking into consideration the atmosphere of the scene and the emotional state of the characters.

"Application example 3"
(Claim 1)
A way to input story settings such as the theme, setting, and characters of the story,
Based on the input story setting, it automatically creates the characters' lines, scenes, and story structure.
It is a way to automatically generate plot twists, mystery elements, and moving scenes to make the story more interesting,
A way to automatically create graphics and music that match each scene in the story you've created,
The story structure, the characters' conversations, the images and music enhance the production effect and effectively shake the viewers' emotions.
A means of analyzing a story scene and extracting the mood of the scene and the emotional state of the characters;
means for generating graphics based on a description of the scene;
means for generating music based on a scene description;
a means for previewing the generated content;
A system that includes a means to export completed content.

(Claim 2)
2. The system according to claim 1, wherein the means for automatically creating a story structure creates a story based on a three-act structure with a beginning, middle, and end.

(Claim 3)
2. The system according to claim 1, wherein the means for automatically creating graphics and music that match each scene of the story creates the graphics and music taking into consideration the atmosphere of the scene and the emotional state of the characters.

"Example 1 of combining emotion engines"
(Claim 1)
A way to input story settings such as the theme, setting, and characters of the story,
A means for analyzing the input story setting;
a means for searching an existing narrative database based on the analyzed story setting;
A means for generating and inputting a prompt sentence to the generative AI model;
A means to automatically create character dialogue, scenes, and story structure using generative AI models,
It is a way to automatically generate plot twists, mystery elements, and moving scenes to make the story more interesting,
A way to automatically create graphics and music that match each scene in the story you've created,
A system that includes means to enhance the production effect through story structure, character dialogue, images and music, and effectively stir the viewer's emotions.

(Claim 2)
2. The system according to claim 1, wherein the means for automatically creating a story structure creates a story based on a three-act structure consisting of a beginning, middle, and end.

(Claim 3)
2. The system according to claim 1, wherein the means for automatically creating graphics and music that match each scene of the story creates the graphics and music taking into consideration the atmosphere of the scene and the emotional state of the characters.

"Application example 1 when combining emotion engines"
(Claim 1)
A way to input story settings such as the theme, setting, and characters of the story,
Based on the input story setting, it automatically creates the characters' lines, scenes, and story structure.
It is a way to automatically generate plot twists, mystery elements, and moving scenes to make the story more interesting,
A way to automatically create graphics and music that match each scene in the story you've created,
The story structure, the characters' conversations, the images and music enhance the production effect and effectively shake the viewers' emotions.
A means for generating a story using a generative AI model based on a story setting input by a user;
A means to visualize and voice the generated story in real time,
A means for users to experience the generated story in a virtual reality space using a head-mounted display,
A system including:

(Claim 2)
2. The system according to claim 1, wherein the means for automatically creating a story structure creates a story based on a three-act structure consisting of a beginning, middle, and end.

(Claim 3)
2. The system according to claim 1, wherein the means for automatically creating graphics and music that match each scene of the story creates the graphics and music taking into consideration the atmosphere of the scene and the emotional state of the characters.

"Example 2 when combining emotion engines"
(Claim 1)
A way to input story settings such as the theme, setting, and characters of the story,
Based on the input story setting, it automatically creates the characters' lines, scenes, and story structure.
It is a way to automatically generate plot twists, mystery elements, and moving scenes to make the story more interesting,
A means for recognizing the user's emotions in real time and adjusting the story development according to those emotions;
A way to automatically create graphics and music that match each scene in the story you've created,
A system that includes means to enhance the production effect through story structure, character dialogue, images and music, and effectively stir the viewer's emotions.

(Claim 2)
2. The system according to claim 1, wherein the means for automatically creating a story structure creates a story based on a three-act structure consisting of a beginning, middle, and end.

(Claim 3)
2. The system according to claim 1, wherein the means for automatically creating graphics and music that match each scene of the story creates the graphics and music taking into consideration the atmosphere of the scene and the emotional state of the characters.

"Application example 2 when combining emotion engines"
(Claim 1)
A way to input story settings such as the theme, setting, and characters of the story,
Based on the input story setting, it automatically creates the characters' lines, scenes, and story structure.
It is a way to automatically generate plot twists, mystery elements, and moving scenes to make the story more interesting,
A way to automatically create graphics and music that match each scene in the story you've created,
The story structure, the characters' conversations, the images and music enhance the production effect and effectively shake the viewers' emotions.
A means for recognizing the user's emotions in real time and adjusting the story development according to those emotions;
A means for automatically generating a story by generating a story prompt using a generative AI model; and
A system including:

(Claim 2)
2. The system according to claim 1, wherein the means for automatically creating a story structure creates a story based on a three-act structure with a beginning, middle, and end.

(Claim 3)
2. The system according to claim 1, wherein the means for automatically creating graphics and music that match each scene of the story creates the graphics and music taking into consideration the atmosphere of the scene and the emotional state of the characters.

"Example 3 when combining emotion engines"
(Claim 1)
A way to input story settings such as the theme, setting, and characters of the story,
Based on the input story setting, it automatically creates the characters' lines, scenes, and story structure.
It is a way to automatically generate plot twists, mystery elements, and moving scenes to make the story more interesting,
A way to automatically create graphics and music that match each scene in the story you've created,
A means for using information such as the user's facial expression, tone of voice, and text input to recognize the user's emotions;
A means for adjusting the development of the story in response to the user's emotions;
A system that includes means to enhance the production effect through story structure, character dialogue, images and music, and effectively stir the viewer's emotions.

(Claim 2)
2. The system according to claim 1, wherein the means for automatically creating a story structure creates a story based on a three-act structure consisting of a beginning, middle, and end.

(Claim 3)
2. The system according to claim 1, wherein the means for automatically creating graphics and music that match each scene of the story creates the graphics and music taking into consideration the atmosphere of the scene and the emotional state of the characters.

"Application example 3 when combining emotion engines"
(Claim 1)
A way to input story settings such as the theme, setting, and characters of the story,
Based on the input story setting, it automatically creates the characters' lines, scenes, and story structure.
It is a way to automatically generate plot twists, mystery elements, and moving scenes to make the story more interesting,
A way to automatically create graphics and music that match each scene in the story you've created,
The story structure, the characters' conversations, the images and music enhance the production effect and effectively shake the viewers' emotions.
means for recognizing a user's emotion and adjusting the story scenes and music in response to the recognized emotion;
A means of recognizing emotions using the user's facial expressions, tone of voice, text input, etc.;
A system including:

(Claim 2)
2. The system according to claim 1, wherein the means for automatically creating a story structure creates a story based on a three-act structure consisting of a beginning, middle, and end.

(Claim 3)
2. The system according to claim 1, wherein the means for automatically creating graphics and music that match each scene of the story creates the graphics and music taking into consideration the atmosphere of the scene and the emotional state of the characters.

Furthermore, one or more elements included in one of the above-described embodiments may be combined with one or more elements included in another of the above-described embodiments.
(Claim 1)
a means for inputting a story setting including information about the theme, setting, and characters of the story;
means for automatically creating a story structure based on the input story setting;
means for generating elements corresponding to the created story structure;
means for automatically creating graphics and music corresponding to said story;
means for recognizing a user's emotion;
means for adjusting the scenes and music of the story in response to the recognized emotions of the user;
A system including:

(Claim 2)
the means for automatically creating a story structure creates the story based on a three-act structure consisting of a beginning, a middle, and an end of the story;
The system of claim 1 .

(Claim 3)
The element is at least one element selected from the group consisting of a foreshadowing, a mystery element, and a moving scene.
The system of claim 1 .

10, 210, 310, 410 Data processing system 12 Data processing device 14 Smart device 214 Smart glasses 314 Headset type terminal 414 Robot

Claims (2)

a means for receiving text data including information on the theme, setting, and characters of a story input by a user and analyzing the data using natural language processing technology;
A means for generating a prompt sentence to be input to the generation AI model based on the analysis result;
A means for automatically creating a story structure based on the prompt sentence using the generative AI model ;
A means for automatically generating at least one of foreshadowing, mystery elements, and moving scenes using the generative AI model ;
means for analyzing a story scene and extracting the mood of said scene and the emotional state of the characters;
a means for automatically creating graphics corresponding to the story using a graphics generation engine and automatically creating music corresponding to the story using a music generation engine based on the extracted atmosphere and emotional state ;
means for recognizing a user's emotion;
a means for generating prompt sentences to be input to the generative AI model based on the recognized emotion of the user, thereby adjusting the scenes and music of the story according to the recognized emotion of the user;
A system including: the means for automatically creating a story structure creates the story based on a three-act structure consisting of a beginning, a middle, and an end of the story;
The system of claim 1 .