JP7840022B2 - Information processing equipment, methods, programs and systems - Google Patents

Description

This disclosure relates to information processing devices, methods, programs, and systems.

A key indicator of whether a meeting was meaningful is whether it was engaging and lively. In recent years, technologies for measuring the level of engagement in a meeting have been proposed (for example, Patent Document 1).

Japanese Patent Publication No. 2016-012216

Patent Document 1 describes a meeting analysis device that calculates the level of engagement in a meeting from meeting data recorded, including audio and/or video. This meeting analysis device extracts feature quantities as a time series along the progress of the meeting from the meeting data, and calculates the level of engagement as an absolute evaluation value, based on these feature quantities, along with the time series along the progress of the meeting.

However, Patent Document 1 describes a system where meeting data is transmitted to a meeting analysis device, and the information discussed in the meeting is stored in the analysis device. Therefore, there is a non-zero possibility that information intended to remain within the meeting could be leaked externally.

The purpose of this disclosure is to quantify the level of excitement in a given space while maintaining security.

This system comprises: means installed in a designated space to collect sounds generated in that space; means installed in the space to classify the types of sounds collected by the sound-collecting means, and to determine the state and degree of excitement based on the classified type and volume; means to aggregate the states and degrees and calculate the time progression of excitement in the space; and means to present information based on the time progression of excitement in the space to the user.

According to this disclosure, it is possible to quantify the level of excitement in a given space while maintaining security.

The overall configuration of System 1 is shown in the block diagram. This is a block diagram showing an example of the functional configuration of the terminal device 10. This is a block diagram showing a functional configuration example of server 20. This is a block diagram showing an example of the functional configuration of the analysis device 30. This diagram shows the data structure of a table. This diagram shows the data structure of a table. This diagram shows the data structure of a table. This diagram shows the data structure of a table. This figure shows an example of the processing flow in System 1. This figure shows an example of the processing flow in System 1. This figure shows an example of the screen in this disclosure. This figure shows an example of the screen in this disclosure. This figure shows an example of the screen in this disclosure. This figure shows an example of the screen in this disclosure. This is a block diagram showing the basic hardware configuration of computer 90.

The embodiments of this disclosure will be described below with reference to the drawings. In all drawings illustrating the embodiments, common components are denoted by the same reference numerals, and repeated descriptions are omitted. The following embodiments are not intended to unduly limit the scope of the claims of this disclosure. Furthermore, not all components shown in the embodiments are necessarily essential components of this disclosure. Also, each drawing is a schematic diagram and not necessarily a strictly accurate representation.

Furthermore, in the following explanation, "processor" refers to one or more processors. At least one processor is typically a microprocessor such as a CPU (Central Processing Unit), but may be another type of processor such as a GPU (Graphics Processing Unit). At least one processor may be single-core or multi-core.

Furthermore, at least one processor may be a broad-sense processor, such as a hardware circuit that performs some or all of the processing (e.g., an FPGA (Field-Programmable Gate Array) or an ASIC (Application Specific Integrated Circuit)).

Furthermore, in the following explanation, the term "xxx table" may be used to describe information that yields an output for a given input. This information can be data of any structure, or it can be a learning model such as a neural network that generates an output for a given input. Therefore, "xxx table" can be referred to as "xxx information."

Furthermore, in the following explanation, the structure of each table is merely an example; one table may be divided into two or more tables, or all or part of two or more tables may constitute a single table.

Furthermore, in the following explanation, while "program" may be used as the subject to describe processing, since a program, when executed by a processor, performs defined processing using the memory and/or interface units as appropriate, the subject of the processing could also be the processor (or a device such as a controller containing that processor).

The program may be installed on a device such as a computer, or it may reside on a program distribution server or a computer-readable (e.g., non-temporary) recording medium. Furthermore, in the following description, two or more programs may be implemented as a single program, or one program may be implemented as two or more programs.

Furthermore, while identification numbers are used as identification information for various objects in the following explanation, other types of identification information (for example, identifiers containing letters or symbols) may also be used.

Furthermore, in the following explanation, when describing similar elements without distinction, a reference code (or a common code among reference codes) may be used. When describing similar elements with distinction, the element's identification number (or reference code) may be used.

Furthermore, in the following explanation, only control lines and information lines deemed necessary for the explanation are shown; not all control lines and information lines are necessarily shown in the actual product. All components may be interconnected.

Each information processing device consists of a computer equipped with an arithmetic unit and a memory device. The basic hardware configuration of the computer and the basic functional configuration of the computer realized by this hardware configuration will be described later. For each of the terminal device 10 and server 20, explanations that overlap with the basic hardware configuration and basic functional configuration of the computer described later will be omitted.

<Overview>
The system according to this embodiment uses edge AI to determine the type of sound, and based on the determined type and volume, determines a numerical value related to the level of excitement. Using the determined numerical value, it estimates the level of excitement in the space.

The following explanation uses the example of estimating the level of engagement in a conference room. However, the space in which this embodiment is implemented is not limited to a closed space such as a conference room; it may also be an open space within an office. An open space within an office may be, for example, a booth, a lounge, or a cafeteria. Hereafter, an open space within an office will be referred to as an "area."

<1. System Configuration Diagram>
Figure 1 is a block diagram showing an example of the overall configuration of System 1. System 1 shown in Figure 1 includes, for example, a terminal device 10, a server 20, an analysis device 30, a microphone 31, a camera 32, and a speaker 33. The terminal device 10, the server 20, the analysis device 30, and the speaker 33 are connected via, for example, a network 40.

Figure 1 shows an example where System 1 includes two terminal devices 10, but the number of terminal devices 10 included in System 1 is not limited to two. System 1 may include fewer than three terminal devices 10, or three or more.

In this embodiment, a collection of multiple devices may be treated as a single server. The method of allocating the multiple functions required to implement the server 20 according to this embodiment to one or more hardware components can be appropriately determined in consideration of the processing capabilities of each hardware component and/or the specifications required for the server 20.

The terminal device 10 shown in Figure 1 is, for example, an information processing device used by a user to grasp the elevation changes in a space. The terminal device 10 may be a mobile device such as a smartphone or tablet, or a stationary PC (Personal Computer) or laptop PC. It may also be a wearable device such as an HMD (Head-Mounted Display) or a smartwatch. The terminal device 10 may, for example, be owned and carried by an individual, or it may be shared among employees and fixed to the wall of a conference room.

The terminal device 10 comprises a communication interface (IF) 12, an input device 13, an output device 14, a memory 15, storage 16, and a processor 19.

The communication interface 12 is an interface for inputting and outputting signals so that the terminal device 10 can communicate with a device within system 1, such as a server 20.

The input device 13 is a device for receiving input operations from the user (for example, a touch panel, touchpad, pointing device such as a mouse, keyboard, etc.).

The output device 14 is a device (display, speaker, etc.) for presenting information to the user.

Memory 15 is for temporarily storing programs and data processed by programs, etc., and is a volatile memory such as DRAM (Dynamic Random Access Memory).

Storage 16 is for storing data and can be, for example, flash memory or an HDD (Hard Disk Drive).

The processor 19 is hardware for executing the instruction set described in the program, and consists of an arithmetic unit, registers, peripheral circuits, etc.

Server 20 is an information processing device that analyzes the rise based on information acquired from, for example, the analysis device 30. Server 20 is implemented, for example, by a computer connected to the network 40. As shown in Figure 1, Server 20 includes a communication interface 22, an input/output interface 23, a memory 25, a storage device 26, and a processor 29.

The communication interface 22 is an interface for inputting and outputting signals so that the server 20 can communicate with devices within system 1, such as the terminal device 10.

The input/output interface 23 functions as an interface to an input device for receiving user input operations and an output device for presenting information to the user.

Memory 25 is for temporarily storing programs and data processed by programs, etc., and is a volatile memory such as DRAM.

Storage 26 is for storing data and may be, for example, flash memory or an HDD.

The processor 29 is hardware for executing the instruction set described in the program, and consists of an arithmetic unit, registers, peripheral circuits, etc.

The analysis device 30 is an information processing device that analyzes audio and images, for example, from the perspective of excitement.

A single analysis device 30 may cover one space, or it may cover multiple spaces. Specifically, a single analysis device 30 may analyze audio and images obtained in one conference room, or it may analyze audio and images obtained in multiple conference rooms. The analysis device 30 is installed in a location where information analysis is possible. For example, the analysis device 30 is placed in a corner of the conference room. Furthermore, specifically, a single analysis device 30 may analyze audio and images obtained in one area, or it may analyze audio and images obtained in multiple areas. The analysis device 30 is installed in a location where information analysis is possible. For example, the analysis device 30 is placed in a corner of the area.

The analysis device 30 receives information transmitted from the microphone 31 and the imaging device 32, analyzes the received information, and transmits the analyzed information to the server 20. The analysis device 30 also transmits information acquired from the server 20 to the microphone 31 and the imaging device 32. The information acquired from the server 20 includes, for example, information for updating the settings of the microphone 31 or the imaging device 32.

Microphone 31 is installed to collect sound within the space. For example, microphone 31 is installed to collect sound from the entire conference room. Specifically, microphone 31 is, for example, a disc-shaped sound-collecting microphone installed on a table in the center of the conference room. Alternatively, microphone 31 is installed to collect sound from the entire area. Specifically, microphone 31 is, for example, a disc-shaped sound-collecting microphone installed on the ceiling in the center of the area. Microphone 31 outputs the acquired audio signal to the analysis device 30. If one microphone 31 cannot capture the sound from the entire space, multiple microphones 31 are installed. In this case, microphone 31 may be, for example, a microphone with pre-set directivity.

The imaging device 32 is installed to capture the situation within the space. For example, the imaging device 32 is installed to capture the situation of the entire conference room. Specifically, the imaging device 32 is, for example, an omnidirectional camera installed on a table in the center of the conference room. Alternatively, the imaging device 32 is installed to capture the situation of the entire area. Specifically, the imaging device 32 is, for example, an omnidirectional camera installed on the ceiling in the center of the area. The imaging device 32 outputs the acquired image signal to the analysis device 30. If one imaging device 32 cannot capture the situation of the entire space, multiple devices are installed. In this case, the imaging device 32 may be, for example, a camera with a pre-set shooting direction.

Speaker 33 is a speaker installed within a space to play music within that space. For example, speaker 33 is installed to deliver music throughout the entire conference room. Specifically, speaker 33 is, for example, an omnidirectional speaker installed on a table in the center of the conference room. Alternatively, speaker 33 is installed to deliver music to an entire area. Specifically, speaker 33 is, for example, an omnidirectional speaker installed on the ceiling in the center of the area. Speaker 33 converts the audio signal provided by server 20 into music and outputs the music into the space. If one speaker 33 is insufficient to deliver music to the entire space, multiple speakers 33 are installed.

<2. Functional Configuration of Terminal Devices>
Figure 2 is a block diagram showing a functional configuration example of the terminal device 10 shown in Figure 1. The terminal device 10 shown in Figure 2 can be implemented as a mobile terminal, a PC, or a wearable terminal. As shown in Figure 2, the terminal device 10 includes a communication unit 120, an input device 13, an output device 14, an audio processing unit 17, a microphone 171, a speaker 172, a camera 161, a location information sensor 150, a storage unit 180, and a control unit 190. Each block included in the terminal device 10 is electrically connected, for example, by a bus.

The communication unit 120 performs modulation and demodulation processing, etc., for the terminal device 10 to communicate with other devices. The communication unit 120 performs transmission processing on the signal generated by the control unit 190 and transmits it to an external source (e.g., server 20). The communication unit 120 performs reception processing on the signal received from the external source and outputs it to the control unit 190.

The input device 13 is a device for a user operating the terminal device 10 to input instructions or information. The input device 13 can be implemented, for example, by a touch-sensitive device 131 that inputs instructions by touching its operating surface. If the terminal device 10 is a PC, the input device 13 may be implemented by a reader, keyboard, mouse, etc. The input device 13 converts the instructions input by the user into electrical signals and outputs the electrical signals to the control unit 190. The input device 13 may also include, for example, a receiving port that accepts electrical signals input from an external input device.

The output device 14 is a device for presenting information to the user operating the terminal device 10. The output device 14 is implemented, for example, by a display 141. The display 141 displays data according to the control of the control unit 190. The display 141 is implemented, for example, by an LCD (Liquid Crystal Display) or an organic EL (Electro-Luminescence) display.

The audio processing unit 17 performs, for example, digital-to-analog conversion of the audio signal. The audio processing unit 17 converts the signal received from the microphone 171 into a digital signal and provides the converted signal to the control unit 190. The audio processing unit 17 also provides the audio signal to the speaker 172. The audio processing unit 17 is implemented, for example, by an audio processing processor. The microphone 171 receives audio input and provides the corresponding audio signal to the audio processing unit 17. The speaker 172 converts the audio signal received from the audio processing unit 17 into audio and outputs the audio to the outside of the terminal device 10.

The camera 161 is a device that receives light using a photodetector and outputs it as an image signal.

The location information sensor 150 is a sensor that detects the position of the terminal device 10, and is, for example, a GPS (Global Positioning System) module. A GPS module is a receiving device used in a satellite positioning system. In a satellite positioning system, signals are received from at least three or four satellites, and based on the received signals, the current position of the terminal device 10, which is equipped with a GPS module, is detected. The location information sensor 150 may also detect the current position of the terminal device 10 from the position of the wireless base station to which the terminal device 10 is connected.

The storage unit 180 is implemented, for example, by memory 15 and storage 16, and stores data and programs used by the terminal device 10.

The control unit 190 is realized when the processor 19 reads the program stored in the memory unit 180 and executes the instructions contained in the program. The control unit 190 controls the operation of the terminal device 10. By operating according to the program, the control unit 190 functions as an operation reception unit 191, a transmission/reception unit 192, and a presentation control unit 193.

The operation reception unit 191 processes instructions or information input from the input device 13. Specifically, for example, the operation reception unit 191 receives information based on instructions input from a touch-sensitive device 131, etc. Instructions input via the touch-sensitive device 131, etc., are, for example, editing instructions from the user.

Furthermore, the operation reception unit 191 receives voice instructions input from the microphone 171. Specifically, for example, the operation reception unit 191 receives voice signals input from the microphone 171 and converted into digital signals by the voice processing unit 17. The operation reception unit 191 obtains instructions from the user by, for example, analyzing the received voice signals and extracting predetermined nouns.

The transmitting/receiving unit 192 performs processing to enable the terminal device 10 to send and receive data with external devices such as the server 20 according to a communication protocol. Specifically, for example, the transmitting/receiving unit 192 transmits editing instructions entered by the user to the server 20. The transmitting/receiving unit 192 also receives information about the user from the server 20.

The display control unit 193 controls the output device 14 to present information provided by the server 20 to the user. Specifically, for example, the display control unit 193 displays the information transmitted from the server 20 on the display 141. The display control unit 193 also outputs the information transmitted from the server 20 through the speaker 172.

<3. Functional Configuration of the Server>
Figure 3 is a block diagram showing an example of the functional configuration of server 20. As shown in Figure 3, server 20 functions as a communication unit 201, a storage unit 202, and a control unit 203.

The communication unit 201 performs processing to enable the server 20 to communicate with external devices.

The storage unit 202 includes, for example, a voice analysis log database (DB) 2021, an image analysis log database (DB) 2022, etc. The tables stored in the storage unit 202 are not limited to these.

The voice analysis log DB 2021 aggregates the contents of the voice analysis log DB 3021 from each analysis device 30. Details will be described later.

The image analysis log DB 2022 aggregates the contents of the image analysis log DB 3022 of each analysis device 30. The explanation will be given later.

The control unit 203 is implemented when the processor 29 reads the program stored in the memory unit 202 and executes the instructions contained in the program. By operating according to the program, the control unit 203 performs the functions indicated as the receive control module 2031, the transmit control module 2032, the analysis control module 2033, and the presentation control module 2034.

The receive control module 2031 controls the process by which the server 20 receives signals from an external device according to a communication protocol.

The transmission control module 2032 controls the process by which the server 20 transmits signals to external devices according to a communication protocol.

The analysis control module 2033 controls the process of analyzing the information received from the analysis device 30. Specifically, the analysis control module 2033 controls the process of analyzing information related to spatial elevation, for example.

The presentation control module 2034 controls the process of presenting information to the user.

<4. Functional Configuration of the Analysis Device>
Figure 4 is a block diagram showing an example of the functional configuration of the analysis device 30. As shown in Figure 4, the analysis device 30 functions as a communication unit 301, a storage unit 302, and a control unit 303.

The communication unit 301 performs processing to enable the analysis device 30 to communicate with an external device.

The memory unit 302 includes, for example, a speech analysis log database (DB) 3021, an image analysis log database (DB) 3022, a first trained model 3023, a second trained model 3024, and the like. The tables stored in the memory unit 302 are not limited to these.

The audio analysis log DB 3021 is a database for storing information about the audio in the space collected by the microphone 31. Further details will be described later.

The image analysis log DB 3022 is a database for storing information about images of the spatial conditions captured by the imaging device 32. Further details will be described later.

The first pre-trained model 3023 is a model that learns to output information for evaluating excitement based on the input audio. For example, the first pre-trained model 3023 learns to distinguish the type of sound in a given space when audio is input. Examples of sound types include conversation, cheers, laughter, snorting, shouting, silence, screams, crying, groans, sneezes, humming, typing, etc.

In this case, the training data would, for example, use audio as input data, and the correct output data would be the judgment regarding the type of sound contained in this input data.

The second pre-trained model 3024 is a model that, upon receiving an image as input, learns to output information for evaluating the level of excitement based on the input image. For example, the second pre-trained model 3024 learns to determine the number of people in a space, their facial expressions, etc., upon receiving an image as input. Examples of facial expressions include smiles, neutral faces, crying faces, etc. Furthermore, facial expressions include gestures such as nodding, shaking heads, frowning, and looking down.

In this case, the training data would, for example, use images of humans as input data, and the judgments made about the facial expressions contained in this input data would be used as the correct output data.

The first pre-trained model 3023 and the second pre-trained model 3024 are models generated by having a machine learning model perform machine learning according to a model training program. The first pre-trained model 3023 and the second pre-trained model 3024 are, for example, parameterized composite functions composed of multiple functions that perform predetermined inference based on input data. A parameterized composite function is defined by a combination of multiple tunable functions and parameters. The pre-trained models in this embodiment may be any parameterized composite function that satisfies the above requirements.

For example, when the first trained model 3023 and the second trained model 3024 are generated using a feedforward multilayer network, the parameterized composite function is defined as a combination of, for example, linear relationships between layers using weight matrices, nonlinear (or linear) relationships using activation functions in each layer, and biases. The weight matrices and biases are called the parameters of the multilayer network. The parameterized composite function changes its form depending on how the parameters are chosen. In a multilayer network, by appropriately setting the constituent parameters, it is possible to define a function that can output desirable results from the output layer.

As the multilayer network according to this embodiment, for example, a deep neural network (DNN), which is a multilayer neural network targeted by deep learning, may be used. As a DNN, for example, a convolutional neural network (CNN) for image processing may also be used.

The control unit 303 is realized when the processor within the analysis device 30 reads the program stored in the memory unit 302 and executes the instructions contained in the program. By operating according to the program, the control unit 303 performs the functions indicated as the receiving control module 3031, the transmitting control module 3032, the analysis control module 2033, and the presentation control module 3034.

The reception control module 3031 controls the process by which the analysis device 30 receives signals from an external device according to a communication protocol.

The transmission control module 3032 controls the process by which the analysis device 30 transmits signals to an external device according to a communication protocol.

The analysis control module 3033 controls the process of analyzing the information received from the microphone 31 and the imaging device 32. Specifically, the analysis control module 3033 analyzes the audio and images from the perspective of excitement. More specifically, for example, the analysis control module 3033 analyzes the audio using the first trained model 3023. Also, for example, the analysis control module 3033 analyzes the images using the second trained model 3024.

The presentation control module 3034 controls the process of presenting information to the user.

<5. Data Structure>
This section describes the data structure of a table. Note that the data structure described is just one example, and does not exclude data not listed. Furthermore, even data within the same table may be stored in separate memory areas.

Figure 5 shows the data structure of the voice analysis log DB 3021. As shown in Figure 5, each record in the voice analysis log DB 3021 includes, for example, the items "Location," "Microphone ID," "Time," "Voice," "Volume," "Status," and "Value." When voice is output by the first trained model 3023, a new record is created in the voice analysis log DB 3021, and the information is stored in the corresponding items. The voice analysis log DB 3021 of each analysis device 30 records information about the space covered by that analysis device 30.

The item "Location" indicates the space covered by the analysis device 30.

The item "Microphone ID" indicates identification information for identifying the microphone 31 installed in the space.

The "Time" field indicates the time when audio was collected by microphone 31.

The item "Speech" indicates the type of speech. For example, the item "Speech" indicates the type of speech detected by the first trained model 3023.

The "Volume" item indicates the volume of the "Sound" item. The "Volume" item can be expressed in, for example, three levels. If the sound intensity acquired by the microphone 31 exceeds a set threshold, the server 20 sets a value corresponding to the exceeded threshold. Note that the "Volume" item is not limited to three levels.

The "State" category indicates the state related to excitement. This category includes, for example, "Excitement," "Downturn," and "Neutral." "Excitement" includes positive sounds such as laughter. "Downturn" includes negative sounds such as shouting. "Neutral" means a state that is neither "Excitement" nor "Downturn." The states related to excitement are not limited to these three types. For example, there could be two types: "Excitement" and "Downturn."

The "Value" item indicates the degree of the state related to the excitement. The "Value" item may be expressed on a scale of, for example, four levels. However, the "Value" item is not limited to four levels.

Figure 7 shows the data structure of the voice analysis log DB 2021. As shown in Figure 7, each record in the voice analysis log DB 2021 includes, for example, the fields "Location," "Microphone ID," "Time," "Voice," "Volume," "Status," and "Value." As will be described later, the voice analysis log DB 2021 aggregates the contents of the voice analysis log DB 3021 of each analysis device 30.

Since each item in the Voice Analysis Log DB 2021 represents the same information as each item in the Voice Analysis Log DB 3021, the explanations for each item in the Voice Analysis Log DB 2021 are omitted.

Figure 6 shows the data structure of the image analysis log DB 3022. As shown in Figure 6, each record in the image analysis log DB 3022 includes, for example, the items "Location," "Camera ID," "Time," "Facial Expression," "Number of People," "State," and "Value." When a facial expression is output by the second trained model 3024, a new record is created in the image analysis log DB 3022, and the information is stored in the corresponding items. The image analysis log DB 3022 of each analysis device 30 records information about the space covered by that analysis device 30.

The item "Location" indicates the space covered by the analysis device 30.

The item "Camera ID" indicates identification information for identifying the imaging device 32 installed in the space.

The item "Time" indicates the time during which the space was photographed by the imaging device 32.

The item "Facial Expression" indicates the type of facial expression. For example, it indicates the type of facial expression detected by the second trained model 3024. The item "Facial Expression" may also include information related to gestures such as nodding.

The "Number of People" field indicates the number of people who made facial expressions. For example, the "Number of People" field may indicate the number of people who made facial expressions as detected by the second trained model 3024.

The "State" item indicates the state related to excitement. The "State" item includes, for example, "Excitement," "Downturn," and "Neutral." "Excitement" includes positive expressions such as smiles. "Downturn" includes negative expressions such as frowning. "Neutral" means a state that is neither "Excitement" nor "Downturn." The states related to excitement are not limited to these three types. For example, there could be two types: "Excitement" and "Downturn."

The "Value" item indicates the degree of the state related to the excitement. The "Value" item may be expressed on a scale of, for example, four levels. However, the "Value" item is not limited to four levels.

Figure 8 shows the data structure of the image analysis log DB 2022. As shown in Figure 8, each record in the image analysis log DB 2022 includes, for example, the fields "Location," "Camera ID," "Time," "Expression," "Number of People," "Status," and "Value." As will be described later, the image analysis log DB 2022 aggregates the contents of the image analysis log DB 3022 of each analysis device 30.

Since each item in Image Analysis Log DB 2022 represents the same information as each item in Image Analysis Log DB 3022, the explanations for each item in Image Analysis Log DB 2022 are omitted.

<6 Operation>
An example of the processing flow in System 1 is described below.

Figure 9 is a flowchart illustrating an example of the operation of the analysis device 30 when analyzing audio and images.

In step S701, the analysis device 30 receives audio and images. Specifically, for example, the receiving control module 3031 receives audio data of the space collected by the microphone 31 from the microphone 31, and image data of the space captured by the imaging device 32 from the imaging device 32.

In step S702, the control unit 303 analyzes the audio data and image data.

Specifically, the analysis and control module 3033 periodically inputs received audio data, i.e., audio data collected by the microphone 31, to the first trained model 3023. The first trained model 3023 outputs information for evaluating the level of excitement based on the input audio. Specifically, the first trained model 3023 outputs the type of audio when audio is input. Specifically, the first trained model 3023 outputs laughter from audio recorded in the second conference room at 10:30:10 AM on April 1, 2024.

The input cycle is based on time, for example. This time can be set based on, for instance, the approximate time it takes for a person to perform a single action. Specifically, it can be set based on the time it takes for a person to laugh out loud, for example. For instance, the analysis control module 3033 inputs audio data to the first trained model 3023 every 2 to 5 seconds.

Specifically, the analysis control module 3033 periodically inputs received image data, i.e., image data captured by the imaging device 32, to the second trained model 3024. The second trained model 3024 outputs information for evaluating excitement based on the input images. Specifically, the second trained model 3024 outputs, for example, the type of facial expression when an image is input. It also outputs the number of people with that type of facial expression when an image is input. For example, the second trained model 3024 outputs two smiling faces from an image taken in the second conference room at 10:30:10 AM on April 1, 2024.

The input cycle is based on time, for example. This time can be set based on, for instance, the approximate time it takes for a person to perform a single action. Specifically, it can be set based on the time it takes for a person to perform a single nodding motion, for example. For instance, the analysis control module 3033 inputs image data to the second trained model 3024 every 2 to 5 seconds. The image data input to the second trained model 3024 may be a single still image or multiple still images from multiple frames.

Next, the analysis control module 3033 evaluates the level of excitement in the space based on the outputted information. More specifically, the analysis control module 3033 determines the state and degree of excitement based on the type and volume of the sound. For example, the analysis control module 3033 evaluates laughter at volume 2 as a state of "excitement" with a value of "3". Furthermore, more specifically, the analysis control module 3033 determines the state and degree of excitement based on facial expressions and the number of people making those expressions. For example, the analysis control module 3033 evaluates two people frowning as a state of "decline" with a value of "2".

In this embodiment, for example, the type of sound and the state related to excitement are pre-associated. Specifically, for example, positive sound types such as cheers, laughter, and belly laughs are associated with the state of excitement as "excitement." Similarly, negative sound types such as shouting, silence, screaming, crying, groaning, and humming are associated with the state of excitement as "decline." Furthermore, commonly occurring sound types such as conversation, sneezing, and typing are associated with the state of excitement as "neutral." Note that the relationship between the type of sound and the state of excitement is not limited to these examples. Also, for example, volume and the degree of excitement (decline) are pre-associated. Specifically, for example, volume 1 is associated with degree of excitement (decline) 1, volume 2 is associated with degree of excitement (decline) 2, and volume 3 is associated with degree of excitement (decline) 3.

In this embodiment, for example, facial expressions and states related to excitement are pre-associated. Specifically, for example, positive facial expressions such as smiles and nods are associated with a state of "excitement." Negative facial expressions such as crying, frowning, and head shaking are associated with a state of "decline." Furthermore, commonly occurring facial expressions such as a neutral expression are associated with a state of "neutral." Note that the relationship between facial expressions and states of excitement is not limited to these examples. Also, for example, the number of people and the degree of excitement (decline) are pre-associated. Specifically, for example, person 1 is associated with degree of excitement (decline) 1, person 2 is associated with degree of excitement (decline) 2, and person 3 is associated with degree of excitement (decline) 3.

In step S703, the control unit 303 stores the information related to the analysis in step S702. Specifically, for example, the analysis control module 3033 records information corresponding to the items "microphone ID," "location," "time," "voice," "volume," "state," and "value" in the audio analysis log DB 3021. Furthermore, specifically, for example, the analysis control module 3033 records information corresponding to the items "camera ID," "location," "time," "facial expression," "number of people," "state," and "value" in the image analysis log DB 3022.

In step S704, the control unit 303 transmits information related to the analysis in step S702. Specifically, for example, the transmission control module 3032 transmits the contents of the audio analysis log DB 3021 and the image analysis log DB 3022 recorded in step S703 to the server 20.

The transmission control module 3032 continuously transmits the information recorded in step S703 to the server 20. Alternatively, the transmission control module 3032 may transmit at regular intervals based on the volume of untransmitted data. Specifically, for example, the transmission control module 3032 transmits untransmitted data to the server 20 each time the volume of untransmitted data reaches a predetermined volume. Alternatively, the transmission control module 3032 may transmit at regular intervals based on the time elapsed since the last transmission. Specifically, for example, the transmission control module 3032 transmits to the server 20 each time a predetermined time has elapsed since the most recent transmission.

Figure 10 is a flowchart illustrating an example of how server 20 operates when analyzing spatial elevation.

In step S801, the control unit 203 receives information related to spatial elevation from each analysis device 30. Specifically, for example, the receiving control module 2031 receives the contents of the audio analysis log DB 3021 and the image analysis log DB 3022 transmitted in step S704 from each analysis device 30.

The control unit 203 records, for example, the contents of the voice analysis log DB 3021 in the voice analysis log DB 2021, and the contents of the image analysis log DB 3022 in the image analysis log DB 2022. In other words, the voice analysis log DB 2021 stores the contents of the voice analysis log DB 3021, and the image analysis log DB 2022 stores the contents of the image analysis log DB 3022.

In step S802, the control unit 203 analyzes the received information regarding the elevation within the space.

(Step S802A: Evaluate the ongoing set of 1)
Specifically, for example, the analysis control module 2033 evaluates the excitement of a gathering in progress within a space. For example, based on information relating to the excitement of a gathering in space, the analysis control module 2033 calculates an index value indicating the excitement of a gathering, either from a vocal perspective or from a facial expression perspective. Specifically, the larger the "value" for the "state" being "excited," the larger the index value, and the larger the "value" for the "state" being "decreased," the smaller the index value. The analysis control module 2033 may also calculate an index value comprehensively from both vocal and facial expression perspectives. Comprehensive calculation is, for example, the sum of the vocal value and the facial expression value. For example, the analysis control module 2033 calculates an index value "4" based on a vocal state "excited" value "3" and a facial state "excited" value "1". For example, the analysis control module 2033 calculates an index value "2" based on a vocal state "excited" value "3" and a facial state "decreased" value "1". Instead of simply adding them up, it would be better to weight at least one of the values before adding them up.

Furthermore, the analysis and control module 2033 continuously evaluates the rise of a group of 1s and calculates the temporal change in the index value of that group's rise.

(Step S802B Obtain the trend related to the surge)
Specifically, for example, the analysis control module 2033 acquires trends related to the level of excitement. For example, the analysis control module 2033 acquires trends related to excitement by comparing the temporal progression of excitement in multiple past gatherings. Specifically, for example, the analysis control module 2033 acquires that a gathering held on Monday from 10 to 11 a.m. in the third meeting room, with the sales manager as the organizer and sales department managers as participants, and that sales reports are elements that contribute to the excitement of the gathering, by comparing multiple gatherings from the perspective of speech. Furthermore, specifically, the analysis control module 2033 acquires that icebreakers and closing talks at the beginning of a gathering are elements that contribute to the excitement of the gathering, by comparing multiple gatherings from the perspective of speech.

The groups being compared may be groups in the same time and different spaces, groups in different times and the same space, or groups in different times and different spaces.

(Step S802C Obtain and utilize exemplary transitions)
Specifically, for example, the analysis and control module 2033 obtains a model trend of the surge. For example, the analysis and control module 2033 obtains a predetermined model trend trajectory by referring to past collections that generally showed high surges. The predetermined model trend trajectory is, for example, a trajectory that generally slopes upward while reaching periodic peaks in the surge.

Specifically, for example, the analysis and control module 2033 utilizes exemplary progressions of excitement. For instance, the analysis and control module 2033 scores the excitement of the ongoing gathering by comparing its progression, evaluated in step S802A, with an exemplary progression. The more closely the ongoing gathering's excitement progression mirrors the exemplary progression, the higher the score calculated. The gathering may also be scored after completion. The scoring could be, for example, a numerical evaluation out of 100 points, or an alphabetical evaluation from A to F.

(Step S802D: Evaluate the business establishment)
Specifically, for example, the analysis and control module 2033 evaluates the activity of the entire business site. For example, based on the index values indicating the level of activity of the group in each space calculated in step 802A, the analysis and control module 2033 calculates an index representing the level of activity of the group in all the spaces that make up one business site. For example, the index is the sum of the index values in each space. However, the index may be calculated using a predetermined statistical method, not limited to a sum. The index may also be the sum of the index values that have been weighted according to the size of each space. For example, even if the original index values of two spaces are the same, if the size of one space is twice that of the other space, the index value of one space is twice that of the other space.

Furthermore, specifically, the analysis and control module 2033 calculates statistical data regarding the activity of the business establishment. For example, the analysis and control module 2033 calculates the average value of the index over defined time periods. These defined time periods may be, for example, mornings, days of the week, weeks, months, or years. Additionally, the analysis and control module 2033 may calculate the median, maximum, and minimum values of the index over defined time periods.

(Step S802E Verify the effect of the action)
Specifically, for example, the analysis control module 2033 verifies the effect of an action on increasing excitement. For example, the analysis control module 2033 verifies the effect of an action on increasing excitement by observing the trend of the excitement index value calculated in step S802A for a certain period of time from the time the action is performed. If the index value rises for a certain period of time from the time the action is performed, the analysis control module 2033 determines that the action has increased excitement and that the action is effective in increasing excitement.

Actions designed to foster enthusiasm are those that stimulate human activity. For example, actions to encourage enthusiasm include morning exercises, coffee distribution, and project kickoff speeches.

(Step S802F: Music and Volume Selection)
Specifically, for example, the analysis control module 2033 selects the music and volume to play in a space according to the index value of the excitement level of the gathering in that space, which was calculated in step 802A. For example, in spaces where the current index value is low, the analysis control module 2033 selects an up-tempo song and a high volume to encourage excitement. Also, for example, in spaces where the current index value is high, the analysis control module 2033 lowers the volume to maintain the excitement level.

The songs may, for example, be downloaded from the internet each time a selection is made, or they may be stored in the memory unit 202.

In step S803, the control unit 203 provides feedback to the space based on the analysis in step S802.

(Step S803A Displayed at the terminal in space)
Specifically, for example, the transmission control module 3032 transmits the analysis results regarding the space to the terminal device 10. For example, the transmission control module 3032 transmits to the terminal device 10 installed in the space the trend of the index value of the crowd's excitement in space, calculated in step S802A, and the exemplary trend of the excitement and the score for the excitement, acquired in step S802C. The transmission control module 3032 may transmit continuously in real time while the crowd is in progress, or it may transmit once when the crowd is completed.

(Step S803B Displayed on the terminal at the business office)
Specifically, for example, the transmission control module 3032 transmits the analysis results regarding the business establishment to the terminal device 10. For example, the transmission control module 3032 transmits the index representing the overall activity of one business establishment, calculated in step S802D, to the terminal device 10 installed at that business establishment.

(Step S803C speaker)
Specifically, for example, the transmission control module 3032 transmits the music to be played in the space to the speaker 33. For example, the transmission control module 3032 transmits the music and volume selected in step S802F to the speaker 33 installed in the space.

<7. Screen Example>
An example of the screen shown in this disclosure is explained below.

Figure 11 is a schematic diagram showing an example of the display screen of the terminal device 10's display 141. Figure 11 illustrates the case where the progress of an ongoing meeting in conference room 201 is displayed.

Text 9001 is a text that describes the space where the gathering is taking place.

Text 9002 is the text indicating the current time.

Graph 9003 is a graph that shows the trend of the indicator value of the level of enthusiasm at a gathering. Graph 9003 consists of a vertical axis showing the indicator value of enthusiasm and a horizontal axis showing the trend over time. A higher value on the vertical axis indicates a higher indicator value of enthusiasm. The origin of the horizontal axis may, for example, represent the start time of the ongoing gathering or the start time of the business establishment.

The solid line 9004 represents the trend of the ongoing surge index value calculated in step S802A. The solid line 9004 may plot the index value in real-time processing, in batch processing, or all at once after the collection is complete. Batch processing is performed, for example, every 30 seconds.

The dashed line 9005 is a dashed line for comparison with the solid line 9004. Specifically, for example, the dashed line 9005 depicts the trend of the average index value for the same time period in the past. Alternatively, for example, the dashed line 9005 may depict the exemplary trend obtained in step S802C.

The dashed line 9005 may depict only the time progression up to the current time, or it may depict the time progression beyond the current time.

Figure 12 is a schematic diagram showing an example of the display screen of the terminal device 10's display 141. Figure 12 illustrates the case where the scores of completed meetings in conference room 201 are displayed.

Text 1001 is a text that describes the space where the gathering took place.

Text 1002 is the text indicating the time the gathering took place.

Icon 1003 is an icon that describes the level of excitement at a gathering. The analysis control module 2033 divides the current gathering into time segments and compares the progress of each segment with the average progress of past gatherings. The analysis control module 2033 determines that the gathering is exciting if it generally exceeds the average progress, and that it is unexciting if it generally falls below the average progress. For example, the analysis control module 2033 outputs a "sunny then cloudy" icon for a gathering that was exciting in the first half but unexciting in the second half. The presentation control module 2034 presents the outputted icons to the user.

Text 1004 represents the score given for the level of enthusiasm at the gathering. For example, text 1004 represents the score calculated in step S802C.

Text 1005 is text representing advice regarding the engagement level of the gathering. The analysis control module 2033 compares the progress of the current gathering to a model progress and analyzes how much the engagement level of the gathering falls short of the model progress. The analysis control module 2033 outputs advice based on the engagement level. For example, for a gathering that was engaging in the first half but declined in the second half compared to the model progress, the analysis control module 2033 would output the advice, "Next time, try to liven things up with the closing talk." The presentation control module 2034 then presents the outputted advice to the user.

Figure 13 is a schematic diagram showing an example of the display screen of the terminal device 10's display 141. Figure 13 illustrates the case where activity levels at the Azabudai business site are displayed.

Text 1101 is a text indicating the business location and the number of employees present at that location today.

Graph 1102 is a graph that represents emotional beat, or excitement, within a workplace. Graph 1102 consists of a vertical axis showing an index of excitement and a horizontal axis showing the progression over time. A higher value on the vertical axis indicates a higher index of excitement. The horizontal axis represents time periods such as one day, one week, one month, three months, six months, or one year. The time period on the horizontal axis may be switched by the user through interface operations.

Graph 1102 may be accompanied by icons representing emotions. For example, scenes in Graph 1102 that exceed a predetermined index may be accompanied by an icon representing a smile.

The solid line 1103 represents the trend of the index of activity for one business establishment, calculated in step S802D. The index may be plotted in real-time or in batch processing. Batch processing is performed, for example, every 30 seconds.

The dashed line 1104 is a comparison line with the solid line 1103. Specifically, for example, the dashed line 1104 shows the trend of the average index for the same time period in the past.

The dashed line 1104 may depict only the progression up to the current time, or it may depict the progression beyond the current time.

Pie chart 1105 shows the emotional rate, that is, the proportion of facial expressions related to the workplace. Specifically, for example, pie chart 1105 is a pie chart that shows the proportion of people with a specific facial expression among the number of people who are present at work. Specifically, the analysis control module 2033 calculates the sum of the number of people whose "State" is "Excited", the sum of the number of people whose "State" is "Depressed", and the sum of the number of people whose "State" is "Neutral", from the records in the image analysis log DB 2022 where the item "Time" is the current time. The analysis control module 2033 also calculates the number of "Other" people by subtracting the sum of the above three sums from the current number of people present at work. Then, specifically, the analysis control module 2033 calculates the proportion of people who are "Excited" relative to the number of people present at work, the proportion of people who are "Depressed", the proportion of people who are "Neutral", and the proportion of people who are "Other" relative to the number of people present at work. Specifically, the analysis control module 2033 outputs the percentage of people who are "excited" with a smiling face icon, the percentage of people who are "disappointed" with an anxious face icon, the percentage of people who are "neutral" with a serious face icon, and the percentage of people who are "other" with a document icon. The presentation control module 2034 then presents the output percentages and icons to the user.

List 1106 is a list of actions and participants to encourage enthusiasm within the workplace.

Figure 14 is a schematic diagram showing an example of the display screen of the terminal device 10's display 141. Figure 14 illustrates an example where the real-time activity level of gatherings in various spaces within a business premises is displayed.

Map 1201 is a map showing the entire business premises and the spaces that comprise them. Each space is a place where gatherings take place and where activity is evaluated, and it is labeled with text indicating its name. Examples of space names include "Meeting Room A," "Salon Table 1," etc. Map 1201 may be drawn in two dimensions or three dimensions.

The volume meters 1202, as exemplified by volume meters 1202-1 to 1202-3, are volume meters that indicate the real-time rise of a cluster in space based on the real-time index value of the cluster calculated in step S802A. For example, volume meter 1202 divides the possible range of the index value from the highest to the lowest value into five stages, and represents the real-time index value with five bars corresponding to these five stages. The higher the index value, that is, the more the cluster rises, the more the bars on volume meter 1202 stack up.

The volume meter 1202 may be marked with an icon indicating the trend of the rise. For example, the volume meter for a space where the rise has been on an upward trend over the past five minutes may be marked with an upward arrow.

<8 Summary>
As described above, in the above embodiment, System 1 includes means for collecting sounds generated in a predetermined space (a space within an office or a conference room) that is installed in the predetermined space; means for classifying the types of sounds collected by the means for collecting sounds installed in the predetermined space and determining the state and degree of excitement based on the classified type and volume; means for aggregating the states and degrees and calculating the time progression of excitement in the space; and means for presenting information based on the time progression of excitement in the space to the user. As a result, it is possible to calculate the excitement in the space without transmitting raw audio and image data to the cloud. Therefore, according to the program, method, information processing device, and system of this embodiment, it is possible to automatically quantify the excitement in a predetermined space from an audio perspective while maintaining security.

Furthermore, in the above embodiment, System 1 is installed in a predetermined space and includes means for capturing images within that space. The determination means classifies the types of facial expressions contained in the images collected by the capturing means, and determines the state of excitement and its degree based on the classified types and the number of people with each type of expression. This eliminates the need to transmit raw audio and image data to the cloud, maintaining security while automatically quantifying the level of excitement in a predetermined space from the perspective of participants' facial expressions.

Furthermore, in the above embodiment, System 1 includes means for comparing the progression of excitement levels in multiple gatherings within a space and acquiring predetermined trends regarding excitement. This allows Server 20 to acquire measures to promote excitement while comparing multiple gatherings. Additionally, in the above embodiment, System 1 includes means for presenting improvement suggestions regarding excitement based on the acquired trends. This allows users to learn about measures to promote excitement and improve the vitality of the gatherings they participate in.

Furthermore, in the above embodiment, System 1 includes means for verifying the effect of an action on increasing excitement based on an index value indicating the level of excitement achieved by the action. This allows users to recognize which actions contribute to increasing excitement. Previously, it was difficult for users to quantitatively verify measures to promote excitement, but this allows users to learn about measures that have been supported by quantitative verification.

Furthermore, in the above embodiment, System 1 includes means for statistically calculating an index representing the overall activity of the business entity based on indicator values showing the level of activity in multiple spaces. This allows for the automatic quantification of the overall activity of the business entity. While visualizing the activity level of a business entity was previously difficult, this embodiment makes it possible to visualize the activity level of the business entity. This allows users to estimate the state of the business entity based on the visualized activity level. For example, if the activity level of the business entity is shown to be low, the business entity's manager can make a decision to increase the activity level of the business entity by launching a new project.

Furthermore, in the above embodiment, System 1 includes means for playing music in the space according to the spatial excitement index value. This allows for the promotion and maintenance of a high level of excitement in the space.

<9. Variant>
In the above embodiment, we described an example where a microphone is used to collect sound within the space and a camera is used to capture images of the space in order to measure the level of excitement. However, beacons may also be used to obtain the number of people and their locations within the space in order to measure the level of excitement.

For example, a user terminal with a dedicated beacon application installed receives a beacon signal transmitted from an analysis device 30 installed in the space. Then, for example, the user terminal transmits its location information to a nearby analysis device 30, and the analysis device 30 transmits the user terminal's location information to the server 20. Alternatively, the user terminal may transmit its location information directly to the server 20. This allows the server 20 to obtain the number of people and their locations in the space. Alternatively, the user may have a device that transmits beacon signals, and the analysis device 30 may respond to beacon signals transmitted from that device.

The analysis device 30 may have functions to analyze the number, position, etc., within the space. Based on the detected numbers, the analysis device 30 determines the state related to the "boom" and the degree of that state. Specifically, the analysis device 30 determines the state related to the "boom" and the degree of that state based on the number, density, change, or at least two combinations of these detected users. For example, if the number of detected users increases by a predetermined number or more in a short period of time, the analysis device 30 evaluates the state "boom" to a value of "3". The analysis device 30 transmits the analysis results to the server 20.

In this modified version, System 1, for example, is installed in a predetermined space and includes means for detecting the user's position within that space. The determination means determines the state of elevation and the degree of that state based on the number of detected points. This allows for the automatic quantification of elevation in a predetermined space from the perspective of the user's position.

Alternatively, individuals within a space may be identified using identification numbers assigned to each person via beacons.

For example, a user terminal with a dedicated beacon application installed receives a beacon signal transmitted from an analysis device 30 installed in the space. Then, for example, the user terminal transmits its location information to a nearby analysis device 30. The analysis device 30 transmits the user terminal's location information to the server 20. Alternatively, the user terminal may transmit its location information directly to the server 20. The transmitted and received information includes an identification number assigned to each person's account in the application. The server 20 identifies individuals in the space based on their identification numbers. Alternatively, a user may have a device that transmits beacon signals, and the analysis device 30 may respond to beacon signals transmitted from that device.

Server 20 analyzes the correlation between the level of activity in a space and individuals identified based on their identification numbers. This allows for the acquisition of correlations between the level of activity in a space and the individuals within that space; for example, the presence of a specific individual in a space can cause that space to become more active.

Furthermore, the above embodiment described an example where the analysis device 30 does not have a display. However, the analysis device 30 may have a display for showing the user information related to the elevation of the space in which the analysis device 30 is installed. For example, the display control module 3034 causes the analysis device 30's display to show the trend of the elevation index value of the cluster in space 1, calculated in step S802A, and the exemplary trend of the elevation and the score for the elevation obtained in step S802C.

<10. Basic Hardware Configuration of a Computer>
Figure 15 is a block diagram showing the basic hardware configuration of computer 90. Computer 90 comprises at least a processor 901, main memory 902, auxiliary storage 903, and a communication interface 991. These are electrically connected to each other by a communication bus 921.

The processor 901 is hardware for executing the instruction set described in the program. The processor 901 consists of an arithmetic unit, registers, peripheral circuits, etc.

The main memory 902 is for temporarily storing programs and data processed by programs, etc. For example, it is a volatile memory such as DRAM (Dynamic Random Access Memory).

The auxiliary storage device 903 is a storage device for storing data and programs. Examples include flash memory, HDD (Hard Disc Drive), magneto-optical disk, CD-ROM, DVD-ROM, and semiconductor memory.

The communication interface IF991 is an interface for inputting and outputting signals for communication with other computers via a network using wired or wireless communication standards.
A network consists of various mobile communication systems, such as the Internet, LANs, and wireless base stations. For example, a network includes 3G, 4G, and 5G mobile communication systems, LTE (Long Term Evolution), and wireless networks that can connect to the Internet via designated access points (e.g., Wi-Fi®). When connecting wirelessly, communication protocols include, for example, Z-Wave®, ZigBee®, and Bluetooth®. When connecting via a wired connection, the network also includes connections made directly via USB (Universal Serial Bus) cables, etc.

Furthermore, by distributing all or part of each hardware configuration across multiple computers 90 and connecting them to each other via a network, a virtual computer 90 can be realized. Thus, the concept of computer 90 includes not only a computer 90 housed in a single enclosure or case, but also a virtualized computer system.

<Basic Functional Configuration of Computer 90>
The functional configuration of the computer realized by the basic hardware configuration of computer 90 will be described. The computer comprises at least one functional unit: a control unit, a memory unit, and a communication unit.

Furthermore, the functional units of computer 90 can also be realized by distributing all or part of each functional unit across multiple computers 90 interconnected via a network. The concept of computer 90 includes not only a single computer 90 but also a virtualized computer system.

The control unit is realized when the processor 901 reads various programs stored in the auxiliary storage device 903, loads them into the main memory device 902, and executes processing according to those programs. The control unit can realize various functional units that perform information processing depending on the type of program. In this way, the computer is realized as an information processing device that performs information processing.

The memory unit is implemented by the main memory 902 and the auxiliary memory 903. The memory unit stores data, various programs, and various databases. The processor 901 can allocate memory areas corresponding to the memory unit in the main memory 902 or the auxiliary memory 903 according to the program. The control unit can also cause the processor 901 to perform addition, update, and deletion operations on data stored in the memory unit according to the various programs.

A database, specifically a relational database, is used to manage and associate data sets called masters, which are structured in a tabular format defined by rows and columns. In a database, tables are called tables, masters are called masters, the columns of tables are called columns, and the rows of tables are called records. In a relational database, relationships can be established and linked between tables and masters.
Typically, each table and master has a primary key column to uniquely identify records, but setting a primary key column is not mandatory. The control unit can instruct the processor 901 to add, delete, or update records in specific tables and masters stored in the storage unit, according to various programs.
Furthermore, by storing data, various programs, and various databases in the memory unit, the information processing device and information processing system related to this disclosure can be considered to have been manufactured.

Furthermore, the databases and masters in this disclosure may include any data structures (lists, dictionaries, associative arrays, objects, etc.) in which information is structurally defined. Data structures also include data that can be considered as data structures by combining data with functions, classes, methods, etc., written in any programming language.

The communication unit is implemented by the communication IF 991. The communication unit provides the functionality to communicate with other computers 90 via the network. The communication unit can receive information transmitted from other computers 90 and input it to the control unit. The control unit can cause the processor 901 to perform information processing on the received information according to various programs. Furthermore, the communication unit can transmit information output from the control unit to other computers 90.

Furthermore, some or all of the above-described configurations, functions, processing units, and processing means may be implemented in hardware, for example, by designing them as integrated circuits. The present invention can also be implemented by software program code that realizes the functions of the embodiments. In this case, a storage medium containing the program code is provided to a computer, and the computer's processor reads the program code stored on the storage medium. In this case, the program code read from the storage medium itself realizes the functions of the embodiments described above, and the program code itself, and the storage medium storing it, constitute the present invention. Examples of storage media used to supply such program code include flexible disks, CD-ROMs, DVD-ROMs, hard disks, SSDs, optical disks, magneto-optical disks, CD-Rs, magnetic tapes, non-volatile memory cards, and ROMs.

Furthermore, the program code that implements the functions described in this embodiment can be implemented in a wide range of programming or scripting languages, such as assembler, C/C++, Perl, Shell, PHP, and Java®.

Furthermore, the program code for the software that implements the functionality of the embodiment may be distributed via a network and stored in a storage means such as a computer's hard disk or memory, or in a storage medium such as a CD-RW or CD-R. The computer's processor may then read and execute the program code stored in the storage means or storage medium.

The functions realized by the components described herein may be implemented in a circuit or processing circuitry, including general-purpose processors, application-specific processors, integrated circuits, ASICs (Application Specific Integrated Circuits), CPUs (a Central Processing Unit), conventional circuits, and/or combinations thereof, programmed to realize the functions described herein. A processor is considered to be a circuit or processing circuitry, including transistors and other circuits. A processor may be a programmed processor that executes a program stored in memory.
In this specification, circuitry, unit, and means are hardware programmed to perform or execute the functions described herein. Such hardware may be any hardware disclosed herein, or any hardware known to be programmed to perform or execute the functions described herein.
If the hardware is a processor that is considered to be a type of circuitry, then the circuitry, means, or unit is a combination of hardware and software used to constitute the hardware and/or processor.

While several embodiments of this disclosure have been described above, these embodiments can be implemented in various other forms, and various omissions, substitutions, and modifications are permitted without departing from the spirit of the invention. These embodiments and their variations are included within the scope and spirit of the invention, as well as within the scope of the claims and its equivalents.

<Note>
The details described in each of the above embodiments are noted below.
(Note 1)
A system comprising: means installed in a predetermined space to collect sounds generated in that space; means installed in the space to classify the types of sounds collected by the sound-collecting means, and determining the state of excitement and the degree of that state based on the classified type and volume; means to aggregate the state and degree and calculate the time progression of excitement in the space; and means to present information based on the time progression of excitement in the space to the user.
(Note 2)
The system described in (Appendix 1), which is installed in a space and includes means for capturing images within the space, wherein the determining means classifies the types of facial expressions contained in the images collected by the capturing means, and determines the state of excitement and the degree of that state based on the classified types and the number of people with that type of facial expression.
(Note 3)
The system described in (Appendix 1) or (Appendix 2), which is installed in space and includes means for detecting the position of a user in the space, wherein the means for determining the position determines the state of elevation and the degree of said elevation based on the number of detected positions.
(Note 4)
The designated space is a space within the office, and the system is one of the systems described in (Appendix 1) to (Appendix 3).
(Note 5)
The designated space is a conference room, and the system is as described in any of (Appendix 1) to (Appendix 3).
(Note 6)
A system according to any one of (Appendix 1) to (Appendix 5), comprising means for comparing the progression of excitement of multiple gatherings in space and obtaining a predetermined trend regarding the excitement.
(Note 7)
The means for acquiring trends is the system described in (Appendix 6), which presents improvement suggestions regarding the excitement based on the acquired trends.
(Note 8)
A system according to any one of (Appendix 1) to (Appendix 7), comprising means for verifying the effect of an action on the level of excitement, based on an index value indicating the level of excitement in response to an action taken to promote excitement.
(Note 9)
A system as described in any of (Appendix 1) to (Appendix 8), comprising means for statistically calculating an index representing the activity of the entire business entity based on indicator values showing the level of activity in multiple spaces.
(Note 10)
A system described in any of (Appendix 1) to (Appendix 9), comprising means for playing music in a space according to an index value of the space's excitement.
(Note 11)
A method for performing operations on a system comprising a microphone, an analysis device, and a server, the method comprising: the steps of: the microphone being installed in a predetermined space and collecting sounds occurring in the space; the analysis device being installed in the space and classifying the types of sounds collected in the sound collection step, and determining the state of excitement and the degree of said state based on the classified type and volume; the server aggregating the states and degrees and calculating the time progression of excitement in the space; and the server presenting information based on the time progression of excitement in the space to the user.
(Note 12)
A program for operating a computer installed in a predetermined space, comprising a processor and memory, the program causing the processor to perform the steps of: acquiring sound generated in the space, collected by a microphone installed in the space; classifying the type of sound acquired; and determining a state related to excitement and the degree of said state based on the classified type and volume.
(Note 13)
An information processing device installed in a predetermined space, comprising a control unit and a storage unit, wherein the control unit performs the steps of: acquiring sound generated in the space, collected by a microphone installed in the space; classifying the type of sound acquired; and determining a state related to excitement and the degree of said state based on the classified type and volume.

1...System 10...Terminal device 12...Communication IF
13...Input device 14...Output device 15...Memory 16...Storage 19...Processor 20...Server 22...Communication interface
23... Input/Output Interface
25...Memory 26...Storage 29...Processor 30...Information Processing Unit 31...Microphone 32...Photography Device 33...Speaker 40...Network

Claims (13)

A system comprising a microphone, an analysis device, and a server,
The aforementioned microphone is
It is installed in a predetermined space and collects sound generated in the said space .
The aforementioned analysis device is
The aforementioned space is installed
A means for classifying the type of sound by inputting the collected sound into a trained model that has been trained to output the type of sound,
A means for determining the state of excitement and the degree of said state based on the classified type and the volume of the collected sound ,
A means for transmitting information regarding the determined state of the rise and the degree of said state to the server,
Equipped with,
The aforementioned server,
A means for aggregating the state and degree and calculating the time progression of the rise in the space,
A system comprising means for presenting to the user information based on the time progression of the rise in the aforementioned space. The space is equipped with a camera that is installed in the space and captures images of the space,
The system according to claim 1, wherein the means for determining the above-mentioned means classifies the types of facial expressions contained in the image captured by the imaging device , and determines the state of excitement and the degree of that state based on the classified types and the number of people with that type of facial expression. Installed in the aforementioned space and equipped with means for detecting the position of a user within the aforementioned space,
The system according to claim 1, wherein the means for determining the above-mentioned state of elevation and the degree of said state are determined based on the number detected. The system according to claim 1, wherein the predetermined space is a space within an office. The system according to claim 1, wherein the predetermined space is a conference room. The system according to claim 1, wherein the server comprises means for comparing the progression of the rise of multiple gatherings in the space and obtaining a predetermined trend regarding the rise. The system according to claim 6, wherein the means for acquiring the aforementioned trend presents improvement suggestions regarding the surge based on the acquired trend. The system according to claim 1, wherein the server includes means for verifying the effect of an action on the level of excitement, based on an index value indicating the level of excitement after an action has been taken to promote excitement. The system according to claim 1, wherein the server comprises means for statistically calculating an index representing the activity of the entire business entity based on indicator values showing the level of activity in multiple spaces. The system according to claim 1, further comprising a speaker that plays music into the space according to an index value of the excitement level of the space. A method performed on a system comprising a microphone, an analysis device, and a server,
The steps include: installing the microphone in a predetermined space and collecting sound generated in the space;
The analysis device is installed in the space, and the steps include: inputting the collected audio into a trained model that has been trained to output types of sounds, thereby classifying the types of sounds collected;
The analysis device performs the steps of determining the state of excitement and the degree of that state based on the classified type and the volume of the collected sound ,
The analysis device transmits information regarding the determined state of the elevation and the degree of that state to the server.
The server performs the steps of aggregating the state and degree and calculating the time progression of the rise in the space,
The server provides the user with information based on the time progression of the rise in the space,
How to do it. A program for operating a computer installed in a predetermined space, comprising a processor and memory, wherein the program is configured to operate the processor,
The steps include acquiring sound generated in the space, collected by a microphone installed in the space,
The steps include: inputting the acquired audio into a trained model that has been trained to output the type of sound, thereby classifying the type of sound acquired;
A step of determining the state related to excitement and the degree of said state based on the classified type and the volume of the acquired sound ,
The steps include transmitting information regarding the determined state of the elevation and the degree of that state to an external device that aggregates and analyzes the information,
A program that executes the command. An information processing device installed in a predetermined space, comprising a control unit and a storage unit, wherein the control unit is
The steps include acquiring sound generated in the space, collected by a microphone installed in the space,
The steps include: inputting the acquired audio into a trained model that has been trained to output the type of sound, thereby classifying the type of sound acquired;
A step of determining the state related to excitement and the degree of said state based on the classified type and the volume of the acquired sound ,
The steps include transmitting information regarding the determined state of the elevation and the degree of that state to an external device that aggregates and analyzes the information,
An information processing device that performs the following actions.