JP7551079B2 - Live event production support system - Google Patents

Description

This disclosure relates to a live event production assistance system that performs various processes to assist in the production of live events.

An example of a system that assists in the production of live events such as concerts (live event production assistance system) is the system that uses a penlight, as described in Patent Document 1.

JP 2015-011981 A

However, in such live event production assistance systems, there is generally a demand for improving the sense of unity between the performers and the audience (participants). It is desirable to provide a live event production assistance system that can improve the sense of unity between the performers and the audience.

A first live event production assistance system according to an embodiment of the present disclosure is a system for assisting in the production of a live event performed by a performer and a plurality of participants including an audience, and includes a first information processing device arranged on the performer's side, a production control device that controls the production device arranged on the performer's side, a second information processing device arranged on the participant's side, an output device arranged on the participant's side that performs a predetermined output operation, and a sensor that detects the degree of excitement of the participants. During the first production assistance process as a process of assisting in the production from the performer's side to the participant's side, the first information processing device transmits a control signal input in response to an instruction from the performer to the second information processing device, and the second information processing device controls the output operation of the output device based on the control signal received from the first information processing device. In addition, during the second performance support processing as a support process for the performance from the participants to the performers, the second information processing device performs a predetermined analysis process based on the detection result of the degree of excitement by the sensor, and transmits the excitement level data obtained by the analysis process to the first information processing device, and the first information processing device reflects the excitement level of a specific participant selected from the multiple participants in the control operation of the performance device by the performance control device based on the excitement level data obtained for the specific participant.
A second live event production assistance system according to an embodiment of the present disclosure is a system for assisting in the production of a live event performed by a performer and a plurality of participants including an audience and a production assistant, and includes a first information processing device arranged on the performer's side, a production control device that controls the production device arranged on the performer's side, a second information processing device arranged on the participant's side, an output device arranged on the participant's side for performing a predetermined output operation, and a sensor that detects the degree of excitement of the participants. During the first production assistance process as a process of assisting in the production from the performer's side to the participant's side, the first information processing device transmits a control signal input in response to an instruction from the performer to the second information processing device, and the second information processing device controls the output operation of the output device based on the control signal received from the first information processing device. In addition, during the second performance support processing as a support process for performance from the participants to the performers, the second information processing device performs a predetermined analysis process based on the detection result of the degree of excitement by the sensor, and transmits the excitement level data obtained by the analysis process to the first information processing device.The first information processing device then reflects the degree of excitement of the participants based on the excitement level data received from the second information processing device, taking into consideration the detection result of the degree of excitement of the performance assistants who are also participants, in the control operation of the performance device by the performance control device.

According to the first and second live event production assistance systems relating to an embodiment of the present disclosure, it is possible to improve the sense of unity between performers and audience, etc.

1 is a block diagram illustrating an example of the overall configuration of a live event production assistance system according to an embodiment of the present disclosure. A block diagram showing an example of operation during the first performance assistance process (performance assistance process from the performer side to the audience side) in the embodiment. A block diagram showing an example of operation during a second performance assistance process (performance assistance process from the audience side to the performer side) in the embodiment. A block diagram showing another example of operation during the second performance assistance processing in the embodiment. A block diagram showing another example of operation during the second performance assistance processing in the embodiment. FIG. 13 is a diagram illustrating an example of application of the analysis process (analysis process using a machine learning model) according to the embodiment. 5 is a characteristic diagram illustrating an example of a state of a multi-stage evaluation value according to the first embodiment. FIG. 5 is a characteristic diagram illustrating an example of discrimination between evaluation values in a plurality of stages according to the first embodiment. FIG. FIG. 11 is a characteristic diagram illustrating an example of a state of a multi-stage evaluation value according to the second embodiment. FIG. 11 is a characteristic diagram illustrating an example of discrimination between evaluation values in a plurality of stages according to the second embodiment.

Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings. The description will be made in the following order.
1. Embodiment (Example of two-way production support processing between performers and participants such as spectators)
2. Modifications

1. Preferred embodiment
[Overall system configuration]
1 is a schematic block diagram showing an example of the overall configuration of a live event production assistance system (production assistance system 3) according to an embodiment of the present disclosure. This production assistance system 3 is a system for assisting in the production of a live event carried out by a performer 51, an audience 52 as participants, and a production assistant 53. An example of such a live event is a concert, as shown in FIG. 1, in which the performer 51 is a musician and the production assistant 53 is, for example, a dancer (such as a back dancer) (see FIG. 1).

As shown in FIG. 1, the production assistance system 3 of this embodiment is a system that is applied to an online event (an online concert in the example of FIG. 1) in which the performer 51 and the audience 52 are located in different locations. Specifically, in the example of the online concert shown in FIG. 1, the performer 51 and the production assistant 53 are each located in a concert venue 61 as the event venue, and the online concert is held with multiple audience members 52 each located in their own homes 62. Note that the location of each audience member 52 when holding such an online event is not limited to their own homes 62 as shown in FIG. 1, and may be, for example, a designated public viewing venue.

As shown in FIG. 1, such a production assistance system 3 includes a server 11 and a production control device 12 located on the performer 51 side (in the concert venue 61), and a smart device 2a located on each audience member 52 side (in their home 62). In particular, in the example of FIG. 1, the production assistance system 3 further includes a smart device 2b located on the assistant director 53 side (in the concert venue 61). A production system for an online concert as a live event is configured including such a production assistance system 3 and a production device 4 located on the performer 51 side (in the concert venue 61) (see FIG. 1).

(Arrangement on the performer 51 side)
The performance device 4 is a device that performs various performance operations in a concert, and examples thereof include musical instruments and lighting devices.

The performance control device 12 is a device that controls the performance device 4, as indicated by the arrow in FIG. 1. Specifically, the performance control device 12 controls various performance operations in the performance device 4 (for example, the operation of musical instruments and lighting devices).

The server 11 is an information processing device that performs various types of information processing. As shown in FIG. 1, the server 11 is capable of communicating (sending and receiving data) with each smart device 2a on the spectator 52 side (described later) via a network 30 (such as Internet communication). As shown in FIG. 1, the server 11 is also capable of communicating (sending and receiving data) with a smart device 2b on the assistant director 53 side (described later) and the performance control device 12 described above via various networks (wireless or wired). A detailed example of the operation of the server 11 will be described later.

Here, such a server 11 corresponds to a specific example of a "first information processing device" in this disclosure.

(Layout of the audience 52 and assistant director 53)
As shown in Fig. 1, the smart devices 2a and 2b are devices arranged on the side of each spectator 52 and the side of the assistant director 53. Specifically, the smart devices 2a and 2b are, for example, held in the palm of the hand or fixed to a part of the body (hand, foot, head, etc.) by each spectator 52 and the assistant director 53. Such smart devices 2a and 2b are, for example, smart watches, smartphones, tablets, etc., as well as penlights, light-emitting wristbands, etc.

Each of the smart devices 2a and 2b includes an information processing device 21, a vibration sensor 22, a vibration device 23a, and a light-emitting device 23b (see FIG. 1). The information processing device 21 also includes a transmission/reception unit (communication unit) 211, an analysis processing unit 212, and a data conversion processing unit 213.

In other words, in the example of FIG. 1, the information processing device 21, the vibration device 23a, the light-emitting device 23b, and the vibration sensor 22 are each integrated and mounted within a single device (each of the smart devices 2a and 2b).

Each of the smart devices 2a and 2b corresponds to a specific example of a "single device" in this disclosure, and the information processing device 21 corresponds to a specific example of a "second information processing device" in this disclosure. The vibration sensor 22 corresponds to a specific example of a "sensor" in this disclosure, and the vibration device 23a and the light-emitting device 23b each correspond to a specific example of an "output device" in this disclosure.

The vibration sensor 22 is a sensor that detects vibrations in three dimensions (vibrations caused by predetermined movements of each audience member 52 and each performance assistant 53) as an example of the degree of excitement during a concert among each audience member 52 and each performance assistant 53. The detection results of the degree of excitement (vibrations in three dimensions) by the vibration sensor 22 are output as vibration data Dv to the analysis processing unit 212 described below, as shown in FIG. 1.

Note that parameters indicating the degree of excitement include, in addition to the vibrations caused by the predetermined (specific) movements of the participants (each spectator 52 and production assistant 53) described above, the following parameters, for example: For example, the body temperature and pulse rate of the participants, the sounds emitted by the participants, the brightness of the environment surrounding the participants, etc. In other words, the degree of excitement may be determined using at least one of these various parameters detected by various sensors (excitement degree sensor).

The vibration device 23a and the light-emitting device 23b are each a device (output device) that performs a predetermined output operation. Specifically, the vibration device 23a is a device that performs a vibration operation as an output operation, and the light-emitting device 23b is a device that performs a light-emitting operation as an output operation. In other words, in the example of FIG. 1, such output operations include at least one of a vibration operation and a light-emitting operation.

The transmission/reception unit 211 communicates (transmits and receives data) with the server 11 via the network 30 described above.

The analysis processing unit 212 performs a predetermined analysis process based on the detection result of the excitement level by the vibration sensor 22 (the vibration data Dv described above). Specifically, as will be described in detail later, the analysis processing unit 212 performs analysis using various machine learning models to determine the specific actions of each spectator 52 and assistant director 53 (for example, determining the movement state based on complex human movements). The excitement level data De1 of each spectator 52 and assistant director 53 generated by such analysis process is output to the data conversion processing unit 213 (see FIG. 1).

The data conversion processing unit 213 performs a predetermined conversion process for reducing the amount of data on the excitement level data De1 output from the analysis processing unit 212, and generates the excitement level data De2 after such conversion process (see FIG. 1). As a result, the excitement level data De2 sent from each smart device 2a, 2b to the server 11 is not the data before such conversion process for reducing the amount of data (excitement level data De1), but the data after the conversion process, as will be described in detail later.

As an example of such a conversion process, the above-mentioned analysis processing unit 212 may convert the action analyzed into an assigned sound or a signal such as MIDI (Musical Instruments Digital Interface: up to 7 bits). In other words, an example of the excitement level data De2 after the above-mentioned conversion process is MIDI data.

[Actions, actions and effects]
Next, with reference to FIG. 2 to FIG. 6 in addition to FIG. 1, the operation, action and effect of the performance assistance system 3 of this embodiment will be described in detail while comparing with a comparative example.

A. Comparative Examples
First, examples of typical performance assistance systems according to the comparative example include the following systems. That is, for example, the system described in the above-mentioned Patent Document 1, which uses a penlight or a specific wristband (Zyloband (registered trademark)), etc., can be mentioned. In such a performance assistance system of the comparative example, a performance is performed in which the performer transmits a signal to change the light emission state (color of light) of the audience's penlights or wristbands. This type of performance allows each audience member to feel a sense of unity with the performer (a shared experience).

However, with this comparative example directing assistance system, performers are unable to directly feel the audience's reactions, making directing difficult. In particular, in recent years, there have been an increasing number of live events such as concerts being held online as mentioned above (the number of online events is increasing). At such online events, performers are unable to see the audience's facial expressions and movements (level of excitement), making it increasingly difficult to understand the audience's reactions. Also, for the audience, simply watching a video of an online event makes it difficult to feel a sense of unity as if they are participating in the event.

In this way, it can be said that it is difficult to improve the sense of unity between performers and the audience (participants) using the production assistance system in the comparative example.

B. Operation
In contrast to this, in the performance assistance system 3 of this embodiment, the following operations are performed in each of the above-mentioned configurations.

Figure 2 is a block diagram showing an example of the operation during the production assistance process Pd1 (production assistance process from the performer 51 to the audience 52) according to this embodiment. Also, Figures 3 to 5 are block diagrams showing examples of the operation during the production assistance process Pd2 (production assistance process from the audience 52 or production assistant 53 to the performer 51) according to this embodiment.

(B-1. Example of operation during performance assistance process Pd1)
First, during the performance assistance process Pd1 shown in FIG. 2, the following operations are performed in the performance assistance system 3.

That is, as shown in FIG. 2, when a control signal Sc1 corresponding to an instruction from the performer 51 (e.g., pressing a specific switch) is input to the server 11, the server 11 transmits this control signal Sc1 to each smart device 2a (transmitter/receiver 211) via the network 30. Then, the information processing device 21 in each smart device 2a controls the output operation (vibration operation or light emission operation) of the vibration device 23a and the light-emitting device 23b based on the control signal Sc1 received from the server 11 in this way. Specifically, for example, various vibration operations are performed, the light emission color is changed, or a specific blinking operation is performed in synchronization with the performance by the performer 51 or the performance device 4.

By performing each of these operations, during the performance assistance process Pd1 (performance assistance process from the performer 51 to the audience 52) shown in Figure 2, each audience member 52 can feel a sense of unity (shared experience) with the performer 51, just as in the comparative example described above.

During this performance assistance process Pd1, the smart device 2b on the assistant performance person 53's side may also control the output operation (vibration operation or light emission operation) of the vibration device 23a and the light emission device 23b based on the control signal Sc1 transmitted from the server 11. In this case, by controlling the vibration operation or light emission operation of the smart device 2b in response to instructions from the performer 51, it becomes possible to control the performance assistance on the assistant performance person 53's side.

(B-2. Example of operation during performance assistance process Pd2)
On the other hand, during the performance assistance process Pd2 shown in Figures 3 to 5, the following operations are performed in the performance assistance system 3.

That is, as shown in FIG. 3 to FIG. 5, first, the information processing device 21 (analysis processing unit 212) in the smart device 2a on the side of each spectator 52 performs the above-mentioned predetermined analysis processing based on the detection result (vibration data Dv) of the excitement level by the vibration sensor 22. Next, the information processing device 21 (transmission/reception unit 211) transmits the excitement level data obtained by such analysis processing (in this example, as described above, the excitement level data De2 after conversion processing for reducing the data amount in the data conversion processing unit 213) to the server 11 via the network 30 (see FIG. 3 to FIG. 5). Then, based on the excitement level data De2 received from each smart device 2a in this way, the server 11 reflects the excitement level of each spectator 52 in the control operation of the production device 4 by the production control device 12. Specifically, the server 11 outputs a control signal Sc2 to the production control device 12 (see FIG. 3 to FIG. 5), and the excitement level of each spectator 52 is reflected in the production operation of the production device 4.

At this time, as shown in FIG. 4, for example, the excitement level of a specific audience member 52 selected by a selection signal Ss from among the multiple audience members 52 may be reflected in the control operation of the performance device 4 based on the excitement level data De2 obtained for the specific audience member 52. In contrast, in the example of FIG. 3, the excitement level of the multiple audience members 52 (the overall excitement level described below) is reflected in the control operation of the performance device 4 based on the multiple excitement level data De2 obtained individually for the multiple audience members 52. Incidentally, the specific audience member 52 is selected in the server 11 based on the selection signal Ss input to the server 11 in response to an instruction from the performer 51 (for example, the operation of pressing a specific switch, etc.), as shown in FIG. 4, for example.

For example, as shown in FIG. 5, during such production assistant process Pd2, the server 11 may also take into consideration the excitement level data De2 (detection result of the excitement level of the production assistant 53) output from the smart device 2b on the production assistant 53 side. In other words, the server 11 may take into consideration the excitement level of the production assistant 53 in addition to the excitement level of the audience 52 as described above, and then reflect this in the control operation of the production device 4 described above.

In the example of FIG. 5, as in the example of FIG. 4 described above, a specific participant (spectator 52 or production assistant 53) may be selected by the selection signal Ss from among the multiple spectators 52 and production assistants 53 as participants. In other words, based on the excitement level data De2 obtained for the specific participant selected in this way, the excitement level of the specific participant may be reflected in the control operation of the production device 4 described above. Note that the specific participant selected in this way is not limited to one person, and may be, for example, multiple people (two or more people).

In this way, during the production assistance process Pd2 shown in Figures 3 to 5, production assistance processing is performed from the audience 52 and the production assistant 53 to the performer 51, unlike the comparative example described above.

(B-3. Application example of analysis processing using machine learning model)
Here, during such performance assistance processing Pd2, the information processing device 21 (analysis processing unit 212) in each smart device 2a, 2b performs analysis processing using, for example, various machine learning models as described above, to generate excitement level data De1. Specifically, the analysis processing unit 212 generates the excitement level data De1 by determining a multi-level evaluation value (motion state) related to the excitement level for the specific actions described above of each spectator 52 or the performance assistant 53.

Figure 6 shows examples of application of the analysis process according to this embodiment (analysis process using a machine learning model) for each of the two methods (methods A and B) described below.

First, in method A (corresponding to the example of FIG. 3 above), the overall excitement level of the multiple spectators 52 is reflected in the control operation of the performance device 4 described above based on the multiple vibration data Dv (excitement level data De2) obtained individually for the multiple spectators 52. At this time, since the above-mentioned multiple-level evaluation value (for example, M-level evaluation value (M: integer of 2 or more)) regarding the excitement level is obtained individually for each of the multiple spectators 52, for example, if the number of spectators 52 is N (N: integer of 2 or more), an evaluation value of (M×N) levels (for example, 5 levels×5 people=25 levels) is obtained. In other words, in method A, such an evaluation value of (M×N) levels is reflected in the control operation of the performance device 4 as the overall excitement level of the multiple spectators 52.

On the other hand, in method B (corresponding to the example in Figure 4 above), the (specific) excitement level of a specific spectator 52 selected from among the multiple spectators 52 is reflected in the control operation of the production device 4 based on the vibration data Dv (excitement level data De2) obtained for that specific spectator 52. Therefore, in method B, a multi-level evaluation value (for example, the M-level evaluation value described above) regarding the excitement level of that specific spectator 52 is reflected in the control operation of the production device 4 as the excitement level of that specific spectator 52.

(C. Actions and Effects)
In this manner, in the performance assistance system 3 of this embodiment, the following operations are performed during the above-mentioned performance assistance process Pd1 (performance assistance process from the performer 51 to the audience 52). That is, the server 11 transmits a control signal Sc1 from the performer 51 to each of the smart devices 2a, 2b (information processing device 21), and each of the information processing devices 21 controls the output operations of the vibration device 23a and the light-emitting device 23b based on the received control signal Sc1.

Meanwhile, during the above-mentioned performance assistance process Pd2 (performance assistance process from the audience 52 or the performance assistant 53 to the performer 51), the following operations are performed. That is, first, each smart device 2a, 2b (information processing device 21) performs a predetermined analysis process based on the excitement level detection result (vibration data Dv) by the vibration sensor 22, and transmits the excitement level data De2 obtained by the analysis process to the server 11. Then, based on the received excitement level data De2, the server 11 reflects the excitement level of the audience 52, etc. in the control operation of the performance control device 12 for the performance device 4.

In this way, in this embodiment, unlike the comparative example (a system in which only one-way production support processing from the performer side to the audience side is performed), two-way production support processing (production support processing Pd1, Pd2) is performed between the performer 51 side and the audience 52 side. As a result, in this embodiment, not only the performer 51 side but also the audience 52 side can participate in the production of the live event (concert), making it possible to perform a new production (audience participation type production) that has never been seen before. In addition, the performer 51 side and the audience 52 side can be mutually connected, and the audience 52 side can have an even greater sense of unity at the live event, and the performer 51 side can feel the reaction (excitement) of the audience 52 side, unlike the comparative example described above, and can have a more exciting experience. For these reasons, in this embodiment, it is possible to improve the sense of unity between the performer 51 and the audience 52, etc., compared to the comparative example.

In addition, in this embodiment, the information processing device 21 (analysis processing unit 212) in each smart device 2a, 2b performs analysis processing using a machine learning model to determine multiple levels of evaluation values (motion states) related to the degree of excitement and generate excitement level data De1, as follows. That is, since it is possible to determine specific actions of each spectator 52 or assistant director 53 (for example, to determine the motion state based on complex human movements), it becomes possible to easily and accurately determine the degree of excitement of the spectators 52, etc.

Furthermore, in this embodiment, if the overall level of excitement of the multiple spectators 52, etc. is reflected in the control operation of the production device 4 based on the multiple excitement level data De2 obtained individually for the multiple spectators 52, etc., the result will be as follows. In other words, in this case, the overall (consensus) level of excitement of the entire spectators 52, etc. can be reflected in the production, which makes it possible to further improve the sense of unity between the performers 51 and the spectators 52, etc.

In addition, in this embodiment, if the excitement level of a specific participant selected from among multiple participants (such as spectators 52) is reflected in the control operation of the performance device 4 based on the excitement level data De2 obtained for that specific participant, the result will be as follows. That is, in this case, since the excitement level of such a specific participant can be reflected in the performance, for example, spectators 52 who are thought to be particularly excited can be individually allowed to participate in the performance, making it possible to realize a variety of performances at a live event.

In addition, in this embodiment, the specific participants are selected within the server 11 based on a selection signal Ss input to the server 11 in response to an instruction from the performer 51, resulting in the following: That is, for example, as described above, the performer 51 can pick out audience members 52 who are thought to be particularly excited and have them participate in the performance, which can further improve the sense of unity between the performer 51 and the audience members 52.

Furthermore, in this embodiment, when the detection result of the excitement level of the assistant director 53 as a participant (excitement level data De2) is taken into consideration during the above-mentioned assistant director process Pd2 and reflected in the control operation of the production device 4, the result is as follows. In other words, since not only the excitement level of the audience 52 but also the excitement level of the assistant director 53 can be reflected in the production, it becomes possible to realize even more diverse productions at live events. Furthermore, since a sense of unity at the live event can be obtained not only from the performers 51 and the audience 52 but also from the assistant director 53, it becomes possible to further improve the sense of unity between the performers 51 and the participants (the audience 52 and the assistant director 53).

In addition, in this embodiment, the excitement level data De2 transmitted and received during the above-mentioned performance assistance process Pd2 is data after conversion processing for reducing the amount of data in the information processing device 21 (data conversion processing unit 213) of each smart device 2a, 2b, so that it is as follows. That is, by performing the performance assistance process Pd2 using such converted excitement level data De2, low-volume data can be transmitted and received between the audience 52 side and the performer 51 side, so that useful information (excitement level, etc.) on the audience 52 side can be quickly and easily provided to the performer 51 side. As a result, for example, even in the case where a large number of audience members 52 participate in a live event, the load on the server 11 can be reduced (no need to reinforce the server 11) and the impact of data delays on the network 30 can be minimized. As a result, the level of excitement among the audience 52 can be provided to the performers 51 in real time, further improving the sense of unity between the performers 51 and the audience 52, and by eliminating the need to expand the server 11, it is also possible to reduce the implementation costs of live events.

Incidentally, in a next-generation network environment (a so-called 5G generation network environment), communication will be possible at much higher speeds than before, but on the other hand, there is a risk that the load on the server will increase and power consumption at the server will increase. In contrast, in this embodiment, contrary to the usual idea, even in such a next-generation network environment, the amount of data sent and received can be minimized, thereby achieving the various effects described above, and it can be said that it is possible to provide optimal production support for live events.

In addition, in this embodiment, the information processing device 21, the vibration device 23a, the light-emitting device 23b, and the vibration sensor 22 are each integrated and mounted in a single device (each smart device 2a, 2b), as follows. That is, for example, compared to a case in which these are provided separately, the device can be made smaller and can be used more easily, which also improves convenience for the participants (spectators 52 and assistant directors 53).

[Example]
Next, examples (examples 1 and 2) of this embodiment will be described in detail with reference to FIGS.

Figure 7 shows an example of the state of the multi-level evaluation value in Example 1 in the form of a characteristic diagram, and Figure 8 shows an example of the discrimination between the multi-level evaluation value in Example 1 in the form of a characteristic diagram. Also, Figure 9 shows an example of the state of the multi-level evaluation value in Example 2 in the form of a characteristic diagram, and Figure 10 shows an example of the discrimination between the multi-level evaluation value in Example 2 in the form of a characteristic diagram.

Example 1
First, in the first embodiment shown in Fig. 7 and Fig. 8, the degree of excitement on the part of the audience 52, etc. is classified into two stages (binary values of excited state and calm state) and judged. Specifically, Fig. 7(A) shows an example of the time change (obtained waveform of reaction) of acceleration (vibration data Dv) in an excited state, and Fig. 7(B) shows an example of the time change of acceleration in a calm state.

Figures 8(A) and 8(B) show examples of machine learning results for such binary classification under the following conditions. In the examples of machine learning results shown in Figures 8(A) and 8(B), it can be seen that the discrimination between the above-mentioned two values (excited state and calm state) can be achieved with nearly 100% accuracy (discrimination accuracy: approximately 100%).

・Axis waveform for about 2 seconds (200 points): 200 points are used as features and discriminated using a supervised learning model ・Number of training data: 400
・Number of test data: 50
・Number of data points: 200 points (about 2 seconds)
・Machine learning model: Neural network ・Activation function: Sigmoid function (final layer: Softmax function)
・Number of times of learning: 40 times

Example 2
On the other hand, in the second embodiment shown in Figures 9 and 10, the degree of excitement on the part of the audience 52 etc. is classified into five more detailed levels (five values: FIXED, CALM, MIDDLE, INTENSE, and CRAZY) and judged. Specifically, Figures 9(A) to 9(E) each show an example of the change in acceleration over time (obtained reaction waveform) in each of the five states described above.

Figure 10 shows an example of machine learning results for such a five-value classification under the following conditions. In the example of the machine learning results shown in Figure 10, the discrimination accuracy for the five values described above is about 90%, but since the loss error (Loss) has not fallen completely to near 0, it can be seen that there is some room for improvement.

・Machine learning model: Neural network ・Number of layers: 4 layers (64 → 32 → 16 → 5)
・Activation function: sigmoid function (output layer: softmax)
・Number of epochs: 1000
・Number of training data: 2000
・Number of test data: 500

2. Modified Examples
Although the technology of the present disclosure has been described above by giving embodiments and examples, the technology is not limited to these embodiments and can be modified in various ways.

For example, the configurations (specific examples, placement positions, number, etc.) of each component described in the above embodiments are not limited, and other configurations may be used. Specifically, in the above embodiments, the second information processing device (information processing device 21), the output device (vibration device 23a and light-emitting device 23b), and the sensor (vibration sensor 22) are integrated and mounted in a single device (smart device 2a, 2b) on the participant (audience 52 and assistant director 53) side, but this example is not limited. That is, for example, the second information processing device, the output device, and the sensor may be provided as separate entities on the participant side. In this case, the second information processing device, the output device, and the sensor are capable of transmitting and receiving data to and from each other via various communications (wireless communications, etc.).

In addition, in the above embodiment, the vibration device 23a and the light-emitting device 23b are given as examples of the "output device" in the present disclosure, and the vibration operation and the light-emitting operation of these devices are given as examples of the "predetermined output operation" in the present disclosure, but the present disclosure is not limited to these examples. That is, examples of such "output devices" and "predetermined output operations" include, for example, a display (image display device) that performs an image output operation, and a transmitting device (remote controller, etc.) that performs an operation (signal transmission operation) of transmitting a predetermined signal such as a control signal for controlling other equipment. Specifically, in such a display, for example, a control signal transmitted from the performance control device 12 on the performer 51 side via the server 11 may be reflected (composite processing is performed) on the image signal (such as the image signal of a live event) transmitted from the server 11 side in the signal processing device 21 in each smart device 2a, 2b, and then output. The remote controller may also transmit control signals for various devices in the audience member's 52 home 62 (e.g., lighting equipment, television receiver, speakers, etc.) or control signals for various devices on the performer's 51 side.

Furthermore, the smart devices 2a and 2b (corresponding to the second information processing device, output device, and sensor in this disclosure) described in the above embodiment may be operated using, for example, power supplied from a vibration power generating element built into these smart devices 2a and 2b (power generated using the movements of the participants).

In addition, other methods may be used for the analysis processing methods described in the above embodiments (such as the number of stages of the evaluation value when using a machine learning model and the method of discrimination). Furthermore, when performing such analysis processing, methods other than the method using the machine learning model described in the above embodiments may be used.

In addition, in the above embodiment, the operation example of the first production assistance process (production assistance process from the performer side to the audience side) and the second production assistance process (production assistance process from the audience side or the production assistant side to the performer side) has been specifically described, but the present invention is not limited to these operation examples. Specifically, in the above embodiment, the operation example of reflecting the excitement level of each audience member 52 and the production assistant 53 in the control operation for the production device 4 based on the excitement level data De2 received from each smart device 2a, 2b has been described. In this way, various production operation examples when reflecting the excitement level of each audience member 52 and the production assistant 53 in the production operation on the performer 51 side (production device 4) include various kinds. Specifically, for example, in addition to the production operation using the above-mentioned musical instruments and lighting devices, production operation using various video output and audio output, and production operation using video output of avatars of each audience member 52 and production assistant 53 (characters modeled in a virtual space on a network as their alter egos) can be mentioned. Also, for example, in the above embodiment, the excitement level data De2 to be transmitted and received is data that has been converted to reduce the amount of data in the information processing device 21, but this is not a limitation. In other words, for example, in some cases, data before conversion to reduce the amount of data (excitement level data De1) may be transmitted and received.

In addition, in the above embodiment, an online event is described as an example in which the performer 51 and the audience 52 are located in different locations, but this is not limited to this example. That is, the live event production assistance system of the present disclosure can also be applied to offline events in which the performer 51 and the audience 52 are located in the same location (live event venue). Furthermore, in the above embodiment, an example in which such a live event is a concert (in which the performer 51 is a performer and the production assistant 53 is a dancer) is described as an example, but this is not limited to this example. That is, the live event production assistance system of the present disclosure can also be applied to various live events other than concerts (for example, various plays, sporting events, exhibitions, expositions, festival events, etc.).

Furthermore, with this technology, it is possible to apply the contents described so far in any combination.

The effects described in this specification are merely examples and are not limiting, and other effects may also be present.

11...server, 12...performance control device, 2a, 2b...smart devices, 21...information processing device, 211...transmitter/receiver, 212...analysis processing unit, 213...data conversion processing unit, 22...vibration sensor, 23a...vibration device, 23b...light-emitting device, 3...performance support system, 30...network, 4...performance device, 51...performer, 52...audience, 53...performance assistant, 61...concert venue, 62...home, Dv...vibration data, De1, De2...excitement level data, Sc1, Sc2...control signal, Ss...selection signal.

Claims (12)

A system for assisting in directing a live event conducted by a performer and a plurality of participants including an audience, comprising:
A first information processing device disposed on the performer side;
A performance control device that controls the performance device disposed on the performer side;
A second information processing device disposed on the participant side;
An output device arranged on the participant side and performing a predetermined output operation;
A sensor for detecting the degree of excitement among the participants,
In the first production assistance process as an assistance process of the production from the performer side to the participant side,
The first information processing device transmits a control signal input in response to an instruction from the performer to the second information processing device,
the second information processing device controls the output operation of the output device based on the control signal received from the first information processing device;
In the second production assistance process as an assistance process of the production from the participant side to the performer side,
The second information processing device performs a predetermined analysis process based on the detection result of the excitement level by the sensor, and transmits excitement level data obtained by the analysis process to the first information processing device,
A live event production assistance system in which the first information processing device reflects the excitement level of a specific participant selected from the plurality of participants in the control operation of the production device by the production control device based on the excitement level data obtained for the specific participant. The live event production assistance system of claim 1, wherein the specific participant is selected within the first information processing device based on a selection signal input to the first information processing device in response to instructions from the performer. A system for assisting in directing a live event conducted by performers and participants including audience members and directing assistants, comprising:
A first information processing device disposed on the performer side;
A performance control device that controls the performance device disposed on the performer side;
A second information processing device disposed on the participant side;
an output device arranged on the participant side and performing a predetermined output operation;
A sensor for detecting the degree of excitement among the participants;
Equipped with
In the first production assistance process as an assistance process of the production from the performer side to the participant side,
The first information processing device transmits a control signal input in response to an instruction from the performer to the second information processing device,
the second information processing device controls the output operation of the output device based on the control signal received from the first information processing device;
In the second production assistance process as an assistance process of the production from the participant side to the performer side,
The second information processing device performs a predetermined analysis process based on the detection result of the excitement level by the sensor, and transmits excitement level data obtained by the analysis process to the first information processing device,
The first information processing device reflects the degree of excitement of the participants based on the excitement level data received from the second information processing device, in a control operation of the production device by the production control device, taking into consideration a detection result of the degree of excitement of the production assistant as the participant.
Live event production support system. The first information processing device,
Based on the plurality of pieces of excitement level data individually obtained for the plurality of participants,
The live event production assistance system according to claim 3 , wherein the overall level of excitement among the plurality of participants is reflected in a control operation of the production device by the production control device. The second information processing device is
By performing the analysis process using a machine learning model,
5. The live event production assistance system according to claim 1, further comprising : determining a plurality of stages of evaluation values relating to the degree of excitement to generate the degree of excitement data. A live event production assistance system as described in any one of claims 1 to 5 , wherein the excitement level data transmitted and received during the second production assistance processing is data that has been subjected to a predetermined conversion processing to reduce data volume within the second information processing device. The live event production assistance system according to any one of claims 1 to 6 , wherein the second information processing device, the output device and the sensor are integrated and mounted within a single device on the participant side. The live event production assistance system according to claim 1 , wherein the output operation in the output device includes at least one of a vibration operation, a light emission operation, a video output operation, and a signal transmission operation. 9. The live event production assistance system according to claim 1, wherein the degree of excitement is determined using at least one parameter of vibrations caused by a predetermined movement of the participant, the body temperature of the participant, the pulse rate of the participant, the sound emitted by the participant, and the brightness of the participant's surrounding environment. The live event production assistance system according to claim 1 , wherein the second information processing device, the output device, and the sensor operate using power supplied from a vibration power generating element. The live event production assistance system according to any one of claims 1 to 10 , wherein the live event is an online event in which the performers and the audience are located in different locations. the live event is a concert;
A live event production assistance system as described in any one of claims 1 to 11, wherein the performers are musicians and the participant production assistants are dancers.

Images