JP7528883B2 - Production control method, production control system, and program - Google Patents

Description

Application of Article 30, Paragraph 2 of the Patent Act A1: May 13, 2020 Demonstration experiment at Ecopa Stadium (2300-1 Aino, Fukuroi City, Shizuoka Prefecture) A2: May 18, 2020 "With the cooperation of Yamaha, Jubilo Iwata and Shimizu S-Pulse, a demonstration experiment of the remote cheering system was conducted at Ecopa Stadium" B1: June 13, 2020 Demonstration experiment at Yamaha Stadium (2500 Shingai, Iwata City, Shizuoka Prefecture) B2: June 5, 2020 "The first public experiment of the remote cheering system was conducted with Jubilo Iwata during a live broadcast match" B3: June 11, 2020 "6/13 (Sat) Practice match vs. Numazu / How to participate in the remote cheering system" C1: June 13, 2020 Demonstration experiment at IAI Stadium Nihondaira (3880-1 Muramatsu, Shimizu Ward, Shizuoka City, Shizuoka Prefecture) C2: June 25, 2020 "Yamaha and Shimizu S-Pulse conduct joint verification for practical use of remote cheering system" C3: June 25, 2020 "First verification of new functions of remote cheering system in collaboration with Yamaha Corporation" D1: June 26, 2020 "Launch of the first remote cheering project with J-League clubs, aiming for a new style of spectating that takes universal design into consideration" D2: June 27, 2020 Demonstration experiment at Yamanashi Chuo Bank Stadium (840 Ozemachi, Kofu City, Yamanashi Prefecture) D3: June 27, 2020 Demonstration experiment at Fukuda Denshi Arena (1-20 Kawasakimachi, Chuo Ward, Chiba City, Chiba Prefecture) D4: June 27, 2020 Demonstration experiment at City Light Stadium (2-1 Izumimachi, Kita Ward, Okayama City, Okayama Prefecture) D5: June 28, 2020 Demonstration experiment at Tapic Kenso Hiyagon Stadium (5-3-1 Hiyagon, Okinawa City, Okinawa Prefecture) D6: June 27, 2020 Demonstration experiment at Toho Minna no Stadium (1 Jinjiba, Sawara, Fukushima City, Fukushima Prefecture) D7: June 27, 2020 Demonstration experiment at Aitaka Regional Park Multipurpose Stadium (202 Ashitaka, Numazu City, Shizuoka Prefecture) D8: June 27, 2020 Demonstration experiment at Gifu Memorial Center Nagaragawa Stadium (2675-28 Nagara Fukumitsu Ono, Gifu City, Gifu Prefecture) D9: June 27, 2020 Demonstration experiment at Egao Health Stadium (2776 Hirayama-cho, Higashi-ku, Kumamoto City, Kumamoto Prefecture) D10: June 28, 2020 Demonstration experiment at Toyama Prefectural General Athletic Park Athletics Stadium (368 Minami-Nakada, Toyama City, Toyama Prefecture) D11: June 28, 2020 Demonstration experiment at Pikara Stadium (830 Kanakura-cho, Marugame City, Kagawa Prefecture)

This disclosure relates to technology for controlling the effects of various events.

Various technologies have been proposed for remotely participating in various events, such as sports or music events. For example, Patent Document 1 discloses a technology for playing back sound from a sound emitting device installed in a facility, such as a stadium where an event is held, by communicating with a terminal device located remotely from the facility.

U.S. Pat. No. 9,131,016

Meanwhile, users outside the facility where the event is held can view the content distributed by recording the event in parallel with the progress of the event. However, delays occur in the distribution of the content due to various circumstances. In other words, the point at which a user viewing the content recognizes a specific event in the event is delayed from the point at which the event actually occurs in the event. Therefore, for example, in the configuration of Patent Document 1, the sound designated by the user for a specific event in the content (e.g., a scoring scene) is reflected in the facility a reasonable amount of time after the event occurred in the actual event. In consideration of the above circumstances, one aspect of the present disclosure aims to reflect the instructions from the user viewing the content of the event in the production effects in the facility without excessive delay in the progress of the event.

In order to solve the above problems, a production control method according to one aspect of the present disclosure outputs a production effect to the facility where the event is held in response to a first instruction that occurs in parallel with the progress of the event, activates a second instruction from a user who is viewing content that is distributed by recording the event in parallel with the progress of the event in response to the first instruction, and controls the production effect in response to the activated second instruction.

In one aspect of the present disclosure, a method of operating a terminal device includes a terminal device capable of communicating with a performance control system that outputs a performance effect to a facility where an event is held in response to a first instruction that occurs in parallel with the progress of an event, and when the terminal device receives first instruction data indicating the first instruction from the performance control system, the terminal device accepts a second instruction from a user who is viewing content that is distributed by recording the event in parallel with the progress of the event, and when the second instruction is accepted, transmits second instruction data for controlling the performance effect to the performance control system.

A performance control system according to one aspect of the present disclosure includes an output control unit that, in response to a first instruction that occurs in parallel with the progress of an event, causes an output device in the facility where the event is held to output a performance effect, and an instruction management unit that, in response to the first instruction, activates a second instruction from a user who is viewing content that is distributed by recording the event in parallel with the progress of the event, and the output control unit controls the performance effect in response to the activated second instruction.

A terminal device according to one aspect of the present disclosure is a terminal device capable of communicating with a performance control system that outputs a performance effect to a facility where an event is held in response to a first instruction that occurs in parallel with the progress of the event, and includes an instruction receiving unit that, when first instruction data indicating the first instruction is received from the performance control system, receives a second instruction from a user who is viewing content distributed by recording the event in parallel with the progress of the event, and a transmission processing unit that, when the instruction receiving unit receives the second instruction, transmits second instruction data for controlling the performance effect to the performance control system.

1 is a block diagram illustrating a configuration of an information system according to a first embodiment. 2 is a block diagram illustrating a configuration of a management terminal. FIG. 2 is a block diagram illustrating an example of the functional configuration of a performance management system and a terminal device. 11 is a flowchart illustrating a procedure of an instruction receiving process executed by a management terminal; 13 is a schematic diagram of a selection screen displayed on a management terminal. FIG. 2 is a block diagram illustrating a configuration of a control system. 11 is a flowchart illustrating an example of a procedure for an output control process. FIG. 2 is a block diagram illustrating a configuration of a terminal device. 10 is a flowchart illustrating a procedure of an instruction receiving process executed by a terminal device. FIG. 13 is a schematic diagram of a selection screen displayed on a terminal device. FIG. 13 is a schematic diagram of a selection screen displayed on a terminal device. FIG. 11 is a schematic diagram of a selection screen displayed on a terminal device according to a second embodiment. 13 is a flowchart illustrating a procedure of an instruction receiving process in the second embodiment. 10 is a flowchart illustrating a procedure of an output control process in a second embodiment. FIG. 13 is a block diagram illustrating a functional configuration of a control system and a terminal device according to a third embodiment. 13 is a flowchart illustrating a procedure of an output control process in a third embodiment. FIG. 13 is a block diagram illustrating a functional configuration of a control system and a terminal device according to a fourth embodiment. 13 is a flowchart illustrating a procedure of an instruction receiving process in the fourth embodiment. 13 is a flowchart illustrating a procedure of an output control process in the fourth embodiment. FIG. 13 is a block diagram illustrating a functional configuration of a management terminal and a control system according to a fifth embodiment. 13 is a flowchart illustrating a procedure of an output control process in the fifth embodiment.

A: First embodiment Fig. 1 is a block diagram illustrating a configuration of an information system 100 according to a first embodiment. The information system 100 is a computer system capable of communicating with each of a plurality of terminal devices 50 via a communication network 300 such as the Internet. Each of the plurality of terminal devices 50 is an information terminal such as a mobile phone, a smartphone, a tablet terminal, or a personal computer.

The information system 100 includes a management terminal 10, a playback system 20, a control system 30, a recording system 41, and a distribution system 42. The management terminal 10, the playback system 20, and the recording system 41 are installed in a facility 200 where various events are held. The facility 200 is a stadium where an event (hereinafter referred to as a "competitive event") is held in which multiple participants compete in a specific sport. Normally, a large number of spectators in the facility 200 watch the competitive event, but due to various circumstances, such as preventing the spread of infectious diseases, a competitive event (i.e., a match without spectators) may be held in a situation where no spectators are present in the facility 200. The control system 30 or the distribution system 42 may be installed in the facility 200.

The management terminal 10 is a terminal device used by an administrator A located in the facility 200. For example, an information terminal such as a mobile phone, smartphone, tablet terminal, or personal computer is used as the management terminal 10. While watching the competitive event in the facility 200, the administrator A can operate the management terminal 10 in parallel with the progress of the competitive event. The functions of the management terminal 10 may be installed in the control system 30. The administrator A is, for example, an employee of the organization that operates the information system 100, or a call leader selected from supporters who cheer on the participants in the competitive event.

The playback system 20 is an audio system composed of audio equipment installed within the facility 200, and reproduces sound within the facility 200. The playback system 20 includes, for example, multiple sound emitting devices (e.g., speaker devices) installed at different locations within the facility 200. The playback system 20 and the control system 30 may be configured as a single device. The playback system 20 is an example of an "output device."

The control system 30 is a computer system that controls the reproduction of sounds by the reproduction system 20. Specifically, the control system 30 causes the reproduction system 20 to reproduce various sound effects. The sound effects are sounds for production related to the competitive event. For example, the singing sounds of cheering songs and instrument sounds for the participants of the competitive event, the cheers of many spectators, and the sighing sounds of spectators when disappointed with an event that occurs in the competitive event (for example, a mistake by a contestant) are exemplified as sound effects. The contestants of the competitive event can hear the sound effects reproduced by the reproduction system 20. By reproducing the sound effects by the reproduction system 20 as described above, even in a situation where there are no spectators or there are few spectators in the facility 200, an atmosphere as if a large number of spectators were watching the competitive event can be created in the facility 200. As exemplified in FIG. 1, the management terminal 10, the reproduction system 20, and the control system 30 function as a production control system 400 that controls the reproduction of sound effects in the facility 200. The playback system 20 may be excluded from the elements of the performance control system 400.

The recording system 41 records the competitive event held within the facility 200. Specifically, the recording system 41 includes an imaging device that captures video of the competitive event, and a sound collection device that collects sound from the competitive event. The sound collected by the sound collection device includes sound produced by the contestants or spectators within the facility 200, as well as sound effects played back by the playback system 20. A video V composed of the video captured by the imaging device and the sound collected by the sound collection device is generated by the recording system 41. The data format of the video V is arbitrary.

The distribution system 42 distributes content C including the video V recorded by the recording system 41 to multiple terminal devices 50. Specifically, the distribution system 42 distributes content C to each terminal device 50 in real time in parallel with the progress of the competition event (i.e., live distribution by streaming). The distribution system 42 generates content C by performing various image processing on the video V recorded by the recording system 41, such as compositing images such as commentary or subtitles, or adjusting image quality.

Each of the multiple terminal devices 50 plays content C distributed by the distribution system 42. A user U of each terminal device 50 carries the terminal device 50 and is located outside the facility 200 (e.g., at home). Each user U watches the content C played by the terminal device 50 in parallel with the progress of the competitive event. However, the playback of content C by the terminal device 50 is delayed with respect to the actual progress of the competitive event. For example, the point at which user U recognizes a specific event in the competitive event (e.g., a scoring scene) by watching content C is delayed with respect to the point at which the event actually occurred in the competitive event in the facility 200.

As can be understood from the above explanation, while the manager A watches the competition event without delay within the facility 200, each user U watches content C with a delay relative to the progress of the competition event. The delay time in playing content C relative to the progress of the competition event depends on various factors, such as the communication environment between the recording system 41 and the distribution system 42, the total number of terminal devices 50 to which the distribution system 42 distributes content C, or the time required for image processing by the distribution system 42 to generate content C.

Figure 2 is a block diagram illustrating the configuration of a management terminal 10. The management terminal 10 comprises a control device 11, a storage device 12, a communication device 13, a display device 14, and an operation device 15. The management terminal 10 may be realized as a single device, or as a collection of multiple devices configured separately from each other.

The control device 11 is composed of one or more processors that control each element of the management terminal 10. For example, the control device 11 is composed of one or more types of processors, such as a CPU (Central Processing Unit), a GPU (Graphic Processing Unit), a DSP (Digital Signal Processor), an FPGA (Field Programmable Gate Array), or an ASIC (Application Specific Integrated Circuit).

The storage device 12 is a single or multiple memories that store the programs executed by the control device 11 and various data used by the control device 11. The storage device 12 is configured from a known recording medium, such as a magnetic recording medium or a semiconductor recording medium. The storage device 12 may be configured from a combination of multiple types of recording media. Note that the storage device 12 may be a portable recording medium that is detachable from the management terminal 10, or a recording medium (e.g., cloud storage) that the management terminal 10 can write to and read from via the communication network 300.

The communication device 13 communicates with the control system 30 via the communication network 300. The communication device 13 and the control system 30 are connected via a communication line 500. As illustrated in FIG. 1, the communication device 13 performs wired communication with the control system 30 via the communication line 500. However, wireless communication may also be performed between the communication device 13 and the control system 30. For example, the communication device 13 performs short-range wireless communication with the control system 30 using Bluetooth (registered trademark) or Wi-Fi (registered trademark), etc.

The display device 14 in FIG. 2 displays an image under the control of the control device 11. The display device 14 is configured with a display panel such as a liquid crystal panel or an organic EL (ElectroLuminescence) panel. The operation device 15 is an input device that accepts instructions from the administrator A. The operation device 15 is, for example, a plurality of operators operated by the administrator A, or a touch panel that is configured integrally with the display device 14.

Figure 3 is a block diagram illustrating the functional configuration of the management terminal 10, the control system 30, and the terminal device 50. The control device 11 of the management terminal 10 realizes multiple functions (an instruction receiving unit 111 and a transmission processing unit 112) by executing a program stored in the storage device 12.

The instruction receiving unit 111 receives an instruction (hereinafter referred to as a "first instruction") Q1 from the manager A to select a sound effect to be played within the facility 200 from among the multiple sound effects. The transmission processing unit 112 transmits first instruction data D1 indicating the first instruction Q1 received by the instruction receiving unit 111 from the communication device 13 to the control system 30. For example, the first instruction data D1 is data including identification information of the sound effect selected by the manager A via the first instruction Q1.

Figure 4 is a flowchart illustrating the procedure of the process Sa (hereinafter referred to as the "instruction reception process") that the control device 11 of the management terminal 10 executes in relation to the first instruction Q1. The instruction reception process Sa is started, for example, in response to an instruction from the administrator A at the start of the competition event, and continues in parallel with the progress of the competition event.

When the instruction reception process Sa is started, the instruction reception unit 111 displays the selection screen Ga of FIG. 5 on the display device 14 (Sa11). The selection screen Ga is an image for the manager A to select one of a plurality of sound effects. Specifically, a plurality of operation images Wa corresponding to different sound effects are arranged on the selection screen Ga. Each operation image Wa is a software button that the manager A can operate using the operation device 15. "Chant" in FIG. 5 means the singing sound and instrument sound of a cheer song for the participants of the competitive event. "Large cheer" means the cheer sound uttered by the audience in a particularly exciting scene (e.g., a score scene) in the competitive event. "Small cheer" means a cheer sound that is smaller than "large cheer". "Disappointment" means the sigh sound uttered by the audience when they are disappointed.

By operating the operation device 15 while watching the competitive event, the manager A can select one of the multiple sound effects from the selection screen Ga. Specifically, the manager A selects an appropriate sound effect from the multiple sound effects for the current situation at an appropriate time according to the situation of the competitive event. For example, if a contestant wins a point in the competitive event, the manager A selects "loud cheers" from the selection screen Ga. Also, if a contestant makes a mistake in the competitive event, the manager A selects "disappointment" from the selection screen Ga. The instruction receiving unit 111 determines whether or not a first instruction Q1 to select a sound effect has been received from the manager A (Sa12).

When the instruction receiving unit 111 receives a first instruction Q1 from the administrator A (Sa12: YES), the transmission processing unit 112 transmits first instruction data D1 indicating the first instruction Q1 from the communication device 13 to the control system 30 (Sa13). On the other hand, when the instruction receiving unit 111 does not receive the first instruction Q1 (Sa12: NO), the transmission of the first instruction data D1 (Sa13) is not executed. Note that the first instruction data D1 transmitted from the management terminal 10 to the control system 30 and the first instruction data D1 transmitted from the control system 30 to each terminal device 50 do not necessarily have to be the same.

The control device 11 determines whether or not the administrator A has instructed the end of the instruction reception process Sa (Sa14). For example, when a competitive event has ended, the administrator A instructs the end of the instruction reception process Sa. If the end of the instruction reception process Sa is not instructed (Sa14: NO), the transmission (Sa13) of the first instruction data D1 is repeated on the condition that the first instruction Q1 is accepted (Sa12: YES). On the other hand, if the end has been instructed by the administrator A (Sa14: YES), the control device 11 ends the instruction reception process Sa.

Figure 6 is a block diagram illustrating the configuration of the control system 30. The control system 30 includes a control device 31, a storage device 32, and a communication device 33. The control system 30 may be realized as a single device, or as a collection of multiple devices configured separately from each other.

The control device 31 is composed of one or more processors that control each element of the control system 30. For example, the control device 31 is composed of one or more types of processors, such as a CPU, GPU, DSP, FPGA, or ASIC. The communication device 33 communicates with the management terminal 10 via a communication line 500. The communication device 33 also communicates with each terminal device 50 via a communication network 300. The communication device 33 may be composed of a communication device that communicates with the management terminal 10 and a communication device that communicates with the terminal devices 50.

The storage device 32 is a single or multiple memories that store the programs executed by the control device 31 and various data used by the control device 31. The storage device 32 is configured from a known recording medium, such as a magnetic recording medium or a semiconductor recording medium. The storage device 32 may be configured from a combination of multiple types of recording media. Note that a portable recording medium that is detachable from the control system 30, or a recording medium (e.g., cloud storage) that the control system 30 can write to and read from via the communication network 300 may be used as the storage device 32.

The storage device 32 stores a plurality of audio signals X corresponding to different sound effects. The audio signal X for each sound effect is a signal that represents the waveform of that sound effect. Each audio signal X is recorded or synthesized in advance and then stored in the storage device 32. The audio signal X is expressed in any format, such as MP3 (MPEG-1 Audio Layer-3). In addition, identification information is registered in the storage device 32 for each of the plurality of terminal devices 50 that use the performance control system 400.

As illustrated in FIG. 3, the control device 31 of the control system 30 realizes multiple functions (a receiving processing unit 311, an instruction management unit 312, and an output control unit 313) by executing a program stored in the storage device 32.

The receiving processing unit 311 receives the first instruction data D1 transmitted from the management terminal 10 by the communication device 33. The receiving processing unit 311 also receives the second instruction data D2 transmitted from each terminal device 50 by the communication device 33. The second instruction data D2 transmitted from each terminal device 50 is data indicating an instruction Q2 from the user U to that terminal device 50 (hereinafter referred to as the "second instruction"). The second instruction Q2 is an instruction by the user U of the terminal device 50 to select the sound effect indicated by the first instruction data D1 (i.e. the sound effect selected by the administrator A). The second instruction data D2 will be described in detail later.

In response to a first instruction Q1 from administrator A, the instruction management unit 312 transmits the first instruction data D1 received by the reception processing unit 311 to each terminal device 50 via the communication device 33. Upon receiving the first instruction data D1, the terminal device 50 becomes capable of validly accepting a second instruction Q2. In other words, the instruction management unit 312 validates the second instruction Q2 from the user U to each terminal device 50. The output control unit 313 controls the playback of sound effects by the playback system 20 in accordance with the first instruction data D1 or the second instruction data D2 received by the reception processing unit 311.

Figure 7 is a flowchart illustrating the procedure of the process Sb (hereinafter referred to as "output control process") that the control device 31 of the control system 30 executes regarding the playback of sound effects. The output control process Sb, like the instruction reception process Sa described above, is started, for example, when the competitive event starts, and is executed continuously in parallel with the progress of the competitive event.

When the output control process Sb is started, the receiving processing unit 311 determines whether the first instruction data D1 has been received from the management terminal 10 via the communication device 33 (Sb11). If the receiving processing unit 311 has received the first instruction data D1 (Sb11: YES), the instruction management unit 312 transmits the first instruction data D1 from the communication device 33 to each terminal device 50 (Sb12). The first instruction data D1 is transmitted to each terminal device 50 registered in the storage device 32 as its destination.

Also, when the first instruction data D1 is received (Sb11: YES), the output control unit 313 causes the reproduction system 20 to reproduce the sound effect indicated by the first instruction data D1 (Sb13). Specifically, the output control unit 313 supplies the sound signal X corresponding to the sound effect indicated by the first instruction data D1, among the multiple sound signals X stored in the storage device 32, to the reproduction system 20, thereby causing the reproduction system 20 to reproduce the sound effect. That is, the sound effect selected by the manager A from the selection screen Ga is reproduced in the facility 200 by the reproduction system 20. When the reception processing unit 311 does not receive the first instruction data D1 (Sb11: NO), the transmission of the first instruction data D1 (Sb12) and the reproduction of the sound effect (Sb13) are not executed. Note that the order of the transmission of the first instruction data D1 (Sb12) and the reproduction of the sound effect (Sb13) may be reversed.

As can be understood from the above explanation, the first instruction Q1 from manager A triggers playback of sound effects by the playback system 20. As described above, manager A watches the competitive event without delay, so appropriate sound effects according to the situation of the competitive event are played within the facility 200 at an appropriate time without delay in the progress of the competitive event. For example, immediately after a contestant wins a point in the competitive event, playback of the sound effect "loud cheers" begins, and immediately after a contestant makes a mistake in the competitive event, playback of the sound effect "disappointment" begins. Note that if manager A selects sound effect 2 from among the multiple sound effects while sound effect 1 is being played, sound effect 1 and sound effect 2 are played overlappingly within the facility 200.

After executing the above process, the reception processing unit 311 judges whether or not the second instruction data D2 has been received from each terminal device 50 via the communication device 33 (Sb14). If the reception processing unit 311 has received the second instruction data D2 (Sb14: YES), the output control unit 313 controls the acoustic characteristics of the sound effect being reproduced by the reproduction system 20 (Sb15). Specifically, the output control unit 313 increases the volume of the sound effect indicated by the second instruction data D2. For example, the output control unit 313 controls the volume of the sound effect according to the total number of second instruction data D2 received by the reception processing unit 311 (hereinafter referred to as the "instruction number"). For example, the output control unit 313 increases the volume of the sound effect by an increase amount according to the instruction number. In other words, the larger the instruction number, the greater the increase amount of the volume. The trajectory of the change in the volume of the sound effect is arbitrary, but for example, a configuration in which the volume is increased over time over a predetermined time is assumed. When multiple sound effects are being played back in an overlapping manner, the volume of the sound effect indicated by the second instruction data D2 among the multiple sound effects is increased. When sound effects with opposite meanings, such as "cheer" and "sigh," are indicated by the user U, the acoustic characteristics of the sound effect that is selected by the majority of users U using the second instruction Q2 may be controlled.

The control device 31 determines whether or not an instruction to end the output control process Sb has been given (Sb16). For example, when a competition event ends, the operator of the control system 30 gives an instruction to end the output control process Sb. If an instruction to end the output control process Sb has not been given (Sb16: NO), the transmission of the first instruction data D1 (Sb12) and the start of playback of sound effects (Sb13) triggered by the first instruction Q1, and the control of the sound effects (Sb15) triggered by the second instruction Q2 are repeated. If an instruction to end the output control process Sb has been given (Sb16: YES), the control device 31 ends the output control process Sb.

Figure 8 is a block diagram illustrating the configuration of a terminal device 50. The terminal device 50 includes a control device 51, a storage device 52, a communication device 53, a playback device 54, and an operation device 55. The terminal device 50 may be realized as a single device, or as a set of multiple devices configured separately from each other.

The control device 51 is composed of one or more processors that control each element of the terminal device 50. For example, the control device 51 is composed of one or more types of processors such as a CPU, GPU, DSP, FPGA, or ASIC.

The storage device 52 is a single or multiple memories that store the programs executed by the control device 51 and various data used by the control device 51. The storage device 52 is configured from a known recording medium, such as a magnetic recording medium or a semiconductor recording medium. The storage device 52 may be configured from a combination of multiple types of recording media. Note that a portable recording medium that is detachable from the terminal device 50, or a recording medium (e.g., cloud storage) that the terminal device 50 can write to and read from via the communication network 300 may be used as the storage device 52.

The communication device 53 communicates with the information system 100 (specifically, the control system 30 and the distribution system 42) via the communication network 300. For example, the communication device 53 receives content C distributed from the distribution system 42.

The playback device 54 plays video and sound under the control of the control device 51. For example, the playback device 54 plays content C received by the communication device 53. The playback device 54 includes a display device 541 and a sound emitting device 542. The display device 541 displays an image under the control of the control device 51. For example, the display device 541 displays the video of content C. The display device 541 is configured with a display panel such as a liquid crystal panel or an organic EL panel. The sound emitting device 542 plays sound under the control of the control device 51. For example, the sound emitting device 542 plays the sound of content C.

The operation device 55 is an input device that accepts instructions from the user U. The operation device 55 is, for example, a plurality of operators operated by the user U, or a touch panel that is integrated with the display device 541.

As illustrated in FIG. 3, the control device 51 of the terminal device 50 realizes multiple functions (a playback control unit 511, a reception processing unit 512, an instruction acceptance unit 513, and a transmission processing unit 514) by executing a program stored in the storage device 52. The playback control unit 511 causes the playback device 54 to play content C received by the communication device 53 from the distribution system 42.

The receiving processing unit 512 receives the first instruction data D1 transmitted from the control system 30 via the communication device 53. The instruction accepting unit 513 accepts a second instruction Q2 for selecting a sound effect from the user U. The transmitting processing unit 514 transmits second instruction data D2 indicating the second instruction Q2 accepted by the instruction accepting unit 513 from the communication device 53 to the control system 30. For example, the second instruction data D2 is data including identification information of the sound effect selected by the user U via the second instruction Q2.

Figure 9 is a flowchart illustrating the procedure of the process Sc (hereinafter referred to as the "instruction reception process") executed by the control device 51 of the terminal device 50 with respect to the second instruction Q2. The instruction reception process Sc is started in response to an instruction from the user U at the start of the competition event, for example, and is executed continuously in parallel with the progress of the competition event. In parallel with the instruction reception process Sc, playback of the content C is executed by the playback control unit 511.

When the instruction reception process Sc is started, the instruction reception unit 513 causes the display device 541 to display the selection screen Gb of FIG. 10 (Sc11). The selection screen Gb is an image that allows the user U to select one of a plurality of sound effects. Specifically, similar to the selection screen Ga described above, the selection screen Gb has a plurality of operation images Wb arranged thereon that correspond to different sound effects. Each operation image Wb is a software button that can be operated by the user U using the operation device 55.

As described above, administrator A can select any one of the multiple sound effects from the selection screen Ga. That is, the multiple operation images Wa on the selection screen Ga are maintained in an enabled state. The enabled state of the operation image Wa is a state in which a selection by administrator A can be accepted. In contrast to the selection screen Ga, the multiple operation images Wb on the selection screen Gb of the terminal device 50 are initially set to an disabled state. The disabled state of the operation image Wb is a state in which a selection by user U is not accepted. That is, the instruction accepting unit 513 ignores an operation from user U on an operation image Wb in an disabled state.

On the other hand, when the administrator A selects a sound effect by the first instruction Q1, as illustrated in FIG. 11, among the multiple operation images Wb on the selection screen Gb, the operation image Wb corresponding to the sound effect selected by the first instruction Q1 is selectively set to an enabled state. The enabled state of the operation image Wb is a state in which a selection by the user U can be accepted. In other words, the second instruction Q2 for selecting the sound effect selected by the first instruction Q1 is enabled. FIG. 11 illustrates a state in which the operation image Wb for the sound effect "chant" is changed to an enabled state, and the other operation images Wb are maintained in an disabled state. The display manner of the operation image Wb in the enabled state and the operation image Wb in the disabled state differs. For example, the operation image Wb in the disabled state is displayed in a grayed-out state, whereas the operation image Wb in the enabled state is displayed in a specific hue.

The reception processing unit 512 determines whether the first instruction data D1 has been received from the control system 30 by the communication device 53 (Sc12). If the reception processing unit 512 has received the first instruction data D1 (Sc12: YES), the instruction acceptance unit 513 sets the operation image Wb corresponding to the sound effect indicated by the first instruction data D1 among the multiple operation images Wb on the selection screen Gb to an active state (Sc13), as described above with reference to Figures 10 and 11. For example, the instruction acceptance unit 513 changes the display mode of the operation image Wb from an inactive state to an active state.

As can be understood from the above explanation, the second instruction Q2, in which the user U selects a sound effect selected by the administrator A from among multiple sound effects, is enabled by receiving the first instruction data D1. For example, if a contestant wins a point in a competitive event and the administrator A selects the sound effect "loud cheers" with the first instruction Q1, the second instruction Q2 for selecting the sound effect "loud cheers" is enabled. Also, if a contestant makes a mistake in a competitive event and the administrator A selects the sound effect "disappointment" with the first instruction Q1, the second instruction Q2 for selecting the sound effect "disappointment" is enabled. As can be understood from the above explanation, the first instruction data D1 is data for enabling the second instruction Q2 from the user U to the terminal device 50.

As described above, the playback of content C is delayed relative to the progress of the competitive event. On the other hand, the time from the generation of first instruction Q1 until the first instruction data D1 arrives at the terminal device 50 is sufficiently short compared with the delay time of the playback of content C. Therefore, the first instruction Q1 triggers the activation of the second instruction Q2 immediately after the occurrence of a specific event that occurs in the competitive event, prior to the time when the user U recognizes the event by viewing content C. Note that, if the administrator A selects sound effect 2 from among the multiple sound effects while sound effect 1 is being played back, the operation image Wb corresponding to sound effect 1 and the operation image Wb corresponding to sound effect 2 are set to an active state in parallel on the selection screen Gb. In other words, the user U is able to select either sound effect 1 or sound effect 2.

The instruction receiving unit 513 determines whether or not a second instruction Q2 for selecting an operation image Wb in a valid state has been received from the user U (Sc14). If the instruction receiving unit 513 has received the second instruction Q2 from the user U (Sc14: YES), the transmission processing unit 514 transmits second instruction data D2 indicating the second instruction Q2 from the communication device 53 to the control system 30 (Sc15). On the other hand, if the instruction receiving unit 513 has not received the second instruction Q2 (Sc14: NO), the transmission of the second instruction data D2 (Sc15) is not executed.

The control device 51 determines whether the user U has instructed the end of the instruction reception process Sc (Sc16). For example, when a competition event has ended, the user U instructs the end of the instruction reception process Sc. If the end of the instruction reception process Sc is not instructed (Sc16: NO), the activation of the second instruction Q2 on the condition that the first instruction data D1 is received (Sc13) and the transmission of the second instruction data D2 on the condition that the second instruction Q2 is received (Sc15) are repeated. On the other hand, if the end is instructed by the user U (Sc16: YES), the control device 51 ends the instruction reception process Sc.

As described above, in the first embodiment, the first instruction Q1 that occurs in parallel with the progress of the competitive event is used as a trigger to start the reproduction of sound effects by the reproduction system 20. Therefore, it is possible to start the reproduction of sound effects in the facility 200 without delay in the progress of the competitive event. In particular, in the first embodiment, since the manager A who watches the competitive event in the facility 200 gives the first instruction Q1, the output of sound effects can be started at an appropriate time in the competitive event. In addition, the second instruction Q2 by the user U is enabled by the first instruction Q1, and the volume of the sound effects being played is controlled by the enabled second instruction Q2. Therefore, even in a situation where the user U watches the content C in an environment where the delay in the delivery of the content C relative to the progress of the competitive event is relatively large, it is possible to reflect the second instruction Q2 from each user U in the sound effects in the facility 200 without excessive delay in the progress of the competitive event.

In the first embodiment, the second instruction Q2 for selecting a sound effect selected by the first instruction Q1 from among the multiple sound effects displayed on the selection screen Gb is selectively enabled. Therefore, the second instruction Q2 from a large number of users U is simultaneously reflected in the sound effects in the facility 200 at a time when there is little delay in the progress of the competition event. Therefore, a special effect is realized in which multiple users U can participate in effective cheering from outside the facility 200, in which each user U can perceive a sense of unity. The sense of unity among users U is particularly important for sound effects such as the above-mentioned chants (cheering songs). In addition, according to the configuration in which the second instruction Q2 is enabled in response to the first instruction Q1, there is also the advantage that the possibility of the user U erroneously selecting a sound effect not selected by the first instruction Q1 from among the multiple sound effects (for example, a sound effect that is not appropriate for the situation of the competition event) can be reduced.

In the first embodiment, the volume of the sound effects played within the facility 200 is controlled in response to the second instruction Q2 from the user U. Therefore, listeners of the sound effects within the facility 200 (e.g., participants in a competitive event) can auditorily grasp the viewing situation regarding the content C of the competitive event, such as the number of users U who give the second instruction Q2.

For example, in a configuration in which content C recording a competitive event is simply distributed to multiple terminal devices 50, the reactions of users U watching the competitive event at locations other than the facility 200 where the competitive event is held (e.g., cheers for the contestants) are not transmitted within the facility 200. According to the first embodiment, the reactions of users U of each terminal device 50 are reflected in the sound effects within the facility 200, making it possible to increase the number of users U who can contribute to the competitive event.

B: Second embodiment A second embodiment will be described. Note that, for elements in the following exemplary aspects that have the same functions as those in the first embodiment, the same reference numerals as those used in the description of the first embodiment will be used and detailed descriptions of each will be omitted as appropriate.

12 is a schematic diagram of the selection screen Gb displayed on the display device 541 of the terminal device 50 in the second embodiment. As illustrated in FIG. 12, the selection screen Gb in the second embodiment includes an area R1 and an area R2. In the area R1, a plurality of operation images Wb corresponding to different sound effects (hereinafter sometimes referred to as "first sound effects") are arranged, as in the first embodiment. On the other hand, in the area R2, a plurality of operation images Wc corresponding to different sound effects (hereinafter sometimes referred to as "second sound effects") are arranged. The user U can select one of the plurality of operation images Wc in the area R2 by operating the operation device 55. The "cheers" exemplified as the second sound effect in FIG. 12 means the cheers uttered by the audience. "Applause" means the applause sound produced by the audience's applause. "Whistle" means the whistle sound produced by whistling. Also, "disappointment" means the sigh sound produced when the audience is disappointed. The first sound effect presented in region R1 and the second sound effect presented in region R2 may be of the same type or different types. The storage device 32 of the control system 30 stores the sound signal X representing each of the second sound effects in addition to the sound signal X representing each of the first sound effects.

As with the first embodiment, the operation image Wb corresponding to the first sound effect selected by the administrator A in the first instruction Q1 is selectively set to the active state among the multiple operation images Wb in the region R1. On the other hand, the multiple operation images Wc in the region R2 are maintained in the active state regardless of the presence or absence of the first instruction Q1. That is, the user U can always operate any of the multiple operation images Wc. The instruction receiving unit 513 receives from the user U an instruction (hereinafter referred to as a "third instruction") Q3 to select any of the multiple operation images Wc in the region R2, in addition to a second instruction Q2 to select any of the multiple operation images Wb in the region R1. The third instruction Q3 is an instruction to select any of the multiple second sound effects. The user U can select a desired sound effect in the region R2 at any time by operating the operating device 55 while viewing the content C.

Figure 13 is a flow chart illustrating the procedure of the instruction reception process Sc executed by the control device 51 of the terminal device 50 in the second embodiment. The instruction reception unit 513 causes the display device 541 to display the selection screen Gb illustrated in Figure 12 (Sc21). As in the first embodiment, the second instruction Q2 is enabled (Sc23) on the condition that the first instruction data D1 is received (Sc22: YES), and the second instruction data D2 is transmitted to the control system 30 on the condition that the second instruction Q2 is received (Sc24) (Sc25).

The instruction receiving unit 513 determines whether or not a third instruction Q3 for selecting any operation image Wb in the region R2 has been received from the user U (Sc26). If the instruction receiving unit 513 has received the third instruction Q3 from the user U (Sc26: YES), the transmission processing unit 514 transmits third instruction data D3 representing the third instruction Q3 from the communication device 53 to the control system 30 (Sc27). The third instruction data D3 is, for example, data including identification information of the second sound effect selected by the user U by the third instruction Q3. On the other hand, if the instruction receiving unit 513 does not receive the third instruction Q3 (Sc26: NO), the transmission of the third instruction data D3 (Sc27) is not executed. The above process is repeated until the user U instructs to end the instruction receiving process Sc (Sc28: NO), which is the same as in the first embodiment.

Figure 14 is a flow chart illustrating the procedure of the output control process Sb executed by the control device 31 of the control system 30 in the second embodiment. As in the first embodiment, the first instruction data D1 is transmitted to each terminal device 50 (Sb22) and sound effects are played (Sb23) on the condition that the first instruction data D1 is received (Sb21: YES), and sound effects are controlled (Sb25) on the condition that the second instruction data D2 is received (Sb24: YES).

The receiving processing unit 311 determines whether the third instruction data D3 has been received from each terminal device 50 by the communication device 33 (Sb26). When the receiving processing unit 311 has received the third instruction data D3 (Sb26: YES), the output control unit 313 causes the reproduction system 20 to reproduce the second sound effect indicated by the third instruction data D3 (Sb27). Specifically, the output control unit 313 causes the reproduction system 20 to reproduce the second sound effect by supplying the sound signal X corresponding to the second sound effect indicated by the third instruction data D3 among the multiple sound signals X stored in the storage device 32 to the reproduction system 20. Note that a configuration is also envisioned in which the second sound effect is reproduced by the reproduction system 20 when the number of times the third instruction data D3 indicating each second sound effect has been received exceeds a predetermined threshold, and the second sound effect is not reproduced when the number of times is below the threshold. As can be understood from the above explanation, the second instruction Q2 is used to control the volume of the first sound effect that is played in response to the first instruction Q1, while the third instruction Q3 is used as a trigger for starting playback of the second sound effect. The above processing is repeated until an instruction to end the output control processing Sb is issued (Sb28: NO), which is the same as in the first embodiment.

In the second embodiment, the same effect as in the first embodiment is achieved. Furthermore, in the second embodiment, regardless of whether the manager A issues the first instruction Q1, the user U can play the second sound effect within the facility 200 by issuing the third instruction Q3.

In addition to the first operation mode in which the operation image Wb in the region R1 is selectively controlled to be in an enabled state in response to the first instruction Q1 as in the second embodiment, a configuration is also envisioned in which the terminal device 50 operates in a second operation mode in which the multiple operation images Wb in the region R1 are forcibly maintained in an disabled state. In the second operation mode, the user U is only permitted to operate the multiple operation images Wc in the region R2.

C: Third embodiment Fig. 15 is a block diagram illustrating the functional configuration of each of the control system 30 and the terminal device 50 in the third embodiment. The performance control system 400 in the third embodiment includes the playback system 20 and the control system 30. That is, in the third embodiment, the management terminal 10 is omitted from the performance control system 400.

The instruction management unit 312 of the third embodiment analyzes the video V generated by the recording system 41. The analysis of the video V by the instruction management unit 312 is performed in parallel with the progress of the competitive event. Specifically, the instruction management unit 312 analyzes the video V of the competitive event to determine whether or not a specific event has occurred within the competitive event. The specific event is, for example, a scoring scene.

The instruction management unit 312 identifies a scoring scene by voice recognition of the sound that constitutes the video V. For example, the instruction management unit 312 determines that a scoring scene has occurred when a character string such as "GOAL!" is recognized by voice recognition of the commentary sound emitted within the facility 200. The instruction management unit 312 also identifies a scoring scene by image recognition of the images that constitute the video V. For example, the instruction management unit 312 determines that a scoring scene has occurred when a specific action that indicates a score is recognized from the video V of a referee participating in a competitive event. The instruction management unit 312 issues a first instruction Q1 when a specific event such as the one exemplified above occurs. That is, the instruction management unit 312 issues a first instruction Q1 according to the result of analyzing the situation of the competitive event.

Figure 16 is a flowchart illustrating the procedure of the output control process Sb in the third embodiment. When the output control process Sb is started, the instruction management unit 312 analyzes the video V of the competition event (Sb31). The instruction management unit 312 determines whether or not a specific event has occurred in the competition event by analyzing the video V (Sb32). If it is determined that a specific event has occurred (Sb32: YES), the instruction management unit 312 generates a first instruction Q1 (Sb33). The first instruction Q1 is an instruction to play a sound effect. The sound effect specified by the first instruction Q1 is selected from multiple sound effects depending on the type of event identified by the analysis of the video V.

Also, when it is determined that a specific event has occurred, the instruction management unit 312 transmits the first instruction data D1 indicating the first instruction Q1 from the communication device 33 to each terminal device 50 (Sb34), similar to step Sb12 in the first embodiment. Also, the output control unit 313 causes the playback system 20 to play the sound effect indicated by the first instruction Q1 (Sb35), similar to step Sb13 in the first embodiment. On the other hand, if a specific event has not occurred (Sb32: NO), the generation of the first instruction Q1 (Sb33), the transmission of the first instruction data D1 (Sb34), and the playback of the sound effect (Sb35) are not executed. As in the first embodiment, the sound effect is controlled (Sb37) on the condition that the second instruction data D2 is received (Sb36: YES), and the above processing is repeated until the end of the output control processing Sb is instructed (Sb38: NO). The other configurations and operations are the same as those in the first embodiment. Therefore, in the third embodiment, sound effects corresponding to the results of analyzing the situation of the competitive event are reproduced within the facility 200 from the reproduction system 20, and the volume of the sound effects is controlled in response to the second instruction Q2 from the user U.

The third embodiment also achieves the same effects as the first embodiment. Furthermore, in the third embodiment, playback of appropriate sound effects can be started at an appropriate time according to the situation of the competitive event, without the need for an administrator A to issue the first instruction Q1. Note that the configuration of the second embodiment, in which a valid third instruction Q3 is accepted from the user U regardless of the presence or absence of the first instruction Q1, may be applied to the third embodiment.

D: Fourth embodiment Fig. 17 is a block diagram illustrating the functional configuration of the control system 30 and the terminal device 50 in the fourth embodiment. The performance control system 400 in the fourth embodiment includes the playback system 20 and the control system 30, as in the third embodiment. That is, in the fourth embodiment, the management terminal 10 is omitted from the performance control system 400.

The terminal device 50 in the fourth embodiment includes a playback control unit 511, an instruction receiving unit 513, and a transmission processing unit 514. As in the first embodiment, the playback control unit 511 causes the playback device 54 to play back the content C that the communication device 53 receives from the distribution system 42.

In the fourth embodiment, the multiple operation images Wb on the selection screen Gb are always maintained in an active state. The instruction receiving unit 513 receives a second instruction Q2 from the user U to select one of the multiple sound effects presented on the selection screen Gb. The transmission processing unit 514 transmits second instruction data D2 indicating the second instruction Q2 received by the instruction receiving unit 513 from the communication device 53 to the control system 30.

Figure 18 is a flowchart illustrating the procedure of the instruction reception process Sc in the fourth embodiment. When the instruction reception process Sc is started, the instruction reception unit 513 displays the selection screen Gb on the display device 541, in which all operation images Wb are set to the valid state (Sc41). The instruction reception unit 513 determines whether or not a second instruction Q2 for selecting an operation image Wb has been received from the user U (Sc42). If the instruction reception unit 513 receives the second instruction Q2 from the user U (Sc42: YES), the transmission processing unit 514 transmits second instruction data D2 indicating the second instruction Q2 from the communication device 53 to the control system 30 (Sc43). If the instruction reception unit 513 does not receive the second instruction Q2 (Sc42: NO), the transmission of the second instruction data D2 (Sc43) is not executed. The above process is repeated until an instruction to end the instruction reception process Sc is issued (Sc44: NO).

As illustrated in FIG. 17, the control system 30 in the fourth embodiment includes a reception processing unit 311 and an output control unit 313. The reception processing unit 311 receives the second instruction data D2 transmitted from each terminal device 50 via the communication device 33. The output control unit 313 controls the reproduction of sound effects by the reproduction system 20 in accordance with the second instruction data D2 received by the reception processing unit 311.

Figure 19 is a flowchart illustrating the procedure of the output control process Sb in the fourth embodiment. When the output control process Sb is started, the reception processing unit 311 judges whether or not the second instruction data D2 is received from each terminal device 50 by the communication device 33 (Sb41). If the second instruction data D2 is received (Sb41: YES), the output control unit 313 judges whether or not a predetermined condition (hereinafter referred to as the "playback condition") is satisfied for the second instruction data D2 (Sb42). For example, the playback condition is a condition regarding the frequency with which the user U of each terminal device 50 instructs each sound effect. For example, the output control unit 313 counts the number of times the second instruction data D2 is received within a predetermined period (hereinafter referred to as the "instruction count") for each sound effect, and judges that the playback condition is satisfied for the sound effect whose instruction count exceeds a predetermined threshold. The instruction count is the total number of second instructions Q2 within a predetermined period. The output control unit 313 may also calculate the rate of increase in the number of indications for each sound effect, and determine that the playback condition is met for a sound effect for which the rate of increase exceeds a predetermined threshold.

When it is determined that the playback condition is met (Sb42: YES), the output control unit 313 causes the playback system 20 to play the sound effect for which the playback condition is met (Sb43). That is, the output control unit 313 supplies the playback system 20 with the sound signal X corresponding to the sound effect for which the playback condition is met, among the multiple sound signals X stored in the storage device 32. If the second instruction data D2 is not received (Sb41: NO) or the playback condition is not met (Sb42: NO), the sound effect is not played (Sb43). The above process is repeated until an instruction to end the output control process Sb is issued (Sb44: NO).

According to the fourth embodiment, the playback of sound effects is controlled according to indicators (number of instructions and rate of increase) relating to the frequency of the second instruction Q2 by the user U of each terminal device 50. Therefore, it is possible to reflect the excitement of the users U watching the content C of the competition event in the sound effects within the facility 200. Another advantage of the fourth embodiment is that it is not necessary to issue the first instruction Q1, which is the trigger for starting the playback of sound effects. However, the first instruction Q1 may be issued when a playback condition is met.

E: Fifth embodiment Fig. 20 is a block diagram illustrating the functional configuration of the management terminal 10 and the control system 30 in the fifth embodiment. The configuration and functions of the management terminal 10 in the fifth embodiment are the same as those in the first embodiment. That is, the management terminal 10 executes an instruction reception process Sa similar to that in the first embodiment, thereby transmitting first instruction data D1 indicating a first instruction Q1 received from the administrator A to the control system 30.

The control system 30 in the fifth embodiment includes a receiving processor 311 and an output controller 313. The receiving processor 311 receives the first instruction data D1 from the management terminal 10 via the communication line 500. The receiving processor 311 also receives the notification data Z transmitted from each terminal device 50. The notification data Z is data representing the attributes of the user U. Specifically, the notification data Z represents the attributes of the user U, such as the gender, age, generation, nationality, and place of residence. Each terminal device 50 transmits the notification data Z to the control system 30, for example, before the start of a competition event. The output controller 313 causes the playback system 20 to play sound effects in accordance with the first instruction data D1, as in the first embodiment. The output controller 313 in the fifth embodiment also controls the volume of the sound effects in accordance with the notification data Z received from each terminal device 50.

Figure 21 is a flowchart illustrating the procedure of the output control process Sb in the fifth embodiment. When the output control process Sb is started, the reception processing unit 311 determines whether or not the first instruction data D1 has been received from the management terminal 10 by the communication device 33 (Sb51). If the reception processing unit 311 has received the first instruction data D1 (Sb51: YES), the instruction management unit 312 causes the playback system 20 to play the sound effect indicated by the first instruction data D1 (Sb52).

The output control unit 313 of the fifth embodiment causes the playback system 20 to play the sound effect selected by the first instruction Q1 with acoustic characteristics according to the attributes of the user U of each terminal device 50. Specifically, the output control unit 313 counts the number of notification data Z received by the reception processing unit 311 (hereinafter referred to as the "number of users") for each attribute of the user U, and causes the playback system 20 to play the sound effect with acoustic characteristics according to the results of the count.

Specifically, sound effects with acoustic characteristics according to the attribute with the largest number of users among the multiple attributes are reproduced. For example, when the attribute (gender) with the largest number of users is female, the output control unit 313 reproduces higher-pitched sound effects compared to when the attribute is male. Also, the output control unit 313 reproduces louder sound effects the lower the attribute (age) with the largest number of users. On the other hand, if the reception processing unit 311 does not receive the first instruction data D1 (Sb51: NO), the sound effects are not reproduced (Sb52). The above processing is repeated until the end of the output control processing Sb is instructed (Sb53: NO). Note that any method can be used to change the acoustic characteristics of the sound effects. For example, a configuration is assumed in which an acoustic signal X according to the attribute of the user U among multiple acoustic signals X with different acoustic characteristics is selectively supplied to the reproduction system 20, or an acoustic signal X with acoustic characteristics according to the attribute of the user U is generated by acoustic processing of the initial acoustic signal X.

According to the fifth embodiment, the playback of sound effects by the playback system 20 is triggered by the first instruction Q1 that occurs in parallel with the progress of the competition event. Therefore, it is possible to start the playback of sound effects in the facility 200 without delay in relation to the progress of the competition event. In addition, since sound effects are played at a volume according to the attributes of the users U, it is possible to reflect the tendency of the attributes of the users U who watch the content C (for example, which attributes there are more users U with) in the sound effects in the facility 200.

In the above explanation, sound effects with a volume according to the attributes of the user U are played, but the playback system 20 may also play sound effects with a volume according to the number of times notification data Z is received from each terminal device 50 (i.e., the total number of users U watching the competitive event). In addition, the configuration of the second embodiment in which the first instruction Q1 is generated by analyzing the status of the competitive event may be applied to the fifth embodiment.

The configuration in which the second instruction Q from the user U to each terminal device 50 is reflected in the sound effect is also applied to the fifth embodiment. For example, in the fifth embodiment, a configuration is assumed in which the output control unit 313 causes the playback system 20 to play back a sound effect according to the total number (number of instructions) of second instruction data D2 received by the reception processing unit 311 from each terminal device 50 and the attributes of the user U of each terminal device 50. For example, the output control unit 313 controls a first acoustic characteristic of the sound effect according to the number of instructions, and controls a second acoustic characteristic of the sound effect according to the attributes of the user U. The first acoustic characteristic and the second acoustic characteristic are different characteristics related to the sound effect. For example, the first acoustic characteristic is volume, and the second acoustic characteristic is pitch. One acoustic characteristic of the sound effect (e.g., volume) may be controlled according to the number of instructions and the attributes of the user U.

F: Modifications Specific modifications to the above-mentioned embodiments are given below. Multiple modifications selected from the following examples may be combined as appropriate to the extent that they are not mutually contradictory.

(1) In the first to third embodiments, the second instruction Q2 for the operation image Wb is invalidated by setting the operation image Wb to an invalid state, and the second instruction Q2 for the operation image Wb is valid by controlling the operation image Wb corresponding to the sound effect selected by the administrator A to an valid state. However, a configuration in which the second instruction Q2 from the user U is uniformly accepted by the terminal device 50 is also assumed. Specifically, the output control unit 313 of the control system 30 effectively reflects the second instruction data D2 indicating the sound effect selected by the administrator A among the multiple second instruction data D2 received by the reception processing unit 311 in the volume of the sound effect, and ignores the second instruction data D2 indicating sound effects other than the selected sound effect. As can be understood from the above examples, the validation of the second instruction Q2 includes a process of effectively accepting the second instruction Q2 from the user U in the terminal device 50, as well as a process of effectively reflecting the second instruction Q2 in the volume of the sound effect in the control system 30.

(2) In each of the above-described embodiments, the manager A located inside the facility 200 issues the first instruction Q1, but the manager A outside the facility 200 may also issue the first instruction Q1. For example, a viewer of content C of the competitive event may issue the first instruction Q1 as the manager A. However, from the perspective of reducing the delay in the playback of sound effects relative to the progress of the competitive event, it is desirable for the manager A to be able to recognize the progress of the competitive event with a smaller delay than the viewing of content C by the user U.

(3) In each of the above-mentioned embodiments, a configuration has been exemplified in which the first instruction data D1 representing the first instruction Q1 is transmitted from the management terminal 10 to each terminal device 50 via the control system 30, but the first instruction data D1 may be transmitted directly from the management terminal 10 to the terminal device 50. For example, a configuration in which the first instruction data D1 transmitted from the management terminal 10 reaches each terminal device 50 via the communication network 300 is also envisioned. In other words, a configuration in which the control system 30 relays the first instruction data D1 is not essential. Note that in a configuration in which the first instruction data D1 is transmitted from the management terminal 10 to the terminal device 50, the instruction management unit 312 in FIG. 3 is omitted.

(4) The processes (Sa, Sb, Sc) exemplified in each of the above embodiments may be executed in parallel for each of the multiple contestants (participating teams) participating in the competitive event. For example, the manager A selects a sound effect from one of the multiple contestants by the first instruction Q1. The control system 30 causes the playback system 20 to play the sound effect selected by the manager A for each of the multiple contestants by the first instruction Q1. Note that a single or multiple managers A may operate the management terminal 10 individually prepared for each contestant. On the other hand, the user U selects in advance the contestant that the user U will support from among the multiple contestants in the competitive event. The control system 30 controls the sound effect of each of the multiple contestants in response to the second instruction Q2 from the user U who selected the contestant.

(5) In each of the above embodiments, a configuration has been exemplified in which the terminal device 50 both receives and plays the content C and accepts the second instruction Q2 from the user U. However, the content C may be played by a playback device separate from the terminal device 50 that accepts the second instruction Q2. The playback device plays the content C distributed by the distribution system 42. For example, in a configuration in which the distribution system 42 distributes the content C via the communication network 300, an information terminal (such as a mobile phone, smartphone, tablet terminal, or personal computer) capable of communicating via the communication network 300 is used as the playback device. Also, in a configuration in which the distribution system 42 is a television broadcasting system, for example, a television receiver is used as the playback device.

(6) In each of the above-described embodiments, the volume of the sound effect is controlled in response to the second instruction Q2 from each user U, but the acoustic characteristics of the sound effect controlled in response to the second instruction Q2 are not limited to the volume. For example, various acoustic characteristics such as the pitch, sound quality (tone), or reverberation characteristics of the sound effect may be controlled in response to the second instruction Q2. The number of sounds in the sound effect is also an example of an acoustic characteristic controlled in response to the second instruction Q2. The number of sounds in the sound effect refers to the total number of acoustic components that make up the sound effect (for example, the total number of sound sources such as speakers or instruments).

(7) In each of the above-described embodiments, the reproduction system 20 reproduces sound effects. However, the production effects output within the facility 200 are not limited to the reproduction of sound effects (i.e., acoustic effects). For example,
The performance control system 400 may control the video effects using a video display system installed in the facility 200. For example, the performance control system 400 causes the video display system to start displaying a predetermined video in response to a first instruction Q1, and controls the content or display characteristics (e.g., brightness, etc.) of the video in response to a second instruction Q2.

The performance control system 400 may control the lighting effects using a lighting system installed in the facility 200. For example, the performance control system 400 causes the lighting system to start emitting light in response to a first instruction Q1, and controls the light emission pattern in response to a second instruction Q2. The light emission pattern is a pattern of temporal changes in the amount of light emitted or the color of the light emitted.

The performance control system 400 may also control a performance effect that utilizes a smoke-producing system for performance installed within the facility 200. For example, the performance control system 400 causes the smoke-producing system to start emitting smoke in response to a first instruction Q1, and controls the smoke pattern in response to a second instruction Q2. The smoke pattern is a pattern of temporal changes in the amount of smoke produced or the color of the smoke.

As can be understood from the above examples, "performance effect" refers to any effect that is output within facility 200. The reproduction of sound effects by playback system 20, the display of images by the video display system, the emission of light by the lighting system, and the production of smoke by the smoke production system are examples of "output of performance effects." In addition, playback system 20, the video display system, the lighting system, and the smoke production system are examples of output devices that output performance effects. Performance control system 400 is expressed as a computer system for controlling the output of performance effects.

(8) An advertising image (e.g., a banner advertisement) may be displayed on the selection screen Gb displayed on the terminal device 50. For example, a configuration is envisaged in which identification information is transmitted to each terminal device 50 by acoustic communication using the sound of content C, and an advertising image corresponding to the identification information is displayed on the display device 541. With the above configuration, advertising images can be presented only to users U who are viewing content C. In addition, the advertising image may be displayed using location information indicating the location of the terminal device 50. For example, a configuration is envisaged in which the advertising image is displayed on the display device 541 on the condition that the location indicated by the location information is within a specific range. To acquire the location information, any known positioning technology such as GPS (Global Positioning System) may be adopted.

(9) The application program (hereinafter referred to as the "remote instruction program") that causes the control device 51 to execute the instruction reception process Sc may be equipped with a function for charging the user U of the terminal device 50. For example, the remote instruction program may be equipped with a function for electronically paying the viewing fee for the user U to view content C.

The remote instruction program may also be equipped with a function for electronically paying for money (so-called tips) provided by user U to participants or organizers of the competitive event. The remote instruction program may also be equipped with a function for electronically paying for the purchase of items related to the competitive event (e.g., merchandise for participants of the competitive event).

The output control unit 313 of the control system 30 may output a performance effect within the facility 200 in response to a payment by the user U using the payment function exemplified above. The output control unit 313 may also control the performance effect according to the amount paid by the user U. For example, the larger the payment amount, the greater the volume of the sound effects may be increased, a sound effect that is easy to perceive auditorily may be played, or a sound effect may be played from a specific sound emitting device among the multiple sound emitting devices that make up the playback system 20. With the above configuration, it is possible to encourage each user U to pay.

(10) A configuration is also envisaged in which the user U can specify a desired location (hereinafter referred to as a "designated location") within the facility 200 by operating the terminal device 50. The designated location is, for example, one of a number of spectator seats installed within the facility 200. The output control unit 313 controls, for example, the acoustic characteristics of the sound effects reproduced by a sound emitting device in the vicinity of the designated location of each user U, among the multiple sound emitting devices constituting the reproduction system 20, in response to a second instruction Q2 from the user U. In addition, the content C recorded in the vicinity of the designated location of each user U may be distributed to the terminal device 50 of the user U. A configuration is also envisaged in which, when the user U selects one of a number of contents C recorded at different locations within the facility 200, the location where the content C was recorded is automatically selected as the designated location. For example, in a situation in which the content C is reproduced by a reproduction device other than the terminal device 50, identification information of a specific location within the facility 200 is transmitted to the terminal device 50 by acoustic communication using the sound of the content C. The terminal device 50 recognizes the location corresponding to the identification information received via acoustic communication as the specified location.

(11) In each of the above-described embodiments, an operation on the operation image Wb arranged on the selection screen Gb is exemplified as the second instruction Q2, but the configuration in which the user U gives the second instruction Q2 is not limited to the above examples. For example, if the terminal device 50 is equipped with an acceleration sensor, the action of the user U shaking the terminal device 50 may be detected by the acceleration sensor as the second instruction Q2. For example, second instruction data D2 indicating a sound effect corresponding to a vibration pattern detected by the acceleration sensor among multiple sound effects is transmitted from each terminal device 50 to the control system 30.

(12) The control system 30 may collect various information regarding the usage status of the performance control system 400 and generate data representing the usage status (hereinafter referred to as "usage status data"). The usage status may be, for example, the total number of users U using the performance control system 400 (number of users), the total number of second instruction data D2 received by the reception processing unit 311 (number of instructions), or the attributes of the users U of each terminal device 50. The usage status data may also be expressed as data representing the situation in which multiple users U are cheering on participants in a competitive event.

The control system 30 transmits the usage status data to the management terminal 10, thereby displaying the information represented by the usage status data on the display device 14 of the management terminal 10. For example, the number of users (total number or numerical value for each attribute) or the number of instructions (total number or numerical value for each attribute) is displayed on the selection screen Ga. With the above configuration, the manager A can grasp the usage status of the performance control system 400 (the cheering status by multiple users U). The control system 30 may also select one of multiple sound effects according to the usage status data and have the playback system 20 play it. For example, one of multiple sound effects representing the chants (cheering songs) of the contestants is selectively played according to the usage status data. The control system 30 may transmit the usage status data to each terminal device 50, thereby displaying the information represented by the usage status data on the display device 541. For example, the number of users (total number or numerical value for each attribute) or the number of instructions (total number or numerical value for each attribute) is displayed on the selection screen Gb.

(13) In each of the above-mentioned embodiments, a competitive event in which multiple contestants compete in a sport is exemplified, but the types of events for which dramatic effects are output are not limited to the above examples. For example, various events held for a specific purpose are included in the concept of an "event", such as a performance event (live) in which a performer such as a singer or dancer performs, an exhibition event in which various items are exhibited, an educational event in which various educational institutions such as a school or a cram school provide lessons to students, or a lecture event in which a lecturer such as an expert gives a lecture on various subjects. In addition, if a creator of a video distributed by a video distribution site records and distributes his/her own activities in real time in a facility 200 such as his/her home and distributes (i.e., broadcasts live), the activity is also included in the concept of an "event". As can be understood from the above examples, an event means various performances in which dramatic effects are output.

(14) The functions of the management terminal 10 (e.g., instruction reception process Sa) are realized by the cooperation of one or more processors constituting the control device 11 and a program stored in the storage device 12. Similarly, the functions of the control system 30 (e.g., output control process Sb) are realized by the cooperation of one or more processors constituting the control device 31 and a program stored in the storage device 32. Furthermore, the functions of the terminal device 50 (e.g., instruction reception process Sc) are realized by the cooperation of one or more processors constituting the control device 51 and a program stored in the storage device 52.

The above-mentioned programs can be provided in a form stored in a computer-readable recording medium and installed in a computer. The recording medium is, for example, a non-transitory recording medium, and a good example is an optical recording medium (optical disk) such as a CD-ROM, but also includes any known type of recording medium such as a semiconductor recording medium or a magnetic recording medium. Note that a non-transitory recording medium includes any recording medium other than a transient, propagating signal, and does not exclude volatile recording media. In addition, in a configuration in which a distribution device distributes a program via a communication network, the recording medium that stores the program in the distribution device corresponds to the non-transitory recording medium described above.

G: Supplementary Note From the above-described exemplary embodiments, the following configurations, for example, can be understood.

A production control method according to one aspect (aspect 1) of the present disclosure outputs a production effect to a facility where an event is held in response to a first instruction that occurs in parallel with the progress of the event, activates a second instruction from a user who is viewing content that is recorded and distributed in parallel with the progress of the event in response to the first instruction, and controls the production effect in response to the activated second instruction.

In the above aspect, the first instruction occurring in parallel with the progress of the event is used as a trigger to output the dramatic effect within the facility, and the second instruction from the user viewing the content of the event is enabled. In a situation in which the distribution of the content is delayed relative to the progress of the event, for example, the first instruction is used as a trigger to enable the second instruction before the user recognizes a specific event in the event by viewing the content. The enabled second instruction is then used as a trigger to control the dramatic effect. Therefore, even in a situation in which the user is viewing the content in an environment in which there is a relatively large delay in distribution relative to the progress of the event, it is possible to reflect the second instruction from the user in the dramatic effect within the facility without excessive delay relative to the progress of the event.

A "facility" is any facility where an event is held. Specifically, the concept of a "facility" encompasses a variety of locations, both indoors and outdoors, such as stadiums where competitive events are held in which multiple contestants compete in a particular sport, audio halls or outdoor live venues where live performance events are held in which performers such as singers or dancers perform, exhibition halls where exhibition events are held to display various items, educational facilities where educational events are held in which various educational institutions such as schools and cram schools offer lessons to students, and lecture facilities where lecture events are held in which lecturers such as experts give lectures on various topics.

"Content" refers to information (digital content) provided to a user's terminal device. For example, the user views any content that includes at least one of video and audio. The method by which the user views the event content is also arbitrary. For example, the user may view the content using a terminal device to which the user gives the second instruction, or the user may view the content using a device separate from the terminal device (for example, a television receiver).

"Enabling" a second instruction means controlling the state so that the second instruction from the user can be effectively received. For example, assuming an operation unit that the user operates to give a second instruction, a specific example of "enabling" would be a process of changing the state of the operation unit from an invalid state in which the user cannot operate the operation unit to an valid state in which the user can operate the operation unit. In addition, in a configuration in which a second instruction given by a user to the operation unit is always accepted, the concept of "enabling" also includes a process of changing the processing for the second instruction from an invalid state in which the second instruction received from the user is ignored to an valid state in which the performance effect is effectively controlled in response to the second instruction.

"Performance effects" are dramatic effects that are output within a facility. For example, the concept of "performance effects" includes various effects that can be perceived within a facility (typically visual or auditory effects), such as sound effects that utilize the facility's audio equipment, visual effects that utilize the facility's display equipment, or lighting effects that utilize the facility's lighting equipment.

In a specific example (Aspect 2) of Aspect 1, the first instruction is given by the administrator in parallel with the progress of the event. According to the above aspect, the output of the dramatic effect can be started at an appropriate point in the event, triggered by the first instruction from the administrator who can recognize the progress of the event with a delay smaller than that of the user viewing the content.

An "administrator" is a person who can recognize the progress of the event with a smaller delay than the user's viewing of the content (including a situation where there is no delay). For example, an "administrator" is a viewer who watches the event directly (i.e., without delay) within the facility, or a user who can view the content with a smaller delay than the environment in which the user views the content.

In a specific example (aspect 3) of aspect 1, the first instruction is generated according to the result of analyzing the situation of the event. According to the above aspect, it is possible to start outputting a dramatic effect at an appropriate time according to the situation of the event without requiring an administrator to generate the first instruction.

In a specific example (Aspect 4) of any of Aspects 1 to 3, the first instruction is an instruction to select one of a plurality of effect effects, and in outputting the effect, the effect selected by the first instruction is output, and in activating the second instruction, the second instruction is activated to select the effect selected by the first instruction from among the plurality of effect effects. In the above aspect, it is possible to output an appropriate effect from among the plurality of effect effects according to the situation of the event. Also, the second instruction is activated to select the effect selected by the first instruction from among the plurality of effect effects. Therefore, it is possible to reduce the possibility that the user will erroneously select an effect that is not appropriate for the actual situation of the event.

In a specific example (Aspect 5) of any of Aspects 1 to 4, the output of the dramatic effect is the reproduction of a sound effect by a reproduction system installed in the facility, and the control of the dramatic effect involves controlling the acoustic characteristics of the sound effect. In the above aspect, the acoustic characteristics of the sound effect reproduced in the facility are controlled in response to a second instruction from a user. Therefore, listeners of the sound effect in the facility (e.g., performers at an event) can auditorily grasp the viewing situation of the content, such as the number of users who give second instructions.

In a specific example (aspect 6) of any of aspects 1 to 5, when a third instruction is given by the user, a different effect from the aforementioned effect is output to the facility, regardless of the presence or absence of the first instruction. In the above aspects, regardless of the presence or absence of the first instruction, the user can cause the facility to output a special effect by issuing a third instruction.

In one aspect (aspect 7) of the present disclosure, a method of operating a terminal device is as follows: when a terminal device capable of communicating with a performance control system that outputs a performance effect to a facility where an event is held is triggered by a first instruction that occurs in parallel with the progress of an event, the terminal device, upon receiving first instruction data indicating the first instruction from the performance control system, accepts a second instruction from a user who is viewing content distributed by recording the event in parallel with the progress of the event, and, upon receiving the second instruction, transmits second instruction data for controlling the performance effect to the performance control system.

The present disclosure may also be realized as a performance control system that realizes the performance control method according to any one of aspects 1 to 6, or as a program that causes a computer system to execute the performance control method. The present disclosure may also be realized as a terminal device that realizes the operation method according to aspect 7, or as a program that causes a computer system to execute the operation method.

100...information system, 200...facility, 300...communication network, 400...performance control system, 500...communication line, 10...management terminal, 11, 31, 51...control device, 111...instruction reception unit, 112...transmission processing unit, 12, 32, 52...storage device, 13, 33, 53...communication device, 14, 541...display device, 15, 55...operation device, 20...playback system, 30...control system, 311...reception processing unit, 312...instruction management unit, 313...output control unit, 41...recording system, 42...distribution system, 50...terminal device, 511...playback control unit, 512...reception processing unit, 513...instruction reception unit, 514...transmission processing unit, 54...playback device, 542...sound emission device, A...administrator A, U...user U, Q1...first instruction, Q2...second instruction, D1...first instruction data, D2...second instruction data.

Claims (6)

In response to a first instruction that occurs in parallel with the progress of the event, playback of a production effect is started in the facility where the event is held;
receiving a second instruction from a user who watches the content recorded at the event and is distributed in parallel with the progress of the event, from the terminal device of the user;
Controlling the performance effect, which has been continuously reproduced since the start of reproduction triggered by the first instruction, in response to the second instruction ;
The start of the reproduction of the dramatic effect triggered by the first instruction, the reception of the second instruction by the user, and the control of the dramatic effect in response to the second instruction are executed in parallel for each of a plurality of contestants participating in the event.
A production control method realized by a computer system. The content includes a video recording of the event;
analyzing the video to determine whether a particular occurrence occurred at the event;
The performance control method according to claim 1 , further comprising the step of: generating the first instruction when it is determined that the specific event has occurred. The method of claim 1 or 2, wherein the effect includes reproducing a sound effect or displaying an image. 4. The method of claim 1, wherein the effect is reproduced at a location designated by a user of the terminal device among a plurality of locations in the facility. an output control unit that starts reproduction of a performance effect in a facility where the event is held, in response to a first instruction that occurs in parallel with the progress of the event;
a reception processing unit that receives a second instruction from a user who watches the content distributed by recording the event in parallel with the progress of the event, from the terminal device of the user;
The output control unit controls the performance effect that has been continuously reproduced since the start of the reproduction triggered by the first instruction in response to the second instruction ,
The start of the reproduction of the dramatic effect triggered by the first instruction, the reception of the second instruction by the user, and the control of the dramatic effect in response to the second instruction are executed in parallel for each of a plurality of contestants participating in the event.
Production control system. an output control unit that starts reproduction of a performance effect in a facility where the event is held, in response to a first instruction that is generated in parallel with the progress of the event;
a reception processing unit that receives a second instruction from a user who watches the content distributed by recording the event in parallel with the progress of the event, from the terminal device of the user;
A program for causing a computer system to function as
The output control unit controls the performance effect that has been continuously reproduced since the start of the reproduction triggered by the first instruction in response to the second instruction ,
The start of the reproduction of the dramatic effect triggered by the first instruction, the reception of the second instruction by the user, and the control of the dramatic effect in response to the second instruction are executed in parallel for each of a plurality of contestants participating in the event.
program.

Images