JP7628595B2 - Content distribution server, system, terminal, method, content distribution method and program - Google Patents

Description

The present invention relates to distance education using ICT (Information and Communication Technology).

Previously, attempts have been made to provide distance education using ICT. For example, in a typical e-learning system, a server distributes recorded educational content to students' terminals via the Internet. This allows students to study without the restrictions of location or time, compared to traditional face-to-face classes.

Patent document 1 also discloses a teaching system that provides ongoing lesson data to student terminals in real time.

Patent No. 6227816

Simply delivering pre-recorded educational content makes it difficult for students to sense the presence of the teacher and other students. In addition, students are not able to participate in the lesson other than watching the video and listening to the audio, and they tend to have a passive attitude.

On the other hand, by delivering educational content in real time as it is being recorded, students can feel the teacher's presence in a real way, and vice versa. Also, by providing communication tools such as chat and comment functions, students can be encouraged to actively participate in class in anticipation of reactions from the teacher and other students.

However, whether providing pre-recorded educational content or providing educational content being recorded in real time, the content provided itself does not change unless it is re-recorded. Therefore, even if a student wants to retake a class on the same topic, they are forced to play the exact same content they previously viewed, which creates a problem of low motivation to review or re-learn.

The present invention aims to increase the interest of educational content provided by a distance education system and encourage students to take the courses.

The content distribution server according to the first aspect of the present invention includes a material data distribution unit, a received data acquisition unit, and a material data change unit. The material data distribution unit distributes to the education terminal first specification data indicating specifications of multiple objects placed in a virtual space constituting educational content for distance learning, and motion data indicating the movements of the multiple objects. The received data acquisition unit acquires a request to change at least a portion of the first specification data received from an external device. The material data change unit changes the first specification data to second specification data different from the first specification data in response to the request. The material data distribution unit distributes the second specification data to the education terminal after the first specification data has been changed to the second specification data.

A content creation device according to a second aspect of the present invention includes a playback unit, an input data acquisition unit, and a material data generation unit. The playback unit plays reference content based on first specification data indicating the specifications of an object related to the presentation of lesson content and first motion data indicating the movement of the object. The input data acquisition unit acquires second motion data based on the movement of a user during the playback of the reference content, and voice data based on the user's speech during the playback of the reference content. The material data generation unit generates material data used to reproduce a virtual space constituting educational content for distance learning, based on the first specification data, second specification data indicating the specifications of an avatar corresponding to the user, the first motion data, the second motion data, and the voice data.

The educational terminal according to the third aspect of the present invention includes an input data acquisition unit and a playback unit. The input data acquisition unit acquires input data corresponding to a request to change at least a part of first specification data indicating the specifications of a plurality of objects to be placed in a virtual space constituting educational content for a distance learning class. The playback unit plays back, together with the lesson audio data, lesson video data that captures the virtual space reproduced based on second specification data in which the first specification data has been changed based on the request and motion data indicating the movements of the plurality of objects.

The present invention can increase the interest of educational content provided by a distance education system, encouraging students to take the courses.

FIG. 2 is a block diagram illustrating an educational terminal according to the first embodiment. 2 is a block diagram illustrating a distance education system including the education terminal of FIG. 1; FIG. 2 is a block diagram illustrating a content distribution server according to the first embodiment. 2 is a diagram illustrating an example of a playback screen of educational content on the educational terminal of FIG. 1; 5 is a diagram illustrating an example of a playback screen of the educational content on the educational terminal of FIG. 1 after the teacher's avatar in the educational content of FIG. 4 has been changed based on a request from a user. 6 is a diagram illustrating an example of a playback screen of the educational content on the educational terminal of FIG. 1 after student avatars have been added to the educational content of FIG. 5 based on a request from a user. 4 is a flowchart illustrating the operation of the education terminal of FIG. 1 . 3 is a flowchart illustrating an example of the operation of the content distribution server of FIG. 2 . FIG. 11 is a block diagram illustrating a content creation device according to a second embodiment. 10 is a flowchart illustrating the operation of the content creating device of FIG. 9 . FIG. 13 is a block diagram illustrating an educational terminal according to a third embodiment. 12 is a flowchart illustrating the operation of the education terminal of FIG. 11 .

Below, the embodiments will be described with reference to the drawings. In addition, hereafter, elements that are the same or similar to elements already described will be given the same or similar reference numerals, and duplicated descriptions will generally be omitted. For example, when there are multiple identical or similar elements, a common reference numeral may be used to describe each element without distinguishing between them, and a subnumber may be used in addition to the common reference numeral to describe each element with distinction between them.

(First embodiment)
The education terminal and content distribution server according to the first embodiment can be incorporated into a distance education system as exemplified in Fig. 2. This system includes education terminals 10-1, 10-2, 10-3, ... according to the embodiments, and a content distribution server 20. The content distribution server 20 is connected to each education terminal 10 via a network such as the Internet, and can transmit and receive data between them. The content distribution server 20 may also be connected to other servers (not shown), such as a web server and a comment server, via the network, and can transmit and receive data between them.

The educational terminal 10 may be, but is not limited to, an electronic device such as a computer, for example, a PC (Personal Computer), a mobile terminal (for example, a tablet, a smartphone, a laptop, a feature phone, a portable game console, etc.), an HMD (Head Mounted Display), a game console, a television set (including Internet television), etc.

Users of the educational terminal 10 are primarily students in classes, but may also include teachers of classes. For example, a teacher may use the educational terminal 10 to record and transmit (upload) educational content for distance learning, add teaching material objects to the educational content, and the like. When the educational terminal 10 is used to record educational content, the educational terminal 10 will incorporate or be connected to a motion detection device and a microphone for capturing the movements of the teacher. The motion detection device may be, for example, an inertial sensor, a Doppler sensor, a 3D camera, and the like.

The content distribution server 20 provides educational content in response to a distribution start request from the educational terminal 10. The content distribution server 20 may provide the educational content in real time while the educational content is being recorded, or may provide previously recorded educational content. The content distribution server 20 may also provide the educational content provided in real time after the educational content has been recorded. The educational content may be recorded by the content distribution server 20 to which, for example, a motion detection device, a microphone, etc. are connected, or may be recorded by the teacher's educational terminal 10 or another terminal and transmitted sequentially to the content distribution server 20.

The educational content may typically include virtual content such as VR (Virtual Reality), AR (Augmented Reality), MR (Mixed Reality), and SR (Substitutional Reality), which allows remote students to experience a realistic lesson. The content distribution server 20 may distribute to the educational terminal 10 lesson video data in which a virtual space constituting such educational content is shot from a certain viewpoint, or may distribute to the educational terminal 10 material data including materials for rendering the lesson video data.

The educational terminal 10 may receive comment data posted by students or teachers on educational content from a comment server (not shown), and may display the comments indicated by the comment data superimposed on the educational content. The comments may be displayed, for example, by moving over a predetermined period of time on the surface of an object for displaying comments that is placed in a virtual space constituting the educational content. On the other hand, the educational terminal 10 may generate comment data to be posted on the educational content based on text input from the user, and transmit the data to the comment server. This allows teachers and students to have synchronous communication (in the case of real-time distribution) and/or asynchronous communication on a text basis.

The educational terminal 10 also receives and plays back lesson audio data as part of the educational content from the content distribution server 20. This lesson audio data includes the voice spoken by the teacher during recording, and may also include (a part of) the voice spoken by the students. In this case, (a part of) the educational terminal 10 may generate audio data based on the voice of the students detected by a microphone (not shown), and transmit it to the content distribution server 20. This allows students to experience a more realistic lesson, and enables teachers and students to have synchronous communication (in the case of real-time distribution) and/or asynchronous communication based on audio.

Note that the number of devices shown in FIG. 2 is merely an example. For example, the number of educational terminals 10 changes from moment to moment, and may be zero or may number in the hundreds or thousands. Furthermore, web servers, comment servers, and the like not shown in FIG. 2 may also be provided.

The configuration of the education terminal 10 will be described in detail below with reference to FIG.
As illustrated in FIG. 1 , the education terminal 10 includes a communication I/F (interface) 101 , an input/output I/F 102 , a processor 110 , and a memory 120 .

The communication I/F 101 communicates with the content distribution server 20 via a network, typically the Internet, using communication technologies such as mobile communications (e.g., 3G, 4G, 5G, mobile WiMAX), Wi-Fi, and WiMAX. The communication I/F 101 can receive various data from the processor 110 and transmit it to the content distribution server 20, and can receive various data from the content distribution server 20 and send it to the processor 110.

Specifically, the communication I/F 101 receives material data from the content distribution server 20 and sends it to the received data acquisition unit 111. Note that the material data may or may not be encoded. In the former case, the material data needs to be decoded by a decoder (not shown) before being read into the material data storage unit 121 or after being read from the material data storage unit 121.

The input/output I/F 102 is an interface terminal (port) used for inputting and outputting data between the processor 110 and an external device, and can also be called an I/O (Input/Output) unit. The input/output I/F 102 can be used, for example, for data exchange between the processor 110 and an input device such as a touch screen, keyboard, mouse, numeric keypad, microphone, motion detection device, etc. Similarly, the input/output I/F 102 can be used, for example, for data exchange between the processor 110 and an output device such as a display (which may include a touch screen) or a speaker, etc.

For example, the input/output I/F 102 receives played lesson video data and lesson audio data from the processor 110, and outputs them to the display and speaker, respectively. The input/output I/F 102 also receives input data (user input) equivalent to a request to change (at least a part of) the specification data of an object described below from an input device, and sends this to the processor 110. In addition, the input data may include voice data based on the user's voice generated by a microphone, comment data generated by a keyboard (which may include a screen keyboard), and requests related to the playback control of educational content, such as specifying the playback time, rewinding, fast-forwarding, changing the playback speed, skipping, etc.

The processor 110 is typically a CPU (Central Processing Unit) and/or a GPU (Graphics Processing Unit), but may also be a microcontroller, an FPGA (Field Programmable Gate Array), a DSP (Digital Signal Processor), or other general-purpose or dedicated processor.

Memory 120 includes memory for temporarily storing programs executed by processor 110 to realize each process, such as an OS (Operating System), firmware, distance education applications, etc., and data used by processor 110, such as material data, lesson video data, etc. The memory may include a RAM (Random Access Memory) having a work area in which such programs/data are expanded.

The processor 110 can function as the received data acquisition unit 111, the renderer 112, the content playback unit 113, the input data acquisition unit 114, and the input data reporting unit 115 in FIG. 1 by executing the programs stored in the memory 120. The memory 120 can also include a material data storage unit 121.

The received data acquisition unit 111 acquires received data from the content distribution server 20 via the communication I/F 101 and stores it in the memory 120. For example, the received data acquisition unit 111 acquires material data from the communication I/F 101 and stores it in the material data storage unit 121.

The renderer 112 reads out the material data stored in the material data storage unit 121 that serves as the material for reproducing the virtual space that constitutes the educational content, specifically, specification data indicating the specifications of multiple objects placed in this virtual space and motion data indicating the movements of the multiple objects. The renderer 112 then reproduces the virtual space (the appearance, movement, etc. of each object placed in the virtual space) based on this specification data and motion data, and generates lesson video data that captures these from a reference viewpoint (virtual camera). The renderer 112 sends the generated lesson video data to the content playback unit 113.

The content playback unit 113 receives the lesson video data from the renderer 112 and reads the lesson audio data from the material data storage unit 121. The content playback unit 113 then plays back the lesson audio data together with the lesson video data, and outputs the educational content to the output device via the input/output I/F 102.

The input data acquisition unit 114 acquires input data from the input/output I/F 102 and sends at least a portion of this to the input data reporting unit 115. The input data acquisition unit 114 sends input data to be reported to the content distribution server 20 or the comment server to the input data reporting unit 115, and can send other input data, such as requests for controlling the playback of educational content, to, for example, the renderer 112 and/or the content playback unit 113.

The input data reporting unit 115 receives input data from the input data acquisition unit 114 and reports it to the content distribution server 20 or the comment distribution server via the communication I/F 101. For example, the input data reporting unit 115 may transmit a request to change the specification data of an object and voice data to the content distribution server 20, and transmit comment data to the comment server. Here, a request to change the specification data of an object may include, for example, a request to replace the appearance, voice character, sound effects, music, etc. of a specific object (which may include an avatar) included in the multiple objects for which specification data is defined, a request to delete the specification data of the specific object, a request to add specification data of a new object (which may include an avatar) not included in the multiple objects, etc.

The material data storage unit 121 stores material data. Here, the material data includes specification data and motion data of objects, as well as lesson audio data. The material data storage unit 121 may store material data for reproducing a virtual space constituting a certain educational content, for example, in association with an identifier that identifies the educational content. The material data storage unit 121 may also store specification data and motion data of each object in association with an identifier that identifies the object.

The specification data may include, for example, appearance data of the object, and sound effect data or music data representing a sound effect or music to be generated when a specific condition is satisfied for the object. Furthermore, for an object that can speak, such as an avatar, voice character data indicating the voice character who provides the voice for the object may also be included in the specification data of the object. Note that at least a portion of the specification data of the object may be preset in the material data storage unit 121, for example, before the start of playback of the educational content.

The motion data represents the movement, orientation (posture), and position coordinates of an object's entire body or each part for each playback time of the educational content. If the object is an avatar that represents a teacher, student, or other related person, the motion data of the object can be generated by a motion detection device based on the person's entire body or each part's movement. To generate motion data based on a human's entire body or each part's movement, known techniques can be used, for example, based on sensing signals from an inertial sensor or Doppler sensor, and/or image data from a 3D camera.

The material data is written to the material data storage unit 121 by the received data acquisition unit 111. Of the material data stored in the material data storage unit 121, the object specification data and motion data are read by the renderer 112. Of the material data stored in the material data storage unit 121, the lesson audio data is read by the content playback unit 113.

When the input data reporting unit 115 reports input data corresponding to a request to change the specification data of an object to the content distribution server 20, the content distribution server 20 can change the specification data of the object in response to this request. Note that the content distribution server 20 does not necessarily need to respond to such requests from all educational terminals 10, and may respond under certain conditions, such as by charging, majority vote, lottery, or first come first served.

When the content distribution server 20 changes the specification data of an object based on a request, it distributes the specification data, which is the original specification data modified based on the request, to each educational terminal 10. This modified specification data is received by the communication I/F 101 together with any unreceived motion data, acquired by the received data acquisition unit 111, and stored in the material data storage unit 121. Note that the motion data received here may be the same regardless of whether the specification data has been modified. Also, the pre-modification specification data stored in the material data storage unit 121 is overwritten by the modified specification data.

The modified specification data and motion data stored in the material data storage unit 121 are read by the renderer 112 and rendered as lesson video data. This allows students to, for example, change the appearance and/or voice character of the teacher or student avatar, add new objects (which may include avatars), and change the sound effects and/or music that can be generated by objects in educational content subsequently distributed by the content distribution server 20.

For example, if a student does not like the appearance or tone of voice of the teacher avatar 31 appearing in the educational content shown in Figure 4, he or she can change to a different teacher avatar 33 with a preferred appearance and tone of voice as shown in Figure 5 and take the class.

Students can also add their own avatars to the virtual space that constitutes the educational content. As a result, as shown in FIG. 6, in addition to the teacher avatar 33, student avatars 34 to 36 appear in the educational content, so that students can sense the presence of other students, and the number of student avatars that appear in the educational content can be an indicator of the popularity of the content. For example, students can roughly understand whether the educational content is popular or not from the number of student avatars displayed in the preview of the educational content. Furthermore, depending on the number of student avatars, the size, appearance, etc. of the classroom object placed in this virtual space may be automatically changed. In other words, the size and/or appearance of the classroom object indicated by the changed specification data may depend on the number of avatars specified by this specification data. For example, as the number of avatars increases, a classroom object with a size/appearance according to the number of occupants may be selected, and further desk and chair objects may be added. Furthermore, as the classroom object increases in size, the size of the blackboard object 32 may also be enlarged.

In the examples of Figures 4 to 6, in addition to the teacher and student avatars, a blackboard object 32 is provided as an object related to the presentation of lesson content, but any object such as a whiteboard or projector can be used as such an object.

Teaching materials such as text, graphs, diagrams, photos, videos, etc. displayed on the blackboard object 32 or other objects related to the presentation of lesson content may be defined as teaching material objects separate from the object in question. For example, a teacher can add, delete, or replace teaching material objects, change the font type, color, size, etc. of text, or change the size and layout of graphs, diagrams, photos, etc., based on the students' comments, voices, etc., during real-time distribution or distribution after recording. Such adjustments to educational content can motivate students to re-view educational content that they have already viewed. To create a classroom-like atmosphere, objects such as dates, the person on duty, graffiti, etc., that would be written on a blackboard in a real classroom may be attached to the blackboard object 32.

Furthermore, objects of student creations such as answer sheets, grade rankings, drawings, etc. may be posted on the walls of the classroom object. The appearance of the answer sheet object may be generated based on the students' answers to questionnaires or other types of tests distributed during the playback of the educational content. The appearance of the grade rankings object may be generated based on the user's scores on such tests and may be updated each time a new student achieves a high score, or periodically. A teacher may add a comment or audio praising a student who has achieved a high score after the fact, or such a presentation may be automatically performed within the educational terminal 10 of the student. Exposing such postings to the eyes of teachers and other students may motivate students to retake the test in order to achieve a higher score or to work on a more complete creation.

The configuration of the content distribution server 20 will be described in detail below with reference to FIG.
As illustrated in FIG. 3 , the content distribution server 20 includes a communication I/F 201 , a processor 210 , and a memory 220 .

The communication I/F 201 communicates with the educational terminal 10 via a network, typically the Internet, using communication technology such as optical communication. The communication I/F 201 can receive various data from the processor 210 and transmit it to the educational terminal 10, and can receive various data from the educational terminal 10 and send it to the processor 210.

Specifically, the communication I/F 201 receives data, such as a request to change specification data, from the education terminal 10 and sends it to the processor 210. The communication I/F 201 also receives material data from the processor 210 and sends it to the education terminal 10. Note that the material data may or may not be encoded. In the former case, the content distribution server 20 may be equipped with an encoder that encodes the material data.

The lesson audio data and motion data among the material data may be uploaded to the content distribution server 20 from, for example, the teacher's educational terminal 10, or may be generated by a microphone and motion detection device (not shown) connected to the content distribution server 20. In the latter case, the content distribution server 20 may include an input/output I/F for connecting to the microphone and motion detection device, and an input data acquisition unit for acquiring the motion data and lesson audio data from these. The initial values of the specification data among the material data may be specified by the teacher or an administrator of the distance education system, for example, before recording the educational content.

Processor 210 is typically a CPU and/or GPU, but may also be a microcontroller, FPGA, DSP, or other general-purpose or special-purpose processor.

Memory 220 includes memory for temporarily storing programs executed by processor 210 to realize each process, such as an OS, firmware, distance education application, etc., and data used by processor 210, such as material data, lesson video data, etc. The memory may include a RAM having a work area in which such programs/data are deployed.

The processor 210 can function as the received data acquisition unit 211, the material data change unit 212, and the material data distribution unit 213 in FIG. 3 by executing a program stored in the memory 220. The memory 220 can also include a material data storage unit 221.

The received data acquisition unit 211 acquires received data from the educational terminal 10 or other external devices, such as the terminal of the administrator of the distance education system, via the communication I/F 201, and sends this to the material data modification unit 212. The received data acquisition unit 211 sends received data (user input) corresponding to a request to change the specification data to the material data modification unit 212. On the other hand, the received data acquisition unit 211 can write other received data, such as motion data and lesson audio data uploaded from the educational terminal 10 for the teacher, audio data uploaded from the educational terminal 10 for the student, etc., to the material data storage unit 221.

The material data modification unit 212 receives received data corresponding to a request to modify the specification data from the received data acquisition unit 211, and modifies the specification data stored in the material data storage unit 221 accordingly. For example, when the material data modification unit 212 is requested to replace the specification data of a specific object, it may rewrite the target specification data in the material data storage unit 221 with the specification data included in the request. When the material data modification unit 212 is requested to add specification data of a new object, it may write the specification data included in the request to the material data storage unit 221 in association with the identifier of the target educational content. When the material data modification unit 212 is requested to delete the specification data of a specific object, it may delete the specification data of the target object from the material data storage unit 221.

As mentioned above, the material data modification unit 212 does not necessarily need to comply with all requests. In other words, the material data modification unit 212 only needs to modify the specification data in response to requests under certain conditions.

The material data distribution unit 213 reads the material data from the material data storage unit 221 and distributes it to the education terminal 10 via the communication I/F 201. After the material data is changed by the material data change unit 212, the material data distribution unit 213 distributes the changed specification data to the education terminal 10.

The material data storage unit 221 stores material data. The material data storage unit 221 may store material data for reproducing a virtual space constituting certain educational content, for example, in association with an identifier that identifies the educational content. The material data storage unit 221 may also store specification data and motion data for each object in association with an identifier that identifies the object.

The material data may be written to the material data storage unit 221 by, for example, the received data acquisition unit 211 or an input data acquisition unit (not shown). The material data may also be rewritten by the material data modification unit 212. The material data stored in the material data storage unit 221 is read out by the material data distribution unit 213.

The operation of the educational terminal 10 will be described below with reference to FIG. 7. FIG. 7 shows an example of the operation of the educational terminal 10 from when the user of the educational terminal 10 starts viewing a certain educational content until the user ends the viewing.

When the operation in FIG. 7 starts, the process proceeds to step S301. In step S301, the received data acquisition unit 111 acquires the material data received by the communication I/F 101 and stores it in the material data storage unit 121.

The material data received in step S301 may include, for example, motion data and lesson audio data associated with a predetermined period of playback time of the educational content that the educational terminal 10 is about to play. Furthermore, this material data may include initial specification data or modified specification data.

Next, the renderer 112 renders the lesson video data based on the specification data and motion data of the material data received in step S301 (step S302).

Then, the content playback unit 113 plays the lesson video data rendered in step S302 together with the lesson audio data from the material data received in step S301 (step S303).

On the other hand, when the input data acquisition unit 114 acquires input data corresponding to a request to change the specification data via the input/output I/F 102, the process proceeds to step S305 (step S304). In step S305, the input data reporting unit 115 reports the input data corresponding to the request to change the specification data to the content distribution server 20.

In the example of FIG. 7, steps S304 and S305 are depicted as being executed after step S303, but this is not essential. In other words, the acquisition of a request to change the specification data may be used as a trigger to report the request to the content distribution server 20, independent of the process from receiving the material data to playing the educational content (steps S301 to S303), which may be executed in a pipeline manner.

The operation of the content distribution server 20 will be described below with reference to FIG. 8. FIG. 8 shows an example of the operation of the content distribution server 20 from when the user of the education terminal 10 starts viewing a certain educational content to when the user ends the viewing.

When the operation of FIG. 8 starts, the process proceeds to step S401. In step S401, the material data distribution unit 213 reads the material data from the material data storage unit 221 and distributes it to the educational terminal 10 via the communication I/F 201.

The material data distributed in step S401 may include, for example, motion data and lesson audio data associated with a predetermined period of playback time of the educational content that the educational terminal 10 is about to play. Furthermore, this material data may include initial specification data or modified specification data.

The content distribution server 20 basically executes this step S401 repeatedly. However, when the communication I/F 201 receives a request to change the specification data from the education terminal 10, the process proceeds to step S403 (step S402).

In step S403, the received data acquisition unit 211 acquires a request to change the specification data received by the communication I/F 201. Next, the material data change unit 212 changes the specification data stored in the material data storage unit 221 in response to the request acquired in step S403 (step S404).

Then, in step S401, which is executed after step S404, the material data distribution unit 213 distributes the material data including the changed specification data to the educational terminal 10.

As described above, the educational terminal according to the first embodiment acquires input data corresponding to a request to change at least a portion of the specification data indicating the specifications of multiple objects placed in a virtual space that constitutes educational content, and transmits this to the content distribution server according to this embodiment. This content distribution server (conditionally) changes the specification data in response to this request, and distributes the changed specification data. Therefore, with these educational terminals and content distribution servers, students, teachers, or administrators of the distance education system can adjust the educational content to suit events that arise during or after its recording, such as the needs and reactions of students, public trends, promotions, etc., to increase its interest. In other words, it becomes possible to encourage students to take the course.

Conventional pre-recorded educational content may be regarded by students as simply a recorded program. In this case, for students, watching such educational content is no different from receiving education through video playback or television broadcasting, so it is difficult to feel a sense of realism, tension, a sense of participation, and a desire to learn. On the other hand, according to this embodiment, for example, answer sheets and report cards of students who have watched the educational content are posted in the classroom, the number of student avatars taking classes in the virtual classroom increases, the teacher's avatar can be changed to an avatar that the students can relate to instead of a character that the students do not know, and exchanges such as questions and answers between the teacher and other students in the classroom can be viewed. Therefore, according to this embodiment, even when pre-recorded educational content is distributed as well as educational content being recorded, students can feel as if they are participating in a live class, which is an exceptional effect that easily generates a sense of realism, tension, a sense of participation, and a desire to learn.

Second Embodiment
In the first embodiment described above, the educational content is made more interesting by allowing the user to modify the educational content. On the other hand, by reusing a part of the educational content, such as the specification data and motion data of the teaching material object, it is possible to create new educational content on the same theme at low cost.

As illustrated in FIG. 9, the content creation device 50 according to the second embodiment includes an input/output I/F 501, a processor 510, and a memory 520.

The input/output I/F 501 is an interface terminal (port) used for inputting and outputting data between the processor 510 and an external device. The input/output I/F 501 can be used for data exchange between the processor 510 and an input device such as a touch screen, keyboard, mouse, numeric keypad, microphone, motion detection device, etc. Similarly, the input/output I/F 501 can be used for data exchange between the processor 510 and an output device such as a display (which may include a touch screen) or a speaker, etc.

For example, the input/output I/F 501 receives the played reference content from the processor 510 and outputs it to the display (and speaker). The input/output I/F 501 also receives motion data based on the user's movements during playback of the reference content from a motion detection device (not shown) and sends this to the processor 510. The input/output I/F 501 receives voice data based on the user's vocalization during playback of the reference content from a microphone (not shown) and sends this to the processor 510.

Processor 510 is typically a CPU and/or GPU, but may also be a microcontroller, FPGA, DSP, or other general-purpose or special-purpose processor.

Memory 520 includes memory for temporarily storing programs executed by processor 510 to realize each process, such as an OS, firmware, content creation application, distance education application, etc., and data used by processor 510, such as material data, etc. The memory may include a RAM having a work area in which such programs/data are deployed.

The processor 510 can function as the renderer 511, the content playback unit 512, the input data acquisition unit 513, and the material data generation unit 514 in FIG. 9 by executing a program stored in the memory 520. The memory 520 can also include a material data storage unit 521.

The renderer 511 reads out specification data indicating the specifications of an object related to the presentation of lesson content, such as a teaching material object, and motion data indicating the movement of this object from the material data stored in the material data storage unit 521. Here, the specification data and motion data of this object may be, for example, part of the material data related to existing base educational content, or may be created for reuse among multiple educational contents on the same theme even if there is no such existing base educational content. Note that the reference content may be rendered in advance, in which case the content creation device 50 does not require the renderer 511.

Then, based on this specification data and motion data, the renderer 511 reproduces at least the appearance, movement, etc. of objects related to the presentation of the lesson content, and generates reference content (which can also be called reference video data) that is shot from a reference viewpoint (virtual camera). This reference content corresponds to a script that allows a user (typically a teacher) to understand in real time the development of the lesson content to be reused. The user will teach the lesson using gestures while checking this reference content. The renderer 511 sends the generated reference content to the content playback unit 512.

The content playback unit 512 receives the reference content from the renderer 511. The content playback unit 512 then plays back the reference content and outputs it to the output device via the input/output I/F 501. The reference content may include audio. In this case, the content playback unit 512 may read out (lesson) audio data stored in the material data storage unit 521. The reference content may also be the existing educational content itself that serves as the base.

The input data acquisition unit 513 acquires input data from the input/output I/F 501 and sends at least a portion of this data to the material data generation unit 514. The input data acquisition unit 513 sends input data corresponding to motion data and movement data to the material data generation unit 514.

The material data generation unit 514 receives the motion data and audio data from the input data acquisition unit 513, and generates material data for the new educational content being created based on the motion data and audio data. Specifically, the material data generation unit 514 assigns to the motion data and audio data the playback time of the reference content at the time of its generation, input, or reception. The material data generation unit 514 then stores the motion data and audio data in the material data storage unit 521 in association with the identifier of the new educational content.

Furthermore, the material data generation unit 514 stores specification data indicating the specifications of the user's avatar and the material data that is the source of the reference content (e.g., specification data and motion data of a teaching material object) in the material data storage unit 521 in association with an identifier of the new educational content being created. Note that the specification data indicating the specifications of the user's avatar may be determined based on input by the user himself, for example, or may be specified by an administrator of the distance education system.

In short, the material data generation unit 514 saves the material data that is the source of the reference content, i.e. the reused material data, the motion data and audio data that indicate the user's movements and audio while the reference content is being played, and the specification data that indicates the specifications of the user's avatar in the material data storage unit 521 as material data for new educational content. In the educational content created in this way, the development of the lesson content is the same as in the reference content, but the teacher can demonstrate his or her own individuality through the movements of the teacher avatar and the teaching audio.

The material data storage unit 521 stores material data. Here, the material data includes specification data and motion data of objects, as well as lesson audio data. The material data storage unit 521 may store material data for reproducing a virtual space constituting certain educational content, for example, in association with an identifier that identifies the educational content. The material data storage unit 521 may also store specification data and motion data of each object in association with an identifier that identifies the object.

Specifically, before new educational content is created, at least the specification data and motion data that are the basis of the reference content are stored in the material data storage unit 521. The specification data and motion data are read by the renderer 511. Then, while this educational content is being created, the user's voice data and motion data are written sequentially to the material data storage unit 521. The user's voice data is used as lesson voice data for this educational content, and the user's motion data is used as motion data for a teacher avatar for this educational content.

The operation of the content creation device 50 will be described below with reference to FIG. 10. FIG. 10 shows an example of the operation of the content creation device 50 from the start to the end of creating new educational content by reusing a certain reference content.

When the operation in FIG. 10 starts, the process proceeds to step S601. In step S601, the renderer 511 reads the specification data and motion data of objects related to the presentation of lesson content from the material data storage unit 521, and renders the reference content based on these.

The material data read in step S601 may include, for example, motion data associated with a predetermined period of playback time of the reference content that the content creation device 50 is about to play. Furthermore, this material data may include specification data.

Then, the content playback unit 512 plays the reference content rendered in step S601 (step S602).

On the other hand, the input data acquisition unit 513 acquires motion data and voice data based on the user's movements and vocalizations while the reference content is being played (step S603). The material data generation unit 514 then generates material data for new educational content based on the motion data and voice data acquired in step S603, and stores the generated material data in the material data storage unit 521 (step S604).

Note that in the example of FIG. 10, steps S603 and S604 are depicted as being executed after step S602, but this is not essential. In other words, new educational content material data may be added sequentially based on the acquired motion data and audio data, independent of the rendering and playback of the reference content (steps S601 to S602), which may be executed in a pipeline manner.

As described above, the content creation device according to the second embodiment plays reference content equivalent to a script that the teacher can refer to in real time for the development of the lesson content to be reused, and generates material data for new educational content based on the teacher's movements and vocalizations during that time. Therefore, with this content creation device, it is possible to easily provide educational content on the same theme in which the development of the lesson content is the same but the movements of the teacher avatar and the lesson audio data are different. Therefore, students can select and view educational content that stimulates their motivation to take the course.

When selecting educational content to take, students may place importance on the sensory elements they receive from the content, such as visual and auditory elements. Therefore, even if the development of the lesson content is the same, educational content in which the teacher's appearance, movements, and speaking style are different may be completely different in popularity. Therefore, it is significant to be able to easily provide educational content on the same theme by reusing the development of the lesson content, as in this embodiment.

The content creation device may upload new educational content to the content distribution server while the new educational content is being created. This allows the new educational content to be distributed in real time. On the other hand, the content creation device may upload the new educational content to the content distribution server after finishing creating the new educational content. Alternatively, the content creation device may be incorporated into the content distribution server.

The second embodiment can also be combined with the first embodiment described above or the third embodiment described below. That is, it may be possible to change the specification data as described in the first or third embodiment for the educational content created by the content creation device according to this embodiment.

Third Embodiment
In the first embodiment described above, the specification data stored in the material data storage unit 521 of the content distribution server 20 is changed in response to a request to change the specification data. That is, the change in the specification data affects all students who view the same educational content. Therefore, when the change in the specification data is reflected globally in this way, it may be necessary to make the conditions for allowing the change stricter.

On the other hand, it is also possible to allow only students who have requested changes to the specification data to view educational content that reflects those changes. If changes to the specification data are reflected locally in this way, their influence is limited, making it easier to relax the conditions for allowing those changes. This allows students to freely customize objects, including avatars, to suit their preferences when attending classes.

As illustrated in FIG. 11, the educational terminal 70 according to the third embodiment includes a communication I/F 701, an input/output I/F 702, a processor 710, and a memory 720.

The communication I/F 701, the input/output I/F 702, the processor 710, and the memory 720 may be the same or similar hardware as the communication I/F 101, the input/output I/F 102, the processor 110, and the memory 120 in FIG. 1, respectively.

The processor 710 can function as the received data acquisition unit 711, the renderer 712, the content playback unit 713, the input data acquisition unit 714, and the material data modification unit 715 in FIG. 11 by executing the programs stored in the memory 720. The memory 720 can also include a material data storage unit 721.

Here, the received data acquisition unit 711, the renderer 712, and the content playback unit 713 may be the same as or similar to the received data acquisition unit 111, the renderer 112, and the content playback unit 113 in FIG. 1, respectively.

The input data acquisition unit 714 acquires input data from the input/output I/F 702 and sends at least a portion of this to the material data modification unit 715. The input data acquisition unit 714 sends input data corresponding to a request to change the specification data to the material data modification unit 715. The input data acquisition unit 714 may also send, for example, a request regarding playback control of educational content to the renderer 712 and/or the content playback unit 713. In addition, the input data acquisition unit 714 may send input data to be reported to the content distribution server or comment server, such as voice data or comment data, to an input data reporting unit (not shown). In this case, the input data reporting unit may report such data to the content distribution server or comment server via the communication I/F 701.

Here, since the content distribution server according to the third embodiment does not receive a request to change the specification data from the educational terminal 70, the content distribution server 20 described above may be configured with the functional unit related to the request, such as the material data change unit 212, removed.

The material data modification unit 715 receives input data corresponding to a request to modify the specification data from the input data acquisition unit 714, and modifies the specification data stored in the material data storage unit 721 accordingly. For example, when the material data modification unit 715 is requested to replace the specification data of a specific object, it may rewrite the target specification data in the material data storage unit 721 with the specification data included in the request. When the material data modification unit 715 is requested to add specification data of a new object, it may write the specification data included in the request to the material data storage unit 721 in association with the identifier of the target educational content. When the material data modification unit 715 is requested to delete the specification data of a specific object, it may delete the specification data of the target object from the material data storage unit 721.

The material data storage unit 721 stores material data. The material data storage unit 721 may store material data for reproducing a virtual space constituting certain educational content, for example, in association with an identifier that identifies the educational content. The material data storage unit 721 may also store specification data and motion data for each object in association with an identifier that identifies the object.

The material data is written to the material data storage unit 721 by the received data acquisition unit 711. The material data can also be rewritten by the material data modification unit 715. Of the material data stored in the material data storage unit 721, the object specification data and motion data are read by the renderer 712. Of the material data stored in the material data storage unit 721, the lesson audio data is read by the content playback unit 713.

Changes to the specification data of an object made by the material data modification unit 715 are applied only to educational content played back by an educational terminal 70 that includes the material data modification unit 715. On the other hand, other educational terminals 70 can play educational content based on material data distributed from a content distribution server, or educational content based on material data that has been modified within the educational terminal 70.

The modified specification data is stored in the material data storage unit 721 together with the motion data received from the content distribution server. Note that the motion data received here may be the same regardless of whether the specification data has been modified. The modified specification data and motion data stored in the material data storage unit 721 are read out by the renderer 712 and rendered as lesson video data. This allows students to freely change the appearance and/or voice characters of teacher or student avatars, add new objects (which may include avatars), and change the sound effects and/or music that objects can generate in the educational content played back on their educational terminals 70.

The operation of the educational terminal 70 will be described below with reference to FIG. 12. FIG. 12 shows an example of the operation of the educational terminal 70 from when the user of the educational terminal 70 starts viewing a certain educational content until the user ends the viewing.

When the operation in FIG. 12 starts, the process proceeds to step S801. In step S801, the received data acquisition unit 711 acquires the material data received by the communication I/F 701 and stores it in the material data storage unit 721.

The material data received in step S801 may include, for example, motion data and lesson audio data associated with a predetermined period of playback time of the educational content that the educational terminal 70 is about to play. Furthermore, this material data may include initial specification data.

Next, the renderer 712 renders the lesson video data based on the specification data and motion data of the material data received in step S801 (step S802).

Then, the content playback unit 713 plays the lesson video data rendered in step S802 together with the lesson audio data from the material data received in step S801 (step S803).

On the other hand, when the input data acquisition unit 714 acquires input data corresponding to a request to change the specification data via the input/output I/F 702, the process proceeds to step S805 (step S804). In step S805, the material data change unit 715 changes the specification data stored in the material data storage unit 721 in response to the request acquired in step S804. Then, in step S802 executed after step S805, the renderer 712 renders the lesson video data based on the material data including the changed specification data.

In the example of FIG. 12, steps S804 and S805 are depicted as being executed after step S803, but this is not essential. In other words, the acquisition of a request to change the specification data may be used as a trigger to change the specification data in response to the request, independently of the process from receiving the material data to playing the educational content (steps S801 to S803), which may be executed in a pipeline manner.

As described above, the educational terminal according to the third embodiment acquires input data corresponding to a request to change at least a portion of the specification data indicating the specifications of multiple objects placed in a virtual space that constitutes educational content, and changes the specification data stored in the educational terminal accordingly. Therefore, with this educational terminal, students can freely modify educational content to suit their own preferences, increasing its interest. In other words, students can customize and view educational content in a way that increases their motivation to take the courses.

It is also possible to combine the first and third embodiments. As an example, a request to replace or delete specification data of a particular object may be reflected locally, and a request to add specification data of a new object may be reflected globally.

(Modification)
In the first and third embodiments described above, the educational terminal renders the lesson video data. However, this is merely an example. The content distribution server may render the lesson video data. In this case, the educational terminal receives and stores the lesson video data from the content distribution server instead of the specification data and motion data, so no renderer is required. On the other hand, the content distribution server transmits the lesson video data to the educational terminal instead of the specification data and motion data, so a renderer is required.

The above-described embodiments are merely illustrative examples to aid in understanding the concept of the present invention, and are not intended to limit the scope of the present invention. Various components may be added, deleted, or modified in the embodiments without departing from the spirit of the present invention.

In the above embodiment, several functional units have been described, but these are merely examples of implementation of each functional unit. For example, multiple functional units described as being implemented in one device may be implemented across multiple separate devices, and conversely, functional units described as being implemented across multiple separate devices may be implemented in one device.

The various functional units described in each of the above embodiments may be realized by using a circuit. The circuit may be a dedicated circuit for realizing a specific function, or may be a general-purpose circuit such as a processor.

At least a part of the processing of each of the above embodiments can be realized by using, for example, a processor installed in a general-purpose computer as basic hardware. The program that realizes the above processing may be provided by being stored in a computer-readable recording medium. The program is stored in the recording medium as a file in an installable format or an executable format. Examples of the recording medium include magnetic disks, optical disks (CD-ROM, CD-R, DVD, etc.), magneto-optical disks (MO, etc.), and semiconductor memories. Any recording medium can be used as long as it can store a program and is computer-readable. In addition, the program that realizes the above processing may be stored on a computer (server) connected to a network such as the Internet, and downloaded to a computer (client) via the network.

REFERENCE SIGNS LIST 10, 70... Educational terminal 20... Content distribution server 31, 33... Teacher avatar 32... Blackboard object 34, 35, 36... Student avatar 50... Content creation device 101, 201, 701... Communication I/F
102, 501, 702... Input/output I/F
110, 210, 510: Processor 111, 211, 711: Received data acquisition unit 112, 511, 712: Renderer 113, 512, 713: Content playback unit 114, 513, 714: Input data acquisition unit 115: Input data report unit 120, 220, 520: Memory 121, 221, 521: Material data storage unit 212, 715: Material data change unit 213: Material data distribution unit 514: Material data generation unit

Claims (13)

A content distribution server capable of providing a predetermined content to a terminal,
a first material data storage unit configured to store material data including first specification data indicating specifications of a plurality of objects to be placed in a virtual space that constitutes the predetermined content;
a received data acquisition unit that acquires a request for changing at least a part of the first specification data from the terminal ;
a material data changing unit that changes the stored first specification data to second specification data different from the first specification data in response to the request from the terminal under a condition;
A content distribution server comprising: The content distribution server according to claim 1, wherein the material data change unit responds to the request with conditions including at least one of charging, majority vote, lottery, or first come, first served. The content distribution server according to claim 1 or 2, wherein the first specification data includes at least one of an object's appearance, a voice character, a sound effect, or music. The content distribution server according to any one of claims 1 to 3, wherein the specified content is educational content for distance learning. The content distribution server according to any one of claims 1 to 4, wherein the request includes at least one of a request to replace specification data of a specific object, a request to add specification data of a new object, or a request to delete specification data of a specific object. The content distribution server according to any one of claims 1 to 5, wherein the material data modification unit modifies the appearance of the avatar, the voice character, or the sound effects and/or music that the multiple objects can generate, or adds new objects. The content distribution server according to any one of claims 1 to 6, wherein the material data includes initial specification data. A system including a content distribution server capable of providing a predetermined content to a terminal ,
The content distribution server includes:
a first material data storage unit configured to store material data including first specification data indicating specifications of a plurality of objects to be placed in a virtual space that constitutes the provided predetermined content;
a received data acquisition unit that acquires a request for changing at least a part of the first specification data from the terminal ;
a material data changing unit that changes the stored first specification data to second specification data different from the first specification data in response to the request from the terminal under a condition;
a material data distribution unit that distributes the material data in response to a distribution request from the terminal ;
Equipped with
The terminal includes :
a received data acquisition unit that acquires the material data from the content distribution server;
A second material data storage unit that stores the material data;
Equipped with
system. A terminal that receives a predetermined content from a content distribution server that can provide the predetermined content,
the content distribution server is configured to store material data including first specification data indicating specifications of a plurality of objects to be placed in a virtual space constituting the predetermined content, and, in response to a request from the terminal to change at least a part of the first specification data under a condition , change the stored first specification data to second specification data different from the first specification data, and distribute the material data ;
The terminal includes:
a received data acquisition unit that acquires the material data from the content distribution server;
A second material data storage unit that stores the material data;
Equipped with
Terminal. A method executed by an apparatus for distributing content to a terminal , comprising:
The apparatus,
storing material data including first specification data indicating specifications of a plurality of objects arranged in a virtual space that constitutes the content to be reproduced;
obtaining , from the terminal, a request to change at least a portion of the first specification data;
and changing the stored first specification data to second specification data different from the first specification data in conditional response to the request from the terminal .
method. A content distribution method in a system including a content distribution server capable of providing a predetermined content to a terminal , comprising:
The content distribution server includes:
storing material data including first specification data indicating specifications of a plurality of objects to be placed in a virtual space constituting the provided predetermined content in a first material data storage unit;
obtaining , from the terminal, a request to change at least a portion of the first specification data;
changing the stored first specification data to second specification data different from the first specification data in conditional response to the request from the terminal ;
and distributing material data including the first specification data in response to a distribution request from the terminal ,
The terminal includes :
receiving the distributed material data;
storing the received material data in a second material data storage unit;
Equipped with
Content delivery methods. A program that causes a processor to execute the processing of each unit of the content distribution server according to any one of claims 1 to 7. A program that causes a processor to execute the processing of each unit included in the terminal according to claim 9.